Despite measurement advances in implementation science, few studies have assessed the psychometric properties of implementation outcome measures in low- and middle-income countries. Reliance on a limited set of tools may constrain understanding. We conducted a psychometric evaluation of brief measures of acceptability, appropriateness, and feasibility from the mental health implementation science tools (mhIST) with youth in two NIH-funded HIV prevention and care projects in Africa.

A consortium working group selected items to generate three shortened measures: a 7-item acceptability scale, a 5-item appropriateness scale, and an 8-item feasibility scale. Data drew from the IMARA-South Africa HIV/STI prevention program for adolescent girls (15–19 years) and their caregivers (n = 514) and CombinADO multicomponent intervention for youth living with HIV (10–24 years) in Mozambique (n = 385). We conducted factor analysis to explore dimensionality, test measurement invariance across two independent samples, and assess internal consistency reliability.

Results supported a one-factor model for acceptability imposing metric invariance across samples with reliability ranging from questionable to acceptable (α = 0.64, 0.77). Results also supported a one-factor model for appropriateness imposing metric invariance, though reliability varied (α = 0.73 vs. 0.53). Scalar invariance was not supported, suggesting differences in item interpretation across samples. No clear factor structure emerged for feasibility. Correlations among measures aligned with theoretical expectations.

The brief mhIST measures of acceptability and appropriateness show promise for use in resource-constrained settings, though the feasibility measure may require refinement. Findings contribute to advancing pragmatic measurement approaches in global implementation science.