The subjective reporting of memory symptoms is helpful in discovering the nature, extent, and functional impact of neurocognitive difficulties experienced in daily life. Clinical and research criteria for diagnoses of some neurocognitive disorders include subjective neurocognitive symptoms (Bondi et al., Reference Bondi, Edmonds, Jak, Clark, Delano-Wood, McDonald and Salmon2014). Elevated rates of subjective memory symptoms are a defining feature of Subjective Cognitive Decline (Jessen et al., Reference Jessen, Amariglio, van Boxtel, Breteler, Ceccaldi and Chételat2014), evident in many clinical populations (Lehrner et al., Reference Lehrner, Kogler, Lamm, Moser, Klug, Pusswald and Auff2015), and can predict incident neurocognitive disorders (Tsutsumimoto et al., Reference Tsutsumimoto, Makizako, Doi, Hotta, Nakakubo, Makino and Suzuki2017). Subjective memory symptoms are associated with dependence in activities of daily living (McAlister & Schmitter-Edgecombe, Reference McAlister and Schmitter-Edgecombe2016) and healthcare noncompliance (Hertzog, Park, Morrell, & Martin, Reference Hertzog, Park, Morrell and Martin2000). Thus, it is important to understand the reliability and validity of the methods with which neuropsychologists ascertain subjective memory symptoms.
A paradoxical issue in assessing subjective memory complaints is that the very failures in episodic memory that the clinician wishes to probe can themselves interfere with the recollection of the target symptoms (Clare, Whitaker, & Nelis, Reference Clare, Whitaker and Nelis2010). Subjective memory symptoms show small and inconsistent associations with performance-based episodic memory (Dhillon, Videla-Nash, Foussias, Segal, & Zakzanis, Reference Dhillon, Videla-Nash, Foussias, Segal and Zakzanis2020). And informant reports are not always available. Thus, measures of subjective memory symptoms may have suboptimal reliability and validity among individuals with objective memory impairment.
Measurement invariance techniques may shed some light on the inconsistent associations between subjective memory symptoms and objective memory performance. Measurement invariance is a statistical property that, if present, indicates that the same underlying construct is measured in the same way across groups (Vandenberg & Lance, Reference Vandenberg and Lance2000). There are multiple sequential steps to establishing measurement invariance: (1) configural invariance is defined as whether the same items assess the same latent variable(s) across groups, (2) weak invariance is defined as whether the factor loadings of items are equivalent across groups, and (3) strong invariance determines whether participants with equal values on the latent construct also have equal values on measure items (Gregorich, Reference Gregorich2006). If a test demonstrates measurement invariance in these three areas across two groups, then that test assesses the same factor structure within both groups and any differences on observed scores are equal to differences on underlying factor scores (e.g., Siedlecki et al., Reference Siedlecki, Manly, Brickman, Schupf, Tang and Stern2010).
Here we evaluate the measurement invariance of the Prospective and Retrospective Memory Questionnaire (PRMQ; Smith, Della Sala, Logie, & Maylor, Reference Smith, Della Sala, Logie and Maylor2000) among persons with and without objective HIV-associated memory impairment. The PRMQ is a measure of subjective memory symptoms that has been cited over 500 times (Google Scholar) and shows evidence of both reliability (Chronbach’s α = .89; Crawford, Smith, Maylor, Della Sala, & Logie, Reference Crawford, Smith, Maylor, Della Sala and Logie2003) and validity (Rönnlund, Mäntylä, & Nilsson, Reference Rönnlund, Mäntylä and Nilsson2008). Multiple factor structures have been described (Crawford et al., Reference Crawford, Smith, Maylor, Della Sala and Logie2003), including a “tripartite” structure with three latent factors: general memory, retrospective memory, and prospective memory. Clinical populations including persons with HIV (PWH) produce elevated scores on the PRMQ (Woods et al., Reference Woods, Carey, Moran, Dawson, Letendre and Grant2007), which are independently associated with medication nonadherence (Woods, Moran et al., Reference Woods, Moran, Carey, Dawson, Iudicello and Gibson2008), instrumental ADLs (Woods, Iudicello et al., Reference Woods, Iudicello, Moran, Carey, Dawson and Grant2008), and health-related quality of life (Doyle et al., Reference Doyle, Weber, Atkinson, Grant and Woods2012).
While some studies show that objective memory impairment relates to elevated PRMQ scores (Marcone et al., Reference Marcone, Gagnon, Desjardins, David, Postuma, Montplaisir and Rouleau2019; Vaskivuo et al., Reference Vaskivuo, Hokkanen, Hänninen, Antikainen, Bäckman, Laatikainen and Ngandu2018), the effect sizes are commonly small. Other studies show no relationship between objective memory impairment and subjective memory symptom reporting on the PRMQ (e.g., Lee et al., Reference Lee, Ong, Pike, Mullaly, Rand, Storey and Kinsella2016), even when considering varying degrees of objective memory severity (Ryu, Lee, Kim, & Lee, Reference Ryu, Lee, Kim and Lee2016). Specifically, within groups with known memory impairment the association between objective memory performances and subjective memory symptoms is weak (e.g., de Mendonça et al., Reference de Mendonça, Felgueiras, Verdelho, Câmara, Grilo, Maroco and Guerreiro2018; Sugden, Thomas, & Kiernan, Reference Sugden, Thomas and Kiernan2021). Previous studies suggest that mood (Yoo-Jeong et al., Reference Yoo-Jeong, Anderson, Rahman, Baumann, McBroom and Waldrop-Valverde2018) and the high base rates of memory symptoms (Reid & MacLullich, Reference Reid and Maclullich2006) may help explain the weak associations between objective memory functioning and subjective symptoms. Here we investigate objective memory functioning as another possible contributor to these weak and inconsistent findings. Although previous studies suggest that factor scores of memory symptom questionnaires show differences in associations with objective memory performance across cognitive impairment status (Cook & Marsiske, Reference Cook and Marsiske2006), and that factor structure of self-reported cognitive symptoms differs across demographic groups (e.g., Filshtein et al., Reference Filshtein, Chan, Mungas, Whitmer, Fletcher, DeCarli and Farias2020), no previous study has explicitly examined the factor structure or properties of a memory symptom inventory across persons with and without objective memory impairment.
There are multiple features of HIV disease that make PWH a particularly informative population for the examination of memory symptoms. First, HIV neuropathogenesis affects primarily prefrontal systems, as well as the hippocampus, both of which affect multiple aspects of declarative memory (Castelo, Sherman, Courtney, Melrose, & Stern, Reference Castelo, Sherman, Courtney, Melrose and Stern2006). Next, deficits in memory are among the most common neurocognitive sequelae of HIV (Rippeth et al., Reference Rippeth, Heaton, Carey, Marcotte, Moore and Gonzalez2004). The memory profile observed in HIV is a mild-to-moderate mixed encoding/retrieval deficit (Woods et al., Reference Woods, Scott, Dawson, Morgan, Carey and Heaton2005), which can adversely affect activities of daily living (Obermeit et al., Reference Obermeit, Morgan, Casaletto, Grant and Woods2015). Memory symptoms are also elevated in PWH (Sheppard, Woods, Massman, & Gilbert, Reference Sheppard, Woods, Massman and Gilbert2019) and tend to not be strongly related to objective memory performance (Hinkin et al., Reference Hinkin, van Gorp, Satz, Marcotte, Durvasula, Wood and Baluda1996). Nevertheless, memory symptoms, including those measured by the PRMQ, are reliably and independently related to poorer daily functioning (Casaletto, Doyle, Weber, Woods, & HNRP Group, Reference Casaletto, Doyle, Weber and Woods2014).
This study therefore sought to (1) replicate the tripartite factor structure of the PRMQ described by Crawford et al. (Reference Crawford, Smith, Maylor, Della Sala and Logie2003) in a sample of PWH, (2) examine model fit of this PRMQ factor structure within PWH with and without objective memory impairment, and (3) test for measurement invariance of the PRMQ by conducting a series of confirmatory factor analysis (CFA) models with increasing levels of cross-group equality constraints imposed on factor loadings and item thresholds. For this final step, we specifically tested for configural, weak, and strong invariance (Gregorich, Reference Gregorich2006) on the PRMQ between groups with intact versus impaired objective memory performance.
Method
Participants
The sample included 505 PWH (aged 18–79 years) enrolled in studies at the HIV Neurobehavioral Research Program (see Table 1). All study procedures adhered to the ethical standards of the institutional research committee and the 1964 Helsinki declaration and later amendments. Participants were recruited from the community and local HIV clinics. HIV serostatus was confirmed using standard Western blot and/or a point-of-care test (MedMira Inc.). Participants were excluded if they had current substance disorders as operationalized by the Composite International Diagnostic Interview, version 2.1 or the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST; WHO ASSIST Working Group, 2002). Additional exclusion criteria consisted of psychotic disorders, major neurological disease (e.g., major head injury, seizure disorder, multiple sclerosis), or an estimated verbal IQ <70 on the Wechsler Test of Adult Reading (Psychological Corporation, 2001).
Demographic, cognitive, mood, substance use, medical, disease, and everyday functioning characteristics for the memory impaired and memory unimpaired groups.
PRMQ = Prospective and Retrospective Memory Questionnaire; CD4 = cluster of differentiation.
a Mean demographically adjusted T score across available objective memory measures.
b Diagnosis of Major Depression Disorder or Generalized Anxiety Disorder (n = 465) or elevated Brief Symptom Inventory (n = 40).
c Any lifetime diagnosis of dependence on alcohol or illicit substances among n = 465 with known lifetime diagnosis.
Data represent M (SD) or percentages.
Materials
Self-reported memory symptoms
Participants completed the PRMQ (Smith et al., Reference Smith, Della Sala, Logie and Maylor2000), which measures the frequency of memory difficulties in everyday life on a scale that ranges from 1 (“never”) to 5 (“very often”). The PRMQ includes eight retrospective memory and eight prospective memory items. Raw scores on each PRMQ item were used for all analyses (sample Chronbach’s α = .94).
Objective memory impairment
Participants were drawn from one of four ongoing studies that used different batteries to accomplish their aims (see Table 2). We included participants across these studies to maximize our sample size and to demonstrate the robustness of the findings, which did not differ meaningfully in the largest subsample (n = 359). To ensure a consistent approach to defining memory impairment across these protocols, individual demographically adjusted T-scores for each measure were converted to deficit scores that range from 0 (normal) to 5 (severely impaired; Carey et al., Reference Carey, Woods, Gonzalez, Conover, Marcotte and Grant2004) using the following conversions: ≥40T = 0 deficit score, 39T-35T = 1, 34T-30T = 2, 29T-25T = 3, 24T-20T = 4, ≤19 = 5. These individual deficits scores were then averaged to generate a memory domain deficit score. Consistent with diagnostic guidelines for HIV-associated neurocognitive disorders (Antinori et al., Reference Antinori, Arendt, Becker, Brew, Byrd, Cherner and Wojna2007), participants with memory domain deficit scores >.5 were classified as impaired (i.e., > 1.0 SD below the normative mean). Note that alternate cut points were used for any person with only a single memory measure (see Table 2). In the current sample, a total of 141 (27.9%) individuals had objective memory impairment, which is consistent with estimates of cognitive impairment in community-dwelling samples of PWH (Wang et al., Reference Wang, Liu, Lu, Farrell, Lappin, Shi and Bao2020).
Objective memory measures and rates of impairment across subsamples of the PWH study sample (n = 505).
WMS-III = Wechsler Memory Scale, 3rd Edition (Psychological Corporation, 1997); CVLT-II = California Verbal Learning Test, 2nd Edition (Delis, Kramer, Kaplan, & Ober, Reference Delis, Kramer, Kaplan and Ober2000); LM = WMS-III Logical Memory; LDFR = CVLT-II Long Delay Free Recall; HVLT-R = Hopkins Verbal Learning Test, Revised (Brandt & Benedict, Reference Brandt and Benedict2001); BVMT-R = Brief Visuospatial Memory Test – Revised (Benedict, Reference Benedict1997); CPAL = Continuous Paired Associate Learning test from the CogState (Collie, Maruff, Darby & McStephen, Reference Collie, Maruff, Darby and McStephen2003); OCL = One Card Learning task from the CogState (Collie, Maruff, Darby & McStephen, Reference Collie, Maruff, Darby and McStephen2003); PSMT = Picture Sequence Memory Test from the NIH Toolbox (Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer and Gershon2013).
a See Tierney et al. (Reference Tierney, Sheppard, Kordovski, Faytell, Avci and Woods2017) for description of subsample.
b See Faytell et al. (Reference Faytell, Doyle, Naar-King, Outlaw, Nichols, Casaletto and Woods2017) for description of subsample. The relatively higher rate of impairment in this subsample of 40 participants is consistent with those reported in previous studies using this subsample of youth (age 18–24 years) and demonstrated comorbidities consistent with other studies of youth PWH (see Loft et al., Reference Loft, Doyle, Naar-King, Outlaw, Nichols, Weber and Woods2014 for details).
c See Woods et al., Reference Woods, Morgan, Loft, Matchanova, Verduzco and Cushman2020 for description of subsample.
d Objective memory impairment was determined by a more conservative deficit score of > 1.0, given that there was only one test outcome used to determine impairment as opposed to two tests in other three subsamples (see Carey et al., Reference Carey, Woods, Gonzalez, Conover, Marcotte and Grant2004).
Procedure
After providing written informed consent, all participants completed psychiatric, neuropsychological, and standardized medical research evaluations, for which they received nominal financial compensation.
Data analysis procedures
All analyses were conducted using Mplus Version 8.1. Models were estimated using weighted least squares with mean and variance adjusted (WLSMV) and theta parameterization. Model fit was determined using Chi-square likelihood ratio test, Root Mean Square Error of Approximation (RMSEA<.10 acceptable, <.05 good fit), and Comparative Fit Index and Tucker-Lewis Index (CFI and TLI >.90 acceptable, >.95 good fit; Byrne, Reference Byrne2012). Model comparisons were conducted using the DIFFTEST procedure in Mplus, which computes a chi-square difference test using derivatives from nested models (henceforth Δχ2 Test) and is the recommended method (as opposed to ΔCFI, ΔRMSEA) for comparing models estimated by WLSMV (Muthén & Muthén, Reference Muthén and Muthén2017; Sass, Schmitt, & Marsh, Reference Sass, Schmitt and Marsh2014). Although not explicitly used to determine change in fit of models, ΔCFI (i.e., invariance if ≥−.010; Chen, Reference Chen2007) and ΔRMSEA (i.e., invariance if ≥.015; Chen, Reference Chen2007) were reported as additional markers of model comparisons. PRMQ scores were nonnormally distributed (Shapiro-Wilk W range = .73-.91, ps < .001). There were no missing data for the PRMQ in the current study.
After confirming the PRMQ factor structure in the full sample, multisample CFA models were then fit separately to the data from memory impaired and memory unimpaired groups. Invariance was subsequently assessed using a series of CFA models with increasing levels of cross-group equality constraints imposed on factor loadings and item thresholds. Specifically, configural invariance was assessed by imposing no equality constraints across groups for any model parameter estimates, weak invariance was tested by constraining factor loadings to be equal across both groups, and strong invariance was evaluated by imposing equality constraints on factor loadings and item thresholds across groups. Across these sequential CFA models, invariance was first assessed in the configural invariance model by estimating the model simultaneously in each group and was determined to be present if this model showed adequate model fit using fit indices. Next, the weak invariance model was compared to the configural invariance model, whereby weak invariance was deemed to be present if the DIFFTEST (Δχ2 test) did not reach statistical significance using Bonferroni-corrected alpha (α = .025; Ottenbacher, Reference Ottenbacher1998). Finally, the strong invariance model was compared to the weak invariance model using the Δχ2 test, whereby strong invariance was judged to be present if it did not reach statistical significance using Bonferroni-corrected alpha set at α = .025.
Results
Clinicodemographic Characteristics of Memory Impaired and Unimpaired Groups
Table 1 presents the clinicodemographic characteristics in persons with (n = 141) and without (n = 364) objective memory impairment. The memory impaired group had fewer men (odds ratio = 1.8 [1.1,3.1]) and a lower proportion of White participants (odds ratio = 2.3 [1.5,3.4]) and consisted of individuals that were younger (Cohen’s d = .23) and obtained fewer years of education (d = .53). Among these variables, only gender was associated with PRMQ total scores (d = .41, p = .002), such that women had more memory symptoms. No other variables that differed between memory impaired and unimpaired groups were significantly related to PRMQ total scores across the entire sample (all ps > .10).
Confirmatory factor analysis
In the entire sample (n = 505), CFA using the tripartite (bifactor) model proposed by Crawford et al. (Reference Crawford, Smith, Maylor, Della Sala and Logie2003) demonstrated adequate fit across indices (χ2[88] = 335.1, RMSEA = .08, CFI = .98, TLI = .97). However, examination of standardized item loadings on each factor revealed that although the general memory factor had adequate loadings (range .58 to .83, all ps < .001), there were nonsignificant loadings on the prospective memory factor (range −.13 to .33, p range < .001 to .901) and trend-level loadings on the retrospective memory factor (range −.21 to .45, p range < .001 to .05). Given that theory would not predict significant positive loadings onto a general memory factor and nonsignificant or significant negative loadings onto the subfactors, we elected to consider the more parsimonious models (i.e., those without bifactor structure) proposed in the original factor analytic study (Crawford et al., Reference Crawford, Smith, Maylor, Della Sala and Logie2003). Among these, a model with two correlated factors (retrospective memory, prospective memory) demonstrated adequate fit (χ2[103] = 567.8, RMSEA = .10, CFI = .97, TLI = .96); however, the two factors were highly correlated (r = .97). Therefore, we elected to examine a model with a single general memory factor (see Figure 1) that has been shown to have generally adequate fit in previous factor analytic studies (Crawford et al., Reference Crawford, Smith, Maylor, Della Sala and Logie2003; Zimprich, Kliegel, & Rast, Reference Zimprich, Kliegel and Rast2011). Fit indices of the CFA models examined in the whole sample, memory impaired group, memory unimpaired group, and measurement invariance models can be found in Table 3. Across the entire sample, the one-factor (general memory) structure indicated adequate fit (χ2[104] = 574.9, RMSEA = .10, CFI = .96, TLI = .96), and item factor loadings ranged from .63 to .84 (all ps < .001; Figure 1). Within the memory impaired group only (n = 141), the same CFA model also demonstrated adequate fit across indices (χ2[104] = 229.0, RMSEA = .09, CFI = .97, TLI = .96). This model in the memory unimpaired group (n = 364) also revealed adequate fit across indices (χ2[104] = 407.3, RMSEA = .09, CFI = .97, TLI = .96).
Confirmatory factor analysis with standardized factor loadings (all ps < .001) for one-factor (general memory) model on the Prospective and Retrospective Memory Questionnaire in the study sample.
Model fit indices and model comparison statistics for measurement invariance models across memory impaired (n = 141) and memory unimpaired (n = 364) groups.
* p < .025 (Bonferroni-corrected alpha).
Δχ2 Test value is derived from the DIFFTEST procedure in Mplus 8.1 (Muthén & Muthén, Reference Muthén and Muthén2017).
Configural invariance
The configural invariance model was assessed simultaneously across memory impaired and memory unimpaired groups without imposing equality constraints across groups for any model parameter estimates. This model revealed adequate fit (χ2[208] = 616.6, RMSEA = .09, CFI = .97, TLI = .96), indicating that the same PRMQ items measured the identified one-factor model of memory symptoms for both memory impaired and memory unimpaired groups (see Table 3).
Weak invariance
The weak invariance model was assessed simultaneously across memory impaired and unimpaired groups while constraining factor loadings to be equal across both groups. This model revealed adequate fit (χ2 [223] = 456.3, RMSEA = .06, CFI = .98, TLI = .98). Comparison of this model to the configural model using the Δχ2 test of model fit provided evidence of weak invariance (Δχ2[15] = 13.1, p = .589) in addition to no worsening in model fit (ΔCFI = .01, ΔRMSEA = −.03), suggesting that factor loadings of items were equivalent across groups.
Strong invariance
Strong invariance was assessed using an identical model to that of the weak invariance model across memory impaired and unimpaired groups while additionally imposing equality constraints on item thresholds across groups. This model showed acceptable fit (χ2[286] = 540.4, RMSEA = .06, CFI = .98, TLI = .98). Comparison of this model to the weak model indicated a lack of strong invariance (Δ χ2[63] = 103.5, p = .001), suggesting that item thresholds did differ across groups (i.e., noninvariance). However, the strong model did not demonstrate different fit compared to the weak model (ΔCFI = .00, ΔRMSEA = .00). Examination of modification indices in the strong invariance model revealed a modification index of 7.8 for PRMQ item 6 thresholds (“Do you fail to recognize a character in a radio or television show from scene to scene?”). Thus, the strong invariance model was re-examined without imposing threshold equality constraints for this item, which resulted in a model that showed acceptable fit (χ2[282] = 527.6, RMSEA = .06, CFI = .98, TLI = .98). Comparison of this model to the weak invariance model still indicated a lack of strong invariance (Δ χ2[59] = 86.5, p = .011) and no change in fit (ΔCFI = .00, ΔRMSEA = .00). Examination of modification indices in this partial strong invariance model revealed a modification index of 5.4 for PRMQ item 2 thresholds (“Do you fail to recognize a place you have visited before?”). Thus, the strong invariance model was re-examined without imposing equality constraints on both item 2 and 6 thresholds, which resulted in a model that showed acceptable fit (χ2 [278] = 518.6, RMSEA = .06, CFI = .98, TLI = .98). Comparison of this model to the weak invariance model was not significant (Δχ2[55] = 73.8, p = .046), although there was no change in model fit (ΔCFI = .00, ΔRMSEA = .00), indicating partial strong invariance. In other words, all items on the PRMQ other than items 2 and 6 have the same expected observed response at the same absolute value of the latent trait of general memory across both groups. Examination of thresholds of items 2 and 6 in the partial strong invariance model demonstrated lower item thresholds for the memory impaired group (item 6 standardized threshold range: −.63 to 1.93; item 2 standardized threshold range: −.22 to 2.16) compared to the memory unimpaired group (item 6 standardized threshold range: .23 to 2.07; item 2 standardized threshold range: −.05 to 2.3), such that the impaired group had a lower probability of reporting better PRMQ scores compared to the memory unimpaired group on these items.
Post Hoc Measurement Invariance: Moderately Impaired Memory Group
To examine the effect of memory impairment using a more conservative cutoff (i.e., deficit score >1.0), we repeated the above measurement invariance testing steps using a smaller group (n = 77) of moderately memory impaired participants compared to those without memory impairment (n = 428). Clinicodemographic characterizations of these groups and detailed findings from invariance testing can be found in supplementary tables S1, S2, and supplementary results. In summary, model fit was adequate in the memory impaired group (χ2[104] = 187.1, RMSEA = .10, CFI = .96, TLI = .95) and the memory unimpaired group (χ2[104] = 471.1, RMSEA = .09, CFI = .97, TLI = .96). There was evidence of configural invariance (χ2[208] = 580.0, RMSEA = .08, CFI = .97, TLI = .97). Compared to the configural model, there was a weak noninvariance (Δ χ2[15] = 44.1, p < .001), which after iteratively freeing item loadings for items 12, 6, and 3 (see supplementary results for details of items and results) there was evidence of partial weak invariance (Δ χ2[12] = 22.5, p = .032), such that these items were more strongly related to the construct of general memory within the memory impaired group compared to those in the memory unimpaired group while all other item loadings demonstrated invariance. Finally, there was a lack of strong invariance (Δ χ2[63] = 94.4, p = .006). After freeing item thresholds for item 2, there was evidence of partial strong invariance (Δ χ2[59] = 73.3, p = .100), such that the impaired group had a lower probability of reporting better PRMQ scores compared to the memory unimpaired group on this item.
DISCUSSION
Prior studies suggest that subjective recall of memory symptoms may be unreliable in the setting of objective memory deficits. The current study addressed this question by using measurement invariance techniques to evaluate whether the PRMQ assesses memory symptoms in the setting of objective memory impairment in PWH disease. All told, our data showed that 1) the PRMQ had a robust factor structure among persons with and without objective mild-to-moderate HIV-associated memory impairment; 2) individuals with memory impairment reported slightly more severe observed memory symptoms on low base rate PRMQ items than their factor score might otherwise reflect, which may therefore overestimate their actual severity; and 3) for individuals with at least moderate HIV-associated memory impairment, some select PRMQ items specifically measuring memory for intentions and recognition memory may be more strongly reflect of the construct of general memory symptoms compared to those in memory unimpaired individuals. These findings are interpreted below in the context of the strengths and weaknesses of the study sample, design, and methods.
A primary finding from this study is that the PRMQ showed a robust one-factor model that had adequate fit when examined in the full sample of PWH and also when examined separately in subsamples of PWH with and without objective mild-to-moderate memory impairment. The latter findings provide support for the configural invariance of the PRMQ in the context of memory deficits; that is, the fundamental factor structure of self-reported memory symptoms in daily life as measured by the PRMQ was broadly consistent across persons with and without HIV-associated memory impairment. Additionally, the PRMQ showed evidence of weak invariance, meaning that the factor loadings were statistically comparable in persons with and without HIV-associated memory impairment. Together, this pattern of findings indicates that the latent construct of memory symptoms measured by the PRMQ is not affected by whether the responder has mild HIV-associated memory impairment on performance-based tasks. This is important because it argues against the notion that individuals with objective memory impairment may be responding on the PRMQ in a manner that reflects a separate construct of level of insight into memory (i.e., ability to recall instances of memory failures) as opposed to perceived memory symptoms.
That said, the PRMQ did show a very modest lack of strong invariance, meaning that some item thresholds on this measure may differ across persons with and without objective HIV-associated memory impairment. This interpretation is also supported by follow-up analyses showing partial-strong invariance by freely estimating only two PRMQ items. In practical terms, a lack of strong invariance can be thought of as reflecting a response bias between groups (Cheung & Rensvold, Reference Cheung and Rensvold2000). This lack of invariance can be problematic when making across-group comparisons because differences in observed scores for two individuals may not accurately reflect differences in true scores. By way of example, a lack of strong invariance is commonly observed for items assessing crying behaviors on depression scales across men and women (Byrne, Baron, & Campbell, Reference Byrne, Baron and Campbell1993; Wu, Reference Wu2010). Specifically, a lack of strong invariance on such items often indicates that a woman who endorses an observed score that is 1 or 2 points higher than a man still reflects the same level of depression.
In the context of the current study, the pattern of findings revealed that the memory impaired group had lower item thresholds (i.e., lower likelihood of reporting better PRMQ scores) compared to the memory unimpaired group. This indicates that the memory impaired group was more likely to report observed scores that were relatively worse (i.e., higher PRMQ scores) than their latent score of memory symptoms for each item on the PRMQ. This effect was most pronounced on item 2 (“Do you fail to recognize a place you have visited before?”) and item 6 (“Do you fail to recognize a character in a radio or television show from scene to scene?”). Thus, the scores of an individual in the memory impaired group who endorsed the frequency of these items as “Quite Often” could conceivably be interpreted as the same severity as a person in the within normal limits group who endorsed them as “Sometimes”. Altogether, this pattern of findings indicates that caution should be taken when interpreting observed scores on the PRMQ, as observed scores may be slightly higher (i.e., worse memory symptom reporting) than expected latent scores for individuals with objective memory impairment. The content assessed by items 2 and 6 are relatively uncommon memory complaints and had the lowest mean observed score across both study groups. Moreover, these two items are the only items on the PRMQ to use the term “recognize” in item wording (vs. “forget” or “fail to recall”). This is notable because the memory profile in PWH is most commonly seen as encoding/retrieval deficits marked by intact recognition (Woods et al., Reference Woods, Scott, Dawson, Morgan, Carey and Heaton2005). Thus, endorsement of this relatively uncommon and potentially more severe symptom by a person with objective memory impairment may be worth closer examination.
The current study utilized a relatively liberal 1.0 SD cutoff for determining objective memory impairment in the primary analyses. However, post hoc analyses using a more conservative 1.5 SD cutoff confirmed configural invariance in this group, but also revealed preliminary evidence of partial weak and partial strong measurement invariance in persons with greater levels of objective memory impairment. That is, they also demonstrated a stronger association between items loading onto the general memory factor compared to those in the memory unimpaired group. These findings were specific to items that assessed failures to (1) perform a task a few moments after it was intended, (2) recognize a TV character after a few moments, and (3) pass information to another person. Two of these three items assess prospective memory, which is commonly affected in PWH (Woods et al., Reference Woods, Moran, Carey, Dawson, Iudicello and Gibson2008). Although these data provide preliminary evidence that more significant memory impairment could contribute to differential item characteristics, it is particularly important to note that the moderately impaired group differed from the nonimpaired group on non-HIV characteristics (e.g., demographics and mood), which together are a limitation for interpreting findings as being specific to HIV-associated memory difficulties. Future studies should include well-matched HIV samples with severe memory impairment (e.g., HIV-associated dementia), as well as clinical groups with different memory profiles (e.g., primary encoding or forgetting), to determine if measurement invariance can be detected more reliably in these groups that are at higher risk for reduced insight or biased responding.
There are a number of limitations in the current study that are important to consider. First, our data did not replicate Crawford et al.’s (Reference Crawford, Smith, Maylor, Della Sala and Logie2003) original tripartite model of the PRMQ factor structure. Nevertheless, the one-factor model that we used here is more parsimonious and showed adequate fit in the original factor analytic validation study (Crawford et al., Reference Crawford, Smith, Maylor, Della Sala and Logie2003). Moreover, while performance-based data have demonstrated that prospective and retrospective memory are separable in PHW samples (e.g., Gupta et al., Reference Gupta, Woods, Weber, Dawson and Grant2010), studies of self-report show very high correlations between these two aspects of declarative memory symptoms (e.g., Woods et al., Reference Woods, Carey, Moran, Dawson, Letendre and Grant2007). Another limitation of the current study was the lack of controlling for mood variables within models, which was purposeful because original factor analytic studies of the PRMQ also did not control for depression or anxiety (see Crawford et al., Reference Crawford, Smith, Maylor, Della Sala and Logie2003). Although the current memory impairment groups did not significantly differ in mood variables (Table 2; note that supplementary analyses using a moderately impaired group did result in the impaired group having elevated anxiety compared to the memory unimpaired group, thus limiting HIV disease-specific interpretations of findings in this moderately impaired memory group), given reliable associations between mood and subjective cognitive symptoms (Dhillon et al., Reference Dhillon, Videla-Nash, Foussias, Segal and Zakzanis2020) future studies may examine samples without depression or anxiety to examine instances of measurement noninvariance on memory symptoms without possible contributions of mood symptoms. Another limitation of the current study is the cross-sectional design, and future studies would benefit from investigating measurement invariance within individuals between time points, particularly among individuals with potentially worsening objective memory difficulties. The current sample also did not have a full representation of ethno/racial groups and had the highest proportion of White individuals, and as such the generalizability to groups underrepresented in the current sample is limited (Avila et al., Reference Avila, Rentería, Witkiewitz, Verney, Vonk and Manly2020).
Notwithstanding these limitations, the current study has multiple strengths, including utilizing a robust CFA statistical approach, inclusion of a large clinical sample of PWH, examining a well-characterized measure of memory symptoms, and using a comprehensive neurocognitive approach to determining objective memory impairment. Of note, the current study expands on previous literature examining the PRMQ and other memory symptom questionnaires by investigating how measurement invariance could explain the tenuous associations between objective memory performance and reported memory symptoms. Future studies would benefit from using Item Response Theory to clarify the specific associations between observed scores and underlying factor scores of memory symptoms among individuals with memory impairment.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/S1355617721000448
ACKNOWLEDGMENTS
The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, nor the United States Government. The authors are grateful to Marizela Verduzco, Katie Doyle, Matthew Dawson, Jessica Beltran, and Javier Villalobos for study coordination, Donald R. Franklin and Stephanie Corkran for data processing, Dr. J. Hampton Atkinson and Jennifer Marquie Beck for participant recruitment, Drs. Scott Letendre, Sara Gianella Weibel, Sharon Nichols, Mark Bondi, and Elizabeth Twamley for serving as coinvestigators on the parent grants. This research also was supported resources from the Department of Veterans Affairs Office of Academic Affiliations, Advanced Fellowship Program in Mental Illness Research and Treatment, and the VA Puget Sound Health Care System, Seattle, Washington.
FINANCIAL SUPPORT
This research was supported by National Institutes of Health grants R01-MH073419, R01-DA034497, and P30-MH062512.
CONFLICT OF INTEREST
The authors report no conflicts of interest.
