Introduction
Mild cognitive impairment (MCI) is a prodromal state that represents a transitional period between normal aging and dementia. In most diagnostic schemes, criteria for MCI include a subjective memory complaint, preferably corroborated by an informant; objective evidence of cognitive impairment (1.5 standard deviations below normative means on one or more cognitive measures); preserved activities of daily living; and a failure to meet criteria for dementia (Petersen & Morris, Reference Petersen and Morris2005; Petersen, 2004; Winblad et al., Reference Winblad, Palmer, Kivipelto, Jelic, Fratiglioni, Wahlund and Petersen2004). Recent research using cluster analytic statistical techniques has shown that MCI cohorts based on these criteria present with heterogeneous cognitive profiles. Some individuals demonstrate deficits primarily in one area of cognitive ability (e.g., memory, executive functioning, language), while others demonstrate impairments in multiple cognitive domains (Bondi et al., Reference Bondi, Edmonds, Jak, Clark, Delano-Wood, McDonald and Salmon2014; Clark et al., Reference Clark, Delano-Wood, Libon, McDonald, Nation, Bangen and Bondi2013; Delano-Wood et al., Reference Delano-Wood, Bondi, Sacco, Abeles, Jak, Libon and Bozoki2009; Edmonds et al., Reference Edmonds, Delano-Wood, Clark, Jak, Nation, McDonald and Bondi2014; Libon et al., Reference Libon, Xie, Eppig, Wicas, Lamar, Lippa and Wambach2010). In addition, a large subgroup of participants actually performs within normal limits on a battery of neuropsychological tests that is independent of the memory test used in making the original MCI diagnosis (Bondi et al., Reference Bondi, Edmonds, Jak, Clark, Delano-Wood, McDonald and Salmon2014; Clark et al., Reference Clark, Delano-Wood, Libon, McDonald, Nation, Bangen and Bondi2013; Edmonds et al., Reference Edmonds, Delano-Wood, Clark, Jak, Nation, McDonald and Bondi2014). Based on their normal cognitive profile, normal cerebrospinal fluid (CSF) biomarkers, lower genetic risk for Alzheimer’s disease (AD), and low rates of progression to dementia in this “Cluster-Derived Normal” MCI subgroup compared to other MCI subgroups (see Bondi et al., Reference Bondi, Edmonds, Jak, Clark, Delano-Wood, McDonald and Salmon2014; Edmonds et al., Reference Edmonds, Delano-Wood, Clark, Jak, Nation, McDonald and Bondi2014), it appears that existing MCI criteria as operationalized by ADNI and others may be over-diagnosing this clinical entity (i.e., as much as one-third or more may be false positive diagnostic errors).
One aspect of the conventional diagnostic criteria for MCI that may contribute to false positive classifications is the inclusion of subjective memory complaints or concern as a feature of the diagnosis. The rationale behind considering subjective complaints is to capture the notion that there had been a change in an individual’s cognitive performance, thus excluding individuals with longstanding cognitive difficulties (e.g., learning disability) from a diagnosis of MCI (Petersen, Reference Petersen2004). However, the utility of this aspect of the criteria has been called into question by studies showing an inconsistent relationship between subjective memory complaints and objective memory performance in MCI (Buckley et al., Reference Buckley, Saling, Ames, Rowe, Lautenschlager, Macaulay and Ellis2013; Lenehan, Klekociuk, & Summers, Reference Lenehan, Klekociuk and Summers2012; Roberts, Clare, & Woods, Reference Roberts, Clare and Woods2009; Studer, Donati, Popp, & von Gunten, Reference Studer, Donati, Popp and von Gunten2013). There are multiple factors that could account for this weak relationship, including the possibility that cognitive complaints are more strongly related to emotional factors (i.e., depression, anxiety, neuroticism), personality features (Reid & MacLullich, Reference Reid and MacLullich2006; Studer et al., Reference Studer, Donati, Popp and von Gunten2013), or knowledge that one carries a risk factor for AD (Lineweaver, Bondi, Galasko, & Salmon, Reference Lineweaver, Bondi, Galasko and Salmon2014) than to actual cognitive ability. In addition, individuals who truly have objective cognitive impairments may report few or no subjective concerns due to reduced awareness (i.e., anosognosia) or an under-appreciation of their cognitive decline (Roberts et al., Reference Roberts, Clare and Woods2009). Finally, subjective memory complaints in the clinic setting may be relatively ubiquitous (i.e., the complaint is typically what generates the referral in the first place). Even in community settings, the prevalence of subjective memory complaints in older adults has ranged as high as 88% (for review, see Reid & MacLullich, Reference Reid and MacLullich2006). Thus, its differential utility may be diminished because of the very high base rate of reporting.
Another difficulty in using subjective memory complaints in the diagnosis of MCI is variability in how they are operationalized. Cognitive complaints are assessed in a variety of ways (e.g., interview, questionnaire) and may be scored as dichotomous or continuous. There is also variability in the source of the subjective complaint or concern, as it can be obtained from either the patient, an informant, or a skilled clinician (Albert et al., Reference Albert, DeKosky, Dickson, Dubous, Feldman, Fox and Phelps2011; Petersen et al., Reference Petersen, Aisen, Beckett, Donohue, Gamst, Harvey and Weiner2010). Finally, “cognitive complaint” and “memory complaint” are often used interchangeably when applying the diagnostic criteria (Petersen, 2004; Petersen & Morris, Reference Petersen and Morris2005), so it is unclear whether one must consider memory complaints per se or whether perceived changes in other cognitive abilities (e.g., subjective word-finding difficulty) fulfill this criterion.
Given these potential problems with considering subjective complaints in the diagnosis of MCI, the aim of the current study was to examine them in subgroups of the MCI cohort from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) to determine if consideration of subjective complaints augments or obscures the diagnosis of MCI. Consistent with our prior work, we hypothesized that a cognitive cluster analysis of the subset of ADNI MCI participants with Everyday Cognition (ECog) data would identify (1) one or more MCI subgroups with deficits in a single cognitive domain (e.g., amnestic, dysexecutive), (2) a mixed MCI subgroup with deficits in multiple domains, and (3) a Cluster-Derived Normal subgroup that performed within normal limits on cognitive testing. We further predicted that the Cluster-Derived Normal subgroup would endorse more cognitive complaints than reported by their informant (i.e., over-report subjective memory complaints), whereas the other MCI subtypes would endorse fewer cognitive complaints than reported by their informants. We also hypothesized that, across all MCI subgroups, self-reported cognitive complaints would be unrelated or only weakly related to objective cognitive performance, but positively related to symptoms of depression. In addition, informant-reported cognitive complaints would be more strongly associated with objective cognitive performance than self-reported complaints. Finally, we predicted that there would be little association between self-reported cognitive complaints and CSF biomarkers of AD or progression to dementia.
Methods
Data were obtained from the ADNI database (adni.loni.usc.edu). ADNI was launched in 2003 by the National Institute on Aging (NIA), National Institute of Biomedical Imaging and Bioengineering (NIBIB), Food and Drug Administration (FDA), private pharmaceutical companies, and non-profit organizations. The primary goal of ADNI is to test whether neuroimaging, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of MCI and early AD. ADNI is the result of efforts of many coinvestigators from a range of academic institutions and private corporations, and subjects have been recruited from over 50 sites across the United States and Canada. For more information, including inclusion and exclusion criteria, see www.adni-info.org.
Participants
Participants included 605 individuals enrolled in ADNI-GO and ADNI-2: 448 classified as MCI and 157 as cognitively normal. Participants represent a subset of the larger ADNI sample who received the ECog questionnaire (the ECog was not administered at baseline in ADNI-1). Participants were diagnosed with MCI based on conventional diagnostic criteria adapted for ADNI (Petersen & Morris, Reference Petersen and Morris2005; Petersen et al., Reference Petersen, Aisen, Beckett, Donohue, Gamst, Harvey and Weiner2010). Criteria for MCI were: (1) subjective memory complaint reported by participant or “study partner” (presumably determined by an interview; dichotomized as yes/no); (2) Mini-Mental State Examination (MMSE) score between 24 and 30 (inclusive); (3) global Clinical Dementia Rating Scale (CDR) score of 0.5; (4) abnormal memory function documented by scoring below education-adjusted cutoffs for delayed free recall (score of ≤11 for 16 or more years of education, ≤9 for 8–15 years of education, ≤6 for 0–7 years of education; maximum score=25) on Story A of the Wechsler Memory Scale-Revised (WMS-R) Logical Memory II subtest (Wechsler, Reference Wechsler1987), and (5) general cognition and functional performance sufficiently preserved to an extent that they could not qualify for a diagnosis of AD. Of the 448 MCI participants, 429 met all five criteria (18 meet four criteria, and 1 met three criteria).
The Normal Control (NC) group consisted of participants who had at least one year of follow-up data and who remained classified as normal for the duration of their participation in the study (range of 1–7 years of follow-up). Criteria for being classified as normal were: (1) no subjective memory complaint; (2) MMSE score between 24 and 30 (inclusive); (3) global CDR of 0; (4) intact memory function based on the WMS-R Logical Memory II; and (5) no significant impairment in cognitive functions or activities of daily living. The NC and MCI groups did not differ significantly in age, education, or gender distribution (p-values >.05). This study was approved by an ethical standards committee on human experimentation at each institution. Written informed consent was obtained from all participants in the study. All MCI and NC participants were required to have a “study partner” who had frequent contact with the participant (an average of 10 hours per week or more), and could accompany the participant to all clinic visits.
Materials and Procedure
Subjective Cognitive Complaints
Detailed information about subjective cognitive complaints was assessed via the ECog, which measures an individual’s ability to perform everyday tasks relative to 10 years ago. This instrument has been validated in MCI and AD samples, and informant-report on this measure has been shown to correlate with established measures of functional abilities and global cognition (Farias et al., Reference Farias, Mungas, Reed, Cahn-Weiner, Jagust, Baynes and Decarli2008). In addition, a recent study found that fewer informant-reported problems on the ECog was associated with better performance on neuropsychological testing, particularly on measures of memory and executive function, and larger brain volumes in the hippocampus, dorsolateral prefrontal cortex, and total brain (Farias et al., Reference Farias, Park, Harvey, Simon, Reed, Carmichael and Mungas2013).
The ECog consists of 39 items rated on the following scale: 1=no change or actually performs better than 10 years ago; 2=occasionally performs the task worse than 10 years ago but not all of the time; 3=consistently performs the task a little worse than 10 years ago; 4=performs the task much worse than 10 years ago; 5=do not know (these responses were treated as missing values). A Global Cognition score was calculated by averaging ratings for all 39 items. Scores were also calculated for six subscales that correspond to specific neuropsychological domains: Memory (eight items), Language (nine items), Visuospatial (seven items), Planning (five items), Organization (six items), and Divided Attention (four items). The current study focused on the Global Cognition scale and the Memory subscale, the latter of which has been shown to be the best subscale for differentiating MCI from cognitively normal individuals (Farias et al., Reference Farias, Mungas, Reed, Cahn-Weiner, Jagust, Baynes and Decarli2008).
Although the ECog was originally designed to be completed by an informant (Farias et al., Reference Farias, Mungas, Reed, Cahn-Weiner, Jagust, Baynes and Decarli2008), both participants and study-partners separately completed the questionnaire in ADNI. This allowed for a comparison between self- and informant-report. Discrepancy scores on the ECog were calculated for each participant by subtracting the informant’s rating from the participant’s rating for each item. Thus, a positive discrepancy score indicates that the participant is over-reporting/overestimating their cognitive decline relative to their informant, while a negative score indicates that the participant is under-reporting/underestimating their cognitive decline relative to their informant. It should be noted that an informant’s report may be affected by their relationship with the patient, recall bias, or emotional factors (depression or caregiver burden/stress in study partners was not assessed). Therefore, the discrepancy score does not represent accuracy of the participant’s self-report per se, but only the correspondence between the ratings of the participant and the informant.
Neuropsychological Battery
All participants completed a battery of neuropsychological tests during their baseline ADNI evaluation. The following six scores were included in the current analyses: (1) Animal Fluency; total score, (2) 30-item Boston Naming Test (BNT) total score; (3) Trail Making Test (TMT), Part A; time to completion, (4) TMT, Part B; time to completion, (5) Rey Auditory Verbal Learning Test (AVLT) 30-min delayed free recall; number of words recalled, and (6) AVLT recognition; number of words correctly recognized. These variables were selected because they assess three different domains of cognitive ability – language (Animal Fluency, BNT), attention/executive function (TMT, Parts A & B), and memory (AVLT recall & recognition), and they were administered to all ADNI participants.
CSF Biomarkers
Biological markers of AD were CSF concentrations of hyperphosphorylated tau (p-tau181p), beta-amyloid (Aβ1-42), and the ratio of p-tau181p/Aβ1-42 proteins. High levels of p-tau181p indicate neurofibrillary tangle pathology, and low levels of Aβ1-42 indicate amyloid plaque pathology. The ratio of these two variables has been shown to predict cognitive decline in individuals with MCI (Landau et al., Reference Landau, Harvey, Madison, Reiman, Foster, Aisen and Jagust2010).
Procedure
Participants underwent a “screening” visit, during which they completed the MMSE, CDR, Logical Memory, and Geriatric Depression Scale. They then underwent a “baseline” visit, at which point they completed the neuropsychological evaluation and the ECog questionnaire, and underwent lumbar puncture for CSF collection. According to the ADNI-GO and ADNI-2 procedure manuals, the window from “screening” to “baseline” was 28 days. Participants were then followed longitudinally (6–12 month follow-up visits).
Statistical Analyses
Cluster and Discriminant Function Analyses
Each MCI participant’s raw scores on the six neuropsychological variables were converted into standardized z-scores based on the means and standard deviations of the NC group. The z-scores were entered into a hierarchical cluster analysis that used Ward’s method, consistent with previous studies of MCI (Clark et al., Reference Clark, Delano-Wood, Libon, McDonald, Nation, Bangen and Bondi2013; Delano-Wood et al., Reference Delano-Wood, Bondi, Sacco, Abeles, Jak, Libon and Bozoki2009). To examine how well the final cluster solution best fit the data, a discriminant function analysis was conducted using the neuropsychological measures as predictors and the clusters as the outcome variable. The stability of the cluster solution was also examined using the leave-one-out cross-validation procedure, a method that reduces the potential bias of using the same individuals to develop the classification matrix and to compute the discriminant function.
Differences between the cluster groups and the NC group were examined using analysis of variance/analysis of covariance (ANOVA/ANCOVA) with post hoc comparisons. Bonferroni correction was used to adjust for multiple comparisons. Chi-square analyses were used to explore differences among the clusters in CSF biomarker characteristics and clinical outcome. Correlational analyses were used to examine the relationship between subjective cognitive ratings and objective cognitive performance or emotional functioning. Finally, independent samples t tests were used to compare participants with a positive or negative CSF biomarker, and those who did or did not progress to dementia.
Results
Cluster and Discriminant Function Analyses
Cluster analysis of the neuropsychological scores from the 448 MCI participants resulted in three distinct subgroups (see Figure 1): (1) Amnestic MCI (n=115; 25.7%) with isolated memory impairment; (2) Mixed MCI (n=148; 33.0%) with impairments in attention/executive function and naming; and (3) a Cluster-Derived Normal group (n=185; 41.3%) with normal group means on all six baseline cognitive measures. Discriminant function analysis using the six neuropsychological measures to predict group membership into the three cluster groups correctly classified 393/448 individuals (87.7% overall classification accuracy). Cross-validation of the three-cluster solution using the leave-one-out method showed only a mild, expected reduction in correct classification (86.6%).
Neuropsychological performance for the cluster groups. Error bars denote standard errors of the mean. The horizontal dotted line indicates the typical cutoff for impairment (−1.5 SDs). BNT=Boston Naming Test; TMT=Trail Making Test; AVLT=Rey Auditory Verbal Learning Test.
Characteristics of the Cluster and Normal Control Groups
Demographics and Neuropsychological Performance
The Cluster-Derived Normal group was significantly younger than the Amnestic, Mixed, and NC groups (p<.001; d=.50, .83, and .62, respectively); see Table 1. The Cluster-Derived Normal and NC groups were more educated than the Mixed group, although effect sizes were small (p=.005; d=.34 and .30). The groups did not differ in gender distribution.
Demographic, neuropsychological, biomarker, and clinical outcome characteristics of the cluster groups and normal control group
Note: GDS=Geriatric Depression Scale; MMSE=Mini Mental State Exam; LM=Logical Memory; BNT=Boston Naming Test; TMT=Trail Making Test; AVLT=Rey Auditory Verbal Learning Test; p-tau181p =hyperphosphorylated tau; Aβ1-42 =beta-amyloid; MCI=mild cognitive impairment; NC=Normal Control.
a Data are summarized as mean (standard deviation), unless otherwise indicated.
b Data are summarized as raw number of participants (% of participants).
c Number of participants with CSF data: Amnestic: n=63, Mixed: n=89, Cluster-Derived Normal: n=117, NC: n=92.
d Number of participants with longitudinal data: Amnestic: n=107, Mixed: n=130, Cluster-Derived Normal: n=170; the NC group was not included in this analysis since participants were selected on the basis of remaining normal (did not progress/revert).
e Cluster-Derived Normal group differed significantly from the Amnestic group.
f Cluster-Derived Normal group differed significantly from the Mixed group.
g Cluster-Derived Normal group differed significantly from the NC group.
h Amnestic group differed significantly from the Mixed group.
i Amnestic group differed significantly from the NC group.
j Mixed group differed significantly from the NC group.
After covarying for age and education, the Cluster-Derived Normal group performed significantly better than the Mixed group on all measures (p<.001; d ranged from .92 to 1.53), and better than the Amnestic groups on all measures except the BNT (p<.001; d ranged from .82 to 2.71 for Animal Fluency, TMT, Part B, and AVLT recall and recognition; p<.01; d=.63 for TMT, Part A). The Amnestic group performed better than the Mixed group on all measures of language and attention/executive function (p<.001; d ranged from .63 to 1.26), but worse on both measures of memory (p< .001; d=.76 and .98). Although the Cluster-Derived Normal group scored lower than the NC group on the memory measure that was used in making the ADNI’s MCI diagnosis (WMS-R Logical Memory II: p<.001; d=1.87), there were no significant differences in performance on the six cognitive measures in the more extensive baseline neuropsychological testing (p >.05).
CSF Biomarkers
CSF data were available for a subset of the sample (see Table 1). Participants were classified into dichotomous groups (high/low) for p-tau181p, Aβ1-42, and p-tau181p/Aβ1-42 based on established CSF concentration cut-points (Shaw et al., Reference Shaw, Vanderstichele, Knapik-Czajka, Clark, Aisen, Petersen and Trojanowski2009). The Amnestic and Mixed MCI groups had a higher percentage of individuals with positive CSF AD biomarkers (i.e., high p-tau181p, low Aβ1-42, high p-tau181p/Aβ1-42) than the Cluster-Derived Normal and NC groups, although effect sizes were small. The percentages of individuals with a positive CSF AD biomarker in the Cluster-Derived Normal and NC groups did not differ significantly for any of the three CSF measures.
Clinical Outcomes
Follow-up data were available for a subset of the MCI sample (see Table 1). For this analysis, the follow-up visit used for each participant was either the point at which their diagnosis changed (i.e., progressed to dementia or reverted to cognitively normal) or, for those with a stable MCI diagnosis, the longest follow-up visit. The cluster-derived MCI groups did not differ in average length of follow-up (p>.05; mean=14 months; range=6–36 months). Forty-three MCI participants progressed to meet National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association (NINCDS/ADRDA) criteria for a diagnosis for probable AD, and 16 reverted to normal (i.e., no longer met criteria for a diagnosis of MCI). A 3 (no change, progression from MCI to AD, reversion from MCI to NC) × 3 (cluster group) chi-square test was significant, with the Cluster-Derived Normal group showing the lowest rate of progression to dementia (2.4%) and the highest rate of reversion to normal (7.1%). The NC group was not included in these analyses since they were selected on the basis of remaining normal (did not progress/revert) throughout the course of their participation in ADNI.
Subjective Complaints on ECog
Global Cognition
Results of an ANCOVA (covarying for age and education) showed that the self-report global ECog scores of the Amnestic, Mixed, and Cluster-Derived Normal groups were not significantly different from each other, although all had higher scores (indicating greater decline) than the NC group (p<.001; d=1.14, 1.18, and 1.07, respectively; see Table 2). In contrast, the cluster groups differed in informant-report global ECog scores, with higher scores in the Amnestic and Mixed groups than in the Cluster-Derived Normal group (p<.001; d=.74 and .57) and the NC group (p<.001; d=1.14 and 1.33). The groups differed in discrepancy scores between self-report and informant-report ECog scores, as the Amnestic group had lower discrepancy scores (i.e., participants underestimated their cognitive decline compared to their informant) in comparison to the Cluster-Derived Normal and NC groups (p≤.001; d=.61 and .56; see Figure 2). There was no significant difference in discrepancy scores between the Cluster-Derived Normal and NC groups (p>.05).
Mean discrepancy scores (self-rating minus informant-rating) for all 39 items on the ECog. A positive score indicates one is overestimating their cognitive decline relative to their study-partner’s report, while a negative score indicates one is underestimating their cognitive decline. Error bars denote standard errors of the mean.
Self- and Informant-reported cognitive decline scores on the ECog for the cluster groups and NC group
Note: Data are summarized as mean (standard deviation). Range of scores is 1 to 4, with higher scores indicating more cognitive decline compared to 10 years ago; MCI=mild cognitive impairment; NC=normal control.
a Cluster-Derived Normal group differed significantly from the Amnestic group.
b Cluster-Derived Normal group differed significantly from the Mixed group.
c Cluster-Derived Normal group differed significantly from the NC group.
d Amnestic group differed significantly from the Mixed group.
e Amnestic group differed significantly from the NC group.
f Mixed group differed significantly from the NC group.
Memory
The Amnestic, Mixed, and Cluster-Derived Normal groups did not differ in self-reported ECog memory scores, although all had higher scores (indicating greater decline) than the NC group (p<.001; d=1.44, 1.32, and 1.28). For informant-reported ECog memory scores, the Amnestic group had higher scores than the Cluster-Derived Normal and NC groups (p<.001; d=.80 and 1.75); the Mixed group had higher scores than the Cluster-Derived Normal and NC groups (p<.001; d=.47 and 1.49); and the Cluster-Derived Normal group had higher scores than the NC group (p<.001; d=1.20). For memory discrepancy scores, the Amnestic group had lower discrepancy scores (i.e., they underestimated their memory decline) relative to the Cluster-Derived Normal group (p<.001; d=.57; see Figure 3). The Amnestic group also had lower discrepancy scores than the NC group (p<.001; d=.53). There was no significant difference in discrepancy scores between the Cluster-Derived Normal and NC groups (p>.05).
Mean discrepancy scores (self-rating minus informant-rating) for specific cognitive domains on the ECog. A positive score indicates one is overestimating their cognitive decline relative to their study-partner’s report, while a negative score indicates one is underestimating their cognitive decline. Error bars denote standard errors of the mean.
Other Cognitive Domains
The groups differed in ECog discrepancy scores for the language, visuospatial, organization, and divided attention domains, although omnibus effect sizes were small (see Figure 3). The ECog planning discrepancy score did not meet the Bonferroni adjusted threshold for statistical significance (p=.02). The Amnestic group had lower discrepancy scores than the Cluster-Derived Normal group for language, visuospatial, organization, and divided attention (p≤.001; d ranged from .45 to .55). The Amnestic group also had lower scores than the NC group for organization (p<.001; d=.55). There were no significant differences between the Cluster-Derived Normal and NC groups (p>.05).
Relationship between Subjective Cognitive Complaints and Objective Cognitive Performance
Global Cognition
There were no significant correlations between MMSE scores and self-report global ECog scores, informant-report global ECog scores, or global ECog discrepancy scores for the cluster groups or NCs (p>.05).
Memory
There were no significant correlations between objective memory scores and self-report ECog memory scores for any of the groups (p>.05). In contrast, there was a weak negative correlation between informant-report ECog memory scores and scores on objective memory tests for all groups (AVLT Delayed Recall: Amnestic: r=−.28; p=.002; Mixed: r=−.26; p=.002; Cluster-Derived Normal: r=−.27; p<.001; NC: r=−.23; p=.004; AVLT Recognition: Mixed: r=−.26; p =.001; NC: r=−.33; p< .001). Lower ECog memory discrepancy scores (i.e., underestimating memory decline relative to one’s informant) were associated with worse memory recall scores in the Amnestic group (r=.29; p=.002).
Language
There were no significant correlations between objective language scores and self-report or informant-report ECog language scores for any of the cluster groups (p>.05). There was a weak negative correlation between self-report (but not informant-report) ECog language scores and Animal Fluency scores in the NC group (r=−.22; p=.006). ECog language discrepancy scores were not related to objective language test scores in any group.
Divided Attention
There were no significant correlations between scores on an objective task requiring divided attention (i.e., TMT, Part B) and ECog divided attention self-report, informant-report, or discrepancy scores for any of the groups (p>.05).
Relationship between Subjective Cognitive Complaints and Emotional Functioning
Inclusion/exclusion criteria for ADNI required a 15-item Geriatric Depression Scale (GDS) score of less than 6 and no history of major depression or bipolar disorder within the past year. Within these parameters, the cluster groups did not differ in self-reported GDS scores (p>.05), but all had higher scores (i.e., more depressive symptoms) than the NC group (p<.001; d ranged from .87 to .95). There were significant correlations between GDS scores and self-report global ECog scores (Amnestic: r=.35; p<.001; Mixed: r=.38; p<.001; Cluster-Derived Normal: r=.21; p=.004; NC: r=.38; p<.001) and self-report ECog memory scores (Mixed: r=.26; p=.002; NC: r=.22; p=.005). In contrast, there were no significant correlations between GDS scores and informant-report global or memory ECog scores (p>.05). Higher GDS scores were associated with overestimation of cognitive decline (i.e., higher global ECog discrepancy scores) in the Mixed (r=.22; p=.009) and NC (r=.24; p=.003) groups.
Relationship between Subjective Cognitive Complaints and CSF Biomarkers
There were no significant differences in self-report global or memory ECog scores in participants who were positive or negative for the CSF biomarker p-tau181p/Aβ1-42. Informant-report ECog scores were higher in those positive for p-tau181p/Aβ1-42 than negative in the Amnestic (Global Cognition: t(61)=−2.61; p=.01; d=.66; Memory: t(61)=−2.52; p=.01; d=.64) and Mixed (Global Cognition: t(82.25)=−3.19; p=.001; d=.73; Memory: t(87)=−3.28; p=.001; d=.73) groups. Individuals in the Cluster-Derived Normal group who were positive for p-tau181p/Aβ1-42 had lower ECog memory discrepancy scores (i.e., underestimated their cognitive decline compared to an informant) than those who were negative (t(115)=2.49; p=.01; d=.58). Discrepancy scores in other domains also showed that those who were positive for p-tau181p/Aβ1-42 had lower ECog language discrepancy scores than those who were negative in the Amnestic (t(56.4)=2.96; p<.005; d=.74) and Mixed (t(87)=2.56; p=.01; d=.45) groups.
Relationship between Subjective Cognitive Complaints and Clinical Outcome
In the Amnestic group, global, language, planning, and organization discrepancy scores were significantly lower (i.e., underestimation of decline compared to an informant) in participants who progressed to dementia (n=13) than in those who remained stable or reverted to normal (n=94 with follow-up; Global: t(105)=3.23; p=.002; d=.91; Language: t(105)=3.67; p<.001; d=1.01; Planning: t(105)=3.14; p=.002; d=.97; Organization: t(105)=3.15; p=.002; d=.94). In the Mixed group, memory and language discrepancy scores were significantly lower in participants who progressed to dementia (n=26) than in those who remained stable or reverted to normal (n=104 with follow-up; Memory: t(128)=3.56; p=.001; d=.83; Language: t(128)=3.09; p=.002; d=.68). In the Cluster-Derived Normal group, there were no differences in discrepancy scores between those who progressed to dementia (n=4) and those who remained stable or reverted to normal (n=166 with follow-up; all p-values>.10).
Discussion
Our results show a striking and somewhat counterintuitive finding: cognitively intact individuals who had been classified as MCI in ADNI overestimated their cognitive problems, whereas individuals with MCI and objective memory impairment on comprehensive testing underestimated their cognitive difficulties. Although there was no relationship between self-reported cognitive ratings and objective cognitive functioning in any domain, there were significant correlations between self-reported cognitive ratings and depressive symptoms (despite its restricted range), consistent with previous studies (Buckley et al., Reference Buckley, Saling, Ames, Rowe, Lautenschlager, Macaulay and Ellis2013; Lenehan et al., Reference Lenehan, Klekociuk and Summers2012; Roberts et al., Reference Roberts, Clare and Woods2009; Studer et al., Reference Studer, Donati, Popp and von Gunten2013). There was also an inverse relationship with CSF biomarkers. Individuals who had been classified as MCI in ADNI and had a positive CSF AD biomarker underestimated their cognitive problems in comparison to those who had a negative CSF AD biomarker. Taken together, these results provide evidence that inclusion of subjective cognitive complaints in the criteria for MCI may cloud diagnosis and result in high rates of misclassification.
Consistent with previous research showing that the conventional diagnostic criteria for MCI are susceptible to false-positive diagnostic errors (Bondi et al., Reference Bondi, Edmonds, Jak, Clark, Delano-Wood, McDonald and Salmon2014; Clark et al., Reference Clark, Delano-Wood, Libon, McDonald, Nation, Bangen and Bondi2013; Edmonds et al., Reference Edmonds, Delano-Wood, Clark, Jak, Nation, McDonald and Bondi2014), cluster analysis of the neuropsychological performance of 448 individuals in ADNI’s MCI cohort revealed a large Cluster-Derived Normal subgroup that comprised over a third of the sample (41.3%). Despite the poor performance on the Logical Memory Test that initially led to their MCI classification, this subgroup’s performance on a more extensive neuropsychological test battery, and their likelihood of having a positive CSF AD biomarker, did not differ from that of a “robust” normal control group that excluded individuals with preclinical dementia based on longitudinal follow-up (Sliwinski, Lipton, Buschke, & Stewart, Reference Sliwinski, Lipton, Buschke and Stewart1996). Individuals in this Cluster-Derived Normal group were also less likely to progress to dementia than those in Amnestic and Mixed MCI groups over an average of 14 months. Only four individuals in the Cluster-Derived Normal group (2.4%) progressed to dementia, a rate comparable to the base rate of progression to dementia in ADNI’s overall sample of participants diagnosed as cognitively normal. Specifically, examination of the base rate of cognitive decline in ADNI’s entire group of 404 normal control participants with neuropsychological and follow-up data (not just the 157 participants retained for the robust normal control group in the present study) was found to be 13%, with 2% of the normal control sample progressing to dementia and 11% progressing to MCI. A larger percentage of individuals in the Cluster-Derived Normal group reverted to an ADNI classification of cognitively normal (7.1%) than progressed to dementia, which is consistent with previous studies reporting high reversion rates in MCI (Ganguli et al., Reference Ganguli, Snitz, Saxton, Chang, Lee, Vander Bilt and Petersen2011; Koepsell & Monsell, Reference Koepsell and Monsell2012; Petersen et al., Reference Petersen, Aisen, Boeve, Geda, Ivnik, Knopman and Jack2013; Summers & Saunders, Reference Summers and Saunders2012). Thus, while it is possible that a subset of the Cluster-Derived Normal group may be at risk for future cognitive decline or may represent an asymptomatic “preclinical AD” phase (Sperling et al., Reference Sperling, Aisen, Beckett, Bennett, Craft, Fagan and Phelps2011), a diagnosis of MCI is not warranted at this time given the group’s intact performances on neuropsychological measures.
The criteria used by ADNI to diagnose MCI are quite liberal and consist of only three factors that distinguish MCI from cognitively normal individuals: a history of memory concerns, a global CDR score of 0.5, and performance on Story A of WMS-R Logical Memory II below education-adjusted cut-off scores for impairment. Two of these three factors, subjective memory complaints and CDR score, rely on the subjective report of the participant and/or their study partner. The CDR is a subjective assessment that characterizes six domains of cognitive and functional performance based on a semi-structured interview (Morris, Reference Morris1993). A CDR of 0.5 indicates significant but mild cognitive decline that does reach the level of dementia. The Cluster-Derived Normal group’s tendency to over-report cognitive problems may have affected their scores on the CDR and contributed to misclassification as MCI. Support for this notion comes from a study by Saxton et al. (Reference Saxton, Snitz, Lopez, Ives, Dunn, Fitzpatrick and DeKosky2009) who found that diagnosis of MCI based on global CDR scores of 0.5 resulted in a high rate of false positive diagnostic errors. Specifically, a large number of individuals in their sample had CDR scores of 0.5 but performed normally on cognitive testing and were less likely to develop dementia in comparison to individuals diagnosed with MCI based on comprehensive neuropsychological testing. Furthermore, they found that CDR ratings of 0.5 were influenced by symptoms of depression and subjective health problems despite the fact that participants endorsed only mild depressive symptoms and did not meet criteria for a diagnosis of depression.
Subjective complaints or a global CDR score of 0.5 are almost certainly not the only factors that contributed to the possible misdiagnosis of the Cluster-Derived Normal group. Another is the use of a single memory test score to determine abnormal memory function, which leads to several problems. First, shortening a test (administering only Story A) and detaching it from its standardized administration, scoring, and normative referencing (applying only education- but not age- or sex-adjustments) likely makes it less sensitive or reliable. Second, recall of a single story may be less sensitive to MCI or an evolving dementia than other tests of memory (e.g., verbal list learning: Bondi, Salmon, Galasko, Thomas, & Thal, Reference Bondi, Salmon, Galasko, Thomas and Thal1999; de Jager, Hogervorst, Combrinck, & Budge, Reference de Jager, Hogervorst, Combrinck and Budge2003; Rabin et al., Reference Rabin, Pare, Saykin, Brown, Wishart, Flashman and Santulli2009; Tierney, Yao, Kiss, & McDowell, Reference Tierney, Yao, Kiss and McDowell2005). Third, use of this single test score ignores base rates of “impaired” scores in neurologically normal populations and violates a psychometric maxim that multiple measures provide a more reliable estimate of a cognitive construct than any single measure (Anastasi & Urbina, Reference Anastasi and Urbina1997). For example, Brooks, Iverson, and White (Reference Brooks, Iverson and White2007) found that 55.5% of healthy older adults had at least one memory score 1 standard deviation (SD) below the mean, and 30.8% had at least one score 1.5 SDs below the mean. In addition, Palmer, Boone, Lesser, and Wohl (Reference Palmer, Boone, Lesser and Wohl1998) found that 73% of their healthy older adult sample had at least one test score that was 1.3 SDs or more below the mean, and 20% had at least two test scores 2.0 SDs or more below.
In contrast to the Cluster-Derived Normal group’s overestimation of subjective concerns, the Amnestic MCI group in our study underestimated the extent of their cognitive deficits relative to their informants. The tendency to under-report problems may result from reduced awareness of cognitive dysfunction (i.e., anosognosia) or an inability to accurately appraise one’s own cognitive abilities (Roberts et al., Reference Roberts, Clare and Woods2009). Available evidence suggests that individuals with amnestic MCI may have impaired insight to the same degree as patients with mild AD (Vogel et al., Reference Vogel, Stokholm, Gade, Andersen, Hejl and Waldemar2004). An unexpected finding from our study was that the Mixed MCI group was generally accurate in evaluating their cognitive abilities. Their self-ratings on the ECog were similar to the ratings made by their informants. Given that this group had impairments primarily in language and attention/executive functioning, it is possible they were more aware of the everyday consequences of these cognitive deficits (e.g., word-finding problems, difficulty multi-tasking) than were those with memory deficits. It is also possible that the Mixed MCI group was more cognizant of their deficits compared to the Amnestic group due to different underlying pathology. Based on previous work showing that mixed cognitive deficits in MCI are associated with a higher burden of cerebrovascular disease, as indexed by white matter lesions (Delano-Wood et al., Reference Delano-Wood, Abeles, Sacco, Wierenga, Horne and Bozoki2008), the Mixed group in the current study may have been comprised of those with primarily cerebrovascular disease rather than AD pathology. Post hoc analysis examining available neuroimaging data offers support for this possibility, as the Mixed group (n=138) showed greater white matter hyperintensity volume relative to the Amnestic (n=107; p=.03; d=.38), Cluster-Derived Normal (n=180; p=.003; d=.40), and NC (n=145; p=.03; d=.29) groups, although effect sizes were small. In addition to the type of pathology, the location of such pathology may also differ across groups, which could also explain our findings to some degree. For example, the Amnestic group may have sustained greater medial temporal lobe/hippocampal damage whereas this region may have been less severely affected in our Mixed group. Further research is needed to examine awareness of cognitive deficits by MCI subtype and its neuroanatomic correlates.
The discrepancy between self- and informant-report on the ECog was related to clinical outcome. The 43 participants who progressed to dementia significantly underestimated their cognitive decline (relative to the estimate of their informant) compared to the 364 individuals with follow-up data who did not progress. This finding extends prior work showing that reduced awareness of functional deficits in patients with MCI predicts development of AD (Tabert et al., Reference Tabert, Albert, Borukhova-Milov, Camacho, Pelton, Liu and Devanand2002). Our results are also consistent with another recent study which found that self-reported cognitive problems in non-demented older adults were not consistently predictive of a future diagnostic outcome of dementia, although informant-reported cognitive complaints were predictive, particularly when combined with self-report (i.e., when there was a “mutual complaint”; Gifford et al., Reference Gifford, Liu, Lu, Tripodis, Cantwell, Palmisano and Jefferson2014).
Informants completing the ECog reported more cognitive problems for the Amnestic and Mixed groups than for the Cluster-Derived Normal group. In addition, there was a correlation between informant-reported memory decline and objective memory performance in the MCI groups, although the relationships were weak. Informants also reported more decline in cognitively impaired individuals who had positive CSF AD biomarkers than in those with negative biomarkers. These results suggest that, although informant-reports of cognitive decline may be limited by the potential for recall bias or the influence of emotional factors, they are generally more accurate than self-reports of decline made by individuals with MCI.
A limitation of our study was the inability to examine false negative diagnostic errors. Individuals classified as “normal” by ADNI but found to have cognitive impairment on more extensive testing or to have subsequently declined were not included in the present study. Instead, we chose to use a “robust” cognitively normal sample. However, research has shown that classifying individuals based on subjective complaints significantly reduces the accuracy of the MCI diagnosis in both directions, producing high rates of false positive and false negative errors (Lenehan et al., Reference Lenehan, Klekociuk and Summers2012). Another limitation is that the validity of the ECog as a self-report instrument has not been demonstrated; it was developed and validated as an informant-rated questionnaire (Farias et al., Reference Farias, Mungas, Reed, Cahn-Weiner, Jagust, Baynes and Decarli2008). Nevertheless, the discrepancy between self- and informant-ratings on functional ability questionnaires in general is a well-validated method (Farias, Mungas, & Jagust, Reference Farias, Mungas and Jagust2005; Roberts et al., Reference Roberts, Clare and Woods2009). Strengths of the current study include using an empirical statistical approach to identify MCI subgroups, using a robust normative reference group, examining discrepancy scores between self- and informant-reported complaints, exploring subjective complaints across multiple cognitive domains, and relating subjective memory complaints to CSF AD biomarkers and clinical outcomes.
The high rate of false positive diagnostic errors (over a third of the ADNI MCI cohort) demonstrates the inadequacy of basing an MCI diagnosis on subjective complaints, subjective rating scales, and a single memory test. This imprecision has implications for clinical research trials of MCI. If a large number of cognitively normal individuals are incorrectly identified as MCI in biomarker or treatment studies, the inclusion of such individuals could significantly dilute or obscure important relationships and effects. Our findings may also have important clinical implications. For instance, 17% of the Cluster-Derived Normal group was receiving anticholinesterase medication, which may have been unnecessary.
In summary, the present results indicate that self-reported subjective complaints should not be relied upon in making a diagnosis of MCI, as they contribute to misdiagnosis. Informant report, on the other hand, appears to be of some utility in making the MCI classification. If the subjective report of the patient is considered in the diagnosis, it should be focused on the discrepancy between the patient and informant with the following heuristic in mind: when a patient reports more cognitive problems than his or her informant, it is more characteristic of normal aging than MCI. This over-reporting, which was observed in both the Normal Control and the Cluster-Derived Normal groups, may reflect individuals being acutely aware of the cognitive changes that they are experiencing as part of the normal aging process, perhaps amplified by emotional factors in some cases. However, if individuals underestimate their cognitive problems, they may be at greater risk for decline. Such a modification to systematize discrepancies in reporting cognitive decline may improve diagnostic accuracy, enhance biomarker relationships, and improve prediction of who will progress to dementia.
Acknowledgments
Conflict of Interest Disclosures: Dr. Galasko serves as Editor for Alzheimer’s Research and Therapy, and as a paid consultant on Data Safety Monitoring Boards for Pfizer, Inc., Elan, Inc., and Balance Pharmaceuticals, Inc. Dr. Salmon serves as a consultant for CHDI Foundation, Novartis, and Bristol-Meyers Squibb. The other authors report no disclosures. Funding/Support: This work was supported by the NIH (M.B., grant number R01 AG012674), (M.B., grant number K24 AG026431), (D.G., grant number P50 AG05131). Additional Information: Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; BioClinica, Inc.; Biogen Idec Inc.; Bristol-Myers Squibb Company; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; GE Healthcare; Innogenetics, N.V.; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Medpace, Inc.; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Synarc Inc.; and Takeda Pharmaceutical Company. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Disease Cooperative Study at the University of California, San Diego. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.
