Introduction
Background
Identifying correlates and risk factors for cognitive and functional decline is an important area of research in aging and geriatric care. Many studies have estimated the effects of risk factors on the rate of decline and evolution from mild cognitive impairment (MCI) to Alzheimer's disease (AD) dementia; however, the measure or measures used to quantify this decline vary from study to study. Some studies have used a single global test (Storandt et al., Reference Storandt, Grant, Miller and Morris2002; Clarke et al., Reference Clarke, Ko, Lyketsos, Rebok and Eaton2010; Cronk et al., Reference Cronk, Johnson and Burns2010; Landau et al., Reference Landau2010; Whitehair et al., Reference Whitehair2010), while others have calculated composite scores (CSs) from several tests (Cosentino et al., Reference Cosentino2008; Wilson et al., Reference Wilson, Leurgans, Boyle and Bennett2011).
Estimated rates of decline may depend on the measurement used to estimate the slope. In addition, the strength of the association between a potential correlate of decline and the estimated rate of decline may be influenced by the choice of outcome measurement used to evaluate the effect. Thus, estimation of the rate of the decline and its correlates may be measurement dependent. Variation in the measures used to characterize decline makes it difficult to compare estimates of decline across studies. Understanding the possible differences among the measures is an essential step toward identifying risk factors for decline to AD dementia. Differences across measures could also have a profound effect on intervention trials where cognitive and functional changes are often the endpoints of interest (Reisberg, Reference Reisberg2007).
Specific aims
The primary aim of this analysis was to estimate and compare the rates of decline to AD dementia for three common measures: Mini-Mental State Examination (MMSE) score (Folstein et al., Reference Folstein, Folstein and McHugh1975), Clinical Dementia Rating sum of boxes (CDR-SB) score (Morris, Reference Morris1993), and a neuropsychological test CS from the Uniform Data Set (UDS) Neuropsychological Test Battery (Weintraub et al., Reference Weintraub2009). The second aim was to test for differences in the strength of association between common predictors (e.g. patient characteristics) on the rate of decline estimated from the three outcomes. For example, increased age may be associated with a faster decline as estimated by the CDR-SB but may not be associated with the rate of decline when the endpoint is the CS.
Methods
Study subjects
The study population was composed of subjects from the National Alzheimer's Coordinating Center (NACC) UDS (Beekly et al., Reference Beekly2007), gathered prospectively from 33 Alzheimer's Disease Centers (ADCs) (29 active and 4 inactive) between September 2005 and June 2011. Subjects underwent annual clinical exams, which included cognitive and functional testing. Standardized forms were used at all Centers and informed consent was given by all subjects and their informants. Research using the NACC database was approved by the Institutional Review Board at the University of Washington.
This analysis focused on subjects with a clinical diagnosis of amnestic MCI (aMCI) defined by the Petersen criteria (Petersen et al., Reference Petersen2001), as these subjects are often considered to have early-stage AD (Morris et al., Reference Morris2001). Subjects with single domain and multidomain aMCI were included (Petersen, Reference Petersen2004). At least one follow-up visit after an aMCI diagnosis was required in order to evaluate change over time. Subjects under the age of 65 were excluded in order to remove those with probable early-onset AD. Of the 5,008 subjects who had a diagnosis of aMCI during at least one visit, 2,899 met the additional follow-up visit and age inclusion criteria.
Measures of cognition and function
The MMSE is a commonly used test of cognitive function. The total score ranges from 0 (too cognitively impaired to answer any questions) to 30 (no detectable decline in cognition). The CDR is a measure of cognition and function used to identify patients who are likely to be on a trajectory toward AD dementia. The CDR grades performance, based on observation and input from a collateral source, in six domains: memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care. Impairment in each of the six domains is evaluated by the clinician as none (0), questionable (0.5), mild (1), moderate (2), and severe (3). Thus, scores range from 0 to 18 with a low score representing no detectable decline in cognition and function and a high score indicating severe impairment and high likelihood of AD dementia.
The CS was calculated from ten tests in the UDS Neuropsychological Test Battery. These tests evaluated five cognitive domains (Weintraub et al., Reference Weintraub2009). Tests of episodic memory were Current Logical Memory IA Story Units Recalled (Wechsler, Reference Wechsler1987), Logical Memory IIA-Delayed Story Units Recalled (Wechsler, Reference Wechsler1987), and WAIS-R Digit Symbol (Wechsler, Reference Wechsler1981). Semantic memory and language were assessed by the Boston Naming Test (Kaplan, Reference Kaplan, Goodglass and Weintraub1983), as well as animal and vegetable naming tests (Morris et al., Reference Morris1989). Digit Span Backwards and Forwards Trials Correct (Wechsler, Reference Wechsler1987) tested working memory and attention. Finally, Trail Making Test Part A evaluated processing speed, and Trail Making Test Part B (Reitan, Reference Reitan and Wolfson1993) evaluated executive function.
For all tests except the two Trail Making tests, a low score indicated poor performance. In order to make the tests more comparable, observed scores from the two Trail Making tests were subtracted from their maximum scores (150 and 300, respectively). A low score on the transformed test, then, indicated poor performance.
Scores on the individual neuropsychological tests were standardized by subtracting the study sample baseline mean from the raw score and dividing by the study sample baseline standard deviation, yielding a Z-score. All ten standardized test scores were averaged at each subject-visit to create a single score. If at least one neuropsychological test score was missing, the CS could not be calculated and was considered missing.
Statistical analysis
All three measures (CDR-SB, MMSE, and CS) were converted into standardized scores by subtracting the baseline mean and dividing by the baseline standard deviation of the study sample (this was a second standardization for the CS). The standardized scores for the three measures were then stacked to create three outcome values for each patient on each visit so that the outcome became Yijk , the standardized score for the ith subject at time tj for measurement type k.
Missing standardized measurement scores were estimated using multiple imputation with chained equations (Little and Rubin, Reference Little and Rubin1987; van Buuren, Reference van Buuren2007). The process of multiple imputation requires that the unavailable data be imputed not just once but multiple times for each missing data point. In this analysis, we imputed the missing values 20 times, resulting in 20 complete data sets. The regression analyses were run on each of the 20 complete data sets.
Variables used to estimate the missing scores included: time since baseline, measurement type, time-measurement type interaction terms, age at visit, education, sex, race, and single versus multiple cognitive domain deficit at the current visit. Potential correlates with any missing data were not imputed and were not used in imputing the missing measurement scores. The main advantage of using multiple imputation over the complete case sample was to anticipate the likely possibility that the presence of missing scores was not completely random, i.e. that the missing scores were systematically higher or lower than observed scores after accounting for known subject characteristics.
Spaghetti plots and cubic spline interpolations of the measures over time suggested that the assumption of linear decline was reasonable. Multiple linear regression fit by Generalized Estimating Equations (GEE) was implemented to simultaneously model all of the standardized scores at every available time point while accounting for clustering of measurement types and visits within subjects (Liang, Reference Liang and Zeger1973; Hubbard et al., Reference Hubbard2010). An independent correlation structure with robust standard errors was employed. Coefficients and standard errors were combined using Rubin's rules (Rubin and Schenker, Reference Rubin and Schenker1991).
The second aim was addressed by adding a potential correlate of decline and including all two- and three-way interaction terms to the regression model. The common predictors of decline to dementia included in this analysis were age, race, education, sex, body mass index (BMI), Hachinski Ischemic score (HIS), apolipoprotein E (APOE) ϵ4 allele, and single- versus multiple-domain impairment. Age, BMI, HIS, and single- versus multiple-domain impairment were treated as time-varying covariates; all other predictors were considered baseline-only measures. A separate regression model was run for each of the predictors, using Rubin's rules to combine estimates across the imputed data sets (Rubin and Schenker, Reference Rubin and Schenker1991). Finally, the relationship between the predictor and the rate of decline was compared across the three measures using a global test for differences. All analyses were performed using R 2.13.1 (R Development Core Team, 2008).
Sensitivity analysis
An alternative calculation of the CS and an analysis without imputation were performed in order to assess the extent to which the results depended on the methods used to analyze the data. First, the CS was calculated by averaging the standardized scores within domain and then averaging across domains. This alternative calculation gave equal weight to each domain but not to each individual test. Second, all available data from subject-visits that met original exclusion criteria (missing the MMSE or any test used to calculate the CS does not warrant deletion of the entire subject-visit) were analyzed. Finally, alternate correlation structures and clustering mechanisms were explored.
Results
Analytic sample characteristics
On average, subjects were followed for 2.4 years (range: 0.5–5.8 years) and completed 3.1 evaluations (range: 2–6 evaluations). Approximately, 15% of the 8,933 visits included in this analysis had at least one missing measurement value. Around 5% of visits were missing MMSE, and 15% were missing at least one neuropsychological test score used in constructing the CS; no subject had a missing CDR-SB score (a required data element in the UDS).
Demographic and clinical characteristics of the analytic sample are described in Table 1. All characteristics are evaluated at the first visit where the subject met all inclusion criteria, except years followed and number of evaluations completed. The BMI, HIS, and APOE ɛ4 allele status were the only characteristics with any missing data (803 visits, 304 visits, and 966 subjects, respectively).
Baseline characteristics of analytic sample (n = 2899 subjects)
BMI = body mass index; APOE = Apolipoprotein E; MCI = mild cognitive impairment; CDR-SB = Clinical Dementia Rating sum of boxes; MMSE = Mini-Mental State Examination.
Calculations are based on observed data only. Years followed and visits are calculated using longitudinal data; all other characteristics are derived only from data observed at baseline, which is defined as the first visit where the subject met all inclusion criteria. Mean and SD are presented for continuous measures. Sample percentages are presented for categorical measures.
At the time of their most recent UDS visit, over half of the subjects were still diagnosed with MCI (n = 1,605, 55%). Of the remaining subjects, 971 (33%) had dementia and 323 (11%) had a diagnosis of normal cognition, failing to meet criteria for MCI.
Differences across measures
Scatterplots of each measurement are displayed in Figure 1. The intercepts and slopes of the lines were estimated from the main effects regression model with GEE using Rubin's rules to combine estimates from the imputed data sets. The rate of decline of the three measures differed significantly in all pairwise comparisons and for a global test for differences (p < 0.0001 for all tests). The CDR-SB had the steepest estimated slope, with a decline of 0.49 standard deviations (SD) per year (95% CI: −0.53, −0.44), followed by the MMSE with 0.22 SD per year (95% CI: −0.24, −0.18), and finally the CS with 0.07 SD per year (95% CI: −0.12, −0.02). The estimated rate of decline can be transformed from SD per year back to the test's original scale by multiplying the estimate by the baseline sample standard deviation. The estimated rate of decline was 0.59 points per year for the CDR-SB, 0.55 points per year for the MMSE, and 0.04 standardized points per year for the CS.
Change in cognitive and functional evaluation over time (slope and intercept of lines are estimated from regression model).
The association between several potential correlates of decline to AD dementia and the rate of decline itself varied across measurement (see Table 2). Age at visit, BMI at visit, APOE ɛ4 allele status, and race (black vs. white) had significantly different relationships with rate of decline in a global test for difference among the three measures. Marginally significant differences were detected for HIS and race (other vs. white). P-values from the global tests of differences, as well as the estimates and 95% confidence intervals are presented in Table 2 for each predictor. The average rate of decline for a specific measure can be calculated by multiplying the effect estimate by the standard deviation of the sample at baseline. For example, keeping in mind that the CDR-SB increases with level of impairment, we estimate the yearly decline in the CDR-SB score of a 90-year old to be 0.2 points per year greater than for an 80-year old (0.15 SD per year × 10 years × 1.2 SD).
Estimated association between potential correlates and rate of decline for each measure
MMSE = Mini-Mental State Examination; CDR-SB = Clinical Dementia Rating sum of boxes; BMI = body mass index; HIS = Hachinski Ischemic score; APOE = Apolipoprotein E; MCI = mild cognitive impairment.
Estimates obtained using multiple linear regression with GEE and multiple imputation run separately for each potential correlate. Age, education, BMI, HIS, and domain are time-varying covariates; all other risk factors are observed only at the first visit where the subject met all inclusion criteria. The p-value presented is from a global test for differences among the three measures. P-values less than 0.05 are in bold. All analyses were performed on 8,933 subject-visits except where noted.
*8,130 subject-visits used in this model due to missing data.
† 5,792 subject-visits used in this model due to missing data.
‡ 2,661 subject-visits used in this model due to missing data.
Sensitivity analysis
The CS was calculated by first averaging within domain and then across all domains. The missing data were then imputed and analyzed. The results of the regression using the alternative CS calculation produced almost identical results to those in the main analysis (see Table 3).
Estimated slopes for measures of clinical decline using three different methods of analysis
MMSE = Mini-Mental State Examination; CDR-SB = Clinical Dementia Rating sum of boxes; CS = composite score.
The primary analysis and analysis using the alternative CS calculation included 26,799 subject-visit-measures, whereas the analysis excluding missing data had 25,084 subject-visit-measures.
An analysis using all available data without imputation included 8,933 subject-visits with 25,084 subject-visit-measures, compared to the imputed analysis which had 26,799 subject-visit-measures. Again, results from the regression model were very similar to those obtained using multiple imputation (see Table 3). Different model specifications for the working correlation and clustering mechanism were also explored. All models produced similar results (not shown).
Discussion
Summary and inference
The present study was carried out to determine if the choice of measurement could influence estimated rates of decline in individuals with aMCI and if potential risk factors for decline were also measurement dependent. Three measures − MMSE, CDR sum of boxes, and a CS composed of ten tests in the UDS Neuropsychological Test Battery − were analyzed. The estimated slopes for all three measures were significantly different from one another. Thus, estimates of the rate of decline to AD dementia and their correlates can be measurement dependent, suggesting that the measurements are capturing different pictures of the true underlying cognitive and functional states.
The results of this analysis are similar to those seen in previous studies of MCI and mild AD. For example, in the Washington Heights and Inwood Columbia Aging Project study, CSs for subjects with at least one APOE ɛ4 allele decreased more than those without an ɛ4 allele by 0.06 SD in the incident sample and 0.07 SD in the prevalent sample (Cosentino et al., Reference Cosentino2008). In our analysis, the effect of having at least one APOE ɛ4 versus not was to increase the rate of decline in the CS by 0.08 SD. Other studies have also shown that the association between rate of decline and a predictor can differ depending on the measurement used to assess decline. In the Alzheimer's Disease Neuroimaging Initiative cohort, the estimated effect of baseline BMI on one year change in cognition varied across outcome measure; significant effects were detected with the CS, MMSE, and ADAS-Cog scores but not with the CDR-SB (Cronk et al., Reference Cronk, Johnson and Burns2010).
In this analysis, the CS was arguably the least responsive measure, having the slowest rate of decline. One possible explanation is that the CS averages equally over many domains so that substantial change in one domain is diluted by a lack of change in other domains. Therefore, using a domain-specific CS may be more sensitive to change than a global CS. For example, it is well known that memory and executive function test scores can predict subsequent decline from MCI to a state of dementia (Albert et al., Reference Albert, Moss, Tanzi and Jones2001). Instead of combining all of the domains together, these two domains could be analyzed separately.
It is also important to keep in mind that a comparatively small slope does not mean that the CS is less sensitive to early changes. The estimated rate of decline for the CDR-SB appears to be much larger than that of the CS; however, the change in observed scores still may not be clinically meaningful. For example, the average change in CDR-SB was less than 1.5 points after three years. Considering that the CDR-SB ranges from 0 to 18, this change is actually not as marked as indicated by the slope estimates alone. Moreover, though the CS showed rather small changes over time, we were still able to detect significant differences in the rate of decline by age, BMI, and race.
Strengths and weaknesses
This study had a large sample size of subjects from 33 ADCs spread across the USA. Although some subjects had four or five follow-up visits, the median number of visits was only three. Ideally, all subjects would be followed through to a diagnosis of dementia; however, less than half of the subjects had a diagnosis of dementia at their most recent visit. Several factors are likely to have contributed to this. First, the UDS has been active since 2005, with many subjects enrolled only recently. Second, subjects with decreased cognitive and functional abilities have a harder time scheduling and attending clinical visits. In addition, cognitive deficits can be due to non-neurodegenerative processes such as depression and medication. While limiting the sample to subjects who were eventually diagnosed with dementia would certainly capture those most likely to have AD pathology, inclusion of all subjects with aMCI allowed us to look at a more general population of subjects meeting aMCI criteria.
It is also possible that a longer follow-up period could give different estimates of rate of decline and even change the shape of decline from a linear trend to a more curvilinear decline or a change point, as other studies have shown (Wilkosz et al., Reference Wilkosz2010; Wilson et al., Reference Wilson, Leurgans, Boyle, Schneider and Bennett2010).
Although the sensitivity analysis suggests that the results are robust to some analysis decisions, the extent of the external validity is unknown. It is difficult to generalize these results to all older adults with aMCI, as those who decline to dementia are more likely to miss visits, drop out, or die. The NACC database itself is best characterized as a large case-series, rather than a statistically based sample of the US population.
Conclusion
In summary, the rate of decline in aMCI subjects differed depending on the measure used to capture progression to AD dementia. The estimated association between the rate of decline and a predictor, such as age, was also influenced by the measurement used to describe progression. Variation in both the strength and direction of the association between rate of decline and its correlates may influence intervention trials where the target is cognitive and/or functional change, as the effect of the intervention could vary depending on the measurement used to estimate decline.
Conflict of interest
None.
Description of authors' roles
S. Monsell carried out the statistical analysis and writing of the paper. D. Liu gave consultation on statistical methodology, interpretation of results, and reviewed the paper. W. Kukull and S. Weintraub provided assistance in design of the study, as well as review of the paper.
Acknowledgments
The NACC database is supported by the NIA grant: UO1 AG016976. The authors thank the NACC Methods Committee members for their suggestions and feedback, as well as George Thomas for database support.
