Introduction
Aging research often focuses on rates of decline in cognitive performance as assessed by neuropsychological instruments, and on the influence of risk factors on these rates of decline. Assessing decline in longitudinal studies is complicated by informative loss to follow-up; that is, study participants who miss scheduled evaluations often perform more poorly than those who complete their visits (Little & Rubin, Reference Little and Rubin2002; Rubin, Reference Rubin1976; Sliwinski et al., Reference Sliwinski, Stawski, Hall, Katz, Verghese and Lipton2006; Wang & Hall, Reference Wang and Hall2010). This problem is well exemplified by cognitive decline.
Cognitive decline in elderly populations may be underestimated by prospective observational studies because the individuals with the greatest change are lost to follow-up. This informative loss to follow-up may bias estimates of the effects of risk factors on cognitive performance. This well-recognized problem in longitudinal aging studies may make it more difficult to discover genetic and environmental risk factors for cognitive decline and may lead to underestimates of the effects of risk factors.
Using “auxiliary” information that is correlated with the missing outcome is an emerging approach to help correct this bias (Daniels & Hogan, Reference Daniels and Hogan2008; Ibrahim, Lipsitz, & Horton, Reference Ibrahim, Lipsitz and Horton2001). When cognitive test data are missing, auxiliary information might include a clinical assessment, an informant interview, or a telephone interview with a participant unable to complete a clinic visit. We have previously shown that auxiliary data from an in-person clinical assessment based on the Clinical Dementia Rating (CDR; Morris, Reference Morris1993) can be used to refine estimates of memory decline based on the free and cued selective reminding test (FCSRT) (Grober & Buschke, Reference Grober and Buschke1987) in the context of a dementia screening study. The results suggested that memory decline in the clinic-based cohort with over a 1-year follow-up was underestimated by approximately 10% (Wang & Hall, Reference Wang and Hall2010). We have not yet examined the use of auxiliary data to estimate the influence of genetic risk factors on cognitive decline.
Herein, we use data from the Einstein Aging Study (EAS) (Katz et al., Reference Katz, Lipton, Hall, Zimmerman, Sanders, Verghese and Derby2012) to assess the rates of memory decline associated with carrying an APOE ε4 allele. APOE ε4 is the major genetic variant that influences rates of memory decline and the incidence of late onset Alzheimer’s disease. We then corrected for informative loss to follow-up using auxiliary telephone cognitive assessments. These approaches may increase our ability to find genetic variants associated with cognitive decline and prove useful in neurologic research and clinical practice. Our a priori hypothesis was that methods that take into account the effect of missing data would show a larger estimated difference in the rate of memory decline between persons with at least one ε4 allele (ε4+), and persons without an APOE ε4 allele (ε4−) than the more commonly used methods that ignore the effect of missing data.
Method
Study Population
The Einstein Aging Study (EAS) is an ongoing longitudinal prospective study of cognitive aging and dementia started in 1993. The sample includes older adults drawn from a multi-ethnic, community-dwelling population in Bronx County, NY, USA, and recruited using Medicare eligibility and voter registration lists. Eligible participants were at least 70 years old, resided in the Bronx, were not living in institutions, and spoke English. Exclusion criteria included visual or auditory impairments that preclude neuropsychological testing, active psychiatric symptomatology that interfered with the ability to complete assessments, and non-ambulatory status. Participants receive comprehensive annual medical and neuropsychological evaluations at approximately 12-month intervals (Katz et al., Reference Katz, Lipton, Hall, Zimmerman, Sanders, Verghese and Derby2012). Oral consent was obtained on first telephone contact and written informed consent was obtained at the initial clinic visit. The local institutional review board approved the study protocol.
Outcome Measure
All EAS participants are administered the Free and Cued Selective Reminding Test (FCSRT) (Grober & Buschke, Reference Grober and Buschke1987), a test of episodic memory that has been shown to be sensitive and specific for dementia, and are periodically evaluated for dementia and mild cognitive impairment (MCI). The FCSRT includes presentation and learning of 16 pictures by having subjects identify each of the 16 pictures. It also consists of three trials of immediate free recall, each of which is followed by cued recall in which a category cue is presented to the subjects to facilitate recall of the items not freely recalled. Total recall is the sum of free and cued recall; free recall (range, 0–48) was used in analyses.
APOE ε4 status
Most participants also agree to submit blood for genotyping. APOE genotyping was performed using methods previously described (Derby et al., Reference Derby, Burns, Wang, Katz, Zimmerman, L’Italiend and Lipton2013; Sanders et al., Reference Sanders, Wang, Katz, Derby, Barzilai, Ozelius and Lipton2010). Individuals were classified as APOE ε4 carriers if they had at least one ε4 allele (ε4+), and non-carriers otherwise (ε4−).
Auxiliary data.
The administration of the Memory Impairment Screen for Telephone (MIS-T) (Lipton et al., Reference Lipton, Katz, Kuslansky, Sliwinski, Stewart, Verghese and Buschke2003) began in March, 2003 regardless of whether the study participant was able to make a clinic visit for in-person cognitive assessment. MIS-T is a brief test of episodic memory that uses procedures similar to the FCSRT. Specifically, the telephone interviewer presents two (of four) target words and asks the subject to repeat the words to ensure that they heard correctly. The interviewer then presents a category cue associate with one of the target words. The subject is asked to select the word that corresponds to the category cue. The interviewer then presents a second category cue that corresponds to the other target word. This procedure is repeated for the next two target items. After an interference task lasting 2–3 min, the subject is then asked to recall as many target items as possible (free recall). For each word not recalled, the interviewer presents a cue category and asks the subject to recall the word (cued recall). The scoring of the MIS-T is calculated as: total MIS-T=((2*free) + cued). The application of the MIS-T is used as auxiliary data for the FCSRT using methods described below. For all analyses in this report, follow-up began with the first administration of the MIS-T and continued through July 2011.
Additional data collected to characterize the cohort
Demographic information including self-reported race/ethnicity as defined by the U.S. Census Bureau in 1994 and education (in years) were obtained during in-person evaluations at baseline and at subsequent 12-month intervals. Global cognition was measured at baseline and annually by the Blessed Information Memory Concentration test (BIMC; Blessed, Tomlinson, & Roth, Reference Blessed, Tomlinson and Roth1968). BIMC scores range from 0 to 33; higher scores reflect greater cognitive impairment. A diagnosis of dementia was based on standardized clinical criteria and required impairment in memory plus at least one additional cognitive domain, accompanied by evidence of functional decline. A diagnosis of amnestic mild cognitive impairment required objective memory impairment as stated above, subjective memory impairment indicated by responses to self or informant reports, absence of functional decline, and absence of dementia. Details are in Katz et al., Reference Katz, Lipton, Hall, Zimmerman, Sanders, Verghese and Derby2012.
Statistical methods
Non-random missingness is likely to be common in longitudinal aging studies. A subset of such situations, called auxiliary variable missing at random (A-MAR) (Daniels & Hogan, Reference Daniels and Hogan2008; Ibrahim et al., Reference Ibrahim, Lipsitz and Horton2001), occurs when the missing data process depends on the unobserved data through the auxiliary data. Here, the free recall score from the FCSRT (Grober & Buschke, Reference Grober and Buschke1987) is the outcome of interest, and the MIS-T (Lipton et al., Reference Lipton, Katz, Kuslansky, Sliwinski, Stewart, Verghese and Buschke2003) is used as auxiliary data. We compared three linear mixed effects models (Laird & Ware, Reference Laird and Ware1982) that modeled cognitive change over time as a function of chronological age: (1) A “traditional” model, ignoring non-random missingness, with random effects used to take into account heterogeneity across study participants and the lack of independence of repeated measurements on those individuals. This is the most common approach used in longitudinal studies of aging, and implicitly assumes that data are missing at random. (2) A correlated random effects model (previously called simply a “random effects model”) (Wang & Hall, Reference Wang and Hall2010), in which separate random effects are used to model the FCSRT and MIS-T, with the correlation between the random effects estimated from the data. (3) A shared random effects model in which the FCSRT and MIS-T are assumed to vary together through a common random effect.
All three models can be described using the following model equations:
$$\eqalignno{ & y_{{ik}} =b_{{0i}} {\plus}\beta _{1} \, {\asterisk} \, age_{{ik}} {\plus}{\varepsilon}_{{ik}} \cr & z_{{ik}} =c_{{0i}} {\plus}\gamma _{1} \, {\asterisk} \, age_{{ik}} {\plus}\delta _{{ik}} \cr & (b_{{0i}} ,\,c_{{0i}} )\,\sim\,BVN\left( {(\beta _{0} ,\gamma _{1} ),\left( {\matrix{ {\sigma _{b}^{2} } & {\rho \sigma _{b} \sigma _{c} } \cr {\rho \sigma _{b} \sigma _{c} } & {\sigma _{c}^{2} } \cr } } \right)} \right) \cr ({\rm bivariate}\,{\rm normal}\,{\rm distribution}) \cr & {\varepsilon}_{{ik}} \,\sim\,N(0,\,\sigma _{e}^{2} )\,({\rm bivariate}\,{\rm normal}\,{\rm distribution}) \cr & \delta _{{ik}} \,\sim\,N(0,\,\sigma _{d}^{2} )\,({\rm bivariate}\,{\rm normal}\,{\rm distribution}) $$
Here, y ik is the value of the outcome of interest for study participant i at assessment k, z ik is the value of the auxiliary variable for study participant i at assessment k, ε ik and δ ik are the residual terms that are assumed to be uncorrelated with each other and with the random effects, β 1 is the effect of age on the outcome of interest – the parameter we are most interested in estimating – and γ 1 is the effect of age on the auxiliary variable. b 0i is the random intercept for the subject i for the outcome of interest, and c 0i is the random intercept for subject i for the auxiliary variable. The random effects are allowed to be correlated with each other with correlation ρ. For model 1 (traditionnal), ρ=0; for model 3 (shared random effects) ρ=1; for model 2 (correlated random effects) 0<ρ<1.
The models are fit separately for ε4+ and ε4− study participants and we compare age-associated rates of decline between the two groups. Akaike’s information criterion (AIC, Akaike, Reference Akaike1974), an information-theory-based measure that takes into account model complexity, was used to compare fit between models. The models were fit via maximum likelihood using SAS version 9.3’s NLMIXED and MIXED procedures (SAS Institute, Cary, NC; http://www.sas.com).
Results
The analysis cohort included 620 EAS participants active since March 1, 2003, who had at least one administration of the MIS-T along with the FCSRT and were genotyped for APOE. A total of 147 of the 620 study participants (24%) were ε4+, a lower prevalence than reported in other studies (Ward et al., Reference Ward, Crean, Mercaldi, Collins, Boyd, Cook and Arrighi2012). Ten study participants had dementia and 77 had amnestic mild cognitive impairment (aMCI) at the time of the first telephone assessment. The sample included 2085 person years of follow-up. During the follow-up period (March 2003 to July 2011), there were 48 incident cases of dementia and 52 incident cases of aMCI based on in-person evaluations.
Table 1 shows the baseline demographic and cognitive characteristics by APOE ε4 status separately for participants with and without missing in person assessments. Significant baseline differences between those who missed an in person assessment and those who did not were few. ε4+participants were a year younger on average, and were more likely to be non-Hispanic Black, but were otherwise similar to ε4− participants. A total of 177 ε4+ participants (37.4%) and 51 ε4− participants (34.7%) had one or more missing in person assessments. For both ε4+ and ε4− participants, 88% of telephone assessments were followed by an in person assessment. FCSRT scores were positively correlated with MIS-T scores (Pearson correlation 0.52 for first assessment, p<.001). Rates of age-related decline on the FCSRT were lower in those missing at least one evaluation (0.354 vs. 0.301 points/year) but differences were not significant.
Cohort characteristics by presence/absence of an APOE ε4 allele, and whether the study participant missed at least one in-person memory assessment.
SD=standard deviation
Table 2 shows the results for the three approaches to modeling rates of decline in free recall in APOΕ ε4 allele carriers and non-carriers. Using the traditional model, the decline in free recall from the FCSRT was 0.284 points per year for ε4− participants and 0.500 points per year for ε4+ participants. In the correlated random effects models rates of decline were 0.337 for APOΕ ε4 negative individuals and 0.632 for APOΕ ε4+ individuals, representing 19% and 27% increases in the rates of decline, compared to the traditional model. In the shared random effects models the corresponding rates of decline were 0.349 and 0.638, representing 23% and 28% increases in the rate of decline compared to the traditional model. The differences in the rate of memory decline (as measured by FCSRT) between ε4+ and ε4− participants were 0.295 and 0.292 for the correlated and shared random effects models, 37% and 35% greater than the 0.216 difference seen in the traditional model. Corresponding changes in the rates of decline in the MIS-T were small. AIC indicated that the correlated random effects model fit best for both groups. High correlations between the random effects in the correlated random effects models indicated a strong association between the FCSRT and MIS-T measures. Sensitivity analyses (not shown) using the ranks of the FCSRT and MIS-T values rather than the actual measurements showed similar results.
Results of linear mixed models estimating rates of memory decline in ε4+ and ε4− participants.
Footnote for Table 2: For all models, the in-person age effect is in units of change in the sum of three trials of free recall from the Free and Cued Selected Reminding Test (FCSRT), and the telephone age effect is in units of the core on the Memory Impairment Screen – Telephone (MIS-T). The Model 1, the naïve model ignores the potential impact of informative missing data. Model 2, the correlated random effects model, models the correlation between the FCSRT and MIS-T, allowing the more complete data from the telephone instrument to influence the rate of memory decline in the in-person instrument. Model 3, the shared random effects model, is a special case of model two with a stronger correlation assumption.
We were concerned that the heterogeneous population that included persons with dementia or mild cognitive impairment at the beginning of the analysis period, might not be typical of all longitudinal cohort studies of cognition, so we repeated the analyses excluding the individuals with dementia or amnestic mild cognitive impairment at the first telephone assessment. The differences in the rate of memory decline (as measured by FCSRT) between 462 ε4+ and 140 ε4− participants were 0.315 and 0.313 for the correlated and shared random effects models, 27% and 26% greater than the 0.248 difference seen in the traditional model, showing robustness of this methodology.
Discussion
We have shown that a telephone assessment of memory that is modestly correlated with a sensitive and specific in-person assessment instrument can be used to address the informative nature of the missing data process in the in-person assessments. Numerous methods now exist for addressing potentially informative missingness situations where the missingness is associated only with the observed data, the “missing at random” (Rubin, Reference Rubin1976) case. When missingness is associated with unobserved outcomes, the “missing not at random” case, sensitivity analyses are sometimes used to estimate the potential for bias, but this approach is of severely limited utility. The framework we present here allows the use of auxiliary data to inform investigators of the magnitude of the missingness biases.
This demonstration is limited to modeling linear decline in cognition as a function of age; more work, with larger samples, is needed to evaluate more complex trajectories. The higher the correlation of the auxiliary data with the missing outcome data of interest, the more accurate the correction should be. In addition, death itself is a cause of non-random missingness, and these analyses did not control for the well-known phenomenon of terminal decline (Dodge, Wang, Chang, & Ganguli, Reference Dodge, Wang, Chang and Ganguli2011; Sliwinski et al., Reference Sliwinski, Stawski, Hall, Katz, Verghese and Lipton2006; Thorvaldsson et al., Reference Thorvaldsson, Hofer, Berg, Skoog, Sacuiu and Johansson2008; Wilson, Beckett, Bienias, Evans, & Bennett, Reference Wilson, Beckett, Bienias, Evans and Bennett2003; Wilson, Beck, Bienias, & Bennett, Reference Wilson, Beck, Bienias and Bennett2007). Additional theoretical and empirical work is needed to address these concerns, and more research is needed to identify brief assessment instruments that can be used as auxiliary data for other cognitive domains as well as other outcomes of interest in longitudinal aging studies such as mobility and frailty.
Despite these limitations, this work may be significant for research in settings where informative loss to follow up is likely. In aging research, there is much interest in identifying genetic and environmental factors that are associated with increased or decreased rates of cognitive decline. Taking missingness into account may provide more robust estimates of decline and increase power for detecting the effects of genetic or environmental risk factors on cognitive trajectories, and this approach would be equally applicable to randomized trials if drop out is associated with unobserved outcomes.
Acknowledgments
Funding was provided by the National Institute of Aging, Grant # P01 AG03949, the Einstein Aging Study (PI: Richard B. Lipton), which provided partial salary support to all authors. Funding for the APOE genotyping was provided by the National Center for Research Resources, Grant # UL1 RR025750, Clinical and Translational Science Award (PI: Harry Shamoon). All authors report no financial or personal conflicts of interest.
