Introduction
In longitudinal studies, it is almost inevitable for some follow-up data to be missing. This is particularly true in studies of older adults, among whom attrition is often related to mortality, which increases dramatically with age. Other reasons for attrition can include participants' reluctance to continue in the study, illness, or relocation from the study area. Different factors can be associated with different causes of attrition (van Beijsterveldt et al., Reference van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002; Matthews et al., Reference Matthews, Chatfield, Freeman, McCracken and Brayne2004; Reference Matthews, Chatfield and Brayne2006). When elderly individuals develop depressive symptoms, they may become particularly unmotivated to participate in research assessments. They also have altered health indices, functional ability, health services utilization, and elevated mortality rates (Ganguli et al., Reference Ganguli, Dodge and Mulsant2002). Individuals may in fact have symptoms shortly before they die (Katona and Shankar, Reference Katona and Shankar2004) or drop out of studies (Casey et al., Reference Casey, Hughes, Waechter, Josephson and Rosneck2008; Ong et al., Reference Ong, Kuo and Manber2008). The course and relationships of depressive symptoms in these individuals may therefore differ substantially from those observed in individuals who complete the study.
Our objective in the present study was to examine the impact of attrition on the analysis of factors associated with depressive symptoms over time, and to compare different statistical approaches to handling attrition effects in population-based data. In this study, we examined factors associated with depressive symptoms over multiple years of follow-up, using three models: a generalized mixed model and two shared parameter models which simultaneously modeled depression and dropout. Specifically, we used three well-established analytical methods for longitudinal data to compare the estimated effect sizes for different covariates of depressive symptoms, and subsequently determine whether the results would change in magnitude or interpretation. The three models are based on different statistical assumptions as to whether or not the dropout data were missing at random (Rubin, Reference Rubin1976; Little and Rubin, Reference Little and Rubin2002).
Methods
Study setting, design and participants
The Monongahela Valley Independent Elders Survey (MoVIES) was conducted from 1987 to 2002 in a largely rural American region with a population that was economically depressed and had relatively low levels of education. Inclusion criteria for study participation included being at least 65 years old and community-dwelling at the time of recruitment, being fluent in English, and having a sixth-grade education or higher (Ganguli et al., Reference Ganguli1993). The initial study cohort consisted of 1681 participants assessed at study entry (wave 1) (Ganguli et al., Reference Ganguli, Lytle, Reynolds and Dodge1998) and reassessed in a series of approximately biennial data collection waves. Between waves 1 and 2, a total of 340 participants died, relocated or dropped out of the study, leaving 1341 participants to be assessed at wave 2.
Data on depressive symptoms were collected for the first time in wave 2 (1989–1991), which we therefore designated as the data baseline for the current analyses. The modified Center for Epidemiological Studies–Depression Scale (mCES-D Scale), in which scores range from 0 to 20, was used to represent the number of self-reported depressive symptoms experienced over most of the preceding week. The score was dichotomized as 0–4 symptoms vs. 5 or more symptoms, with this threshold representing the 90th percentile of the cohort (Ganguli et al., Reference Ganguli, Gilby, Seaberg and Belle1995). The scale was administered at waves 2 through 6. Here, an mCES-D score <5 indicates the absence of substantial depressive symptoms (“not depressed” for the purpose of this article) while a score ≥5 indicates the presence of substantial depressive symptoms (“depressed”).
To ensure validity of responses to the mCES-D depression questions and to avoid confounding by dementia, we excluded from these analyses 48 participants who were classified as having dementia at baseline, and 12 participants who did not complete the Mini-mental State Examination (MMSE; Folstein et al., Reference Folstein, Folstein and McHugh1975), a screening measure of global cognitive functioning. We further excluded 21 participants whose mCES-D depression data were incomplete.
For the current analyses, we used data from the remaining 1260 participants, whose demographic and clinical characteristics have been described previously (Andreescu et al., Reference Andreescu, Chang, Mulsant and Ganguli2008). Of these 1260 participants, 48 (3.81%) completed waves 2 and 6 but skipped one or more waves in between. For these participants, we imputed the missing mCES-D score by using the average of the scores derived from the waves immediately before and after the missing wave.
Statistical methods
Descriptive statistics
The overall cohort, and the subgroup that dropped out over the course of the study, were characterized using means and proportions with regard to demographic characteristics (age, sex, and education); mCES-D depression symptom scores (dichotomized as <5 vs. ≥5); number of prescription drugs being used (dichotomized as <4 vs. ≥4), as a measure of overall morbidity; whether antidepressant drugs were being used; and functional impairment with regard to instrumental activities of daily living (IADLs). We used the χ2 or Fisher's exact test to compare the percentage with presence of substantial depressive symptoms between people who dropped out before or at wave 6 and people who completed the study, and also to compare the percentage with presence of substantial depressive symptoms at baseline between subgroups with different characteristics (e.g. female vs. male).
Overview of missing data mechanisms
Rubin (Reference Rubin1976) and Little and Rubin (Reference Little and Rubin2002) classified the missing data process into three mechanisms: missing completely at random (MCAR), missing at random (MAR), and non-ignorable missing (NIM). In the study of depressive symptoms over time, missing data will be MCAR if the probability of attrition does not depend on the presence or severity of depression (i.e. the number of depressive symptoms). It will be MAR if the probability of attrition depends on the severity of depression observed at some time point(s) before dropout. Finally, it will be NIM if the probability of attrition depends on the severity of depression at the time of dropout, but the severity is unobserved, i.e. was not measured at the time of dropout. Preliminary analysis suggested that attrition here depended on the magnitude of depressive symptoms (Table 1). Therefore methods under the MCAR assumption were ruled out.
Baseline characteristics for individuals who completed all follow up (n = 591) and those who did not complete all follow up (n = 669)
*Based on a χ2 or Fisher's exact test.
Models for longitudinal outcome with missing data
Traditionally, the models for analyzing longitudinal data include the multivariate analysis of variance (MANOVA), the model with generalized estimating equation (GEE), and the mixed effect model. These models operate under the assumption of MCAR (MANOVA and GEE) or MAR (weighted version of GEE, GEE with imputations, and mixed effect model). Several methods have been developed specifically for NIM data under the framework of selection models or pattern-mixture models (Little, Reference Little1995; Roy, Reference Roy2003). Shared parameter models are a popular class of selection models that use common unobserved random effects to predict the courses of outcome and dropout over time.
Modeling approaches
We fit three separate models to evaluate and compare the impact of missingness / nonresponse on parameter estimates. The first was the generalized linear mixed model that assumes dropout as MAR. The other two methods were shared parameter models that jointly modeled the outcome (depressive symptoms at each assessment wave) and the likelihood of attrition, through a common unobserved random effect component in order to account for the NIM (Wu and Carroll, Reference Wu and Carroll1988; Wulfsohn and Tsiatis, Reference Wulfsohn and Tsiatis1997; Lin et al., Reference Lin, McCulloch and Mayne2002; Roy, Reference Roy2003; Tsiatis and Davidian, Reference Tsiatis and Davidian2004; Beunckens et al., Reference Beunckens, Molenberghs and Kenward2005; Vonesh et al., Reference Vonesh, Greene, Mark and Schluchter2006). Shared parameter models have two key advantages: they provide a flexible framework for handling intermittently missing patterns, and they can be applied when study participants do not all follow the same schedule of follow-up interviews or examinations.
In the generalized mixed model, depressive symptom scores were fit by a logistic regression model with random intercepts representing the subject-specific depression status at baseline. In the shared parameter models, the time to dropout was modeled by a regression model in which a random effect from the generalized linear mixed model acted as a predictor that represents the association between the repeatedly measured mCES-D scores and the dropout process. We fit two shared parameter models, a Weibull accelerated failure time (AFT) model and a discrete failure time model, both of which were conditional on the subject-specific random effect, which served as a subject-specific covariate. We compared the results from the shared parameter models (which jointly modeled both depression and dropout as outcomes) with the results from a generalized logistic random-effects model (which only modeled depression as outcome). Note that dropout was identified consistently at each follow-up wave (waves 3–6) and not in between waves; thus it was a discrete rather than a continuous type of event. However, several researchers still use continuous-type survival models to fit discrete-type survival data. Compared with the previous literature, we used both continuous-type and discrete-type survival regressions to model the dropout process, and compared the results as a sensitivity analysis. We selected the Weibull AFT model among various continuous-type survival models since our data met the assumption of monotonic hazards for the Weibull model (the hazard function showed increasing trend).
The generalized mixed model and the longitudinal part (depressive symptoms at multiple waves) of the shared parameter models included the following baseline (wave 2) covariates: gender, age (65–74 years, 75–84 years, and ≥85 years), education level (less than high school vs. at least high school completion), total number of prescription drugs used (<4 or ≥4), use of antidepressant drugs (yes or no), and functional impairment (any or no IADL impairment). To measure time to dropout, we defined the failure date as the date of dropout and the censoring date as the study end date (wave 6). If the actual date of dropout was not available, we approximated the date as 2.4 years after the last interview date, this being the average interval between two consecutive interview dates, in this cohort.
To determine how well the models fit the data, we first checked whether a NIM model performs better than a MAR model by testing the variance of the random effects component in the shared parameter model being significantly different from zero. We further used the Akaike information criteria (AIC) to compare the goodness of fit between the two shared parameter models. A smaller AIC value indicates a better fit (Pawitan, Reference Pawitan2004).
One of the complications of using a generalized linear mixed model is the computational efficiency (Jiang, Reference Jiang1998), specifically the time (and therefore the cost) of running the models. A shorter central processing unit (CPU) running time indicates a computationally more efficient model.
The notation and shared parameter models, as well as the corresponding estimation procedures, are provided in Appendix S1 (available as supplementary material attached to the electronic version of this paper at www.journals.cambridge.org/jid_IPG).
Results
At baseline, the 1260 participants eligible for these analyses had a mean (SD) age of 74.6 (5.3) years. Women comprised 60.79% (n = 766) of the cohort, and 61.11% (n = 770) had at least high school education. Of the 1260 participants, a total of 669 (53.10%) dropped out before or at wave 6, with 171 (13.57%) of the dropouts occurring between waves 2 and 3, 154 (12.22%) between waves 3 and 4, 164 (13.02%) between waves 4 and 5, and180 (14.29%) between waves 5 and 6. The reasons for dropouts are shown in Figure 1. We found that 81 of 669 participants (12.11%) who dropped out before or at wave 6, compared to 46 of 591 participants (7.78%) who completed wave 6, had substantial depressive symptoms (mCES-D ≥5). This difference was significant (χ2 with 1 degree of freedom = 6.47, P = 0.01). As Figure 2 shows, there was a rise in the percentage of participants who had substantial depressive symptoms before their dropouts occurred, and depression was least common in the participants who completed the most waves.
Flow diagram of reasons for attrition between data collection waves.
Proportion of cohort with at least 5 depressive symptoms among subgroups with different lengths of follow up.
For interested readers, the unadjusted relationships between presence of depressive symptoms and each covariate at each wave are provided in Appendix S2 (available as supplementary material attached to the electronic version of this paper at www.journals.cambridge.org/jid_IPG). Regardless of the number of completed waves, depression at baseline was significantly more frequent in women than in men (χ2 with 1 degree of freedom = 11.63, P = 0.001); in participants aged 75–84 years than in those aged 65–74 years (χ2 with 1 degree of freedom = 4.37, P = 0.001); in participants who did not complete high school than in those who did (χ2 with 1 degree of freedom = 17.21, P <0.001); in participants who used <4 prescription drugs than in those who used ≥4 drugs (χ2 with 1 degree of freedom = 30.59, P <0.001); in participants who did use antidepressant drugs than in those who did not (Fisher's exact test P <0.001); and in participants who had any functional impairment in IADL than in those who had none (χ2 with 1 degree of freedom = 24.83, P <0.001). Note that none of the nonsignificant findings were due to insufficient power (<80%). The percentages with ≥5 depressive symptoms by each level of each factor can be found in Appendix S2 online.
The multivariable relationship between factors and depression is summarized in Table 2, which includes results from all three models (the generalized mixed model, the shared parameter model with Weibull dropout, and the shared parameter model with discrete failure dropout). The probability of having substantial (five or more) depressive symptoms was not affected by age, but it was affected by gender, education, number of regularly used prescription drugs, and functional limitations in IADL. In all three models, the direction of significance was the same: factors that increased the probability of depressive symptoms were being female, having less than a high school education, using four or more prescription drugs, using antidepressant drugs, and IADL impairment. However, for each of the factors, the strength of the association, as represented by the odds ratios, were similar in the two shared parameter models and differed from those in the generalized mixed model. For example, in the shared parameter models, the odds of depression for people using antidepressant drugs was 16 times greater than that for people not using these drugs. However, in the generalized mixed model, the odds were only about three times greater. Being male and using antidepressant drugs were positively associated with time to dropout in the Weibull model but not in the discrete time model.
Comparison of three modelsa in terms of their covariate effects on the presence of concurrent depression (having five or more depressive symptoms)
aAll models are multivariable models with intercept (estimate not shown).
† Effect of wave at which data were collected (with wave 2 as the reference group).
OR = odds ratio; CI = confidence interval; CPU = central processing unit (shorter is computationally faster); AIC = Akaike information criteria (smaller indicates a better fit); N/A = not applicable.
In both shared parameter models, the estimated variances were significantly different from zero (estimated variance = 18.70 and P = 0.001 for the Weibull shared model; estimated variance = 18.94 and P = 0.001 for the discrete time model). Therefore, a generalized linear mixed model did not sufficiently account for the total variability. This suggested that an NIM model was a better fit. Comparing the two shared parameter models, the Akaike information criteria (AIC) showed that the discrete time dropout model had a better fit than the Weibull model (AIC of 3453.9 vs. 3599.7). This was as expected because our dropout process was actually discrete type time-to-event data.
To estimate computational efficiency, we measured the CPU times of the two models. The discrete time dropout model used approximately 31 minutes of CPU time compared to 15 minutes of CPU time use for the Weibull model, running on a server with 8 Xeon processors of 2.66 GHz with 32 GB of RAM and a 4-disk striped RAID array.
Discussion
We found that the presence of substantial depressive symptoms in older adults was associated over time with female gender, less education, overall morbidity as reflected in number of prescription medications, use of antidepressant drugs, and functional limitations. These findings are consistent with the literature on late-life depression (Katona and Shankar, Reference Katona and Shankar2004). Our study also showed that the presence of depression is significantly associated with attrition from the study, confirming previous findings that mortality is often preceded by depression or drop in mood (Jorm et al., Reference Jorm, Henderson, Kay and Jacomb1991; Katona and Shankar, Reference Katona and Shankar2004; Andreescu et al., Reference Andreescu, Chang, Mulsant and Ganguli2008). This drop parallels observations regarding a decrease in cognitive function before death, the phenomenon referred to as “terminal decline” (Wilson et al., Reference Wilson, Beck, Bienias and Bennett2007; Lavery et al., Reference Lavery, Dodge, Snitz and Ganguli2008). These findings raise the question of the potential impact of attrition on studies of factors that are themselves related to attrition.
Some degree of attrition is virtually unavoidable in longitudinal studies and must be handled appropriately. Our approach was to compare the results of a model that was unadjusted for non-ignorable attrition (the mixed model) with the results of two models that were adjusted for attrition (the shared parameter models). We found that the direction of the covariate effects was the same in all three models but that the strength of the association was not the same. Information available on the dropouts in this study suggests that attrition was non-ignorable and that the study participants showed an increase in depressive symptoms before they dropped out. Therefore, it appears reasonable to conclude that a shared parameter model provides a less distorted picture of depression symptoms over time than does a mixed model that assumes that attrition occurred randomly. In studies of other populations, or studies examining other relationships, both the direction and the strength of the associations may vary when different models are used to account for attrition. Replications in other datasets should determine under which circumstances the estimation results or the inference results – or both – will vary. Further investigations are also needed to determine under what conditions the bias caused by non-ignorable dropout can and cannot be eliminated after adjusting for it in the shared parameter model framework. These are important but frequently ignored aspects of the analysis of longitudinal data with substantial attrition.
Although most investigators make strenuous efforts to minimize attrition in longitudinal studies, many often choose to ignore attrition mechanisms while carrying out data analysis. For example, some investigators impose their analysis on participants who completed the study, which makes the de facto assumption that attrition took place randomly, or the type of attrition can be ignored. As demonstrated in several simulation studies, any given reason for attrition is associated with multiple factors that could themselves be under investigation in the same study. Therefore, mishandling missing data at the analysis stage could cause the estimated effect sizes to be biased and lead to a distorted understanding of the outcome over time. Touloumi et al. (Reference Touloumi, Babiker, Pocock and Darbyshire2001) compared the impact on estimation bias and efficiency (the variability of the estimator) among different models for analysis of longitudinal data with various attrition mechanisms. Kristman et al. (Reference Kristman, Manno and Côté2005) showed that when data are NIM, imputation and weighting methods may not reduce bias due to attrition efficiently. Eerola et al. (Reference Eerola, Huurre and Aro2005) found poor predictive ability when the attrition mechanism is ignored. Funatogawa et al. (Reference Funatogawa, Funatogawa and Takeuchi2008) showed that linear mixed effect models could give biased estimators when data were simulated under an NIM mechanism. Therefore it is important to choose a model that accounts for the missing data mechanism appropriately.
In order to choose the correct models, the first step is to determine the correct attrition mechanism for the given data. To distinguish between MCAR and NIM, researchers usually compare the outcome measures between subjects who do and do not complete the study. Where possible, the judgment should also be made on the basis of prior knowledge and experience as to whether participants' propensity to complete the study is likely to be highly correlated with the outcome. There is no systematic method to distinguish between MAR and NIM. Researchers tend to make this judgment based on experience but can also use the Index of Local Sensitivity to Non-Ignorable dropout (ISNI) method as an analysis of sensitivity (Troxell et al., Reference Troxell, Ma and Heitjan2004). If the data justify the assumption of MCAR, MANOVA models or models with GEE are typically used. When data are MAR, GEE with weighting, GEE with imputations, or mixed models are the common choices. Selection models and pattern-mixture models are specifically designed for data with NIM. Recently, the joint modeling of longitudinal outcomes and attrition process has gained popularity because of the ease of building the models into standard statistical packages and natural interpretation. van Beijsterveldt et al. (Reference van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002) has indicated that even though the effect of attrition on the evolution of the outcome could be small, careful selection of analytical methods is helpful in correcting the estimation bias and allowing a better interpretation of results.
Matthews et al. (Reference Matthews, Chatfield, Freeman, McCracken and Brayne2004) recommends that all longitudinal studies should investigate attrition, which may help with aspects of design and with the testing of specific hypotheses. In our study, factors associated with overall attrition were age, sex, education, depressive symptoms, IADL impairment, use of more prescription drugs, and use of antidepressant drugs, most of which were also associated with the presence of depressive symptoms. Previous studies have identified different factors associated with different reasons for attrition. Those who are older, male, smokers, have poor cognitive ability, or have impaired ADL have higher attrition rates due to death (van Beijsterveldt et al., Reference van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002; Matthews et al., Reference Matthews, Chatfield, Freeman, McCracken and Brayne2004). Participants who are women, have lower educational levels, or have lower baseline scores on neurocognitive tests have higher rates of dropout due to refusal to continue (van Beijsterveldt et al., Reference van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002; Matthews et al., Reference Matthews, Chatfield, Freeman, McCracken and Brayne2004; Reference Matthews, Chatfield and Brayne2006). Those who are single, smokers, demented or depressed have higher attrition rates due to relocation or becoming uncontactable (Matthews et al., Reference Matthews, Chatfield, Freeman, McCracken and Brayne2004).
The strengths of the MoVIES dataset are the study cohorts that are representative of the community, with large sample size and relatively long period of follow-up with repeated measures. Potential limitations include the fact that assessments were carried out biennially rather than annually, and that attrition increased with time; however, the latter allowed us to carry out the analyses reported here. We measured self-reported depressive symptoms and did not diagnose depressive illness clinically. The strengths of the shared parameter models are that they are a powerful, flexible and easily implemented tool for analyzing longitudinal data while minimizing attrition bias. We used parametric models to fit both the longitudinal and dropout parts of the shared parameter models. Therefore, it was not difficult to incorporate them into standard software packages. We used the NLMIXED procedure in SAS, and have provided our codes in Appendix S1 (available in the online version of this paper at www.journals.cambridge.org/jid.ipg). Depending on the equipment used, the sample size and the likelihood function derived from the data, the calculation of results could be extremely computationally demanding. Therefore, if multiple shared parameter models are used to fit the data, the performance of the models should be judged both by how well a given model fits the data (goodness-of-fit statistics) and by its computational efficiency.
Attrition is often related to the outcomes of interest in longitudinal studies. Given the potential for attrition bias, investigators should make every effort to maximize cohort retention and minimize attrition in longitudinal studies. Since some attrition is inevitable, efforts should be made to minimize the resulting bias. Any information that can be obtained about the reasons for dropouts will help to determine the extent to which attrition can be ignored. At the level of data analysis, appropriate statistical methods should be employed to minimize bias. Intervention trials might also find the new approaches more appropriate to handling dropouts than merely carrying the last observation forward (Gomeni et al., Reference Gomeni, Lavergne and Merlo-Pich2009). The shared parameter models presented here are valuable to other researchers engaged in longitudinal studies, particularly studies of older adults.
Online supplementary materials
The SAS codes used to generate Table 2 and the unadjusted relationships between the presence of depressive symptoms and each covariate at each wave are available as online supplementary material (Appendix S1 and Appendix S2, respectively) at www.journals.cambridge.org/jid_IPG.
Conflict of interest
None.
Description of authors' roles
Dr. Chang had full access to the data in the study, confirmed the accuracy of the data analysis, drafted the manuscript, and provided critical revision of the manuscript for important intellectual content. Ms. Yang performed the statistical analyses and drafted the manuscript. Dr. Tang drafted the manuscript, confirmed the accuracy of the methods used in the study, and provided intellectual contributions to the missing data background reviews and interpretations of the data. Dr. Ganguli had full access to the data in the study and takes responsibility for the integrity of the data. In addition, she drafted the manuscript and provided critical revision of the manuscript for important intellectual content, and was the Principal Investigator of research grant R01 AG07562 that funded the MoVIES study.
Acknowledgments
The work reported here was supported in part by research grant R01 AG07562 and career development grants K24 AG022035 and K25 DK059928 from the National Institutes of Health, U.S. Department of Health and Human Services. The authors thank the editor and two referees for providing insightful comments and suggestions.
