Attrition is a particularly salient problem for longitudinal research on aging. The extent that participants who are lost to follow-up for any reason are different from full study participants can bias the results of the study and reduce generalizability. This issue is compounded because older participants and participants with lower cognitive scores are more likely to be lost to follow-up (Chatfield et al., Reference Chatfield, Brayne and Matthews2005). In their study, “Predictors of attrition in a longitudinal population-based study of aging,” Jacobsen et al. (Reference Jacobsen, Ran, Liu, Chang and Ganguli2020) investigated differences between participants who completed the study versus those who were lost to follow-up for any reason.

Jacobsen and colleagues considered a single outcome that represents all-cause attrition. That is, participants who did not leave the study were compared to participants who left the study for any reason including death, refusal, too sick, relocation, and unreachable for follow-up. They considered a broad range of variables as predictors of attrition. In the fully adjusted model, a number of variables were significantly associated with attrition: being older, male, physically inactive, dependent in at least one instrumental activity of daily living (IADL), leaving the house less than once a week, and having a clinical dementia rating >0 were associated with higher odds of attrition, while living with a partner who was also in the study, higher cognitive scores (but more subjective memory complaints), having a hobby or interest, and various forms of social participation were associated with lower odds of attrition. The logistic regression and the four machine learning approaches yielded very similar areas under the curve (AUC = 0.62 to 0.68) for predicting attrition.

The researchers presented both unadjusted results and the results from a multivariable logistic regression. Both sets of results are informative, although the researchers focused their interpretation on the adjusted results. For instance, lower education was associated with higher odds of attrition in the unadjusted model, consistent with some research (Jacomb et al., Reference Jacomb, Jorm, Korten, Christensen and Henderson2002; Van Beijsterveldt et al., Reference Van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002; Young et al., Reference Young, Powers and Bell2006), but was not statistically significant in the full model. Included covariates vary across studies and this could explain some of the inconsistency. Jacobsen et al. considered a fairly large number of social variables compared to other studies on attrition and these, along with the variables assessing cognition, may have sufficiently overlapped with education to render it not statistically significant. However, education likely influences social factors and cognition so, even though it was not significant in the full model, it may still be a useful variable in understanding attrition.

Another difference between the unadjusted and multivariable regression results was for subjective memory complaints. As expected, in the unadjusted results more subjective memory complaints were associated with higher odds of attrition; however, the association was in the opposite direction in the fully adjusted model. Lower objective cognitive scores were, as expected, associated with higher odds of attrition even in the full model. This suggests that once actual cognition is controlled, those with more subjective complaints are more likely to participate in future waves of the study. This finding goes against the theory that lower cognitive scores may be associated with attrition because people with cognitive difficulties may be afraid to return for fear of a dementia diagnosis or noticeable decline (Levin et al., Reference Levin, Katzen, Klein and Llabre2000). In fact, this finding suggests that among people with the same cognitive level, the ones who perceive more memory complaints may have more motivation and desire to continue with the study.

A meta-analysis examining preventable non-response (i.e., not death) found patterns that were fairly consistent with Jacobsen et al.’s study (Chatfield et al., Reference Chatfield, Brayne and Matthews2005). Older age, poor functioning, cognitive impairment, living alone, and not being married were associated with more dropout. However, the authors of the meta-analysis also noted that the association between social factors and attrition has not been as thoroughly investigated. Jacobsen et al.’s study included a number of social factors that can be considered components of social vulnerability (Godin and Andrew, Reference Godin, Andrew, Gu and Dupre2019). Social vulnerability is a construct that comprehensively considers a broad range of social factors that can leave individuals vulnerable to physical and mental health issues and it has been found to be associated with both cognition (Andrew and Rockwood, Reference Andrew and Rockwood2010) and mortality (Andrew et al., Reference Andrew, Mitnitski, Kirkland and Rockwood2012). If the results of longitudinal studies on aging do not generalize to those who are most socially vulnerable, policies based on these results could lead to further disadvantage. In Jacobsen’s et al.’s study a number of the individual components of social vulnerability were associated with attrition in the unadjusted models: lower education, unmarried, lives alone, low frequency of leaving home, fewer confidants, and no hobbies, computer use, social activity, or social engagement. A number of these remained statistically significant in the multivariable model: fewer confidants and no hobbies, social activity, or social engagement. Oversampling from participants who are likely to be socially vulnerable may help mitigate this issue to some degree. However, it is still crucial for researchers to understand how results from longitudinal studies on aging may be biased in this way and to be cautious when making policy recommendations.

Researchers seeking to understand the aging process are often also interested in frailty (e.g., Godin et al., Reference Godin, Armstrong, Wallace, Rockwood and Andrew2019; St John et al., Reference St John, Tyas, Griffith and Menec2017). Given the link between frailty and cognition (Godin et al., Reference Godin, Armstrong, Rockwood and Andrew2017), Jacobsen et al.’s results likely generalize to research on frailty. Many of the variables considered in Jacobsen et al.’s study are related to frailty, which can be conceptualized as the number of health deficits an individual has accumulated (Searle et al., Reference Searle, Mitnitski, Gahbauer, Gill and Rockwood2008). Although any single health deficit may fail to predict an adverse outcome, a frailty index (FI) that combines all health deficits into a single score is often a much stronger predictor of adverse outcomes, including mortality (Wallace et al., Reference Wallace, Theou, Kirkland, Rockwood, Davidson, Shimbo and Rockwood2014). Possibly, a frailty index would be a better predictor of non-death attrition than individual health variables. Working with fewer variables may be useful in designing methodologies to prevent attrition.

The authors made an interesting choice in combining all forms of attrition into a single outcome. Analyzed in this way, the findings provide good guidance on how results may be biased if listwise deletion is used to handle missing data for both participants who died and for those who were non-respondents in subsequent waves. This is critically important information because, despite its limitations, listwise deletion is a common method for handling missing data (Burton and Altman, Reference Burton and Altman2004; Eekhout et al., Reference Eekhout2012; Geva et al., Reference Geva, Shahar, Harris, Tepper, Molenberghs and Friger2013). There are some limitations to considering all forms of attrition as a single outcome. Most notably, there are likely different predictors of the various mechanisms of attrition, the results are not as useful for designing strategies to prevent attrition, and the results may be conflated with variables that are known to be associated with mortality.

There are many useful ways to categorize attrition including death versus other types of attrition, preventable versus not preventable attrition, death and ill health versus other types of attrition. Many types of attrition are common in longitudinal studies on aging and there tend to be different predictors for attrition due to death and other reasons (Brilleman et al., Reference Brilleman, Pachana and Dobson2010). In one study on attrition, there was a significant difference in processing speed between full participants and those who died, but no significant difference between full participants and those who refused further participation (Van Beijsterveldt et al., Reference Van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002); however, recall scores of both participants who died and those who dropped out were lower than the recall scores of continuing participants. Further, men were more likely to die but women were more likely to refuse future participation (Van Beijsterveldt et al., Reference Van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002). Possibly, Jacobsen et al. could have achieved higher AUCs had they considered different types of attrition separately.

Death and illness occur in both the sample and the target population whereas other types of attrition are limited to the sample (Brilleman et al., Reference Brilleman, Pachana and Dobson2010). This is important to note for two reasons: methodologies to minimize attrition should focus on preventable attrition, such as refusals and loss of contact and approaches to dealing with missing data may need to be adapted for specific reasons for attrition. Strategies such as offering transportation to the testing site are often successful in reducing attrition in older participants (Van Beijsterveldt et al., Reference Van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002); however, these strategies will not likely prevent attrition due to ill health or death.

Including all types of attrition in one outcome can artificially enhance some predictors of attrition. Although age is commonly found to be associated with attrition, one study found that when death was taken into consideration, attrition was similar across age groups (Van Beijsterveldt et al., Reference Van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002). In another study that examined attrition from young through to old womanhood, older women had comparatively good rates of continued participation (Young et al., Reference Young, Powers and Bell2006). Women of middle age were the best responders. Young women were more likely to drop out due to loss of contact, whereas older adults could be contacted but either chose to withdraw or did not return or died. In Jacobsen et al.’s study just over one-third of the attrition was due to death and another 20% due to being too ill. As such, for some variables such as age, it is difficult to untangle whether the association is between age and attrition or age and mortality and morbidity.

Research into attrition in longitudinal research typically indicates that the most vulnerable, sick, and cognitively impaired are more likely to be lost to follow-up due to death and other reasons (Burke et al., Reference Burke2019; Slymen et al., Reference Slymen, Drew, Elder and Williams1996; Young et al., Reference Young, Powers and Bell2006). These same individuals are also more likely to be non-participants even at the first wave of study (Helliwell et al., Reference Helliwell, Aylesworth, McDowell, Baumgarten and Sykes2001). As such, the loss of these participants in subsequent waves of longitudinal studies is conflating an already existing problem. Attrition issues can be particularly problematic in aging research as health problems and social vulnerability tend to increase with age (Andrew et al., Reference Andrew, Mitnitski and Rockwood2008; Mitnitski and Rockwood, Reference Mitnitski and Rockwood2016). A number of studies have shown that, after adjusting for a number of variables, cognitive impairment had the strongest association with dropping out of future survey waves (Chatfield et al., Reference Chatfield, Brayne and Matthews2005). This is particularly problematic for studies on cognitive function and dementia. Research into understanding predictors of attrition in longitudinal studies on aging is critical to developing unbiased results on cognition and aging. Jacobsen et al.’s work provides insight into how attrition may be biasing results in longitudinal studies on aging, particularly those studies investigating cognition as an outcome.

In addition to understanding what factors predict attrition, more research is needed to understand how attrition is impacting the conclusions drawn from longitudinal studies on aging. One study used a combination of simulations and imputation methods to investigate how attrition might be influencing results (Salthouse, Reference Salthouse2019). In young adults there was no difference in cognition or change in cognition between participants who dropped out and those who returned. However, in older adults, returning participants had a higher cognitive ability, but the rate of change in cognition over time was the same for participants who continued with the study and those who dropped out. This finding is in line with another study, which also found that though returning participants had higher cognitive levels, the rate of change in cognition was the same for both groups (Van Beijsterveldt et al., Reference Van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002). These findings also carry over to frailty. With appropriate statistical techniques such as linear mixed models under the missing at random assumption, attrition did not have an impact on frailty trajectories until age 80 and only modestly underestimated frailty after that age (Stolz et al., Reference Stolz, Mayerl, Rásky and Freidl2018).

An important strength of the study is that the Jacobsen et al. study used multivariable logistic regression and four different machine learning techniques, which all had very similar predictive ability; although, it would have also been informative to know whether variable importance was similar across all techniques. Importantly, Jacobsen et al.’s study sheds light on factors associated with overall attrition and the inclusion of a number of social factors fills an important gap in this area of research.