Longitudinal contrasts are essential for providing information about within-person change. However, both the generalizability and the validity of longitudinal comparisons can be compromised if the individuals who return for subsequent occasions differ in meaningful ways from the individuals who do not return. In fact, there are numerous reports that attrition in longitudinal studies is selective, usually in the direction of higher levels of functioning in participants who continue in the study compared to those who do not continue (e.g., Chatfield et al., Reference Chatfield, Brayne and Matthews2005; Matthews et al., Reference Matthews, Chatfield, Freeman, McCracken and Brayne2004; Rabbitt et al., Reference Rabbitt, Lunn, Wong and Cobain2008; Riegel et al., Reference Riegel, Riegel and Meyer1968; Salthouse, Reference Salthouse2014a; Schaie et al., Reference Schaie, Labouvie and Barrett1973; Van Beijsterveldt et al., Reference Van Beijsterveldt, van Boxtel, Bosma, Houx, Buntinx and Jolles2002). Although the relation of attrition on level of cognitive functioning is well established, less is known about the relation of attrition on change in cognitive functioning. A few studies have reported more negative cognitive change in participants who discontinued participation compared to participants who continued (e.g., Cooney et al., Reference Cooney, Schaie and Willis1988; Siegler & Botwinick, Reference Siegler and Botwinick1979), but inconsistent results have also been reported (e.g., Kennison & Zelinski, Reference Kennison and Zelinski2005).
The lack of definitive information about the relation of attrition on cognitive change is unfortunate because the implications of attrition could be quite different depending on whether the effects are primarily on the level of cognitive functioning or on the rate of change in functioning. That is, if the people who drop out of a longitudinal study have lower levels of cognitive functioning than people who continue, generalizability of the results may be limited to high-functioning individuals, but interpretations of the longitudinal age trends would not necessarily be affected. In contrast, inferences about the direction and magnitude of longitudinal change could be distorted if the individuals who drop out of a longitudinal study have more negative cognitive change prior to dropping out than the individuals who continue. Specifically, cognitive decline might be underestimated in longitudinal comparisons if the returning participants have less negative change than the participants who drop out.
A major goal of the current project was to investigate the relation between both level and change in cognitive functioning and the number of longitudinal occasions completed by the participants. In keeping with prior research, participants who returned for subsequent longitudinal occasions were expected to have higher levels of cognitive functioning on the first occasion than participants who did not return. Change in cognitive functioning was assessed in terms of slopes relating cognitive performance to the number of completed occasions. If attrition is selective with respect to cognitive change, these slopes would be expected to be more negative in participants who completed fewer occasions. In contrast, no relation between the slopes and number of completed occasions would be expected if attrition was unrelated to cognitive change.
The data for the current project were based on an ongoing longitudinal study that began in 2001 (Salthouse, Reference Salthouse2014b; Salthouse et al., Reference Salthouse, Pink and Tucker-Drob2008). New participants were recruited between 2001 and 2015, and prior participants were invited to return after intervals averaging about 3 years. Factors associated with the attrition from the first to the second occasion were discussed in Salthouse (Reference Salthouse2014a).
METHOD
Sample
Participants were recruited with newspaper advertisements, flyers, and referrals from other participants. The first participants were recruited in 2001, with new samples of participants recruited nearly every year through 2015. Returning participants were tested between 2004 and 2017, and were invited to return after an average of about 3 years from the previous occasion. The research was conducted in compliance with the Helsinki Declaration and was approved by the University of Virginia Institutional Review Board for Social and Behavioral Sciences (Protocol # 2009-0108-00).
Table 1 contains characteristics of participants in three age groups (young: ages 18–44, middle: ages 45–64, and old: ages 65–85), who had completed between one and five longitudinal occasions. It is important to emphasize that there was no overlap in participants across groups, as none of the participants with only one occasion were also in the group with two occasions, none of the participants with either one or two occasions were included in the group with three occasions, so on.
Demographic characteristics of participants in three age groups who completed different numbers of longitudinal occasions
Note: *p < .01. Health is a self rating on a scale from 1 for “excellent” to 5 for “poor.” MMSE is the Mini-Mental State Exam (Folstein et al., Reference Folstein, Folstein and McHugh1975). Estimated IQ was based on a regression equation predicting full-scale IQ from three cognitive tests (Salthouse, Reference Salthouse2014a). T1–Tn int. is the number of years between the first (T1) and the last (nth) occasion. NA indicates that the estimate was not available. The column labeled β contains the standardized regression coefficient for the number of occasions effect in a regression analysis, and the column labeled F/eta2 contains the results of the analysis of variance treating number of occasions as a categorical variable.
In the young group, participants who had completed more occasions were older than those who had completed fewer occasions, but the reverse was the case in the old group. In the middle and old groups, the participants completing more occasions had more years of education, higher estimated IQ scores, and among the old participants, higher MMSE scores, than participants completing fewer occasions. However, there were no differences in these characteristics across participants with different numbers of occasions in the young group. Although not reported in the table, the intervals between successive occasions (e.g., T1–T2, T2–T3, etc.) were shorter among participants completing more occasions.
Tests
A total of 16 cognitive tests, representing five cognitive domains, were administered in the same order to all participants. Episodic memory was represented by word recall, paired associates, and story memory tests. Perceptual speed ability was represented by a digit symbol substitution test, and pattern comparison and letter comparison tests. Reasoning was represented by a matrix reasoning test, a letter sets test, and a series completion test. Spatial visualization (space) ability was represented by a spatial relations test, a paper folding test, and a form boards test. Finally, vocabulary was represented by a provide-the-definition test, a picture naming test, and multiple-choice synonym and antonym tests. Details of the tests, including reliabilities and results of confirmatory factor analyses supporting the hypothesized ability structure, are reported in other publications (e.g., Salthouse, Reference Salthouse2010; Salthouse et al., Reference Salthouse, Pink and Tucker-Drob2008).
All scores were converted to z-scores based on the mean and standard deviation of the distribution of scores on the first occasion completed by all participants. Composite scores were then created by averaging z-scores for the tests representing each cognitive domain. It is unlikely that the range of scores was restricted by measurement floor or ceiling effects because the means and standard deviations of the raw scores were not close to the minimum or maximum possible, and the aggregation of three or more separate scores into composite scores also served to minimize measurement artifacts.
RESULTS
Figures 1 and 2 portray mean composite cognitive scores as a function of number of occasions in the three age groups. The memory scores are portrayed in Figure 1, and the speed, reasoning, space, and vocabulary scores are portrayed in the four panels of Figure 2. Within each cognitive domain there was relatively small variation in cognitive performance across number of occasions in the young group, moderately large variation in the old group, and an intermediate level of variation in the middle group. Furthermore, the relations between cognitive performance and number of occasions were generally positive in the young group, negative in the old group, and with the exception of speed, nearly flat in the middle group. Prior research comparing participants tested either once or twice and comparing participants from the same birth cohort tested at different ages in different years suggests that the positive relations in the young group are likely attributable to practice, or test experience, effects (Salthouse, Reference Salthouse2010; Reference Salthouse2018).
Mean composite z-score (and standard errors) for memory ability in adults in three age groups who had completed between one and five occasions. Successive occasions are separated by arbitrary units and do not correspond to actual intervals.
Mean composite z-scores (and standard errors) for speed, reasoning, space, and vocabulary in adults in three age groups who had completed between one and five occasions. Successive occasions are separated by arbitrary units and do not correspond to actual intervals.
The first set of analyses investigated the relation between scores on the first occasion and the number of completed occasions. In one of the analyses, number of occasions was treated as a continuous variable and entered as a predictor of composite score in linear regression analyses. Because participants completing different numbers of occasions may have differed qualitatively and not merely quantitatively, analyses of variance were also conducted in which number of occasions was treated as a categorical variable. In order to investigate the role of demographic measures, in both types of analyses the relations between number of occasions and T1 composite scores were examined with and without demographic measures as covariates.
Inspection of the entries in Table 2 reveals that there were significant relations between number of completed occasions and T1 composite scores in the middle and old groups when no covariates were included in the analyses. However, the effects were much smaller, and sometimes not significant, when demographic variables were controlled. This pattern of results implies that large proportions of the relations between number of completed occasions and T1 composite scores were general and associated with the controlled demographic variables, and not specific to particular cognitive domains.
Results of regression analyses (β) and analyses of variance (F and eta2) on first-occasion (T1) composite z-scores associated with number of completed occasions
Note: *p < .01. Entries in the columns labeled No Cov. are from analyses with no covariates, and those in the columns labeled Cov. are from analyses with age, sex, self-rated health, years of education, and estimated IQ as covariates.
Mean slopes, based on least squares regression equations relating the composite cognitive score to number of occasions carried out for each individual participant, are summarized in Table 3. The slopes are based on the T1 and T2 scores in participants completing only two occasions, on the T1, T2, and T3 scores in participants completing only three occasions, so on. It can be seen that nearly all of the slopes were positive in the young adult group, negative in the old group, and close to zero in the middle group.
Slopes of change in composite z-score units per occasion in participants in three age groups with different numbers of completed occasions and results of regression analyses and analyses of variance
Note: *p < .01. Covariates in the regression analyses and in the analyses of variance were the relevant composite score on the first occasion, age, sex, self-rated health, years of education, and estimated IQ.
Because the slope results were similar with and without covariates, only the regression and analysis of variance results with covariates, including T1 score to control for initial level of performance, less negative slopes with more occasions for vocabulary in the old group. In each of these cases, the effects were most pronounced in participants completing only two occasions, and the standardized regression coefficients were not significantly different from zero in participants completing three or more occasions. In addition, the slope was small for memory in older participants completing five occasions, but as just noted, the regression analysis comparing slopes in participants completing three, four, or five occasions was not significant.
DISCUSSION
A finding apparent in both figures and in each table was that the patterns of attrition were different at different ages. That is, relatively little selective attrition was evident in adults under 45 years of age, and when it occurred it was in the direction of lower, rather than higher, levels of functioning for participants who returned for subsequent occasions. However, there was clear selective attrition in adults over 65, with higher levels of performance in participants who returned for more subsequent occasions. Selectivity of attrition was intermediate between the young and old groups for adults between 45 and 64 years of age. Smaller selectivity of attrition at younger ages has also been reported by Riegel et al. (Reference Riegel, Riegel and Meyer1968) and Schaie et al. (Reference Schaie, Labouvie and Barrett1973), and in a subset of the current participants by Salthouse (Reference Salthouse2014a).
Although selectivity of attrition varied with age, within the older adults it was evident in all cognitive domains in Table 2, and with the estimated IQ and MMSE measures in Table 1. Findings from earlier studies suggested that attrition varied across cognitive abilities (e.g., Cooney et al., Reference Cooney, Schaie and Willis1988; Riegel et al., Reference Riegel, Riegel and Meyer1968; Siegler & Botwinick, Reference Siegler and Botwinick1979), but there was no evidence in the current study that selective attrition was restricted to a particular type of cognitive functioning.
As expected on the basis of prior research, the results in Figures 1 and 2 and in Table 2 indicate that performance on the initial occasion was higher for middle-aged and older participants who returned for subsequent occasions than for participants who did not return. Furthermore, there were nearly monotonic relations between performance on the initial occasion and the number of subsequent occasions completed. This finding, which has also been reported by Siegler and Botwinick (Reference Siegler and Botwinick1979) and Siegler et al. (Reference Siegler, McCarty and Logue1982), indicates that one’s initial level of cognitive ability is an important predictor of how long an individual will remain in a longitudinal study.
A key question in the current study was whether individuals completing different numbers of occasions had different rates of cognitive change. The approximately parallel functions in Figures 1 and 2, and the small effect sizes in both the regression analyses and the analyses of variance in Table 3, after controlling for first occasion performance, indicate that the patterns of cognitive change were fairly similar in participants who completed between two and five longitudinal occasions. Although the slopes in memory and vocabulary were slightly more negative for older adults who dropped out after two occasions than for older adults who completed three or more occasions and were smaller in memory for older adults who completed five occasions, there were no significant slope differences among participants completing between three and five occasions. Most of the attrition effects can therefore be inferred to be on the level of functioning and not on rate of change.
The current results focusing on change across two or more occasions extend those of an earlier study (Salthouse, Reference Salthouse2014a) comparing performance of participants who completed either one or two longitudinal occasions. Longitudinal changes in the prior study were examined in returning participants who had the same initial levels of performance as the nonreturning participants and between returning participants and nonreturning participants whose second occasion scores were imputed based on their first occasion scores, and the first and second occasion scores of the returning participants. The major findings were that similar magnitudes of cognitive change were evident with both procedures. These results were interpreted as suggesting that selective attrition from the first to the second occasion is primarily related to the level, rather than the change, in cognitive functioning, which is consistent with the interpretation of the current results.
Several limitations of the study should be acknowledged. First, most of the participants reported themselves to be in very good or excellent health and were high functioning in terms of estimated IQ and MMSE, which could restrict generalizability of the results. Second, the analyses of rates of change were relatively crude because they were based on comparisons of different people with each number of occasions. Third, in addition to level of performance, the participants completing different numbers of occasions may have differed in other respects that could have influenced the results. And fourth, the attrition effects in this study may have been underestimated if some participants might have subsequently returned for additional occasions. However, it is important to note that the pattern of results was very similar in two additional analyses. One involved participants initially tested no later than 2008, and who thus had at least 9 years to have participated in additional assessments. The second supplementary analysis involved participants whose last occasion was 2014, and who therefore had 3 or more years to have participated again. The standardized regression coefficients in these analyses closely resembled those in Table 3 based on the complete sample of participants, which suggests that the results were not distorted by limited opportunities to have returned for additional assessments. Despite the limitations, the study also has important strengths, such as moderately large samples of adults across a wide age range who completed between one and five longitudinal occasions spanning an interval averaging over 10 years, multiple cognitive tests representing five different cognitive domains, and a consistent pattern of results across two different analytical methods.
To summarize, the results of this study confirm earlier findings that at least among adults over about 45 years of age, individuals who drop out of a longitudinal study tend to have lower levels of performance in several different types of cognitive functioning. Moreover, the differences on the initial occasion were monotonically related to the number of longitudinal occasions subsequently completed, which suggests that continued participation in a longitudinal study is partially influenced by one’s initial level of cognitive functioning. However, with the exception of more negative change in memory and vocabulary from the first to the second occasion among older participants completing only two occasions, the rates of longitudinal change were not significantly different across participants completing different numbers of longitudinal occasions. Selective attrition therefore appears to threaten the generalizability of longitudinal comparisons more than the validity of the age–cognition relations derived from those comparisons.
ACKNOWLEDGMENTS
This research was supported by National Institute on Aging grant R01AG024270. The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health. There are no conflicts of interest.
