Introduction
Cognitive declines in multiple domains occur with normal aging (Christensen, Reference Christensen2001). It is not surprising, therefore, that many interventions have been developed to delay or remediate age-related cognitive declines (Buitenweg et al., Reference Buitenweg, Murre and Ridderinkhof2012). Lovden et al.'s theoretical framework for cognitive plasticity (Lovden et al., 2010) provides an excellent context for how such interventions might achieve success. According to Lovden et al. (2010, p. 661), “. . .plastic reactions . . . [reflect] altered [in the case of successful interventions, improved] attainable processing efficiency.” Such improvements in processing efficiency are achieved based on the “mismatch model” which asserts that adaptive multitasking training with progressively increasing challenge levels for all of the tasks being trained (i.e. demand) allows “for some form of rewiring” resulting in improved functional resources and abilities (i.e. supply). Furthermore, Lovden et al. noted that this improved processing efficiency was distinct from the acquisition of new knowledge, which does not result in “rewiring” or improved processing efficiency.
One of the most-studied and successful interventions to delay or remediate age-related cognitive declines is the cognitive SOPT program developed by Ball and Roenker (Ball et al., Reference Ball, Beard, Roenker and Miller1988; 1993; Roenker et al., Reference Roenker, Cissell, Ball, Wadley and Edwards2003; Wadley et al., Reference Wadley, Benz, Ball, Roenker, Edwards and Vance2006). Their SOPT program is designed to improve the efficiency and accuracy of visual information processing and the ability to perform complex visual attention tasks. This is done using a computerized adaptive multitasking platform that uses a divided attention format to focus on the speed and accuracy of identifying and locating visual information. A baseline assessment first tailors the difficulty and complexity of each task to the individual participant, and then increases the degree of difficulty and complexity on those tasks as that participant reaches prescribed success thresholds. Difficulty and complexity increments are brought about by decreased stimuli duration, added visual or auditory distractors, increased similarity between target and distractor stimuli, and an expanded visual field (Wadley et al., Reference Wadley, Benz, Ball, Roenker, Edwards and Vance2006).
Results from the multisite NIH-funded ACTIVE (Advanced Cognitive Training for Independent and Vital Elderly), randomized controlled trial (RCT) revealed substantial beneficial effects of this SOPT intervention on trained cognitive abilities (UFOV) over one to ten years of follow-up compared to no-contact controls (Ball et al., Reference Ball2002; Willis et al., Reference Willis2006; Rebok et al., Reference Rebok2014). ACTIVE, however, provided little evidence of transfer to non-trained cognitive outcomes. Using a second-generation version of Ball and Roenker's SOPT platform, we conducted an NIH-funded RCT known as the Iowa Healthy and Active Minds Study (IHAMS) (Wolinsky et al., Reference Wolinsky2011; Reference Wolinsky, Vander Weg, Howren, Jones and Dotson2013). IHAMS was designed to address several of ACTIVE's shortcomings, most notably by using an attention-control group (computerized crossword puzzles), and including a broader array of cognitive outcomes to test for transfer to non-trained cognitive outcomes. Statistically significant small- to medium-sized immediate post-training effects on the UFOV (Edwards et al., Reference Edwards2005) were initially reported (Wolinsky et al., Reference Wolinsky2011), and these were followed by a report (Wolinsky et al., Reference Wolinsky, Vander Weg, Howren, Jones and Dotson2013) of statistically significant small- to medium-sized effects at one year on the UFOV, along with marginally statistically significant small-sized effects on Trail-making A and B (Reitan and Wolfson, Reference Reitan and Wolfson1993), Symbol Digit Modalities (Smith, Reference Smith1982), and Stroop Word (Golden, Reference Golden1978) tests.
In this paper we tested the hypothesis that SOPT would lead to statistically significant and clinically meaningful improvements on a composite neuropsychological outcome at one year. That composite consisted of six equally weighted tests – UFOV (Edwards et al., Reference Edwards2005), Trail-making (Reitan and Wolfson, Reference Reitan and Wolfson1993), Symbol Digit Modalities (Smith, Reference Smith1982), Stroop Color and Word (Golden, Reference Golden1978), Controlled Oral Word Association (Benton et al., Reference Benton, Hamsher and Silvan1994), and Digit Vigilance (Lewis and Rennick, Reference Lewis and Rennick1979). Compared to our previous report on those individual tests (Wolinsky et al., Reference Wolinsky, Vander Weg, Howren, Jones and Dotson2013), reporting SOPT effects at one year on this equally weighted neuropsychological composite increases statistical power and accuracy, addresses the multiplicity problem that occurs when several appropriate outcome measures are available (without requiring adjustment to the type 1 error rate), and defines a summary outcome that can be used for determining efficacy and establishing more generalizable effect sizes (Montori et al., Reference Montori2005; Goldberg et al., Reference Goldberg, Gore, Barton and Gurwitz2014).
Methods
Ethics and trial registration
IHAMS was reviewed and approved by the University of Iowa Institutional Review Board (IRB Protocol 200908789) and was supported by an NIH Challenge Grant (RC1 AG035546). We obtained written informed consent from all participants at baseline, and verbal consent at all follow-up interviews. The clinical trial registration process was initiated on February 11, 2010 but due to miscommunications with the Protocol Registration System Account (PRSA) official, the University of Iowa did not finalize (“release”) the registration process until July 13, 2010 (NCT-01165463 (http://clinicaltrials.gov/). As a result, the first 241 participants were enrolled prior to finalization of the trial registration process. This delay was an innocent but clearly regrettable error with no intent to circumvent the ICMJE (International Committee of Medical Journal Editors) requirements adhered to by this journal. After this error, all principal investigators at the University of Iowa have been authorized to finalize (“release”) the registration of their own clinical trials. The authors confirm that all of their ongoing and related trials are properly registered.
Protocol adjustments
We made four protocol adjustments before enrollment began. First, we randomized directly to four study arms within two age-bands rather than to three with second-level randomization to boosters within the on-site SOPT group. Second, we specified UFOV as the primary outcome because no data existed from ACTIVE for the other neuropsychological outcomes. Third, we trained the on-site participants in two-hour rather than one-hour (as in ACTIVE) sessions because of the logistical and financial constraints of the NIH Challenge Grant funding mechanism. Fourth, we used repeated measures linear mixed models for comparability to prior studies.
Study design
The CONSORT flow chart is shown in Figure 1. We used the electronic health record to identify 5,743 potential participants that were attending either the General Internal or Family Medicine clinics at the University of Iowa Hospital and Clinics and who were 50 years old or older, had at least two clinic visits in 2009, lived within a 30-mile radius, and did not have indications of cognitive impairment or disease. Weekly random samples (without replacement) of 250 potential participants were then sent letters co-signed by the two clinic medical directors and the principal investigator (FDW) between February 2010 and November 2010 inviting them to participate. Interested patients were asked to call the IHAMS project office and take a brief telephone screen to determine further eligibility. The telephone screen involved self-reported access to a personal computer (PC) at home with an internet connection, no self-reported vision problems that could not be corrected by eyeglasses or contact lenses, and making fewer than three errors on a modified version of the Short Portable Mental Status Questionnaire (Pfeiffer, Reference Pfeiffer1975), none of which could be ascertained from the electronic health record. Based on an analysis of the one-year follow-up data from ACTIVE, we determined that enrolling 650 participants with a 15% attrition rate would provide 82% statistical power to detect an effect on UFOV (the primary IHAMS outcome) that was about half that reported in ACTIVE and in a review of other previous SOPT studies (Ball et al., Reference Ball, Edwards and Ross2007). IHAMS was powered to find a smaller effect size because of the use of an attention-control rather than a no-contact control group (as used in ACTIVE).
The CONSORT flow diagram for the Iowa Healthy and Active Minds Study (IHAMS).
Of the 5,743 potential participants, 5,062 were excluded for the following reasons. First, 1,627 could not be reached by mail. Second, 2,079 declined to participate. Third, 390 did not pass the telephone eligibility screen. Finally, 966 letters had not been sent out before the enrollment period closed. As a result, we enrolled 681 participants between March and November 2010, and the last one-year assessment occurred in January 2012. Of these 681 participants, 620 (91.0%) completed their 12-month follow-up interviews, and 587 (86.2%) of these had complete data and were included in the analytic sample.
Randomization
IHAMS was a parallel RCT. Using a computer program with random permuted blocks of sizes 4, 8, and 12 the study statistician (MPJ) randomized 681 patients separately within two age-bands (50–64, N = 455; and ≥ 65, N = 226) to four groups using a 3:3:4:4 allocation ratio. This unequal allocation ratio was used because for the immediate post-training assessments the two on-sight SOPT groups could be combined due to the fact that booster training (the only characteristic on which they would eventually differ) would not occur for another nine months. The four groups were: 10 hours of initial on-site SOPT during the first 5–6 weeks; 10 hours of initial on-site SOPT during the first 5–6 weeks with 4 hours of boosters at 11 months post-baseline; 10 hours of initial at-home SOPT during the first 5–6 weeks; or, 10 hours of initial on-site attention-control using crossword puzzles during the first 5–6 weeks. All baseline assessments occurred in private offices. Group assignments were sealed in opaque envelopes (double-blinded) until the baseline assessments were completed, but were then un-blinded to participants by the project coordinator (MMD). At the one-year follow-up interviews, which also occurred in private offices, assessor formal un-blinding occurred at the end of the interview in order to assess the assigned intervention's acceptability. About three-fourths of participants had different assessors at baseline than at the one-year follow-up. All participants received the SOPT software after their one-year assessments to use on their home PCs.
Interventions
The interventions, including selected screen shots, have been described elsewhere (Wolinsky et al., Reference Wolinsky2011; Reference Wolinsky, Vander Weg, Howren, Jones and Dotson2013) and trial versions of the rebranded SOPT program (originally named Road Tour and delivered from a CD platform when used in this study, but subsequently named Double Decision and moved to a web-based platform; Posit Science Corporation, San Francisco, CA; http://www.brainhq.com/why-brainhq/about-the-exercises/attention/double-decision) and the crossword puzzles program (Boatload of Puzzles; Boatload Puzzles, LLC, Yorktown Heights, NY; http://www.boatloadpuzzles.com/playcrossword) are available online. Participants were invited back to the IHAMS lab at least one day after completing their baseline assessments. At this visit they first received a scripted, hands-on, 15 minute presentation on how to use their assigned training program. These presentations were given privately to each participant in a room physically separate from the two identically configured training rooms (one for on-site SOPT, and one for on-site attention-control using crossword puzzles) which each had five computer work stations separated by room dividers. Participants assigned to the at-home SOPT group received an additional scripted, hands-on, 5–10 minute presentation on how to load the SOPT program onto a PC, and were then given the program CD with the loading instructions and asked to go home, load the software on their home PCs, and use it for 10 hours over the next 5–6 weeks. In contrast, after their scripted, hands-on, 15 minute presentation on how to use their assigned training program, participants assigned to the three on-site training groups immediately went to their assigned training rooms to begin their first weekly 2-hour session. The training rooms were staffed by personnel other than the assessors. Initial training for the on-site groups was also completed within 5–6 weeks, with the boosters (two additional 2-hour sessions) for the one on-site group completed at 11 months post-baseline. All three SOPT groups used Road Tour on PCs in one project laboratory or on their home PCs, while the attention-control group used Boatload of Crosswords on PCs in another project laboratory.
The neuropsychological composite
As noted above, a good composite outcome can increase statistical power and accuracy, address the multiplicity problem that occurs when several appropriate outcome measures are available, and provide a more generalizable efficacy effect size estimate (Freemantle et al., Reference Freemantle, Calvert, Wood, Eastaugh and Griffin2003; Montori et al., Reference Montori2005; Goldberg et al., Reference Goldberg, Gore, Barton and Gurwitz2014). The composite outcome used here was derived from six neuropsychological tests: UFOV (Edwards et al., Reference Edwards2005), Trail-making A and B (Reitan and Wolfson, Reference Reitan and Wolfson1993), Symbol Digit Modalities (SDMT; Smith, 1082), Stroop Color and Word (Stroop; Golden, Reference Golden1978), Controlled Oral Word Association (COWAT; Benton et al., Reference Benton, Hamsher and Silvan1994), and Digit Vigilance (DVT; Lewis and Rennick, Reference Lewis and Rennick1979), each of which has good validity and reliability, and is known to be sensitive to cognitive changes in older adults. UFOV includes three sub-tests tapping processing speed, selective attention, and divided attention with each scored from 17–500 ms reflecting the shortest exposure time at which the participant could correctly perform each sub-test, with a composite score ranging from 51–1500 ms . Trail-making assesses visual scanning ability, processing speed, and set-shifting/executive functioning, and is coded as s to correctly complete connecting the number (Trail-making A) and number-letter (Trail-making B) sets. SDMT is a substitution task that captures divided attention and processing speed, and is based on how many of 110 possible digit-symbol pairs are correctly scored in 90 s . Stroop assesses the effect of interference with and inhibition on cognitive flexibility and processing speed, and other aspects of executive functioning, and is scored as the correct number of words, colors, and color-words identified in 45 s on each subtest. The COWAT is a phonemic test that assesses verbal fluency based on the number of unique words beginning with the letter C (or F or L) generated by the participant during 60 s , with a composite score of the number of correct words across the three letter trials. DVT is a visual cancellation task that assesses sustained attention and psychomotor speed, and is performed by crossing out randomly placed 6's in 59 rows of numbers, and is scored as the error (commission and omission) and time ( s ) totals.
The composite was created as follows. We first converted each of the six neuropsychological tests into standardized z -scores based on the overall mean and standard deviation ( SD ) at baseline. This was done for the 587 IHAMS participants (86.2%; 138 in the on-site SOPT group; 119 in the on-site SOPT with boosters group; 163 in the at-home SOPT group; and 167 in the attention-control group) with complete data on all components at both the baseline and one-year assessments. For the three tests for which only the summary score is traditionally reported (i.e. UFOV, SDMT, and COWAT), we standardized these summary scores. For the three tests with multiple components that are traditionally reported separately (i.e. Trail-making, Stroop, and DVT), however, we first standardized each component (e.g. Trail-making A), then the standardized component scores (e.g. Trail-making A and B) were summed, and finally we standardized those summed scores. The composite cognitive outcome is the standardized sum of the six standardized neuropsychological test sums, which equally weighted their contributions.
This approach yields overall means and SDs at baseline for the neuropsychological composite of 0 and 1. The treatment group means and SDs at baseline, however, are not so constrained and are free to reflect any baseline differences that remained despite strict adherence to the randomization protocol. This approach has the very beneficial intuitive advantage in that changes in treatment group means from baseline to one-year reflect improvements (or declines) calibrated in baseline SDs , and are thus indicators of relative effect sizes that may be directly compared across groups.
Attrition
Although attrition was modest (91.0% of patients completed their one-year assessments) and 587 patients (86.2%) had complete data at both baseline and one-year, we conducted attrition analyses. Using multiple logistic regression methods, we modeled inclusion in the analytic sample (i.e. the 587 participants with complete data versus the 94 participants without complete information) using three dummy variables reflecting each of the SOPT groups versus the attention-control group (referent), and an indicator for age-band. That model revealed that only participants in the SOPT with boosters group had marginally statistically significantly different (lower) odds of being in the analytic sample ( AOR = 0.54, p = 0.045) compared to the attention-control group. Based on these results, we made no adjustments for attrition or missing data.
Analytic approach
The primary analysis was modified intent-to-treat, with the only modification being that the analysis was restricted to those with complete data. Two of the 587 participants with complete data were assigned to the attention-control group, but were inadvertently trained on SOPT at their first session. These two participants were allowed to continue SOPT, but were analyzed as belonging to the attention-control group. To determine whether the neuropsychological composite outcome met appropriate guidelines for use (Freemantle et al., Reference Freemantle, Calvert, Wood, Eastaugh and Griffin2003; Montori et al., Reference Montori2005; Goldberg et al., Reference Goldberg, Gore, Barton and Gurwitz2014), we first reviewed the literature and determined that the six component tests were of similar importance, and that the underlying etiologic mechanism of SOPT (positive brain plasticity) applied to the cognitive domains that each test measured. To determine whether each of the components mapped to the same latent construct with approximately equal magnitude we conducted exploratory factor analysis to demonstrate a simple structure, and reliability analyses to demonstrate internal consistency. One-way analysis of variance ( ANOVA ) with least significant difference ( LSD ) tests for pairwise comparisons was used to examine baseline treatment group differences for demographic factors, socioeconomic characteristics, health status, training hours, and the ease of use and enjoyability of the training programs. One-way ANOVA with LSD tests was also used to examine treatment group differences on the six neuropsychological tests and the neuropsychological composite at baseline and at one year post-baseline, as well as the change from baseline to one year post-baseline. The primary analyses of changes in the standardized composite neuropsychological outcome used a repeated measures linear mixed model that included between-person main effects for treatment group and age-band, within-person main effects for time, and all of the two- and three-way interaction terms involving time. Dunnett tests were used to compare baseline to one-year changes for each SOPT group relative to the attention-control group with statistical significance set at p < 0.05.
Three secondary analyses were conducted to evaluate the robustness of the findings from the primary analyses. All three used the same repeated measures linear mixed model approach. The first was a treatment-received analysis, because the University of Iowa IRB required that participants be informed that they were allowed to discontinue their training at any point in the study if they chose to do so. Repeated attempts were made, however, to obtain the one-year follow-up assessments for all participants. The treatment-received analysis included only the 496 IHAMS participants (72.8%; 120 in the on-site SOPT group; 113 in the on-site SOPT with boosters group; 111 in the at-home SOPT group; and 152 in the attention-control group) who received at least half of the targeted initial training dose (i.e. five or more hours). The second analysis included all 587 IHAMS participants from the primary analysis, but added sex, education (an indicator for having completed college), self-rated health at baseline, self-rated health at baseline compared to a year ago, and baseline counts of difficulties with activities of daily living (ADLs), difficulties with instrumental ADLs (IADLs), and depressive symptoms as covariates.
The third secondary analysis, which also used the same repeated measures linear mixed model approach, examined the effect of SOPT just on the UFOV composite because it was the primary outcome in the IHAMS RCT. Unlike our original report on the UFOV composite (Wolinsky et al., Reference Wolinsky, Vander Weg, Howren, Jones and Dotson2013) which used Blom rank-transformations of the composite score within age-bands, here we standardized each of the three sub-test scores (processing speed, selective attention, and divided attention) separately before summing them into the composite score and then standardizing that composite. To gauge the SOPT relative effect sizes we used unstandardized ( b ) coefficients obtained from a linear regression of the standardized UFOV composite on a set of three indicators for the active treatment groups (the attention-control group was the reference), along with an indicator for age-band. Finally, to determine which of the three sub-test scores were most affected by SOPT, we also conducted the repeated measures linear mixed model approach separately at the sub-test level.
Results
Descriptive
Table 1 contains means (or percentages within selected categories) for the demographic factors, socioeconomic characteristics, and health status for the 587 IHAMS participants with complete data at the baseline and one-year assessments, by treatment group. The mean age at baseline was 61.9 years old overall ( SD = 8.1, range = 50–86), and was 57.2 years old in the younger age-band and 71.3 years old in the older age-band. About two-thirds of the participants (63.5%) were women, 95.4% were Caucasian, and 71.4% were married. The participants were highly educated, with 71.2% having completed college. About half of the participants (55.0%) were working, but 35.8% had retired. Half of the participants (50.7%) had annual incomes of $75,000 or more. Means for self-rated health (2.17; “very good”), health compared to a year ago (2.75; “about the same”), and the number of seven ADLs (0.28) and eight IADLs (0.32) with reported difficulty reflected a healthy community-dwelling cohort. As shown in Table 1, none of these participant characteristics was statistically significantly different across treatment groups, and there was no pattern of disadvantage for the attention-control group.
Means or percentages for the demographic factors, socioeconomic characteristics, and health status of the 587 IHAMS participants by treatment group
1 P values obtained from one-way ANOVAs .
Descriptive statistics for the number of hours spent on training and its acceptability are shown in Table 2 by treatment group. Statistically significant ( p < 0.001) differences were observed in the number of treatment hours completed in a one-way ANOVA . In addition to the expectedly larger means (and medians) for those in the on-site SOPT with boosters group (who by design were targeted to receive 14 hours of training), those in the on-site SOPT without boosters and the at-home SOPT groups had significantly ( p ≤ 0.018) lower means (and medians) than those in the attention-control group. The at-home SOPT group also had higher (21.5, 31.9, and 42.3) percentages of participants receiving less than 1, 5, and 8 hours of training because (a) 16.7% of them reported being unable to load the software on their home PCs, and (b) the next training session for all on-site group participants was scheduled immediately upon completion of their current session (encouraging adherence to continued training).
Means, medians, and percentages for the number of hours trained and the acceptability of the intervention programs among the 587 IHAMS participants by treatment group
1When the overall p values from these one-way ANOVAs are < 0.10, p values obtained from least significant differences ( LSD ) tests for pairwise comparisons of the first column versus the second, third, and fourth columns are indicated by a, b, and c superscripts, for the second column versus the third and fourth columns by d and e superscripts, and for the third column versus the fourth column by an f superscript.
2For the at-home SOPT group, this percentage was calculated only among those able to load the SOPT program on their home PC.
Statistically significant differences were also observed in the ease of use ( p < 0.001) and enjoyability ( p = 0.043) of the training programs. Ease of use was highest ( p ≤ 0.032) in the at-home SOPT group (among those who were able to load the program onto their PCs) and lowest ( p ≤ 0.008) in the on-site SOPT group without boosters. Enjoyability was also highest in the at-home SOPT group (among those who were able to load the program onto their PCs), and significantly higher in that group than in either of the on-site SOPT groups ( p ≤ 0.032).
Table 3 contains the means by treatment group for the six neuropsychological tests (standardized to the overall baseline means and SDs ) and their composite at baseline and at one-year. At baseline there were statistically significant differences across treatment groups for Trails ( p = 0.006) and the neuropsychological composite ( p = 0.018), and in both cases the attention-controls were disadvantaged relative to all three SOPT groups. At one-year there were statistically significant differences across treatment groups for Trails ( p = 0.013), UFOV ( p < 0.001), and the neuropsychological composite ( p = 0.002), and a marginally statistically significant difference for SDMT ( p = 0.054). In all four cases the attention-controls were disadvantaged relative to all three SOPT groups.
Means for the six standardized cognitive tests and the neuropsychological composite for the 587 IHAMS participants at baseline and at one-year, by treatment group
1When the overall p values from these one-way ANOVAs are < 0.10, p values obtained from least significant differences ( LSD ) tests for pairwise comparisons of the first column versus the second, third, and fourth columns are indicated by a, b, and c superscripts, for the second column versus the third and fourth columns by d and e superscripts, and for the third column versus the fourth column by an f superscript.
Exploratory factor analysis
Results from the exploratory factor analysis at baseline revealed a simple unidimensional structure that explained 53.6% of the variance in the six component tests. The approximate χ2 for the Bartlett’s sphericity test was 1,188 at 15 df ( p < 0.001), and the Kaiser-Meyer-Olkin measure of sampling adequacy was robust at 0.84. In descending order the factor loadings were 0.82 for SDMT, 0.79 for Trail-making, 0.79 for Stroop, 0.76 for DVT, 0.68 for UFOV, and 0.50 for COWAT. Cronbach's α was 0.82. Results at one-year were comparable. Thus, the neuropsychological composite met the empirical criteria for use. Because the explained variance and slope estimates for equal weighting when used to predict regression weighting (based on the factor loadings used to generate factor scores) of the six standardized component scores were near unity ( r2 = 0.995, b = 0.995, p < 0.001), the more straightforward equal weighting was used.
The neuropsychological composite
Figure 2 shows the unadjusted changes from baseline to one-year on the neuropsychological composite by treatment group. Those unadjusted results revealed improvements on the neuropsychological composite for all four treatment groups that ranged from 0.195 baseline SDs for the attention-control group to 0.266 baseline SDs for the on-site SOPT with boosters group. Although the improvements for all three SOPT groups exceeded that for the attention-controls (by 0.038, 0.071, and 0.057 baseline SDs for the on-site SOPT, on-site SOPT with boosters, and at-home SOPT groups, respectively), these differences were not statistically significant overall ( p = 0.401), nor were any of the Dunnett multiple comparison tests statistically significant ( p ≥ 0.285).
Unadjusted mean changes on the neuropsychological composite outcome from baseline to one-year by treatment group.
Primary analyses
Results from the repeated measures (at the baseline and one-year assessments) linear mixed model revealed a large statistically significant time effect ( F = 189.18, df = 1, p < 0.001, ηp 2 = 0.246) indicating improvement in all groups. The time by treatment group ( F = 1.14, df = 3, p = 0.331, ηp 2 = 0.006), time by age-band ( F = 0.54, df = 1, p = 0.463, ηp 2 = 0.001), and time by treatment group by age-band ( F = 0.88, df = 3, p = 0.564, ηp 2 = 0.004) effects, however, were not statistically significant.
Plots of changes in the estimated marginal means by treatment group (not shown) reflected a pattern similar to that for the unadjusted mean differences between baseline and one-year shown in Figure 2. That is, they reveal parallel improvement slopes for the baseline to one-year changes in the neuropsychological composite even though the attention-control group was noticeably disadvantaged relative to the SOPT groups at baseline ( p = 0.018). Plots of changes in the estimated marginal means by age-band (not shown) reflected parallel improvement levels for both age-bands. That is, although participants in the 65 year old or older age-band had lower initial scores than participants in the 50–64 year old age-band on the neuropsychological composite, the magnitude of improvement was equivalent for both age-bands.
Secondary analyses
Results from the treatment-received analysis that was conducted only among the 496 IHAMS participants who received at least half of the targeted initial training dose (i.e. five or more hours) revealed results comparable to those from the primary analysis. That is, a large statistically significant time effect ( F = 183.26, df = 1, p < 0.001, ηp 2 = 0.273) was observed, along with non-significant time by treatment group ( F = 1.23, df = 3, p = 0.290, ηp 2 = 0.008), time by age-band ( F = 0.13, df = 1, p = 0.723, ηp 2 = 0.000), and time by treatment group by age-band ( F = 0.92, df = 3, p = 0.431, ηp 2 = 0.006) effects. Adjustment for the covariates (sex, education, self-rated health, changes in self-rated health, ADLs, IADLs, and depressive symptoms) also revealed comparable effects, with a statistically significant time effect ( F = 6.72, df = 1, p < 0.010, ηp 2 = 0.012; reduced in magnitude due to the inclusion of the covariates), and non-significant time by treatment group ( F = 1.06, df = 3, p = 0.367, ηp 2 = 0.006), time by age-band ( F = 0.80, df = 1, p = 0.367, ηp 2 = 0.001), and time by treatment group by age-band ( F = 0.66, df = 3, p = 0.575, ηp 2 = 0.004) effects.
Finally, the results from the analysis of the standardized UFOV composite, the primary outcome in the IHAMS RCT, revealed statistically significant treatment effects for all factors. That is, there was a large significant time effect ( F = 297.66, df = 1, p < 0.001, ηp 2 = 0.340), a significant time by treatment group effect ( F = 5.94, df = 3, p = 0.001, ηp 2 = 0.030), a significant time by age-band effect ( F = 15.48, df = 1, p < 0.001, ηp 2 = 0.026), and a significant time by treatment group by age-band effect ( F = 4.40, df = 3, p = 0.04, ηp 2 = 0.022). The unstandardized coefficients ( b ) obtained from the linear regression analysis suggested relative effect sizes (vs. the attention-control group) of SOPT on the standardized UFOV composite that were 0.216 ( p < 0.001) for on-site SOPT without boosters, 0.441 ( p < 0.001) for on-site SOPT with boosters, and 0.247 ( p < 0.001) for at-home SOPT. Analysis of the standardized UFOV sub-tests revealed that these treatment group differences accrued from significant improvements on the processing speed ( F = 4.15, df = 1, p = 0.006, ηp 2 = 0.021) and selective attention ( F = 9.84, df = 1, p < 0.001, ηp 2 = 0.049) sub-tests, but not from the divided attention sub-test ( F = 297.66, df = 1, p = 0.192, ηp 2 = 0.008).
Discussion
In this paper we tested the hypothesis that SOPT would lead to statistically significant and clinically meaningful improvements between baseline and the one-year follow-up on a neuropsychological composite outcome. The neuropsychological composite consisted of six equally weighted tests – UFOV (Edwards et al., Reference Edwards2005), Trail-making (Reitan and Wolfson, Reference Reitan and Wolfson1993), Symbol Digit Modalities (Smith, Reference Smith1982), Stroop Color and Word (Golden, Reference Golden1978), Controlled Oral Word Association (Benton et al., Reference Benton, Hamsher and Silvan1994), and Digit Vigilance (Lewis and Rennick, Reference Lewis and Rennick1979). IHAMS included three SOPT (Ball et al., Reference Ball, Beard, Roenker and Miller1988; 1993; Roenker et al., Reference Roenker, Cissell, Ball, Wadley and Edwards2003; Wadley et al., Reference Wadley, Benz, Ball, Roenker, Edwards and Vance2006) groups – a targeted ten-hour dose delivered on-site, a targeted ten-hour dose delivered at-home, and a targeted ten-hour dose delivered on-site with four hours of targeted booster training delivered on-site at 11 months – compared to an on-site attention-control that used computerized crossword puzzles on-site for a targeted ten-hour dose. Complete data were available and analyzed for 587 (86.2%) of the 681 IHAMS participants.
Results from the repeated measures linear mixed model revealed a large statistically significant time effect ( F = 189.18, df = 1, p < 0.001, ηp 2 = 0.246) indicating improvements in all groups, but non-significant time by treatment group, time by age-band, and time by treatment group by age-band effects. Thus, no evidence of transfer to cognitive outcomes measured more broadly by the neuropsychological composite was demonstrated, and therefore the alternative hypothesis was rejected. Moreover, this null finding was robust when the analyses were restricted to those who had received at least half of the initially-targeted training dose (five hours or more), and when covariate adjustments were made for sex, education, self-rated health, changes in self-rated health, ADLs, IADLs, and depressive symptoms. There was, however, a significant treatment group effect on the standardized UFOV composite by itself, with this effect accruing from improvements in the processing speed and selective attention sub-tests.
These results are consistent with those recently reported from the Mental Activity and Exercise (MAX) RCT (Barnes et al., Reference Barnes2013). MAX used a two-by-two factorial design of intensive mental (visual and auditory processing training vs. educational DVD viewing for the mental attention-controls) and exercise (dance-based aerobics vs. stretching and toning for the exercise attention-controls) activities with targeted training doses of 36 hours in all groups. Like IHAMS, MAX used a composite neuropsychological outcome that equally weighted six measures, including the same SDMT (Smith, Reference Smith1982), Trail-making (Reitan and Wolfson, Reference Reitan and Wolfson1993), and UFOV (Edwards et al., Reference Edwards2005) tests used in IHAMS. Thus, the comparability of the MAX and IHAMS composites is good but not complete, and the comparison of the MAX intensive mental activity (vs. mental attention-controls) effects with the IHAMS SOPT (vs. crossword attention-controls) effects is reasonable although the former were based on a targeted dose nearly three times as large. Results from the MAX RCT revealed a significant time effect indicating improvements for all groups (the overall mean improvement was 0.16 baseline SDs ), but no significant treatment effects were observed even though the intense mental activity group improved the most with an improvement nearly two-thirds again the size of that detected for the mental attention-controls ( p = 0.17; see their Figure 2; Barnes et al., Reference Barnes2013).
What is not clear from IHAMS (or from MAX), however, is what accounts for the improvements observed for the attention-control groups. It would seem most plausible that either training or practice effects, or both were present. Although the IHAMS design does not permit parsing out the nature of these effects, recent results from a small RCT examining the effect of daily puzzle use by older adults support a crosswords training effect in that statistically significant improvements on the COWAT (one of the six tests used in the IHAMS neuropsychological composite, with a comparable verbal fluency test used in the MAX neuropsychological composite) were observed (Murphy et al., Reference Murphy, O’Sullivan and Kelieher2014). According to Lovden et al.'s (2010) cognitive plasticity framework, potential crossword puzzle training effects should most likely reflect plasticity that alters representations like knowledge, rather than plasticity that alters processes and their efficiency. The reason for this is that the crossword puzzle attention-control task used in IHAMS did not involve adaptive multitasking training that progressively increased challenge levels for any tasks. Thus, the null effects reported here may partially reflect the inappropriate use of crossword puzzles as the attention-control task.
In terms of practice effects, the literature clearly suggests that these (a) occur for most neuropsychological tests, (b) may be as large as a quarter of a SD, but that these effects (c) generally decline with increased age, (d) the length of the interval between test administrations, and (e) with fewer repeated administrations (Calamia et al., Reference Calamia, Markon and Tranel2012). Data from the MAX RCT (Barnes et al., Reference Barnes2013) provides an initial approximation of the potential magnitude of practice effects on a neurological composite outcome like that used in IHAMS. In MAX, the magnitude of practice effects was approximated by recruiting an additional no-contact control group that followed the same testing protocol used in the main study. That no-contact control group improved by 0.08 baseline SDs on the neuropsychological composite, which was half the size of the improvement observed across the four main MAX study groups (0.16 baseline SDs ). Those results are consistent with the above interpretation that the null effects reported here for IHAMS may partially reflect the inappropriate use of crossword puzzles as the attention-control task. Further research is needed, however, to determine whether our use of crossword puzzles was an appropriate attention-control mechanism, or whether it was a less-effective alternative training intervention. Such studies might consider using either a double-baseline approach or both no-contact and attention-control groups to resolve these issues.
It is also not clear why the larger improvements observed in IHAMS for the SOPT groups were not statistically significantly different from those for the attention-controls. Four possible explanations warrant brief mention. The first and most straightforward is that the SOPT intervention simply did not have the anticipated transfer effect on the broader neuropsychological composite measure. That is, acceptance of the null hypothesis is an accurate reflection of empirical reality. The second possibility is that the targeted dose, even in the SOPT with boosters group (14 hours), was inadequate. Some support for this may be found in the fact that in three recent brain plasticity intervention trials that have reported evidence supporting some degree of transfer effects, the targeted doses were substantially larger (40 hours or more) than those used in IHAMS (Mahncke, et al., Reference Mahncke2006; Smith et al., Reference Smith2009). Similarly, the improvements shown here for the SOPT groups were substantially smaller than those previously reported for UFOV from either ACTIVE (Ball et al., Reference Ball2002; Willis et al., Reference Willis2006; Rebok et al., Reference Rebok2014) or IHAMS (Wolinsky et al., Reference Wolinsky2011; Reference Wolinsky, Vander Weg, Howren, Jones and Dotson2013), as well as for five other SOPT studies that included several of the same neuropsychological assessments reported here (Ball et al., Reference Ball, Edwards and Ross2007). Those studies also generally involved larger treatment doses.
The third possibility involves the fact that despite strict adherence to the randomization protocol, the attention-control group was disadvantaged (i.e. it had a significantly lower mean and a larger SD ; see Table 3) relative to the SOPT groups on the neuropsychological composite at baseline, potentially biasing the results to the null. The fourth possibility involves the potential that a ceiling effect may have imposed bias to the null arising from the high level of cognitive functioning among the IHAMS participants at baseline. As in ACTIVE (Ball et al., Reference Ball2002; Willis et al., Reference Willis2006; Rebok et al., Reference Rebok2014), IHAMS targeted well-functioning older adults residing in the community, and therefore longer follow-up periods may be required for the benefits of SOPT to achieve statistical significance.
Despite its strengths, IHAMS is not without limitations, and four warrant mention here. First, the proportions of minority, less educated, and socioeconomically disadvantaged participants were small (see Table 1). Thus, further investigation of the effectiveness of SOPT in those disadvantaged groups is needed. Second, although the crossword puzzle game used by the attention-controls was computerized, it was not an adaptive task like SOPT. That is, it did not progressively increase the challenge level to the participant. Therefore, it is possible that an adaptive version of the computerized crossword puzzle game may have resulted in an even larger improvement in the neuropsychological composite for the attention-controls, which might have further minimized the advantage observed for participants randomized to any of the three SOPT groups. Third, while the breadth of the neuropsychological tests used in the composite outcome was greater than that reported for the ACTIVE RCT (Ball et al., Reference Ball2002; Willis et al., Reference Willis2006; Rebok et al., Reference Rebok2014), it was nonetheless limited and somewhat over-represented by measures tapping processing speed. For that reason, future studies should consider greater breadth and depth in the neuropsychological outcomes that they assess. Finally, because it was conducted by telephone, we relied on self-reported vision problems (rather than a vision performance test) during the eligibility screening process as an exclusion criterion. During the baseline assessments, however, we did ask participants if they had vision issues that were uncorrectable even when using their corrective lenses, and only 2.7% gave a positive response. Only one participant was dropped from the study after randomization, which was when she first reported that her vision problem (macular degeneration) would not let her continue.
In conclusion, our analysis revealed significant one-year improvements ranging from 0.195 baseline SDs among the attention-control group to 0.266 baseline SDs among the SOPT with boosters group on a neuropsychological composite outcome. Although each of the SOPT groups improved more than the attention-control group, those modest improvements (0.038 baseline SDs for the on-site, 0.071 baseline SDs for the on-site with boosters, and 0.057 baseline SDs for the at-home training groups) were not statistically significant. Thus, IHAMS does not provide evidence of transfer to a broader set of non-trained cognitive outcomes. This suggests that future plasticity-based cognitive training interventions might do well to consider broadening their focus from single cognitive domains like SOPT, to training that targets multiple cognitive domains including processing speed, attention, language, working memory, episodic memory, and executive function.
Finally, at least three crucial questions remain unanswered in the broader context of age-related cognitive declines (Howren et al., Reference Howren, Vander Weg and Wolinsky2014; National Institutes of Health, 2014). First, while numerous types of training programs (including cognitive, moderate to vigorous exercise, mindfulness-based stress reduction, movement therapies, and manipulation therapies) have been developed to delay, avoid, or remediate age-related cognitive decline, there have been no head-to-head comparative effectiveness trials of these different types of interventions. Similarly, there have been no head-to-head comparisons between different interventions of the same type, and none of these interventions have shown meaningful transfers across non-trained cognitive domains or positive impacts on everyday functioning in large RCTs. Second, the etiologic mechanisms by which these various types of training programs operate has not been established, nor have neuroplasticity changes been documented using appropriate biomarker and/or imaging assessment methods. Third, heterogeneity of treatment effects – different subgroups of participants responding differently to these various interventions – have yet to be effectively examined and identified. Thus, much work remains to be done to reduce the burden of age-related cognitive decline.
Conflict of interest
None
Description of authors’ roles
Fredric. D. Wolinsky conceived and designed the study, performed the analyses, and drafted and revised the manuscript. Mark W. Vander Weg, M. Bryant Howren, and Michael P. Jones contributed to the design of the study, and participated in manuscript review and revision. Megan M. Dotson directed study operations, protocol piloting, quality control, and participated in manuscript review and revision.
Acknowledgments
This work was supported by the National Institute on Aging (grant number RC1-AG-010436 to Fredric D. Wolinsky). The IHAMS was reviewed and approved by the University of Iowa (IRB Protocol 200908789), and registered with the NIH Clinical Trials registry (NCT-01165463; http://clinicaltrials.gov). Address correspondence to Fredric D. Wolinsky (fredric-wolinsky@uiowa.edu), Department of Health Management and Policy, College of Public Health, the University of Iowa, 145 North Riverside Drive, 100 College of Public Health Building, Room N-211, Iowa City, Iowa 52242-2007.
Trial Registration: The Iowa Healthy and Active Minds Study (IHAMS) was registered with the NIH Clinical Trials registry (NCT-01165463; http://clinicaltrials.gov).
