INTRODUCTION
Examining the effect of dietary changes on cognitive health has become an important target in the pursuit of brain maintenance strategies. As such, the literature on dietary interventions has grown over the last several years, providing evidence that dietary changes hold promise in improving cognitive function and reducing the risk of cognitive decline. Despite this progress, our ability to draw conclusions may be limited because of the great variability in how the outcome of cognitive function is measured across studies.
A recent systematic review included 56 studies examining the association between diet and cognition (van den Brink et al., Reference van den Brink, Brouwer-Brolsma, Berendsen and van de Rest2019) and revealed substantial variability in the measurement of cognitive outcomes across studies. Fewer than half of the observational studies used a cognitive battery with multiple tests (e.g., ≥ 4 tests). Of the remaining studies, outcome measures included cognitive status only (e.g., neurocognitive diagnosis), a single global screener [e.g., Modified Mini-Mental State Examination (3MS): Teng & Chui, Reference Teng and Chui1987], subjective reports of cognitive function, or mortality. Six randomized control studies were described (Knight et al., Reference Knight, Bryan, Wilson, Hodgson, Davis and Murphy2016; Maratinez-Lapiscina, Clavero, Toledo, Estruch, et al., Reference Martinez-Lapiscina, Clavero, Toledo, Estruch, Salas-Salvado, San Julian, Sanchez- Tainta and Martinez-Gonzalez2013a; Martinez-Lapiscina, Clavero, Toledo, San Julian, et al., Reference Martinez-Lapiscina, Clavero, Toledo, San Julian, Sanchez-Tainta, Corella and Martinez-Gonzalez2013b; Smith et al., Reference Smith, Blumenthal, Babyak, Craighead, Welsh-Bohmer, Browndyke and Sherwood2010; Valls-Pedret et al., Reference Valls-Pedret, Sala-Vila, Serra-Mir, Corella, de la Torre, Martinez-Gonzalez and Ros2015; Wardle et al., Reference Wardle, Rogers, Judd, Taylor, Rapoport, Green and Nicholson Perry2000) and all of them, with one exception (Maratinez-Lapiscina, Clavero, Toledo, Estruch, et al., Reference Martinez-Lapiscina, Clavero, Toledo, Estruch, Salas-Salvado, San Julian, Sanchez- Tainta and Martinez-Gonzalez2013a), included batteries of multiple tests. All the studies with batteries of multiple tests included individual tests with established psychometric properties; however, only one of those studies (Knight et al., Reference Knight, Bryan, Wilson, Hodgson, Davis and Murphy2016) assessed the properties of the entire battery. Specifically, Knight and colleagues (Reference Knight, Bryan, Wilson, Hodgson, Davis and Murphy2016) used a principal components analysis with oblique rotation and found that their battery of 11 tests loaded onto 4 factors. However, they provided limited information regarding the validity of their battery. Given the importance of cognitive function as an outcome variable, increased knowledge of how cognitive batteries perform in different populations is necessary for accurate association with predictors of interest, such as diet.
In the present study, we evaluated the construct validity of a cognitive function battery that was used in the MIND Diet Intervention. The MIND Diet Intervention is a randomized control trial to determine the relative effectiveness of the MIND diet in reducing markers of underlying biological mechanisms for AD and vascular dementia. The MIND diet, Mediterranean-DASH Intervention for Neurodegenerative Delay, where DASH stands for Dietary Approaches to Stop Hypertension, can best be described through its component parts of 10 brain healthy food groups (green leafy vegetables, other vegetables, nuts, berries, beans, whole grains, seafood, poultry, olive oil, and wine) and 5 unhealthy food groups (red meats, butter and stick margarine, cheese, pastries and sweets, and fried/fast foods) (Morris, et al., Reference Morris, Tangney, Wang, Sacks, Barnes, Bennett and Aggarwal2015). A MIND diet score, defined previously (Morris, et al., Reference Morris, Tangney, Wang, Sacks, Barnes, Bennett and Aggarwal2015), was developed to facilitate the understanding of these nutrients on health outcomes, including cognition. The MIND diet has been associated with less cognitive decline (Morris, et al., Reference Morris, Tangney, Wang, Sacks, Barnes, Bennett and Aggarwal2015) and lower risk of AD in several studies (van den Brink et al., Reference van den Brink, Brouwer-Brolsma, Berendsen and van de Rest2019). Additionally, there is evidence that the MIND diet may reduce the risk of cognitive impairment and dementia in populations outside the USA that are younger and are culturally distinct from the US older adult population (Hosking, et al., Reference Hosking, Eramudugolla, Cherbuin and Anstey2019).
The cognitive function battery was chosen to include four relevant domains for measuring the change in aging: executive function, perceptual speed, episodic memory, and semantic memory. We hypothesized that the 12 individual tests of the battery would group into 4 theoretical cognitive domains and would have expected associations with age, education, and late-life cognitive activity.
METHODS
Participants
Participants were 604 community volunteers, who enrolled in the MIND Diet Intervention to Prevent Alzheimer’s Disease. The majority of participants were White (89%) and female (65%), and had a mean age of 70 (SD = 4.2). The mean years of education, ascertained with the question “What is the highest grade or year of school you completed”, was 17 (SD = 2.6), as described in Table 2. One-half of the participants (N = 302) were randomized to the control arm and the other half to the intervention arm. The inclusion criteria were 65–84 years old, body mass index > 25 k/m2, family history of dementia but without personal cognitive impairment as measured by the Montreal Cognitive Assessment (MoCA: Nasreddine et al., Reference Nasreddine, Phillips, Bedirian, Charbonneua, Whitehead, Collin and Chertkow2005) ≥ 22, suboptimal diet as measured by a MIND score (Morris et al. Reference Morris, Tangney, Wang, Sacks, Barnes, Bennett and Aggarwal2015) ≤ 8 out of 14, agree to not take any non-prescribed vitamins or supplements, and live in a household in which they are the only MIND participant.
Data Collection and Randomization
Participants completed a battery of tests at baseline before being randomized into one of the two dietary intervention counseling sessions: the MIND diet plus weight loss group or the standard diet plus weight loss group. The two groups utilized the same multipronged strategy for diet compliance, which included: (1) Personalized diet plans and strategies, (2) Multiple compliance aids (e.g., refrigerator charts), (3) Group motivation strategies (e.g., group cooking classes), and (4) Frequent weight monitoring and check-ins. Dieticians lead a similarly structured program for both groups in order to promote equivalent adherence. Trained research assistants administered the battery of cognitive function tests to participants in one of two site locations (Boston or Chicago); all batteries were administered in person and in a quiet setting, free from major distractions. The study received approval from the Institutional Review Boards of Rush University Medical Center and Harvard School of Public Health.
Designing the Battery
We designed a cognitive battery that would fit several criteria for this randomized control trial in a community-based sample. First, we selected tests that have been shown in previous research to detect changes in cognition of older persons over time. Second, the entire battery had to be administered in 45 min or less, to not place an undue burden on participants. Third, we wanted tests to measure a wide range of abilities across different domains of function, in order to minimize possible floor and ceiling effects. Finally, we wanted tests that could be reliably administered by trained research assistants in a standardized manner across the two study sites.
The battery included 12 public domain tests that were categorized into 4 theoretical domains, as Table 1 demonstrates: executive function, perceptual speed, episodic memory, and semantic memory. Two tests in the battery were chosen to measure executive function: Trail Making Test B (Reitan & Wolfson, Reference Reitan and Wolfson1985) and the Flanker Inhibitory Control test from the NIH Toolbox (Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer and Gershon2013). Three tests were chosen to assess perceptual speed: the Oral Symbol Digit Modality Test (Smith, Reference Smith1982), the Pattern Comparison test (Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer and Gershon2013), and Trail Making Test A (Reitan & Wolfson, Reference Reitan and Wolfson1985). Five tests were administered to assess episodic memory: Word List Memory, Word List Recall, Word List Recognition (Morris et al., Reference Morris, Heyman, Mohs, Hughes, van Belle, Fillenbaum and Clark1989), East Boston Story Immediate Recall, and East Boston Story Delayed Recall (Albert et al., Reference Albert, Smith, Scherr, Taylor, Evans and Funkenstein1991). Two tests measured semantic memory: Category Fluency (Animals/Fruits and Vegetables) (Morris et al., Reference Morris, Heyman, Mohs, Hughes, van Belle, Fillenbaum and Clark1989) and the Multilingual Naming Test (Ivanova et al., Reference Ivanova, Salmon and Gollan2013). For each of the tests, we calculated Z scores, using the mean and standard deviation of all participants. We then averaged the Z scores of the component tests for each domain. Table 2 lists the mean, standard deviation, and range for each of the tests according to their domains.
The MIND cognitive battery
Characteristics of the MIND study population
Late-Life Cognitive Activity Assessment
During the baseline interview, participants reported on the frequency of engagement in late-life cognitive activities. They were asked to rate: 1. How much time they spent reading each day? 2. How often they visited a library in the last year? 3. How often they read the newspaper in the past year? 4. How often they read magazines in the past year? 5. How often they read books in the past year? 6. How often they wrote letters in the past year? 7. How often they played games such as checkers, or other board games, cards, puzzles, etc. Participants rated each item on a 5-point scale with higher scores indicating more frequent cognitive activity. We then created a composite cognitive activity score by averaging the seven items as previously described (Wilson, et al., Reference Wilson, Segawa, Boyle and Bennett2012).
Data Analysis
We specified a conceptually based grouping of the tests into functional domains: executive function, perceptual speed, episodic memory, and semantic memory. Next, we conducted a principal components factor analysis with varimax rotation on the 12 cognitive tests to evaluate whether factor loadings – or empirically based groupings – were consistent with the conceptually based groupings. Tests with loadings of 0.5 or greater were grouped on a common factor, and Rand’s statistic (Rand, Reference Rand1971) was used to assess the fit between the theoretical and empirically based groupings. Figure 1 shows the steps in the theoretical and statistical processes.
Validation process of the MIND cognitive battery.
We ran multiple linear regression models in two steps in order to examine the associations of the composite measures of each cognitive domain with late-life cognitive activity. We first regressed each domain against the demographic variables of age, sex, race, and education. In the second step, we included cognitive activity in the same model. Statistical analyses were conducted using R programming version 3.6.
RESULTS
Psychometric Properties of the Cognitive Battery
The means and standard deviations of each of the 12 cognitive function tests used in the analyses are shown in Table 3. We included all 12 tests in a principal component factor analysis with a varimax rotation to examine the empirically based groupings of these tests and compared them with the theoretical groupings referred to in Table 1: executive function, perceptual speed, episodic memory, and semantic memory. Rand’s statistic was 0.82 (p < 0.001), indicating a good fit between the factor analytic results and the hypothesized grouping.
Psychometric information on 12 cognitive tests
The empirically based groupings revealed four factors, as demonstrated in Table 3. Four factors emerged that were generally consistent with the theoretical groupings but were different in minor ways. The first factor that emerged included all tests from the theoretical domains of executive function (Trail Making Test B, Flanker Inhibitory Control) and perceptual speed (Oral Digit Symbol Modality Test, Pattern Comparison, Trail Making Test A). The second factor emerged that included the word list tests (Word List Memory, Word List Delayed Recall, Word List Recognition) from the episodic memory domain. The third factor contained the two tests from the episodic memory domain (East Boston Immediate Recall, East Boston Delayed Recall), and the fourth factor contained the two tests in the semantic memory domain, consistent with the theoretical conceptualization.
Association of the Cognitive Battery with Other Variables
Table 4 shows the associations of demographic variables and frequency of cognitive activity with the theoretical cognitive domains. In the multivariable-adjusted model, age was associated with lower cognitive scores in all domains. Compared to women, men had a significantly lower cognitive score in episodic memory β = −0.269, SE = 0.056, p < 0.001) and perceptual speed (β =−0.182, SE = 0.060, p = 0.003). No significant sex differences were noted in executive function (p = 0.076) and semantic memory (p = 0.663). Education was associated with higher scores for episodic memory (β = 0.024, SE = 0.011, p = 0.028) and semantic memory (β = 0.029, SE = 0.053, p < 0.001), whereas the frequency of cognitive activity was associated with executive function ( β = 0.144, SE = 0.055, p = 0.009), perceptual speed (β = 0.208, SE = 0.051, p < 0.001), and semantic memory (β = 0.196, SE = 0.053, p < 0.001).
Association of theoretical cognitive domain with demographic variables
*Age was centered to 75 years and education was centered to 12 years.
DISCUSSION
We validated a cognitive function battery that is being used in a clinical trial examining the effect of diet on cognition in older adults. Twelve individual tests were included and grouped into four theoretical domains: executive function, perceptual speed, episodic memory, and semantic memory. We then conducted a principal component analysis of all tests and found that our theoretical groupings were similar to the results from the principal component analysis, strengthening our confidence that the measures are capturing the cognitive domains of interest. Similarly, we found that the battery was associated with age, education, and late-life cognitive activity in the expected directions, further demonstrating the validity of our battery.
Although highly concordant, the domains that emerged from the principal component analysis varied in minor but important ways from our theoretical domains. First, the tests that represented our theoretical episodic memory domain were classified into two groups in the PCA, with the word list learning, recall, and recognition loading on one factor and the immediate and delayed story recall loading on a separate factor. Word lists, stories, or both are commonly used to measure memory and are commonly recognized as tasks that tap different abilities within episodic memory. Specifically, deficits on a word list task presume higher order encoding and retrieval deficits, associated with subcortical deficiency, whereas poor performance on a story memory task implies underlying deficits in encoding and storage with mesial temporal implications (Zahonde et al., Reference Zahonde, Bowers, Price, Bauer, Nisenzon, Foote and Okun2011). Studies comparing word list tasks with story tasks have also provided evidence that the two tasks are distinct in terms of practice effects (Gavett et al., Reference Gavett, Gurnani, Saurman, Chapman, Steinberg, Martin and Stern2016), performance of non-demented persons with Parkinson’s disease (Zahonde et al., Reference Zahonde, Bowers, Price, Bauer, Nisenzon, Foote and Okun2011), and their ability to distinguish causes of dementia in certain samples (Perri et al., Reference Perri, Fadda, Caltagirone and Carlesimo2013). Given substantial evidence that these two memory tasks perform differently, it is not unexpected that they would load on different factors in our study.
The second finding that varied from our theoretical conceptualization was the collapsing of the domains of executive function and perceptual speed onto one factor. This finding was somewhat unexpected, given the last 20 years of research that has examined executive function following the influential article by Miyake and colleagues (Reference Miyake, Friedman, Emerson, Witzki, Howerter and Wager2000). They revealed a model in which the three factors of “shifting,” updating,” and “inhibition” emerged as separable factors, but had moderate correlations with each other (Miyake et al., Reference Miyake, Friedman, Emerson, Witzki, Howerter and Wager2000). A recent systematic review of this body of executive function literature revealed that the unity and diversity in how the domain of executive function abilities manifest themselves may change across the lifespan (Karr et al., Reference Karr, Areshenkoff, Rast, Hofer, Iverson and Garcia-Barrera2018). Perhaps most importantly, the results from this systematic review reinforced our imperfect understanding of executive function as a domain. In our study, we examined tasks that tap inhibition (Flanker Inhibitory Control) and shifting (Trail Making Test B), but not updating and these tests loaded on one factor along with other tests, all of which were timed. As such, it is possible that the loadings on this factor reflect a time element that may override the influence of the individual tasks and underlying latent variables. The empirical findings regarding the domain of semantic memory were consistent with the conceptual groupings. Thus, despite minor discrepancies between conceptual and empirical findings, our study provides evidence that this cognitive battery is measuring distinct domains that are comparable to our theoretical understanding of these abilities.
We found several important associations of our battery with variables of interest in aging. We found an interesting interplay in the relationship between cognitive activity, education, and cognitive performance. Specifically, we found that frequency of cognitive activity in late life, compared with education, was more strongly associated with cognitive performance in all domains, with the exception of episodic memory. Finding a stronger association with late-life cognitive activities compared to years of education is not surprising, given the implication from recent research that years of formal education have been found to contribute less to cognitive function than late-life cognitive activities (Wilson, et al., Reference Wilson, Yu, Lamar, Schneider, Boyle and Bennett2019). However, it is not clear why episodic memory would have a stronger association with education and further research is needed to understand this finding. We also found sex differences, consistent with previous research. The most outstanding finding was that women outperformed men in episodic memory, a finding that is consistent with sex differences found earlier in life and has also been observed in the oldest old (Golchert, et al., Reference Golchert, Roehr, Luck, Wagner, Fuchs, Wiese and Riedel-Heller2019).
Important progress has been made in examining cognitive batteries or cognitive composites in studies examining the preclinical stage of dementia (Weintraub et al., Reference Weintraub, Carrillo, Tomaszewski Farias, Goldberg, Hendrix, Jaeger and Randolph2017). To measure the effect of treatments to reduce risk or prevent cognitive decline, cognitive batteries must be able to detect small changes before a diagnosis of dementia. Thus far, there is evidence that it is feasible to detect a cognitive change in the preclinical stage (Mortamais et al., Reference Mortamais, Ash, Harrison, Kaye, Kramer, Randolph, Pose and Ritchie2017), and researchers in the European Prevention of Alzheimer’s Dementia project have developed guidelines for cognitive outcomes to be used in future studies on preclinical Alzheimer’s disease (Ritchie et al., Reference Ritchie, Ropacki, Albala, Harrison, Kaye, Kramer and Ritchie2017). Future directions of dietary studies may benefit from heeding recommendations from the studies on preclinical Alzheimer’s disease, in addition to examining the validity of their study-specific battery.
Our study has important limitations. Our sample was predominantly White with an average of 17 years of education and was ethnically and linguistically homogenous. As such, these results will need to be validated in more diverse groups of people to generalize the results. Additionally, we did not compare the performance of this battery to an objective sample or control group. Nor did we evaluate the battery over time, in a population with disease progression, from cognitively normal to dementia. Future longitudinal studies are needed to assess the ability of this battery to measure change over time.
ACKNOWLEDGMENTS
We would like to acknowledge all the participants and staff who dedicated time, goodwill, and energy to this study. We acknowledge the astute insight provided by Dr. Martha Clare Morris that led to this work and express our utmost gratitude for her direction and support, which has allowed us to continue advancing the field of diet and cognitive aging.
FINANCIAL SUPPORT
The study was funded by the National Institute on Aging, RO1AG052583.
CONFLICTS OF INTEREST
We have no conflicts of interest to report.
