Executive functioning (EF) is an umbrella term composed of complex and multidimensional processes such as working memory and mental flexibility, initiation and maintenance of behavior, inhibition and response selection, and meta-tasking (MT) (Suchy, Reference Suchy2015). EF has consistently been linked to requisite skills for independent living (Jefferson, Paul, Ozonoff, & Cohen, Reference Jefferson, Paul, Ozonoff and Cohen2006; Kraybill & Suchy, Reference Kraybill and Suchy2011; Marshall et al., Reference Marshall, Rentz, Frey, Locascio, Johnson and Sperling2011; Uemura, Yamada, Nagai, & Ichihashi, Reference Uemura, Yamada, Nagai and Ichihashi2011), including financial management (Nakhla, Reference Nakhla2019; Schillerstrom et al., Reference Schillerstrom, Birkenfeld, Yu, Le, Goldstein and Royall2013), medication or medical regimen management (Manning, Clarke, Lorry, Weintraub, Wilkinson, Duda, & Moberg, Reference Manning, Clarke, Lorry, Weintraub, Wilkinson, Duda and Moberg2012; Stoer et al., Reference Stoehr, Lu, Lavery, Bilt, Saxton, Chang and Ganguli2008; Suchy et al., Reference Suchy, Turner, Queen, Durracio, Wiebe, Butner and Berg2016; Suchy et al., Reference Suchy, Ziemnik, Niermeyer and Brothers2020; Ziemnik & Suchy, Reference Ziemnik and Suchy2019), driving (Brown et al., Reference Brown, Ouimet, Eldeb, Tremblay, Vingilis, Nadeau, Pruessner and Bechara2016; Hayashi, Rivera, Modico, Foreman, & Wirth, Reference Hayashi, Rivera, Modico, Foreman and Wirth2017; Pope, Bell, & Stavrinos, Reference Pope, Bell and Stavrinos2017), and meal preparation (Craik & Bialystok, Reference Craik and Bialystok2006; Jefferson, Paul, Ozonoff, & Cohen, Reference Jefferson, Paul, Ozonoff and Cohen2006; Kosowicz & MacPherson, Reference Kosowicz and MacPherson2017).
Unfortunately, tests of EF have been criticized for having poor ecological validity, as evidenced by patients’ lapses in everyday functioning despite intact performance on traditional measures of EF (Burgess, Reference Burgess2000; Eslinger & Damasio, Reference Eslinger and Damasio1985; Penfield & Evans, Reference Pensfield and Evans1935; Shallice & Burgess, Reference Shallice and Burgess1991). It has been suggested that one key cause of EF tests’ poor ecological validity is the fact that traditional tests only assess abilities that are needed for performance of discrete tasks in a structured setting, while failing to assess abilities needed for successful execution of multiple complex tasks in the unstructured, real-world environment (Burgess et al., Reference Burgess, Veitch, de Lacy Costello and Shallice2000; Kearney, Harwood, Gladman, Lincoln, & Masud, Reference Kearney, Harwood, Gladman, Lincoln and Masud2013; Manchester, Priestley, & Jackson, Reference Manchester, Priestley and Jackson2004; Schmitter-Edgecombe, Parsey, & Cook, Reference Schmitter-Edgecombe, Parsey and Cook2011; Wilson, Reference Wilson1993). One such ability that is not adequately captured by traditional measures of EF is MT.
MT refers to the ability to complete, in a coordinated manner, multiple tasks over a somewhat extended period of time (Suchy, Reference Suchy2015). MT involves not only the planning and organization of the necessary steps needed for completion of each daily task, but also the ability to interleave these steps in a staggered fashion, while inhibiting impulses to mentally or behaviorally engage in distractions throughout the process. MT relies heavily on “branching,” or the ability to respond to contextual cues by temporarily interrupting one task so as to complete another task. Such contextual cues are sometimes referred to as “if-then” rules. There are many examples of if-then rules in daily life: “if you see a bank while driving home from work, then stop to pick up some cash, and then continue on your way home”; or “if you hear a buzzer while sweeping the floor, then go take the laundry out of the dryer, and then return to sweeping the floor”; and so on. Studies have shown a strong link between MT and the fronto-polar cortex (FPC), the most anterior portion of the frontal lobes (Koechlin & Hyafil, Reference Koechlin and Hyafil2007; Charron & Koechlin, Reference Charron and Koechlin2010). While lateral frontal regions select actions based on reward, the FPC enables branching by maintaining mental representations of postponed tasks and resisting distractions while performing the interleaved task at hand (Sakai, Rowe, & Passingham, Reference Sakai, Rowe and Passingham2002; Koechlin & Hyafil, Reference Koechlin and Hyafil2007). Importantly, these representations enable re-engagement of the interrupted task in the absence of external cues or associations between the current and the pending task. Not surprisingly, MT has been shown to play a role in daily functioning (Alderman, Evans, Burgess, & Wilson, Reference Alderman, Evans, Burgess and Wilson1993; Bennett, Ong, & Ponsford, Reference Bennett, Ong and Ponsford2005; Fortune, & Richards, Reference Fortune and Richards2017; Frisch, Förstl, Legler, Schöpe, & Goebel, Reference Frisch, Förstl, Legler, Schöpe and Goebel2012; Renison, Ponsford, Testa, Richardson, & Brownfield, Reference Renison, Ponsford, Testa, Richardson and Brownfield2012; Wilson et al., Reference Wilson, Alderman, Burgess, Emslie and Evans1996). Thus, attempts have been made to enhance our approaches to the assessment of this capacity.
One test designed to measure the ability to apply if-then rules is the fourth trial of the Color-Word Interference Test (CWIT) within the Delis–Kaplan Executive Function System (D-KEFS; Delis et al., Reference Delis, Kaplan and Kramer2001a) battery. Similar to the classic Stroop test (Stroop, Reference Stroop1935), the CWIT includes trials that assess processing speed (Conditions 1 and 2) and the ability to inhibit a pre-potent response (Condition 3). Unlike the traditional Stroop test, the CWIT includes a fourth trial (Condition 4) designed to augment the test by measuring the ability to alternate between cognitive tasks in the absence of explicit cues. Specifically, examinees are instructed to name the color of ink in which the words are printed (similar to the traditional Stroop, or CWIT Condition 3). However, they are further instructed to simply read the word (i.e., not name the color of ink) if they encounter a word that is placed inside a box. The boxes are placed randomly throughout the trial, requiring participants to switch at irregular intervals back and forth between naming the color of ink and reading the word.
Research shows that Condition 4 is more difficult than Condition 3 in terms of both time to completion and number of errors (Cato, Delis, Abildskov, & Bigler, Reference Cato, Delis, Abildskov and Bigler2004; Delis et al., Reference Delis, Kaplan and Kramer2001b). Condition 4 is also more sensitive than Condition 3 to certain types of neuropathology (Clark et al., Reference Clark, Schiehser, Weissberger, Salmon, Delis and Bondi2012; Fine et al., Reference Fine, Delis, Wetter, Jacobson, Jak, Mcdonald and Bondi2008; Wetter et. al., Reference Wetter, Delis, Houston, Jacobson, Lansing, Cobell and Bondi2005). Performance on Condition 4, rather than Condition 3, predicted global cognitive decline among older adults (Fine et al., Reference Fine, Delis, Wetter, Jacobson, Jak, Mcdonald and Bondi2008) and differentiated between at-risk individuals with apolipoprotein E4 compared to none-E4 individuals (Wetter et al., Reference Wetter, Delis, Houston, Jacobson, Lansing, Cobell and Bondi2005). It also predicted progression from mild cognitive impairment (MCI) to Alzheimer’s disease (AD) above and beyond Condition 3 and beyond measures of episodic memory (Clark et al., Reference Clark, Schiehser, Weissberger, Salmon, Delis and Bondi2012). Additionally, Condition 4, but not Condition 3, differentiated between individuals with AD and Huntington’s disease (Delis et al., Reference Delis, Kaplan and Kramer2001b), and between individuals with a history of attention-deficit hyperactivity disorder (ADHD) and controls (Halleland, Haavik, & Lundervold, Reference Halleland, Haavik and Lundervold2012). While research has shown evidence of the CWIT’s ability to assess task/set-shifting (Adólfsdóttir et al., Reference Adólfsdóttir, Haász, Wehling, Ystad, Lundervold and Lundervold2014; Cato, Delis, Abildskov, & Bigler, Reference Cato, Delis, Abildskov and Bigler2004; Delis et al., Reference Delis, Kaplan and Kramer2001b), as well as of its diagnostic utility (Delis et al., Reference Delis, Kaplan and Kramer2001b; McKinley, Grace, Dalrymple-Alford, & Roger, Reference McKinley, Grace, Dalrymple-Alford and Roger2009; Wodka et al., Reference Wodka, Loftis, Mostofsky, Prahme, Larson, Denckla and Mahone2008), no studies to date have examined whether Condition 4 measures the construct of MT.
The purpose of this study was to examine whether performance on Condition 4 of the CWIT (performance of which relies on the ability to branch, or apply if-then rules) is a better predictor of MT than performance on Condition 3 of the CWIT (which does not require branching). Additionally, because it could be argued that Condition 4 of the CWIT also relies on switching abilities (i.e., switching between naming color of ink and reading the word), we were further interested in examining whether Condition 4 of the CWIT relates to MT above and beyond those D-KEFS tests that contain a switching condition but technically do not require branching. To these ends, we administered a validated task of MT, as well as CWIT and switching conditions of, not on other D-KEFS tests, to a sample of community-dwelling older adults. We were particularly interested in examining an older adult sample, since older adults experience declines in EF (Craik & Bialystok, Reference Craik and Bialystok2006; Garden, Phillips, & MacPherson, Reference Garden, Phillips and MacPherson2001; Lin, Chan, Zheng, Yang, & Wang, Reference Lin, Chan, Zheng, Yang and Wang2007) and, consequently, are at an increased risk for experiencing lapses in IADLs in daily life (Burton, Strauss, Hultsch, & Hunter, Reference Burton, Strauss, Hultsch and Hunter2006; Chapman, Duberstein, & Lyness, Reference Chapman, Duberstein and Lyness2007; Martyr & Clare, Reference Martyr and Clare2012; Suchy, Kraybill, & Franchow, Reference Suchy, Kraybill and Franchow2011). We hypothesized that performance on the CWIT Condition 4 would be associated with MT above and beyond CWIT Condition 3, as well as beyond the switching conditions on the D-KEFS Trail Making, Verbal Fluency, and Design Fluency subtests.
METHODS
Participants
Participants were drawn from a sample of 101 non-Hispanic white community-dwelling adults between the ages of 60 and 93 years who participated in a larger study on the associations between contextual factors, cognition, and falls (Niermeyer & Suchy, Reference Niermeyer and Suchy2020a; Niermeyer & Suchy, Reference Niermeyer and Suchy2020b). Participants were recruited from courses for older adults, the University of Utah’s Center on Aging database, newspaper advertisements, and flyers placed at community centers. Interested older adults were screened for eligibility via telephone. Exclusion criteria were self-reported diagnosis of MCI, dementia, and/or a history of other major neurological or psychiatric conditions, uncorrected vision or hearing impairment, motor limitations that would prevent performance, left-handedness, and less than 8 years of education. Two participants were excluded due to missing data, and one participant was excluded due to unusually low scores on a screener of cognitive status (Dementia Rating Scale – Second Edition [DRS-2] total raw score = 123), suggesting possible mild dementia. The final sample consisted of 98 participants. Participants completed a 3.5-hour visit, for which they received $35, a list of community resources, and, if interested, feedback on their performance/responses on screening measures of cognition and depression.
Procedure
This study was approved by the University of Utah Institutional Review Board, and all procedures were in compliance with APA ethical standards. Informed consent was obtained before testing.
Measures
Meta-Tasking (MT)
MT was measured using the Modified Six Elements Test (MSET) from the Behavioral Assessment of the Dysexecutive Syndrome battery (Wilson, Alderman, Burgess, Emslie, & Evans, Reference Wilson, Alderman, Burgess, Emslie and Evans1996). The MSET is modeled after the Multiple Errands Test (Shallice & Burgess, Reference Shallice and Burgess1991), which was designed to tap into typical MT demands in daily life in the absence of external structure and was performed in a real-world environment (e.g., a shopping mall). The MSET is a paper-and-pencil adaptation of the Multiple Errands Test, while still maintaining the lack of structure. The MSET challenges examinees to complete six separate tasks within 10 minutes while adhering to a set of rules. The six tasks are composed of two sets of three activities: writing two stories, completing two sets of arithmetic problems, and writing the names of two sets of pictures printed on cards. While it is not possible to complete all six tasks, examinees are instructed that (a) they must attempt each of the tasks, (b) they should get as many of the early items completed as possible to maximize their points, since the early items are valued more than the items that occur later on, and (c) a certain order of task completion is prohibited. Examinees must self-cue to switch from one task to another in accordance with the rules and to re-engage in interrupted tasks appropriately to maximize their points. Other than the initial instructions, no cues are provided. The test is scored based on the examinees’ ability to follow the rules and the efficiency with which they work on the six tasks. Points are deducted for rule breaks and inefficient use of time. Total score consists of the number of tasks attempted (maximum of six is possible) minus the number of rule breaks. Thus, total possible scores range from 0 to 6. The MSET has been shown to validly assess deficits in EF among individuals with traumatic brain injury (Emmanouel, Kessels, Mouza, & Fasotti, Reference Emmanouel, Kessels, Mouza and Fasotti2014; Gouveia, Brucki, Malheiros, & Bueno, Reference Gouveia, Brucki, Malheiros and Bueno2007), schizophrenia (Liu et al., Reference Liu, Chan, Chan, Tang, Chiu, Lam and Chen2011), MCI and AD (Espinosa et al., Reference Espinosa, Alegret, Boada, Vinyes, Valero, Martinez-Lage and Tarraga2009), and substance abuse (Fernandez-Serrano, Perez-Garcia, Schmidt Rio-Valle, & Verdejo-Garcia, Reference Fernández-Serrano, Pérez-García, Schmidt Río-Valle and Verdejo-García2010). Additionally, the MSET has been shown to be ecologically valid, predicting daily functioning in individuals with ADHD (Clark, Prior, Clark, & Kinsella, Reference Clark, Prior, Clark, Prior and Kinsella2000), brain injury (Fortune, & Richards, Reference Fortune and Richards2017; Frisch, Förstl, Legler, Schöpe, & Goebel, Reference Frisch, Förstl, Legler, Schöpe and Goebel2012; Renison, Ponsford, Testa, Richardson, & Brownfield, Reference Renison, Ponsford, Testa, Richardson and Brownfield2012), and stroke (Frisch et al., Reference Frisch, Förstl, Legler, Schöpe and Goebel2012). Total raw scores on the MSET were used in analyses.
Executive Functioning (EF)
EF was measured using the CWIT, Trail Making, Verbal Fluency, and Design Fluency subtests from the D-KEFS; Delis et al., Reference Delis, Kaplan and Kramer2001a). For each of the four subtests, all available conditions were administered. However, only CWIT Conditions 3 and 4, Trail Making Test Condition 4 (Number-Letter Switching), Verbal Fluency Condition 3 (Category Switching), and Design Fluency Condition 3 (Design Fluency Switching) were used in analyses. Raw scores were used in all analyses.
As described earlier, CWIT Condition 3 (CWIT-3) consists of a list of color words printed in different color print. Examinees are required to name the color of ink in which the words are printed. CWIT Condition 4 (CWIT-4) also consists of a list of words printed in different color print, but some of the words are placed inside a black ink box. Examinees are required to name the color of print, with the exception of the words placed in a box, in which case the examinee is to read the words. For both conditions, the raw amount of time needed for completion of the list of words and the total number of errors (the sum of self-corrected and non-self-corrected errors) were used in analyses.
Trail Making Test Condition 4 (TMT-4) requires that examinees draw a line between letters and numbers placed randomly on an 11 × 17-inch page. Examinees are required to switch back and forth between sequencing numbers and letters (i.e., 1, A, 2, B, etc., to 16, P). The raw time needed for time to completion (in seconds) and the total number of errors (the sum of set-loss and sequencing errors) were used in analyses.
Verbal Fluency Condition 3 (VF-3) requires examinees to generate as many words as possible while simultaneously switching between two semantic categories. The total number of correct words provided within a 60-second period and the total number of errors (sum of the set-loss and repetition errors) were used in analyses.
Design Fluency Condition 3 (DF-3) requires examinees to create unique designs by using four straight lines to make connections among 10 dots. Examinees are required to alternate between filled dots and empty dots. The total number of correct designs and the total number of errors (the sum of set-loss and repetition errors) were used in analyses.
Characterizing the Sample
To characterize the sample, we screened for general cognitive status using the DRS-2 (Jurica, Leitten, & Mattis, Reference Jurica, Leitten and Mattis2001). The DRS-2 is a validated and widely used measure of cognition in adults aged 56 to 105 years old. The DRS-2 is sensitive to mild cognitive difficulties and has been shown to detect individuals with amnestic-MCI (Matteau, Simard, Jean, & Turgeon, Reference Matteau, Simard, Jean, Turgeon and Tsai2008; Matteau et al., Reference Matteau, Dupré, Langlois, Jean, Thivierge, Provencher and Simard2011) and to discriminate between AD and other forms of dementia (Aaarsland et al., Reference Aarsland, Litvan, Salmon, Galasko, Wentzel-Larsen and Larsen2003; Paolo, Troster, Glatt, Hubble, & Koller, Reference Paolo, Tröster, Glatt, Hubble and Koller1995). The DRS-2 includes five subscales measuring attention, initiation/perseveration, construction, conceptualization, and memory, scores of which are summed to determine total raw scores, used in supplementary analyses.
Data Analysis
Preliminary Analyses
Descriptive statistics and box plots were used to examine outliers for each variable. When identified, outlying values were Winsorized such that all values were within 3 SDs of the sample mean. Pearson correlations were utilized to assess relationships among demographic and cognitive variables.
Principal Analyses
To determine whether CWIT-4 accounted for variance in MSET above and beyond CWIT-3, we conducted two hierarchical regressions, using MSET total score as the dependent variable. In the first regression, CWIT performance speed variables were used as predictors, while in the second regression, CWIT performance accuracy variables were used as predictors. CWIT-3 was entered on Step 1, and CWIT-4 on Step 2.Footnote 1 To determine whether CWIT-4 accounted for MSET variance beyond other switching conditions of other D-KEFS subtests, we conducted a second set of two hierarchical regressions, using the MSET as the dependent variable, and TMT-4, VF-3, and DF-3 as predictors in Step 1; CWIT-3 and CWIT-4 were entered as predictors on Steps 2 and 3, respectively.
Supplementary Analyses
To examine whether the association between CWIT-4 and MSET performance was accounted for by demographic factors or cognitive status, we repeated the principal analyses, using age, years of education, and total score on the DRS-2 as covariates on Step 1. Additionally, to examine the ability of CWIT-4 to classify participants based on their MT abilities, we conducted a series of Receiver Operating Characteristics curve analyses, using in turn as criterion MSET performances with zero, <2, and <3 errors.
RESULTS
Preliminary Analyses
Descriptive statistics for the demographics and the cognitive performances of the sample are shown in Table 1. Winsorization was applied to MSET and resulted in decreasing the distance between units for scores 1 through 3, whereas distances between 3 and 6 remained unchanged. Zero-order correlations among the demographics and cognitive variables are presented in Table 2. As seen in Table 2, MSET correlated with all CWIT variables, as well as with the Trail Making test and the Design Fluency tests switching conditions (timed variables). Additionally, age, education, and DRS-2 all correlated with some of the cognitive variables, suggesting that they should be examined as covariates in supplementary analyses.
Demographic and cognitive characteristics of the sample
Note: N = 98. Age, age of participants in years; Education, education of participants in years; DRS-2, Dementia Rating Scale-2nd Edition total raw score; MSET, Modified Six Elements Test raw score; CWIT-3 Time, CWIT Condition 3 raw time; CWIT-4 Time, CWIT Condition 4 raw time; TMT-4, TMT Condition 4 raw time; VF-3 Raw, VF Condition 3 total correct raw; DF-3 Raw, DF Condition 3 total correct raw; TMT-4 Error, TMT Condition 4 total errors (set-loss + sequencing errors); VF-3 Error, VF Condition 2 total errors (set-loss + repetition errors); DF-3 Error, DF Condition 3 total errors (set-loss + repetition errors); CWIT-3 Error, CWIT Condition 3 total errors (self-corrected + non-self-corrected errors; CWIT-4 Error, CWIT Condition 4 total errors (self-corrected + non-self-corrected errors).
Zero-order correlations among demographic, dependent, and independent variables
Note: N = 98. Age, age of participants in years; Education, education of participants in years; DRS-2, Dementia Rating Scale-2nd Edition total raw score; MSET, Modified Six Elements Test raw score; CWIT, Color-Word Interference Test; CWIT-3 Time, CWIT Condition 3 raw time; CWIT-4 Time, CWIT Condition 4 raw time; TMT-4, TMT Condition 4 raw time; VF-3 Raw, VF Condition 3 total correct raw; DF-3 Raw, DF Condition 3 total correct raw; TMT-4 Error, TMT Condition 4 total errors (set-loss + sequencing errors); VF-3 Error, VF Condition 2 total errors (set-loss + repetition errors); DF-3 Error, DF Condition 3 total errors (set-loss + repetition errors); CWIT-3 Error, CWIT Condition 3 total errors (self-corrected + non-self-corrected errors; CWIT-4 Error, CWIT Condition 4 total errors (self-corrected + non-self-corrected errors).
Principal Analyses
The results of two separate hierarchical regressions examining whether CWIT-4 accounted for variance in MSET above and beyond CWIT-3 are summarized in Table 3. As seen in the table, CWIT-4 accounted for variance in MSET above and beyond CWIT-3, regardless of whether performance speed (i.e., “time”) or performance accuracy (i.e., “errors”) were used as predictors. Additional hierarchical regressions were conducted to determine whether CWIT-4 accounted for variance in MSET beyond other switching conditions of D-KEFS subtests. The results are presented in Table 4. As seen in the Table, CWIT-4 once again accounted for variance in MSET above and beyond all other D-KEFS switching conditions, as well as beyond CWIT-3, suggesting that the branching aspect of the task uniquely taps into MT abilities. Of note, when all predictor variables in these two models were examined simultaneously (i.e., non-hierarchically), both CWIT-4 speed (β = −.348, t = 2.54, p = .013) and TMT-4 (β = −.264, t = 2.01, p = .047) emerged as unique timed-based predictors; however, CWIT-4 was the only accuracy-based predictor (β = −.284, t = 2.68, p = .009).
Hierarchical linear regressions predicting Six Elements Test by Color-Word Interference Test Conditions 3 and 4 time and errors
Note: N = 98; CWIT, Color-Word Interference Test; CWIT-3 Time, CWIT Condition 3 raw time; CWIT-4 Time, CWIT Condition 4 raw time; CWIT-3 Error, CWIT Condition 3 total errors (self-corrected + non-self-corrected errors; CWIT-4 Error, CWIT Condition 4 total errors (self-corrected + non-self-corrected errors).
Hierarchical linear regressions predicting Six Elements Test by executive function
Note: N = 98. TMT-4, TMT Condition 4 raw completion time; VF-3 Raw, VF Condition 3 total correct raw; DF-3 Raw, DF Condition 3 total correct raw; CWIT, Color-Word Interference Test; CWIT-3 Time, CWIT Condition 3 raw time; CWIT-4 Time, CWIT Condition 4 raw time; TMT-4 Error, TMT Condition 4 total errors (set-loss + sequencing errors); VF-3 Error, VF Condition 2 total errors (set-loss + repetition errors); DF-3 Error, DF Condition 3 total errors (set-loss + repetition errors); CWIT-3 Error, CWIT Condition 3 total errors (self-corrected + non-self-corrected errors; CWIT-4 Error, CWIT Condition 4 total errors (self-corrected + non-self-corrected errors).
Supplementary Analyses
Because age, education, and general cognitive status (as measured by the DRS-2) are related to cognition, we wanted to ensure that the CWIT-4’s association with MSET cannot be accounted for by these demographic and cognitive factors. The results of the analyses are summarized in Tables 5 and 6. As seen in the tables, CWIT-4 time to completion continued to account for variance in MSET beyond all other variables, while CWIT-4 errors was reduced to a trend.
Hierarchical linear regressions predicting Six Elements Test
Note: N = 98. Age, age of participants in years; Edu, education of participants in years; DRS-2, Dementia Rating Scale-2nd Edition total raw score; CWIT, Color-Word Interference Test; CWIT-3 Time, CWIT Condition 3 raw time; CWIT-4 Time, CWIT Condition 4 raw time; CWIT-3 Error, CWIT Condition 3 total errors (self-corrected + non-self-corrected errors; CWIT-4 Error, CWIT Condition 4 total errors (self-corrected + non-self-corrected errors).
Hierarchical linear regressions predicting Six Elements Test
Note: N = 98. Age, age of participants in years; Edu, education of participants in years; DRS-2, Dementia Rating Scale-2nd Edition total raw score; CWIT-3 Time, CWIT Condition 3 raw time; CWIT-4 Time, CWIT Condition 4 raw time; TMT-4, TMT Condition 4 raw time; VF-3 Raw, VF Condition 3 total correct raw; DF-3 Raw, DF Condition 3 total correct raw; TMT-4 Error, TMT Condition 4 total errors (set-loss + sequencing errors); VF-3 Error, VF Condition 2 total errors (set-loss + repetition errors); DF-3 Error, DF Condition 3 total errors (set-loss + repetition errors); CWIT, Color-Word Interference Test; CWIT-3 Error, CWIT Condition 3 total errors (self-corrected + non-self-corrected errors; CWIT-4 Error, CWIT Condition 4 total errors (self-corrected + non-self-corrected errors).
Lastly, we examined whether CWIT-4 time to completion (as the only variable that continued to predict MSET beyond demographics and all other variables) was able to reliably identify participants with MT difficulties. The results of these analyses are summarized in Table 7.
Receiver operating characteristics curve analyses classifying participants based on MSET performance using CWIT-4 time to completion raw scores
Note: N = 98. CWIT-4, Color-Word Interference Test Condition 4; AUC, Area under the Curve; MSET, Modified Six Elements Test.
DISCUSSION
MT is an aspect of EF that involves the ability to branch (i.e., to apply “if-then” rules) and to effectively interleave sub-goals of one task with sub-goals of another task. MT is crucial for the ability to independently complete instrumental activities of daily living. Traditional measures of EF do not adequately assess the MT capacity, which has been suggested to be a main limitation of these tests’ ecological validity. The present study examined whether Condition 4 of the CWIT-4, which requires periodic switching from naming the color of ink to reading the word, uniquely relates to MT, as assessed via the Modified Six Elements Test (MSET). The key finding of the present study is that CWIT-4 performance speed and accuracy accounted for variance in MSET above and beyond CWIT-3 as well as above and beyond switching conditions of three other EF tests. In addition, these associations held for CWIT-4 speed above and beyond age, education, and cognitive status; however, for CWIT-4 accuracy, its association with MSET was reduced to a trend once demographics and general cognition were included as covariates. Last, CWIT-4 time to completion of >60 seconds identified 75% of participants who made even just one mistake on the MSET (specificity 65%), and >66 seconds identified 71% of participants who made one or more mistakes (72% specificity).
Clinical Implications
Accurate assessment of MT during a neuropsychological evaluation is crucial in older adults, who often exhibit decrements in EF (Craik & Bialystok, Reference Craik and Bialystok2006; Garden, Phillips, & MacPherson, Reference Garden, Phillips and MacPherson2001; Lin, Chan, Zheng, Yang, & Wang, Reference Lin, Chan, Zheng, Yang and Wang2007), and are at increased risk for IADL lapses (Burton, Strauss, Hultsch, & Hunter, Reference Burton, Strauss, Hultsch and Hunter2006; Chapman, Duberstein, & Lyness, Reference Chapman, Duberstein and Lyness2007; Martyr & Clare, Reference Martyr and Clare2012; Suchy, Kraybill, & Franchow, Reference Suchy, Kraybill and Franchow2011). MT difficulties may result in mistakes or inefficient completion of tasks (Cook, Reference Cook2008), due to the inability to mentally maintain and behaviorally re-engage in postponed tasks. As mentioned previously, branching (a key aspect of MT) relies on “if-then” rules, such as “if you hear the timer, then you turn off the oven.” Effective measures of MT can help identify potential safety risks and inform treatment planning for older adults and/or other populations who may experience challenges with MT, such as individuals with acquired brain injury. Treatment recommendations may include additional caregiver support or explicit reminders of contextual cues (e.g., readily-visible written reminders, phone alarms with visual reminders, or phone calls from loved ones). Such recommendations are particularly relevant to complex multi-step tasks that require interleaving of subtasks.
To our knowledge, the present study is the first to specifically examine whether CWIT-4 is uniquely associated with MT in older adults. The strong association (medium to large effect size) between performance on the MSET and CWIT-4 observed in this study supports the use of CWIT-4 as a measure of MT and, as such, may represent a useful tool for predicting patients’ ability to function independently and successfully in daily life. While CWIT-4’s ability to predict actual daily functioning above and beyond other EF tests awaits empirical validation, the fact that CWIT-4 was able to reliably classify participants based on whether or not they made any MSET mistakes is encouraging. In fact, the MSET itself has been consistently found to predict daily functioning as measured by informant questionnaires in individuals with ADHD (Clark, Prior, Clark, & Kinsella, Reference Clark, Prior, Clark, Prior and Kinsella2000), traumatic brain injury (Alderman, Evans, Burgess, & Wilson, Reference Alderman, Evans, Burgess and Wilson1993; Bennett, Ong, & Ponsford, Reference Bennett, Ong and Ponsford2005; Fortune, & Richards, Reference Fortune and Richards2017; Frisch, Förstl, Legler, Schöpe, & Goebel, Reference Frisch, Förstl, Legler, Schöpe and Goebel2012; Renison, Ponsford, Testa, Richardson, & Brownfield, Reference Renison, Ponsford, Testa, Richardson and Brownfield2012; Wilson et al., Reference Wilson, Alderman, Burgess, Emslie and Evans1996), stroke (Frisch et al., Reference Frisch, Förstl, Legler, Schöpe and Goebel2012), and schizophrenia (Chan, Chen, Cheung, & Cheung, Reference Chan, Chen, Cheung and Cheung2004; Liu et al., Reference Liu, Chan, Chan, Tang, Chiu, Lam and Chen2011). The MSET also predicted MT on a real-life cooking task in patients with acquired brain injury (Frisch et al., Reference Frisch, Förstl, Legler, Schöpe and Goebel2012). Lastly, longitudinal studies have shown that MSET performance predicted alcohol consumption 6–9 months post-brain injury (Ponsford et al., Reference Ponsford, Tweedly and Taffe2013), and social/occupational functioning at 1, 2, and 3 years post injury (Liu et al., Reference Liu, Chan, Chan, Tang, Chiu, Lam and Chen2011).
Importantly, previous literature also suggests that MSET is a better predictor of daily functioning than some traditional measures of EF, including the WCST (Bennett, Ong, & Ponsford, Reference Bennett, Ong and Ponsford2005), letter and semantic fluency (Bennett, Ong, & Ponsford, Reference Bennett, Ong and Ponsford2005; Fortune & Richards, Reference Fortune and Richards2017), TMT-B (Bennett, Ong, & Ponsford, Reference Bennett, Ong and Ponsford2005; Fortune & Richards, Reference Fortune and Richards2017), and a test of figural fluency (Frisch et al., Reference Frisch, Förstl, Legler, Schöpe and Goebel2012). In contrast, multiple studies found no significant relationship between functional impairment (as reported by caregivers) and TMT-B (Marshall et al., Reference Marshall, Rentz, Frey, Locascio, Johnson and Sperling2011), the Stroop test (Back-Madruga, et al., Reference Back-Madruga, Boone, Briere, Cummings, McPherson, Fairbanks and Thompson2002), letter fluency, D-KEFS Sorting Test Condition 1, D-KEFS Tower Test, and the CWIT Condition 3 (Razani et al., Reference Razani, Casas, Wong, Lu, Alessi and Josephson2007). The latter three also did not significantly relate to performance on tests of MT (Razani et al., Reference Razani, Casas, Wong, Lu, Alessi and Josephson2007).
Taken together with prior literature, the current findings suggest that CWIT-4 may be a better predictor of daily functioning than other traditional measures of EF, since (a) CWIT-4 is more strongly associated with MSET than are other traditional tests of EF, and (b) MSET is more strongly associated with daily functioning than traditional EF measures. Thus, clinicians may consider prioritizing CWIT-4 as part of their EF battery when examining possible functional decline in older adults. That said, it is important to note that the association between the MSET and the total errors on CWIT-4 was reduced to a trend once demographics and general cognition, as measured by the DRS-2, were included as covariates. This loss of statistical significance likely simply reflects the fact that the univariate association between the MSET and CWIT-4 accuracy appeared weaker than the association between the MSET CWIT-4 speed (which survived the covariate analyses), especially since covariates reduced the association of the MSET with both the speed and the accuracy on the CWIT-4 to a similar degree. Thus, ultimately, it simply appears that the speed CWIT-4 variable is a more powerful predictor of MT than is the error-based score and should be relied upon more heavily in clinical decision-making. This suggestion is in line with the typical clinical practice wherein clinicians tend to rely on the speeded scores more heavily than on error scores and is consistent with the designation of the CWIT time-based scores as “primary” and the CWIT error scores as “optional” in the test manual (Delis et al., Reference Delis, Kaplan and Kramer2001b).
Of course, clinicians could opt to utilize the MSET itself as a measure of MT; however, the MSET has not yet been normed on a North American sample and as such may not be appropriate for clinical use in the US and in Canada. Aside from the lack of North American norms, the advantage of the CWIT-4 is that it is co-normed with CWIT-3, as well as with eight additional EF measures that are frequently administered in clinical settings. Lastly, the administration time for the MSET is somewhat longer than that of CWIT, even when all four CWIT conditions are administered, as is customary. The MSET administration time may be particularly protracted for patients who have difficulty understanding the instructions and require extra or repeated explanations. Again, future research is needed to translate the current findings to the clinic. As part of this, future studies might consider pitting CWIT-4 and MSET against each other as predictors of actual daily functioning.
Theoretical Considerations
As seen in Table 1, CWIT-4 scores (both time-based and error-based) have a wider range than CWIT-3 scores. Thus, it could be argued that CWIT-4’s wider range explains its stronger association with the MSET as compared to CWIT-3 (which could be viewed as having a constricted range). However, the absolute range of raw scores does not, in and of itself, reflect whether a range is constricted or not; rather, a range is constricted if it fails to capture the full range of a given ability. To ascertain that the CWIT-3 and CWIT-4 scores captured the same range of abilities, we examined the corresponding scaled score ranges from the test’s normative tables (Delis et al., Reference Delis, Kaplan and Kramer2001b). We found ranges were roughly comparable, with the widest range being 15 scaled scores and the narrowest range being 13 scaled scores. Also, see Supplementary Figures 1 and 2 for scatterplots.
The finding that CWIT-4 performance speed and accuracy accounted for variance in MSET not only beyond CWIT-3 performance but also beyond the switching conditions of the three other EF tests is important. This finding lends additional evidence to the idea that branching (i.e., applying “if-then” rules) is an aspect of EF that is distinct from simply switching between two different tasks. It also suggests that the unique association between CWIT-4 performance and MSET is unlikely to be explained by task difficulty alone. It has been argued that one way to think about the difference between switching and branching is that, unlike switching, branching requires event-based prospective memory (Suchy, Reference Suchy2015).
Like many aspects of EF, prospective memory has been shown to decline with age in some individuals (McAlister & Schmitter-Edgecombe, Reference McAlister and Schmitter-Edgecombe2013; Uttl, Reference Uttl2008), and in and of itself is an aspect of EF with multiple subcomponents. Specifically, event-based prospective memory (applying “if-then” rules when an event or stimulus occurs) is dissociable from time-based prospective memory, where an individual must remember to switch to a new task after a certain amount of time has passed (Picton, Stuss, Shallice, Alexander, & Gillingham, Reference Picton, Stuss, Shallice, Alexander and Gillingham2006). For example, time-based prospective memory may be required for an older adult to remember to refill a prescription after a month has passed or return to the kitchen after an hour when making a meal so that they can hear the alarm that will indicate the meal is ready. Theoretically, optimal performance on the MSET (and successful functioning in daily life) requires the use of both time-based and event-based prospective memory. In some neurodegenerative conditions, such as Parkinson’s disease, time-based prospective memory may be more impaired than event-based prospective memory (e.g., Raskin et al., Reference Raskin, Woods, Poquette, McTaggart, Sethna, Williams and Tröster2011). Time-based prospective memory (or time estimation abilities for that matter) may play a key role in why low performances on multiple errands type tests (e.g., the MSET) are associated with reduced functional outcomes in some populations. If this is the case, then CWIT-4 performance may not be as effective as multiple errands style tests in predicting functional outcomes for those individuals. This possibility warrants additional empirical investigation.
Limitations and Future Directions
The present results need to be interpreted in the context of study limitations. Most notably, our sample consisted of cognitively healthy older adults, all of whom were Caucasian and well-educated. Thus, replications with more diverse populations and in older adults with confirmed cognitive decline are needed. Since previous studies have shown MT impairment in individuals with MCI (Beaver & Schmitter-Edgecomb, Reference Beaver and Schmitter-Edgecombe2017; Schmitter-Edgecombe, McAlister, & Weakley, Reference Schmitter-Edgecombe, McAlister and Weakley2012), AD (Esposito et al., Reference Esposito, Rochat, Van der Linden, Lekeu, Quittre, Charnallet and Van der Linden2010; Paula, Malloy-Diniz, & Leandro, Reference Paula and Malloy-Diniz2013), and behavioral variant frontotemporal dementia (Roca et al., Reference Roca, Manes, Gleichgerrcht, Watson, Ibáñez, Thompson, Torralva and Duncan2013), demonstration of CWIT-4 deficits in these population is needed.
Additionally, although ecological validity of MSET has been demonstrated (Alderman, Evans, Burgess, & Wilson, Reference Alderman, Evans, Burgess and Wilson1993; Bennett, Ong, & Ponsford, Reference Bennett, Ong and Ponsford2005; Fortune, & Richards, Reference Fortune and Richards2017; Frisch, Förstl, Legler, Schöpe, & Goebel, Reference Frisch, Förstl, Legler, Schöpe and Goebel2012; Renison, Ponsford, Testa, Richardson, & Brownfield, Reference Renison, Ponsford, Testa, Richardson and Brownfield2012), the current study is limited by not assessing MT in the naturalistic environment. Thus, future research should examine the association between CWIT-4 and real-world multi-step tasks. Another limitation lies in our cross-sectional design, necessitating future studies that would examine if worsened performance on CWIT-4 prospectively predicts decline in MT and daily functioning over time. Finally, future research should compare the relative effectiveness of MSET and CWIT-4 in predicting real-world functioning. If MSET outperforms CWIT-4, efforts to norm the MSET in countries outside of the UK would be needed. If CWIT-4 outperforms MSET, such information further substantiates the use of CWIT-4 as an ecologically valid measure of MT.
Acknowledgments
The first author (NEK) wishes to thank Dr. Suchy for her mentorship, generosity, and guidance on this project and Dr. Niemeyer for her collaboration, comments, and suggestions. The authors have no conflicts to disclose. This research was supported by internal funding from the University of Utah College of Social and Behavioral Sciences and received no specific grant from any other funding agency, commercial, or not-for-profit sectors.
CONFLICT OF INTEREST
The authors have nothing to disclose.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/S1355617720001381
