Introduction
Amnestic mild cognitive impairment (aMCI) is characterized by a decline in memory (and other cognitive processes in the case of multi-domain aMCI) that is beyond what would be expected for age and education, but not of a magnitude to meet criteria for dementia (Petersen, Reference Petersen2004). Individuals with aMCI are at increased risk for dementia, especially of Alzheimer’s disease (AD) pathology (Albert et al., Reference Albert, DeKosky, Dickson, Dubois, Feldman, Fox and Phelps2011). An important distinction between aMCI and dementia is that individuals with aMCI have generally intact instrumental activities of daily living (IADL) - tasks that enable people to function independently in the community such as banking, shopping, meal preparation, and navigating transportation. In contrast, individuals with dementia require support with IADLs, often in the form of assisted-living. IADLs draw upon higher-order cognitive processes such as planning and problem solving, mental flexibility, and memory (e.g., Brown, Devanand, Liu, & Caccappolo, Reference Brown, Devanand, Liu and Caccappolo2011; Jefferson et al., Reference Jefferson, Byerly, Vanderhill, Lambe, Wong, Ozonoff and Karlawish2008; Tuokko, Morris, & Ebert, Reference Tuokko, Morris and Ebert2005). One way to conceptualize the transition from aMCI to dementia is that cognitive functioning has declined to the point of obstructing the safe and efficient accomplishment of everyday activity. And so, targeted interventions that enhance or remediate IADLs may allow individuals with aMCI to sustain functional independence and thereby delay the onset of clinical dementia.
In the last decade, there has been mounting evidence indicating that individuals with aMCI enact IADL-type activities less well than healthy older adults, although better than individuals with dementia (Bailey, Kurby, Giovannetti, & Zacks, Reference Bailey, Kurby, Giovannetti and Zacks2013; Farias et al., Reference Farias, Mungas, Reed, Harvey, Cahn-Weiner and DeCarli2006; Reppermund et al., Reference Reppermund, Brodaty, Crawford, Kochan, Draper, Slavin and Sachdev2013). Given the weight of these findings, researchers have called for clarification regarding how much restriction in IADLs can still be permitted in a diagnosis of aMCI before it has reached the threshold of dementia (Reppermund et al., Reference Reppermund, Sachdev, Crawford, Kochan, Slavin, Kang and Brodaty2011). This has led to variability in the differential diagnosis of dementia compared to aMCI, disparate classification standards across research groups and clinics, and a lack of consensus about the nature of the functional impairment that coincides with the cognitive changes (Jekel et al., Reference Jekel, Damian, Wattmo, Hausner, Bullock, Connelly and Frölich2015). A better understanding of how IADLs decline from the early stages of aMCI through to dementia is needed to refine classification guidelines, develop targeted cognitive or pharmaceutical interventions, and enhance environmental adaptations to support functional autonomy. Understanding subtle declines in IADLs in early aMCI may also help to more reliably distinguish it from normal aging.
Questionnaires completed by the patient and/or informant are most often used to assess IADL, and they have merits and disadvantages. Performance-based measures are another way of assessing IADLs, and encompass domain-specific tasks such as financial capacity (Griffith et al., Reference Griffith, Belue, Sicola, Krzywanski, Zamrini, Harrell and Marson2003) or driving safety (Wadley et al., Reference Wadley, Okonkwo, Crowe, Vance, Elgin, Ball and Owsley2009), or more general tests of solving everyday problems (Bangen et al., Reference Bangen, Jak, Schiehser, Delano-Wood, Tuminello, Han and Bondi2010; Burton, Strauss, Bunce, Hunter, & Hultsch, Reference Burton, Strauss, Bunce, Hunter and Hultsch2009; Goldberg et al., Reference Goldberg, Koppel, Keehlisen, Christen, Dreses-Werringloer, Conejero-Goldberg and Davies2010); they can be completed with paper and pencil questions, demonstration with props such as a medicine bottle, or a combination of the two (Loewenstein et al., Reference Loewenstein, Amigo, Duara, Guterman, Hurwitz, Berkowitz and Gittelman1989; Pereira et al., Reference Pereira, Yassuda, Oliveira, Diniz, Radanovic, Talib and Forlenza2010). Some of the drawbacks of performance-based measures include administration and assessment time as well as artificially imposing structure in the testing room that would otherwise need to be supported by everyday cognitive abilities such as executive function (i.e., reduced distractions, increased structure) or memory (i.e., remembering the location of items). Certain performance-based measures have the benefit of affording greater experimental control in the assessment of IADLs, reflecting the person’s current capacity to perform a task while avoiding some of the drawbacks of certain questionnaire-based assessments such as the potential inaccuracy of someone with cognitive impairment reporting on their IADL status, the tendency of collaterals to misestimate functional disability, and the use of scales that rate the presence or absence of changes with broad gradations of change that may not have the range to capture subtle declines in daily functioning (for systematic reviews, see Gold, Reference Gold2012; Jekel et al., Reference Jekel, Damian, Wattmo, Hausner, Bullock, Connelly and Frölich2015; Sikkes, de Lange-de Klerk, Pijnenburg, Scheltens, & Uitdehaag, Reference Sikkes, de Lange-de Klerk, Pijnenburg, Scheltens and Uitdehaag2009). For instance, Giovannetti and colleagues (Reference Giovannetti, Bettcher, Brennan, Libon, Burke, Duey and Wambach2008) found that caregivers or informants may not detect or report mild changes in the performance of everyday action in MCI individuals because some questionnaire measures are not able to capture subtle errors in the enactment of everyday activities. To address this limitation, informant questionnaires with better psychometric properties have been validated in recent years (Farias et al., Reference Farias, Mungas, Harvey, Simmons, Reed and DeCarli2011) and adopted by the Alzheimer’s Disease Neuroimaging Initiative (ADNI), revealing large effect size differences between MCI and control groups (Rueda et al., Reference Rueda, Lau, Saito, Harvey, Risacher and Aisen2014). Studies that directly compare informant questionnaires and performance-based methods report that the effect size difference between MCI and control groups is significantly greater on the performance-based measure (Burton et al., Reference Burton, Strauss, Bunce, Hunter and Hultsch2009; Goldberg et al., Reference Goldberg, Koppel, Keehlisen, Christen, Dreses-Werringloer, Conejero-Goldberg and Davies2010; Pereira et al., Reference Pereira, Yassuda, Oliveira, Diniz, Radanovic, Talib and Forlenza2010; Puente, Terry, Faraco, Brown, & Miller, Reference Puente, Terry, Faraco, Brown and Miller2014). Once again though, the differences in performance are often subtle on measures commonly used in the clinic such as the Institute of Living Scales (Loeb, Reference Loeb1996), as aMCI groups still perform within normal limits on this task (Bangen et al., Reference Bangen, Jak, Schiehser, Delano-Wood, Tuminello, Han and Bondi2010), underscoring the need for more sensitive IADL indices and better normative data. In particular, given the relative strengths and weaknesses of the different methods of assessing IADLs, approaches that characterize changes in IADLs with greater resolution, such as error analyses, may be needed to reliably distinguish healthy older adults from those with aMCI.
Performance-based IADL measures vary in the extent to which they resemble everyday IADL tasks. One approach is to have participants enact tasks that are highly familiar to them. Routine naturalistic actions (NAs) are goal-directed tasks such as preparing coffee or mailing a letter. Of importance, NA performance is correlated both with IADLs and measures of functional independence, thus providing a link between a laboratory task that is more closely associated with everyday functioning than cognitive measures such as tests of complex attention (Schwartz, Segal, Veramonti, Ferraro, & Buxbaum, Reference Schwartz, Segal, Veramonti, Ferraro and Buxbaum2002). NAs may be able to capture additional variability in the prediction of everyday functioning because they contain task demands that extend over several minutes such as interweaving between hierarchical goal and subgoal structures, memory, and cognitive control (Gold, Park, Troyer, & Murphy, Reference Gold, Park, Troyer and Murphy2015). Unlike certain neuropsychological tests, NAs do not contain direct feedback from the experimenter or external cues for how to proceed. We recently examined NA performance in a group with aMCI and found that individuals with aMCI reliably enacted more errors of omission (not performing a step) and commission (inaccurate execution, improper ordering, incorrect tool use) than control participants (Gold et al., Reference Gold, Park, Troyer and Murphy2015). However, it is not known if these subtle differences in NA performance can reliably distinguish healthy older adults from those with aMCI. Furthermore, given the different task demands of NAs relative to neuropsychological tests, it is not known if they are capturing unique or overlapping variability in the prediction of healthy aging relative to aMCI. Either finding would be useful. For instance, if NAs can account for unique variability they could serve as a separate outcome measure in various intervention studies to examine the efficacy of a treatment. If NAs are contributing overlapping variability, they may provide convergent validity for a field that requires more sensitive measures of everyday functioning with better psychometric properties (e.g., ecological validity).
In the current study, we examined the extent to which errors of omission and commission in NA performance could predict membership in the aMCI group or the healthy control group. Group classification was determined by established diagnostic criteria for aMCI (Albert et al., Reference Albert, DeKosky, Dickson, Dubois, Feldman, Fox and Phelps2011; Petersen, Reference Petersen2004), including neuropsychological testing, clinical interview, and a questionnaire-based IADL assessment. We also administered a separate battery of cognitive tests that surveyed memory, semantic knowledge, and executive function to provide a meaningful contrast for the predictive utility of NA.
Prior work indicated that group membership (MCI vs. healthy aging) can be predicted with 73% accuracy on a performance-based measure of money management and health and safety awareness (Bangen et al., Reference Bangen, Jak, Schiehser, Delano-Wood, Tuminello, Han and Bondi2010). Using different performance-based measures of IADL that also require paper-and-pencil responses to everyday scenarios and demonstrations with props, comparable classification of healthy older adults and aMCI groups (Goldberg et al., Reference Goldberg, Koppel, Keehlisen, Christen, Dreses-Werringloer, Conejero-Goldberg and Davies2010) or heterogeneous MCI groups (Burton et al., Reference Burton, Strauss, Bunce, Hunter and Hultsch2009; Pereira et al., Reference Pereira, Yassuda, Oliveira, Diniz, Radanovic, Talib and Forlenza2010) have been reported. These studies did not evaluate the predictive utility of a performance-based IADL measure relative to measures of cognition. Compared to other types of performance-based measures (Bangen et al., Reference Bangen, Jak, Schiehser, Delano-Wood, Tuminello, Han and Bondi2010; Burton et al., Reference Burton, Strauss, Bunce, Hunter and Hultsch2009; Pereira et al., Reference Pereira, Yassuda, Oliveira, Diniz, Radanovic, Talib and Forlenza2010), we hypothesized that our detailed assessment of error performance in the enactment of highly familiar NAs would allow us to capture additional variability in the prediction of group membership given the extent to which NAs overlap with the task demands of IADLs (i.e., a driving simulator would likely be a better predictor of driving safety than a visual search task). We also anticipated that the best prediction of group membership would include both cognitive test performance and detailed error tendencies in the enactment of NAs, capturing both the subtle cognitive and functional changes in aMCI (Albert et al., Reference Albert, DeKosky, Dickson, Dubois, Feldman, Fox and Phelps2011). Neuropsychological testing is influenced by demographic variables such as age and education, presenting challenges for assessing extreme or under-represented groups. By contrast, NAs are not as closely associated with demographic variables (Schwartz et al., Reference Schwartz, Segal, Veramonti, Ferraro and Buxbaum2002). If it is possible to facilitate diagnostic group membership (e.g., aMCI vs. healthy aging) based on NA performance, this information could be used for convergent diagnostic validity as well as to better predict functioning in the community.
Method
Participants and Group Designation
Individuals with memory complaints were recruited from Baycrest Health Sciences, community talks, and advertisements. Healthy older adults were recruited from the volunteer group at Baycrest, community talks, and advertisements. A total of 66 individuals were screened for the study, and 48 individuals participated (Gold et al., Reference Gold, Park, Troyer and Murphy2015). Participants in the healthy control and aMCI groups were required to meet the following study criteria, as determined by consensus opinion of two clinical neuropsychologists (A.T. and K.M.) reviewing the cognitive test results from the diagnostic battery (described subsequently) together with clinical history. Participants were required to be relatively fluent in their comprehension of English, defined by self-report Likert scale (“1” indicated no fluency and “7” indicated complete fluency) of at least 6 out of 7 and clinical impressions from interview (two participants were excluded because performance on the verbal measures of the diagnostic battery was greater than 2 SDs below their visuospatial measures and suggestive of inadequate fluency in English from clinical interview as well). Participants had to display intact perception of objects in the visual-spatial domain (Warrington & James, Reference Warrington and James1991), and to perform within normal limits on an apraxia screening test (Almeida, Black, & Roy, Reference Almeida, Black and Roy2002), and no participants were excluded on this basis. Participants were excluded if they had elevations on either of the depression or anxiety subscales of the Hospital Anxiety and Depression Scale (Zigmond & Snaith, Reference Zigmond and Snaith1983), or a history of medical or psychiatric conditions that could significantly affect cognition (three participants were excluded for elevations in anxiety or depression).
Control Group
The 24 control participants performed within the expected range based on age and education corrected published normative data for measures in the diagnostic cognitive battery (see Table 1) and were independent in their daily functioning as assessed by Lawton-Brody IADL items during interview (see criteria below, although only the aMCI group required collateral information). Three control participants were excluded for lower than anticipated performance on more than one measure in the diagnostic cognitive battery.
Diagnostic cognitive battery performance
Note. aMCI=amnestic mild cognitive impairment. For ease of comparison, all values are presented as age corrected scaled score. MMSE=Mini Mental Status Exam; Folstein, Folstein, & McHugh, Reference Folstein, Folstein and McHugh1975. WAIS-III=Wechsler Adult Intelligence Scale-III; Wechsler, Reference Wechsler1997. BNT=Boston Naming Test [Kaplan, Goodglass, & Weintraub, Reference Kaplan, Goodglass and Weintraub1983; normative data from the MOANS (Ivnik, Malec, Smith, Tangalos, & Petersen, Reference Ivnik, Malec, Smith, Tangalos and Petersen1996)]. D-KEFS=Delis-Kaplan Executive Function System (Delis, Kaplan, & Kramer, Reference Delis, Kaplan and Kramer2001). ROCF=Rey Osterrieth Complex Figure Test [(Spreen & Strauss, Reference Spreen and Strauss1998, normative data from Fastenau, Denburg, & Hufford, Reference Fastenau, Denburg and Hufford1999). HVLT-R=Hopkins Verbal Learning Test Revised, Brandt & Benedict, Reference Brandt and Benedict2001. WMS-R=Wechsler Memory Scale-Revised [Wechsler, Reference Wechsler1987; normative data drawn from Mayo’s Older Americans Normative Studies (MOANS; Ivnik et al., Reference Ivnik, Malec, Smith, Tangalos, Petersen, Kokmen and Kurland1992)].
*Significant difference between control and aMCI group (p<.003).
Diagnostic Criteria for aMCI
Classification criteria outlined in Petersen (Reference Petersen2004) and Albert et al (Reference Albert, DeKosky, Dickson, Dubois, Feldman, Fox and Phelps2011) were adopted. Specifically, participants were required to have a self-reported change in cognitive functioning that was corroborated by an informant. As determined by interview, the memory decline had to be gradual, stable, and at least six months in duration. Participants were also required to have normal general cognitive status for age and education based on performance on the diagnostic cognitive battery relative to both age and education corrected normative data (three participants were removed for having decline across most cognitive domains). Participants were required to have intact basic activities of daily living and generally intact IADL as assessed by consensus clinical judgment of self and informant-rated questionnaires (Lawton & Brody, Reference Lawton and Brody1969) and interview. Although no strict cutoff score was used, individuals could not show complete dependence on others for non-physical purposes in one of their IADLs (e.g., no longer able to manage finances), or partial dependence on others for more than one IADL (e.g., mild restriction in navigating transportation and medication management). Two participants were excluded for no longer having intact IADL. Memory impairment was defined as obtaining scores on at least two of the three memory measures [Hopkins Verbal Learning Test-Revised (Brandt & Benedict, Reference Brandt and Benedict2001), Wechsler Memory Scale-Revised Logical Memory I and II (Wechsler, Reference Wechsler1987), and Rey Osterrieth Complex Figure Test immediate incidental recall (Bylsma, Carison, Schretlen, Zonderman, & Resnick, Reference Bylsma, Carison, Schretlen, Zonderman and Resnick1997)] that were considerably lower than expected based on age, education, and estimated verbal IQ (Wechsler Assessment of Intelligence Scale-III Vocabulary subtest; Wechsler, Reference Wechsler1997). To capture the neuropsychological pattern of “forgetting” that is often found in aMCI, the memory decrements could not be better explained by difficulty with learning in the case of the verbal measures, or solely on the basis of difficulty with effortful retrieval in consideration of the discrepancy between the free recall and recognition trial of list learning. A specific cutoff score was not implemented to accommodate intra-individual variability and clinical judgment (Petersen, Reference Petersen2004). On average, participants in the aMCI group fell more than 2 SD below their mean estimated verbal IQ on the indices of memory (see Table 1). In addition, the aMCI group scored significantly lower on the memory measures than the control group. Four participants were excluded for demonstrating impairment on only one out of the three memory measures, and an additional participant was excluded for showing a non-amnestic cognitive profile.
By definition, all aMCI participants showed impairment in the memory domain. Ten of these participants did not show impairment in any other domain (i.e., had single-domain aMCI) and 14 had additional mild impairments in either visuospatial, language, or executive domains (i.e., had multiple-domain aMCI). Because no a priori hypotheses were made about the role of additional areas of cognitive decline in aMCI, the single- and multiple-domain aMCI groups were not examined separately consistent with the approach of others (e.g., Goldberg et al., Reference Goldberg, Koppel, Keehlisen, Christen, Dreses-Werringloer, Conejero-Goldberg and Davies2010).
Materials
Performance-based IADL measure
The NA tasks have been described in detail elsewhere (Gold et al., Reference Gold, Park, Troyer and Murphy2015; Park et al., Reference Park, Lombardi, Gold, Tarita-Nistor, Gravely, Roy and Black2012). Briefly, the NA tasks consisted of preparing a sandwich, making coffee using a drip filter machine, and preparing a card to be mailed. Each NA had an average of 60 discrete actions required to complete the task. Each NA had 6 target items and 3 distracter items; the distracters were similar to a target item in either physical appearance (a spatula instead of a fork) or function (tape for a stapler).
Experimental cognitive measures
The experimental cognitive measures used in the regression analyses were distinct from the diagnostic cognitive battery, and consisted of composite measures of episodic memory, semantic knowledge, and executive function (see Table 2), similar to the approach used by Bangen and colleagues (Reference Bangen, Jak, Schiehser, Delano-Wood, Tuminello, Han and Bondi2010). Composite scores for the experimental cognitive measures were calculated by summing the raw scores on the constituent tests and deriving the average standardized score within each domain.
Descriptives of experimental cognitive measures
Procedure
The relevant institutional review boards approved this study and all participants provided informed consent. Participants were tested individually either in a quiet testing room at Baycrest Health Sciences or in their home, based on their preference. Participants completed a familiarity questionnaire (Park et al., Reference Park, Lombardi, Gold, Tarita-Nistor, Gravely, Roy and Black2012) to ensure that the NAs were performed weekly and at least 48 times per year. The two most familiar NAs were selected for each participant because we were interested in the enactment of highly familiar activities. All participants met criteria for completing the task of preparing a card (n=48), while there was a relatively even split in the natural frequency of preparing coffee (n=22) or a sandwich (n=26). No differences were found in performance by task, or the interaction of group by task.
Scoring and Treatment of Data
Details about stimulus development and inter-rater reliability have been published previously (e.g., Park et al., Reference Park, Lombardi, Gold, Tarita-Nistor, Gravely, Roy and Black2012). Briefly, NA performance was measured using an adaptation of the Action Coding System (Schwartz, Reed, Montgomery, Palmer, & Mayer, Reference Schwartz, Reed, Montgomery, Palmer and Mayer1991) that classifies detailed types of errors (omission, reversal, perseveration, object substitution, gesture substitution, grasp-spatial mis-estimation, tool omission, quality, and action addition). These error types comprised the broader measure of the rates of omission (any action that was not attempted) and commission (any error committed excluding omissions and action additions) that were the dependent measures used in this study. The total omission or commission rate represents the total number of omission or commission errors divided by the total number of actions in either of the two NA protocols that an individual performed (ratios are used to control for the different number of actions in the NA tasks that an individual could enact in error). Thus, an omission rate of .08 would indicate that a participant omitted an average of 8% of the total possible correct actions across the two NAs enacted.
Statistical Approach
To facilitate interpretation of regression analyses, predictor measures were negatively coded so that lower scores were indicative of better performance on the experimental cognitive variables or the NA variables (omission and commission rate). These regressors were entered simultaneously in a block of a binary logistic regression to predict group membership (aMCI or control). The classification threshold of predicted probability in the target group (aMCI) was set at 0.5 (Meyers, Gamst, & Guarino, Reference Meyers, Gamst and Guarino2013). The classification success rates and the values for sensitivity (true positive) and specificity (true negative) did not warrant adjustment of the classification threshold. Nagelkerke’s pseudo R 2 was used to estimate the amount of variability in group membership that was accounted for by the predictor variables (Meyers et al., Reference Meyers, Gamst and Guarino2013). Beta weights were assessed with the Wald statistic to evaluate the significance of individual predictor variables. These variables were regressed on group membership (aMCI vs. control). Bonferroni correction was used to adjust for multiple pairwise comparisons.
Results
The groups were matched on demographic variables, and did not differ statistically in age, education, or gender ratio (see Table 1). Table 3 shows that the groups differed in memory and semantic knowledge, as well as omission and commission rate. Table 4 shows the correlation matrix of the demographic variables, composite cognitive variables, and NA error rates. Unexpectedly, neither age nor education was reliably associated with the composite cognitive variables or NA performance. This finding may have been attributable to intrinsic characteristics of our sample and inadequate variability in our demographic variables.
Regressor variables used in the prediction of group membership
Note. aMCI=amnestic mild cognitive impairment. Regressor variables were negatively coded as needed so that higher scores on any predictor variables were reflective of poorer performance.
* Significant difference between control and aMCI group (p<.01).
Intercorrelation matrix for demographic, naturalistic action, and experimental cognitive variables
Note. Pearson correlation values for demographic and regression predictor variables (n=48). Regressor variables were negatively coded as needed so that higher scores on any predictor variables were reflective of poorer performance *p<.005, **p<.001
To What Extent Is NA Performance Predictive of Diagnostic Status?
With omission and commission rate entered simultaneously, the model was significantly better than the null model in predicting group membership, χ2 (2, N=48)=49.68, p<.0001. The model accounted for 68% of the variability in group membership, with commission rate contributing unique variability (B=4.03; SE=1.24; p=.001), and omission rate approaching significance (B=0.94; SE=0.50; p=.059). The sensitivity of the model was.88 (95% CI=.67–.98), while the specificity was .88 (95% CI=.67–.97). The classification rates for this and all subsequent analyses are presented in Table 5.
Classifications of group membership
Note. Count data of predicted diagnostic status by NA error rate (88% classification accuracy), by experimental cognitive variables (90% classification accuracy), and both (98% classification accuracy).
Can NA Performance Predict Unique Variability in Diagnostic Status Beyond Cognitive Measures?
With the composite memory, semantic knowledge, and executive function variables entered simultaneously in a block, the model was significantly better than the null model, χ2 (3, N=48)=37.05, p<.0001. The model accounted for 72% of the variability in group membership. The memory composite contributed unique variability (B=5.78; SE=2.07; p=.005), but not the semantic (B=0.61; SE=0.92; p=.511), or executive composites (B=1.68; SE=1.04; p=.106). As anticipated, this finding only confirmed that the groups differed the most on memory. The sensitivity of the model was .88 (95% CI=.67–.97), while the specificity was .91 (95% CI=.70–.98).
With the omission and commission rate entered in the next step of the model, it added significant unique variability above and beyond the cognitive measures, χ2 (2, N=48)=19.40, p<.0001, and the null model χ2 (5, N=48)=56.44, p<.0001. The full model accounted for 92% of the variability in group membership. A similar pattern emerged for the unique predictors as in the separate models of NA errors and composite cognitive variables, although this final model was underpowered to detect the relative contribution of the NA errors. Nevertheless, the commission rate approached significance in its unique contribution to the model (B=10.43; SE=5.44; p=.055), and to a lesser extent, so did the omission rate (B=2.51; SE=2.01; p=.212). The sensitivity of the model was .96 (95% CI=.78–1.0), while the specificity was 1.0 (95% CI=.82–1.0).
Discussion
In this study, we examined the degree to which performance on familiar everyday tasks (NAs) is predictive of membership in an aMCI group and healthy older adult group. The participants with aMCI had no marked impairment in IADLs as assessed through clinical interview and informant report, including questionnaire-based assessment of IADLs to differentiate the groups, consistent with current research guidelines for the assessment of aMCI (Albert et al., Reference Albert, DeKosky, Dickson, Dubois, Feldman, Fox and Phelps2011) and our own clinical practices. Previous research indicated that even on these highly familiar NA tasks, individuals with aMCI showed a slight decrement in their performance (Gold et al., Reference Gold, Park, Troyer and Murphy2015). In particular, we found that the aMCI group had a higher omission and commission rate than control participants. Previous NA research with mixed MCI groups has also found mild decrements in NA performance relative to healthy older adults, but not as pronounced as the impairments found in dementia groups (Giovannetti et al., Reference Giovannetti, Bettcher, Brennan, Libon, Burke, Duey and Wambach2008). The pattern of findings with NAs are consistent with the convergent evidence that individuals with aMCI show subtle decrements on a wide range of IADL-type tasks (e.g., Bangen et al., Reference Bangen, Jak, Schiehser, Delano-Wood, Tuminello, Han and Bondi2010; Burton et al., Reference Burton, Strauss, Bunce, Hunter and Hultsch2009; Goldberg et al., Reference Goldberg, Koppel, Keehlisen, Christen, Dreses-Werringloer, Conejero-Goldberg and Davies2010). We wanted to explore whether or not the subtle difference in NA error rates between individuals with aMCI and control participants could facilitate differential diagnosis.
Findings from our study indicate that based on omission and commission rate alone, group membership could be predicted with 88% accuracy. This is a similar level of accuracy (90%) to what we found with composite cognitive measures that have been shown to be sensitive to the cognitive changes in aMCI, such as associative memory (Troyer et al., Reference Troyer, Murphy, Anderson, Hayman-Abello, Craik and Moscovitch2008). Furthermore, the NA errors contributed unique variability over and above cognitive measures in the prediction of group membership to yield a model with 98% accuracy. As hypothesized, the best model capitalized on both cognitive and NA error analyses, reflecting both the neuropsychological and functional changes that are found in aMCI (Albert et al., Reference Albert, DeKosky, Dickson, Dubois, Feldman, Fox and Phelps2011). Taken together, these findings indicate that NA enactment can facilitate the differential diagnosis between healthy aging and aMCI.
Decrements in IADL are a significant predictor of subsequent cognitive decline in older adults (Peres et al., Reference Peres, Helmer, Amieva, Orgogozo, Rouch, Dartigues and Barberger-Gateau2008) and those with aMCI (Reppermund et al., Reference Reppermund, Brodaty, Crawford, Kochan, Draper, Slavin and Sachdev2013), underscoring the importance of characterizing the nature of the changes in IADL enactment early in the disease process. Of the investigations that have used performance-based IADL tasks to predict current group membership, Bangen and colleagues (Reference Bangen, Jak, Schiehser, Delano-Wood, Tuminello, Han and Bondi2010) used the Managing Money and Health and Safety subtests from the Independent Living Scales (ILS; Loeb, Reference Loeb1996) to predict membership in their MCI and control group. Their findings revealed that based on these ILS subtests, they could predict group membership with 73% accuracy (95% specificity and 24% sensitivity). Managing Money was a significant unique predictor, but not the Health and Safety subtest. Relative to this investigation, NAs predicted group membership with 88% accuracy (88% specificity and 88% sensitivity). The NAs in the current study also had slightly better accuracy (74%) than an investigation that used the Everyday Problems Test (Willis, Jay, Diehl, & Marsiske, Reference Willis, Jay, Diehl and Marsiske1992) together with a questionnaire-based assessment of IADLs to predict group membership in healthy control participants and single and multiple domain MCI (Burton et al., Reference Burton, Strauss, Bunce, Hunter and Hultsch2009). The NAs also performed well relative to a similar performance-based measure of IADL (58% specificity and 88% sensitivity) in the classification of aMCI and healthy aging (Goldberg et al., Reference Goldberg, Koppel, Keehlisen, Christen, Dreses-Werringloer, Conejero-Goldberg and Davies2010). These studies used measures of IADLs that predominately assess how to respond to a health and safety concern or solve an everyday problem. In contrast, NAs address a different element of functional capacity: the detailed analysis of subtle errors during the physical enactment of a familiar task. The higher classification success we found with NAs may also be related to the extent to which NAs make contact with the processes similar to everyday tasks like using household tools and objects, planning an activity, and completing a goal-directed task in the face of distracters. NAs are related to both questionnaire-based assessments of IADLs as well as measures of functional independence (see Schwartz et al., Reference Schwartz, Segal, Veramonti, Ferraro and Buxbaum2002), and their utility in clinical and research settings may provide an important compromise between internal and external validity in the assessment of IADL. Informant-based questionnaires are the most common way of assessing IADL. Although our findings are preliminary and limited to aMCI populations, NA error analyses may provide comparable or slightly better classification accuracy than informant-based questionnaires. For instance, a recently validated informant-based questionnaire, the Everyday Cognition scale, yielded an area under the curve (AUC) of .79 (61% specificity at a sensitivity of .80) in distinguishing MCI from normal controls (Rueda et al., Reference Rueda, Lau, Saito, Harvey, Risacher and Aisen2014). In large-scale ADNI investigations, the Everyday Cognition scale classified “early” MCI from normal control groups with an AUC of .83 (71% specificity at a sensitivity of .80) and “late” MCI from normal control groups with an AUC of .88 (82% specificity at a sensitivity of .80).
Previous research has also focused on how individuals with aMCI may have greater difficulty with more complex IADLs such as banking and medication management than they do with activities such as food preparation or cleaning (e.g., Peres et al., Reference Peres, Helmer, Amieva, Orgogozo, Rouch, Dartigues and Barberger-Gateau2008; Perneczky et al., Reference Perneczky, Pohl, Sorg, Hartmann, Tosic, Grimmer and Kurz2006; Reppermund et al., Reference Reppermund, Brodaty, Crawford, Kochan, Draper, Slavin and Sachdev2013). Our tasks of preparing coffee or making a sandwich are most similar to the less complex IADLs in these investigations. Interestingly, our careful examination of these highly familiar everyday tasks revealed a mild decrement in their performance as well (Gold et al., Reference Gold, Park, Troyer and Murphy2015). Healthy older adults omitted 9% of the actions and committed errors on 5% of the actions, while individuals with aMCI omitted 16% of the actions and committed errors on 14% of the actions. These findings are consistent with the more error-prone performance that has been reported in group studies of individuals with aMCI on a wide range of IADL-type tasks (e.g., Griffith et al., Reference Griffith, Belue, Sicola, Krzywanski, Zamrini, Harrell and Marson2003; Schmitter-Edgecombe, McAlister, & Weakley, Reference Schmitter–Edgecombe, McAlister and Weakley2012; Wadley et al., Reference Wadley, Okonkwo, Crowe, Vance, Elgin, Ball and Owsley2009). We extended these findings to simpler IADL-type tasks and demonstrated that errors were indeed predictive of group membership.
What are the nature of these changes in IADLs? Others have speculated that everyday task performance in aMCI is slower, less efficient, and more error prone (Okonkwo, Wadley, Griffith, Ball, & Marson, Reference Okonkwo, Wadley, Griffith, Ball and Marson2006; Schmitter-Edgecombe et al., Reference Schmitter–Edgecombe, McAlister and Weakley2012; Wadley et al., Reference Wadley, Okonkwo, Crowe, Vance, Elgin, Ball and Owsley2009). On NA tasks, other groups have reported that individuals with MCI show a selective increase in commission errors (Giovannetti et al., Reference Giovannetti, Bettcher, Brennan, Libon, Burke, Duey and Wambach2008). We hypothesized that both errors of omission and commission would be predictive of group membership because individuals with aMCI demonstrated increases in both their rates of omissions and commissions on the NA tasks (Gold et al., Reference Gold, Park, Troyer and Murphy2015). Contrary to hypotheses, commission errors more so than omission errors were predictive of unique variability in group membership. Although further research is needed, commission errors may be a more sensitive indicator of meaningful decline in IADL.
Despite the strengths of this investigation, our study had a modest sample size compared to larger epidemiological investigations with MCI populations. Curiously, we did not find a strong relationship between age and our dependent variables. The most parsimonious explanation is that we sampled a narrow age band of older adults who showed above average performance on most of the diagnostic cognitive battery and may not be representative of the true population of older adults and those with aMCI. Furthermore, participants were also excluded if they had any health condition that could impact cognition aside from processes related to aMCI (see Gold et al., Reference Gold, Park, Troyer and Murphy2015). As such, it may be possible that we failed to find larger effects of cognitive aging because we minimized the influence of comorbid health conditions that may account for significant variability in age-related cognitive decline (Meusel et al., Reference Meusel, Kansal, Tchistiakova, Yuen, MacIntosh, Greenwood and Anderson2014). We also did not compare the classification rate of NAs to other types of IADL tasks to examine if they provide additional sensitivity over and above a separate assessment measure of IADLs. Furthermore, a longitudinal design, or the inclusion of a group with AD would have allowed us to comment on a prominent question in the aMCI literature that surrounds the nature and degree of functional restriction in IADL that separates healthy aging from aMCI, and aMCI from AD. Extending the findings to other populations, such as those with non-amnestic MCI, would be an important contribution.
Mild IADL decrements in cognitively normal adults is predictive of later conversion to aMCI (Reppermund et al., Reference Reppermund, Brodaty, Crawford, Kochan, Draper, Slavin and Sachdev2013) and dementia (Peres et al., Reference Peres, Helmer, Amieva, Orgogozo, Rouch, Dartigues and Barberger-Gateau2008). These changes in the efficient enactment of IADLs may even precede detectable changes in cognitive functioning using neuropsychological testing (Purser, Fillenbaum, Pieper, & Wallace, Reference Purser, Fillenbaum, Pieper and Wallace2005). Of importance, NAs had similar classification accuracy as the composite cognitive measures. In the clinical setting, healthcare providers are reminded of the modest relationship between cognitive measures and functional independence in the community (e.g., Reppermund et al., Reference Reppermund, Sachdev, Crawford, Kochan, Slavin, Kang and Brodaty2011). NAs are more closely related to everyday functioning than measures of attention (Schwartz et al., Reference Schwartz, Segal, Veramonti, Ferraro and Buxbaum2002), and careful examination of performance-based IADL-type tasks can facilitate the differential diagnosis between healthy aging and aMCI. In particular, the extra time required to administer and score these types of tasks may be worthwhile for detecting aMCI in clients that do well on traditional neuropsychological tests or when the cognitive pattern alone is not conclusive. Furthermore, NAs may be a useful outcome measure to evaluate the efficacy of various interventions such as memory training or pharmaceutical trials. For instance, cognitive training programs for aMCI require distinct outcome measures from both the targeted aspect of cognition (e.g., explicit memory) and classification tests (e.g., list learning) to evaluate the efficacy of an intervention and its impact on daily functioning and disease progression. Thus far, the transfer of gains from the intervention setting to everyday scenarios that require the application of a learned strategy has been one of the limitations of memory training programs (Belleville, Reference Belleville2008). Naturalistic actions are not only associated with psychometric measures of cognition and IADLs, but they are also sensitive to aMCI. Given the paramount role of functional independence in the diagnosis of aMCI and AD, an outcome measure such as naturalistic actions may serve as an ecologically valid indicator of the efficacy of interventions and their functional impact.
Acknowledgments
The authors do not have any conflicts of interest to disclose related to this manuscript. This research was conducted in partial fulfillment of the requirements for DG’s doctoral degree and was supported in part by the Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council, and an Ontario Graduate Scholarship Doctoral Award.
