Introduction
The evaluation of functional decline in activities of daily living (ADLs) is pivotal in diagnosing cognitive disorders in older persons. A loss of autonomy in ADLs is a core feature of dementia and is required to establish a diagnosis (McKhann et al., Reference McKhann2011). Moreover, it is recognized that mild changes in advanced (a-)ADLs, which covers activities that go beyond independent living and express a personal engagement, such as leisure, self-development activities, or (semi-) professional work (Reuben et al., Reference Reuben, Laliberte, Hiris and Mor1990), and instrumental (i-)ADLs, including activities essential to maintain independent living, such as cooking, shopping, and managing medication, are already experienced in the earlier stages of the disease, as in mild cognitive impairment (MCI) (Gold, Reference Gold2012). MCI is conceptualized as a transitional period between healthy cognitive aging and mild dementia (Petersen, Reference Petersen2004; Petersen et al., Reference Petersen, Caracciolo, Brayne, Gauthier, Jelic and Fratiglioni2014) and its identification is helpful for a timely diagnosis, offering opportunities for early intervention (Dubois et al., Reference Dubois, Padovani, Scheltens, Rossi and Dell'Agnello2016). Assessing ADLs is not only useful in diagnostics but recent evidence also states that disabilities – especially in a- and i-ADLs – are strong predictors of future dementia (Fauth et al., Reference Fauth, Schwartz, Tschanz, Ostbye, Corcoran and Norton2013), and contribute to a higher and faster risk for conversion from MCI to mild dementia (Anstey et al., Reference Anstey2013).
Many attempts have been made to find sensitive but easy-to-administer and inexpensive assessments to identify functional decline, and hence to allow the differentiation between MCI and mild dementia (Gold, Reference Gold2012; Marshall et al., Reference Marshall, Amariglio, Sperling and Rentz2012; Jekel et al., Reference Jekel2015). In clinical practice, everyday functioning is most commonly assessed through interviews with patients or proxies.
Report-based scales, such as the Lawton Scale (Lawton and Brody, Reference Lawton and Brody1969) and many other instruments, are usually quick and easy to administer (Jekel et al., Reference Jekel2015; Wesson et al., Reference Wesson, Cemson, Brodaty and Reppermund2016). Self-report assessment in which a patient is asked to report on everyday functioning is the most direct and simple method for gathering information (Sikkes and Rotrou, Reference Sikkes and Rotrou2014). However, for individuals with cognitive disorders, an informant-report seems to be superior as patients may be too impaired to report accurately on their ADLs and may over- or underestimate functional limitations (Marshall et al., Reference Marshall, Amariglio, Sperling and Rentz2012; Puente et al., Reference Puente, Terry, Faraco, Brown and Miller2014). Therefore, an informant-report is considered more reliable, providing a good representation of everyday performance (Marshall et al., Reference Marshall, Amariglio, Sperling and Rentz2012; Schmitter-Edgecombe et al., Reference Schmitter-Edgecombe, Parsey and Cook2014). However, informant-report scales might also be prone to error, and reporter bias has been found due to the caregiver burden and distress (Puente et al., Reference Puente, Terry, Faraco, Brown and Miller2014; Schmitter-Edgecombe et al., Reference Schmitter-Edgecombe, Parsey and Cook2014). Thus, most report-based tools have limitations for diagnostic purposes. Moreover, most tools have scoring systems with poor sensitivity in detecting mild functional impairments (Fieo et al., Reference Fieo, Austin, Starr and Deary2011) and show a lack of psychometric value (Gold, Reference Gold2012; Jekel et al., Reference Jekel2015; Wesson et al., Reference Wesson, Cemson, Brodaty and Reppermund2016).
To address the concerns related to report-based scales, performance-based measures, such as the Assessment of Motor and Process Skills (AMPS) (Fisher, Reference Fisher1999), the Large Allen's Cognitive Levels Screen-5 (Allen et al., Reference Allen, Austin, David, Earhart, McCraith and Riska-Williams2007), and the Naturalistic Action Test (NAT) (Schwartz et al., Reference Schwartz, Segal, Veramonti, Ferraro and Buxbaum2002), can be used. In performance-based evaluations, individuals are asked to perform everyday tasks while being observed by a trained rater. In this way, it is argued that performance-based scales provide a more objective measure of ADLs (Puente et al., Reference Puente, Terry, Faraco, Brown and Miller2014; Schmitter-Edgecombe et al., Reference Schmitter-Edgecombe, Parsey and Cook2014; Jekel et al., Reference Jekel2015; Wesson et al., Reference Wesson, Cemson, Brodaty and Reppermund2016). However, performance-based scales can also be influenced by the motivation and behavior of the patient, and they are time-consuming and expensive to administer (Schmitter-Edgecombe et al., Reference Schmitter-Edgecombe, Parsey and Cook2014; Jekel et al., Reference Jekel2015; Wesson et al., Reference Wesson, Cemson, Brodaty and Reppermund2016). Moreover, performance-based scales often only allow observation of a small excerpt of everyday functioning, can sometimes solely be administered by trained occupational therapists (e.g. the AMPS) (Fisher, Reference Fisher1999), and are mostly performed in artificial environments outside the daily routine of individuals (Gold, Reference Gold2012). Some of them provide limited information on psychometric properties and have ceiling effects, and the differentiation between older persons with normal cognition, MCI, and mild dementia is not sufficiently studied (Wesson et al., Reference Wesson, Cemson, Brodaty and Reppermund2016).
The advantages and limitations of report-based and performance-based measures have led to discussions regarding the most suitable method for evaluating ADLs. Currently, there is no gold standard that is time efficient and easy to administer (Reppermund et al., Reference Reppermund2016). Only a limited number of studies have compared report-based and performance-based measures in the same sample of older persons and have found limited correlations between the two measures (Schmitter-Edgecombe et al., Reference Schmitter-Edgecombe, Parsey and Cook2014; Rycroft et al., Reference Rycroft, Giovanetti, Divers and Hulswit2017). This results in significant differences between report-based and performance-based information on ADLs (Sager et al., Reference Sager, Dunham, Schwantes, Mecum, Halverson and Harlowe1992; Bean et al., Reference Bean, Olveczky, Kiely, LaRose and Jette2011; Nielsen et al., Reference Nielsen, Kirkegaard, Ostergaard, Bovbjerg, Breinholt and Maribo2016; Roedersheimer et al., Reference Roedersheimer, Pereira, Jones, Braz, Mangipudi and Platts-Mills2016). Recently, Wesson et al. (Reference Wesson, Cemson, Brodaty and Reppermund2016) recommended the need for studies to refine the psychometric properties of existing instruments and Jekel et al. (Reference Jekel2015) stressed the need for studies comparing report-based and performance-based evaluations in the same sample to allow a valid and reliable assessment of ADLs in MCI and mild dementia.
Therefore, in this study a report-based and performance-based evaluation, both with good psychometric values for diagnosing mild cognitive disorders, were compared within the same study sample. For the report-based evaluation, this study used the recently developed and validated advanced (a-)ADL tool (De Vriendt et al., Reference De Vriendt, Gorus, Cornelis, Bautmans, Petrovic and Mets2013; De Vriendt et al., Reference De Vriendt, Mets, Petrovic and Gorus2015) and the instrumental (i-)ADL tool (Cornelis et al., Reference Cornelis, Gorus, Beyer, Bautmans and De Vriendt2017). For the performance-based evaluation, this study used the NAT, which evaluates a broad range of performance skills in the a- and i-ADLs frameworks. The NAT is feasible for administration in clinical settings (by any trained clinician and without the need for expensive materials), has been validated in dementia patients (Giovannetti et al., Reference Giovannetti, Libon, Buxbaum and Schwartz2002a), and has recently been used to discriminate MCI from mild dementia.
This study aimed to evaluate the convergent and concurrent validity of the a- and i-ADL tools and the NAT in order to compare their accuracy in making a diagnostic differentiation between cognitively healthy older persons and those with MCI and mild dementia. Based on prior research findings, we might expect to find poor to moderate correlations between the report-based and the performance-based measurement tools, wherein, the latter having the best diagnostic accuracy. However, in our previous studies (De Vriendt et al., Reference De Vriendt, Gorus, Cornelis, Bautmans, Petrovic and Mets2013; De Vriendt et al., Reference De Vriendt, Mets, Petrovic and Gorus2015; Cornelis et al., Reference Cornelis, Gorus, Beyer, Bautmans and De Vriendt2017) the a- and i-ADL tools present good to excellent ability in discriminating between HCs, MCI, and Alzheimer's disease (AD) patients; so, we hypothesized that both the a- and i-ADL tools and the NAT might have similar diagnostic power.
Methods
Participants and procedure
Three groups of community-dwelling older persons (≥65 years) were recruited consecutively through the geriatric day hospital of an academic teaching hospital (UZ Brussel, Belgium): patients with MCI, patients with mild to moderate AD, and cognitively healthy comparisons (HCs).
Patients with MCI and AD underwent a standardized multidisciplinary diagnostic procedure, including a physical and neurological examination, clinical history taking, functional evaluation of basic (b-) and i-ADLs, using the Katz index (Katz, Reference Katz1963) and the Lawton scale (Lawton and Brody, Reference Lawton and Brody1969), respectively, neuropsychological assessment, extensive laboratory blood testing, and imaging of the brain by CT or MRI scan, considered good clinical practice (Hugo and Ganguli, Reference Hugo and Ganguli2014). Diagnosis was based on the clinical consensus of the multidisciplinary team. Patients with MCI fulfilled the diagnostic criteria for a-MCI (Petersen, Reference Petersen2004). Patients with AD fulfilled the criteria of the National Institute of Neurological and Communicative Disorders and Stroke and the AD and Related Disorders Association (NINCDS-ADRDA) (McKhann, Reference McKhann1984). When the presence of major depression was presumed, this ruled out the diagnosis of MCI or AD at that stage.
HCs represented geriatric patients who visited the geriatric day hospital for diagnosis or the treatment of conditions other than cognitive disorders (e.g. osteoporosis). They were recruited separately from the diagnostic process for MCI and AD, but were evaluated using the same neuropsychological assessment and functional evaluation as for MCI and AD.
For all participants, exclusion criteria were any acute pathology, sensory, or communicative impairments that precluded them from participating in the assessments, and a history of major psychiatric illness or any pathology of the central nervous system other than MCI or AD (e.g. stroke, epilepsy). Patients with MCI or AD were not included if they were not accompanied by a reliable informant. Informants were considered reliable when they were spouses, family, or close friends who could provide accurate information about the patients’ ADL. The informant's ability to provide accurate information was operationalized by asking each person with MCI or AD if his or her informant was someone who knew them well and could provide accurate information about their daily life. For HCs, additional exclusion criteria were a score <26/30 on the Mini-Mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and McHugh1975), a score <80/105 on the Cambridge Cognitive Test (CAMCOG) (Gallagher et al., Reference Gallagher2010), or a score <18/27 on the CAMCOG memory section and a self- or informant-based history of functional or cognitive deficits suggestive of MCI or AD.
After the participants completed the diagnostic procedure, a trained occupational therapist conducted the NAT, and another trained occupational therapist conducted the a- and i-ADL tools, in which self-report was used for HCs, and an informant-report was obtained for MCI and AD. The occupational therapists were blinded to the outcomes of the other evaluations and the diagnosis. Both the NAT and the a- and i-ADL tools were always administered by the same occupational therapist on the same day. For the HCs, the a- and i-ADL tools and the NAT were administered consecutively. For MCI and AD, the a- and i-ADL tools and NAT were conducted at the same time.
The naturalistic action test
The NAT comprises three tasks of increasing difficulty, including preparing toast and coffee, wrapping a present, and packing a lunch box and a schoolbag (see Table 1). Instructions, cues, termination procedures, and the placement of objects are standardized (Schwartz et al., Reference Schwartz, Segal, Veramonti, Ferraro and Buxbaum2002). Participants are scored for accomplishing each task (i.e. the steps they perform). In addition, errors during performance are recorded and combined with the accomplishment score into a score ranging 0–6 for each task (higher scores signify better performance). The maximum total score is 18. Based on a study with a sample of persons with dementia, Sestito et al. (Reference Sestito, Schmidt, Gallo, Giovannetti and Libon2005) suggested a score of ≤13 as being in the impaired range. Recently, Bruce et al. (Reference Bruce, Ntholang, Crosby, Cunningham and Lawlor2016) recommended a cut-off point of ≥11 to differentiate MCI from mild dementia (AUC of 0.808). The inter-rater reliability of the NAT is high, both for scoring accomplishment of tasks (median weighted kappa of 0.98) and recording errors (median percentage agreement 95%) (Schwartz et al., Reference Schwartz, Segal, Veramonti, Ferraro and Buxbaum2002).
Description of the content of the NAT and the a- and i-ADL tools
The a- and i-ADL tools
The a-ADL tool (De Vriendt et al., Reference De Vriendt, Gorus, Cornelis, Bautmans, Petrovic and Mets2013; De Vriendt et al., Reference De Vriendt, Mets, Petrovic and Gorus2015) and i-ADL tool (Cornelis et al., Reference Cornelis, Gorus, Beyer, Bautmans and De Vriendt2017), both comprise semi-structured interviews that use the terminology and scoring system of the International Classification of Functioning (ICF) (World Health Organization, 2001). The a-ADL tool evaluates complex activities, such as hobbies or using technologies, whereas the i-ADL tool evaluates household activities, such as preparing a meal and doing groceries (see Table 1). These two tools only take relevant activities into account, meaning activities currently or recently (the tools suggest by thinking back to the years since retirement) performed by the individual. If activities have not been carried out during the past years because they were not relevant for an individual, they are not taken into account. There is no cut-off for the total number of relevant activities. The relevant activities are evaluated on present difficulties, based on the narratives of the patient or proxy. The scoring system adapted the performance qualifiers of the ICF, consisting of a five-point scale ranging from 0 (no difficulty to perform) to 4 (complete difficulty or unable to perform). If a score of 1 or higher is assigned, the interviewer also determines the underlying causes of limited performance, such as physical (e.g., mobility problems) and cognitive problems (e.g., memory problems). Both tools result in a set of disability indices. A “global disability index” (DI) is calculated by taking into account all relevant activities related to all the activities that are limited and the severity of the limitation, determining a “cognitive disability index” (CDI), considering only activities limited due to cognitive problems, and a “physical disability index” (PDI), considering activities limited due to physical problems. All indices are expressed as percentages, with higher scores representing greater disability. The a- and i-ADL tools both show good to excellent psychometric properties and have the ability to discriminate between cognitively healthy older persons, MCI, and mild dementia (AUC in the range 0.791–0.968) (De Vriendt et al., Reference De Vriendt, Gorus, Cornelis, Bautmans, Petrovic and Mets2013; De Vriendt et al., Reference De Vriendt, Mets, Petrovic and Gorus2015; Cornelis et al., Reference Cornelis, Gorus, Beyer, Bautmans and De Vriendt2017). De Vriendt et al. (Reference De Vriendt, Mets, Petrovic and Gorus2015) recommend a cut-off point of 47.9% for the a-ADL-DI (AUC of 0.802) and 27.4% for the a-ADL-CDI (AUC of 0.804) to differentiate MCI from mild dementia. Cornelis et al. (Reference Cornelis, Gorus, Beyer, Bautmans and De Vriendt2017) recommend a cut-off point of 44.1% for the i-ADL-DI (AUC of 0.736) and 23.6% for the i-ADL-CDI (AUC of 0.805) to differentiate MCI from mild dementia. The inter-rater reliability is excellent for the a-ADL-DI (ICC = 0.996, 95% CI (0.991–0.998)), a-ADL-CDI (ICC = 0.979, 95% CI (0.952–0.991)), i-ADL-DI (ICC = 0.986, 95% CI (0.968 ± 0.994)), and i-ADL-CDI (ICC = 0.986, 95% CI (0.969 ± 0.994)) (all p < 0.001) (De Vriendt et al., Reference De Vriendt, Gorus, Cornelis, Bautmans, Petrovic and Mets2013; Cornelis et al., Reference Cornelis, Gorus, Beyer, Bautmans and De Vriendt2017).
Statistical analysis
Statistical analyses were performed in IBM SPSS for Mac (version 22.0) (SPSS Inc., Illinois, USA), with an α-level set two sided at p < 0.05 for all analyses. There were no missing data. Since most datasets were not normally distributed (Kolmogorov–Smirnoff goodness-of-fit test p < 0.05), non-parametric tests were used. Data are reported as medians and interquartiles. The demographic and clinical characteristics, NAT scores, and indices of the a- and i-ADL tools are described. Differences between HCs, MCI, and AD were calculated using the Kruskal–Wallis test for continuous variables and the Chi-Squared test for categorical variables. Differences between HCs versus MCI, MCI versus AD, and HCs versus AD were tested with the Mann–Whitney U test. Convergent validity was assessed using correlation analyses performed with Spearman's r between the indices of the a- and i-ADL tools and the NAT total score. Convergent validity was considered poor if the correlation coefficient was below 0.30, moderate if between 0.30 and 0.59, and good if greater than 0.60 (Andresen, Reference Andresen2000). Concurrent validity was evaluated by first calculating the receiver operating characteristic (ROC) curves and the AUC for the indices of the a- and i-ADL tools and the scores of the NAT to compare the ability of the instruments to differentiate between HCs, MCI, and AD. Finally, the ROC curves and the AUC for the indices of the a- and i-ADL tools and the NAT total score were compared using the method of DeLong et al. (Reference DeLong, DeLong and Clarke-Pearson1988) in MedCalc (version 14.8.1.0) (MedCalc Software, Mariakerke, Belgium).
Results
Participants’ characteristics
The demographic and clinical characteristics of the HCs (n = 21), MCI (n = 20), and AD (n = 20) groups are shown in Table 2. There were no significant differences between the groups in terms of age, gender, education, or the Katz index. As expected, the MMSE, CAMCOG, and memory score of the CAMCOG showed significantly higher scores among HCs than among MCI and AD participants, and MCI participants compared to AD participants. The Lawton scale showed significantly higher scores for HCs and MCI participants compared to AD participants.
Participants' characteristics
HC: Healthy comparison; MCI: mild cognitive impairment; AD: Alzheimer's disease; MMSE: Mini-Mental State Examination; CAMCOG: Cambridge examination for mental disorders of the elderly, cognitive part.
b-ADL according to the Katz-Index has a minimum score of 6/24; lower scores are indicating higher autonomy; i-ADL according to the Lawton & Brody Scale has a minimum score of 9/27; lower scores are indicating lower autonomy.
aDifferences between groups tested with Kruskal–Wallis test for continuous variables and χ 2 for categorical variables.
bGroup by group tested with Mann–Whitney U test.
NAT scores
Performances on the NAT were significantly poorer for those with AD compared to MCI, and for those with MCI compared to the HCs group (see Table 3). Tasks 1 and 3 of the NAT differed significantly between the groups: persons with AD performed worse than those with MCI, and those with MCI performed worse than the HCs. Task 2 differed significantly between the HCs and those with MCI, and between the HCs group and those with AD, but there was no significant difference in performance between MCI and AD participants.
NAT scores and indices of the a- and i-ADL tools
HC: Healthy Comparison; MCI: Mild Cognitive Impairment; AD: Alzheimer's disease; NAT: Naturalistic Action Test; a-ADL-DI: advanced activities of daily living-disability index; a-ADL-CDI: advanced activities of daily living-cognitive disability index; a-ADL-PDI: advanced activities of daily living-physical disability index; i-ADL-DI: instrumental activities of daily living-disability index; i-ADL-CDI: instrumental activities of daily living-cognitive disability index; i-ADL-PDI: instrumental activities of daily living-physical disability index.
aDifferences between groups tested with Kruskal–Wallis test.
bGroup by group tested with Mann–Whitney U test.
Indices of the a- and i-ADL tools
The median performed a-ADL was 27 activities (25th percentile = 23 activities; 75th percentile = 30, 5 activities). The median performed i-ADL was 9 activities (25th percentile = 8 activities; 75th percentile = 9 activities). The indices for the a-ADL tool were derived from at least 10 activities (which was the case for 2 participants). The indices for the i-ADL tool were derived from at least 6 activities (which was the case for 4 participants). All other participants reported more activities. As illustrated in Table 3, the DI and CDI of both a- and i-ADL showed significantly more severe deficits in the AD group than MCI patients, and in the MCI patients than the HCs group. The PDI of both a- and i-ADL showed no significant differences between the groups.
Convergent validity between the a- and i-ADL tools and the NAT
The total scores of the NAT showed a significant and strong correlation with the a-ADL-DI, a-ADL-CDI, and i-ADL-CDI (r ranging from −0.634 to −0.663; all p < 0.01), and a significant moderate correlation with the i-ADL-DI (r = −0.583; p < 0.01). The PDI of a- and i-ADL did not correlate significantly with the total score of the NAT (all p >0.05).
Concurrent validity between the a- and i-ADL tools and the NAT
Table 4 presents the results of the ROC curves for the NAT, and the indices of the a- and i-ADL tool. The AUC of the NAT total score ranges from 0.809 to 1.000 but has no significantly better accuracy than the DI and CDI of both a- and i-ADL (AUC ranges from 0.739 to 0.964) for differentiating between the three groups. The PDI of a- and i-ADL did not show significantly better accuracy than the NAT.
Concurrent validity between the NAT and the a- and i-ADL tools
NAT: Naturalistic Action Test; a-ADL-DI: advanced activities of daily living-disability index; a-ADL-CDI: advanced activities of daily living-cognitive disability index; a-ADL-PDI: advanced activities of daily living-physical disability index; i-ADL-DI: instrumental activities of daily living-disability index; i-ADL-CDI: instrumental activities of daily living-cognitive disability index; i-ADL-PDI: instrumental activities of daily living-physical disability index.
ap-values indicate the significant difference between the indices and the total score of the NAT. Differences were calculated by comparing ROC curves with the method of DeLong et al. (Reference DeLong, DeLong and Clarke-Pearson1988).
Discussion
Report-based and performance-based measures have been developed to evaluate ADL in underpinning the diagnosis of cognitive disorders and to distinguish MCI from the earliest manifestations of dementia. Nevertheless, until now there has been no consensus on the most accurate measurement method. Studies comparing report-based and performance-based methods remain scarce (Jekel et al., Reference Jekel2015). In this study we examined the convergent and concurrent validity and diagnostic accuracy between the NAT – as a performance-based method – and the a- and i-ADL tools – as report-based methods – in a sample of HCs, persons with MCI, and persons with mild dementia.
First, the convergent validity was high, with strong significant correlations between the total score of the NAT and the a-ADL-DI, a-ADL-CDI, and i-ADL-CDI, and a moderate significant correlation with the i-ADL-DI. Second, the concurrent validity showed that both the NAT and the a- and i-ADL tools were able to provide an accurate diagnostic differentiation between HCs, MCI, and AD. Neither the AUC of the NAT total score nor the AUC of the a- and i-ADL tools showed significantly better accuracy. Consequently, it can be assumed that the a- and i-ADL tools and the NAT are equally able to assess everyday functioning, and that these tools have analogous accuracy in differentiating between HCs, MCI, and AD.
Until now, performance-based evaluations have been considered more likely to detect deficits in everyday functioning in persons with cognitive disorders than report-based scales (Puente et al., Reference Puente, Terry, Faraco, Brown and Miller2014). However, this study indicates that a report-based measure might be equal to a performance-based measure. This contradicts prior studies showing different estimates, or at most medium correlations between report-based and performance-based measures (Schmitter-Edgecombe et al., Reference Schmitter-Edgecombe, Parsey and Cook2014; Rycroft et al., Reference Rycroft, Giovanetti, Divers and Hulswit2017). Possibly, the strong correlation between the a- and i-ADL tools and the NAT might be explained by important differences between the a- and i-ADL tools and other report-based methods. By conducting a semi-structured interview, the a- and i-ADL tools use a different assessment approach to many other report-based methods for which the scoring is solely based on the success or failure of completing a task or on the presence or absence of a skill (Rodakowski et al., Reference Rodakowksi2014). Moreover, the a- and i-ADL tools offer the possibility of taking causes of limitations into account, which is crucial in determining the extent to which cognitive decline is responsible for functional impairment (Rockwood, Reference Rockwood2007; Wadley et al., Reference Wadley, Okonkwo, Crowe and Ross-Meadows2008; De Vriendt et al., Reference De Vriendt, Mets, Petrovic and Gorus2015). Through discriminating and evaluating reasons of limitations, the a- and i-ADL tools can detect similar limitations in performance as when observing everyday functioning using a performance-based tool. Other report-based instruments, such as the Disability Assessment in Dementia (DAD) – 6 (de Rotrou et al., Reference de Rotrou2012), the Functional Activities Questionnaire (FAQ) (Pfeffer et al., Reference Pfeffer, Kurosaki, Harrah, Chance and Filos1982), and the 18-item AD Cooperative Study/ADLs Inventory for patients with MCI (ACDC-MCI-ADL-18) (Pedrosa et al., Reference Pedrosa2010), do not take reasons of impairment into account, or focus only on executive components of activities. Also the NAT does not record other than cognitive reasons of impairment.
The a- and i-ADL tools and the NAT take different approaches in evaluating ADL, which consequently results in gathering other information on everyday functioning. The NAT records errors in performance and takes those errors into account in the total score. This provides unique and useful information that cannot be derived by interviewing an individual with the a- and i-ADL tools. On the other hand, the a- and i-ADL tools might have greater content validity by only taking those activities into account that are relevant for an individual, in contrast to the NAT. Wrapping a present and packing a lunch box and a schoolbag are activities not embedded in the usual daily routine of older persons and are especially difficult for persons with cognitive disorders (Gold, Reference Gold2012). Even individuals with MCI loose accuracy and efficiency in completing activities outside their typical routine (De Vriendt et al., Reference De Vriendt, Gorus, Cornelis, Bautmans, Petrovic and Mets2013; Seelye et al., Reference Seelye, Schmitter-Edgecombe, Cook and Crandall2013). This might explain why, in our study, task 2 of the NAT could not discriminate between MCI and AD since both groups performed equally worse on this task. In addition, knowledge of being monitored for errors in performance also increase cognitive demands on participants (Seelye et al., Reference Seelye, Schmitter-Edgecombe, Cook and Crandall2013), which might result in worse performance. Therefore, only evaluating relevant activities and listening explicitly to the subjective experiences of an individual pursues a client-centered approach not only of benefit diagnostically but also facilitates more appropriate therapy, better health outcomes, and greater patient satisfaction (Stewart et al., Reference Stewart2011).
Since this study found a similar diagnostic accuracy for the NAT and the a- and i-ADL tools, one could argue that it does not make any difference which instrument is used in clinical practice. However, choosing one or another method depends on the clinical setting and the purpose of the diagnostic procedure. As time and resources are often scarce in health services, it might be recommended to start with the feasible a- and i-ADL tools since they take the least time to administer, offer greater diagnostic accuracy than other report-based measures such as the Lawton scale (Cornelis et al., Reference Cornelis, Gorus, Beyer, Bautmans and De Vriendt2017). Subsequently, a performance-based method such as the NAT can be used. As suggested by Bruce et al. (Reference Bruce, Ntholang, Crosby, Cunningham and Lawlor2016), the NAT may be of particular interest to increase confidence in a clinical diagnosis or in situations in which there is no informant history available or there is doubt regarding the accuracy of the history. In this way, the NAT may be used in addition to the a- and i-ADL tools, and adds unique diagnostic information that might be missing when only the report-based approach is applied. By using multiple methods, the sensitivity of a diagnosis will certainly be maximized (Hunsley and Meyer, Reference Hunsley and Meyer2003).
This study has the strength of using different methods of assessment in the same sample, which mitigates the potential risk of comparing measures across different samples. Only few studies have done this. Nevertheless, it should be noted that this study also has some weaknesses. First of all, the sample is relatively small, which results in low statistical power, although the sample size is comparable to similar recent studies studying the psychometric value of the NAT (Jung et al., Reference Jung, Kim, Kang and Kim2013; Bruce et al., Reference Bruce, Ntholang, Crosby, Cunningham and Lawlor2016). Second, a measurement bias might have occurred in reporting ADLs. For HCs, only self-report was used since prior research suggested that self-report evaluations are generally accurate indicators of ADLs for cognitively healthy older persons, demonstrating insight into their functional abilities (Farias et al., Reference Farias, Mungas and Jagust2005; Suchy et al., Reference Suchy, Kraybill and Franchow2011). For patients with MCI and AD, on the other hand, a reliable informant was questioned since this is generally preferred in order to control over- or under-estimation of functional abilities. Although each informant provided accurate information about the daily life of the patients with MCI and AD, we could not objectively determine how well patients with MCI and AD were known to the informants. Furthermore, we could not rule out that the informants were not mildly cognitively impaired themselves. However a previous study (De Vriendt et al., Reference De Vriendt, Gorus, Cornelis, Bautmans, Petrovic and Mets2013) indicated a strong patient-proxy agreement of the a-ADL tool, this might still have influenced the results. Nevertheless, this resembles clinical reality in which healthcare professionals also have to work with the information that is available. Third, similar to the study of Bruce et al. (Reference Bruce, Ntholang, Crosby, Cunningham and Lawlor2016), only the overall scores of the NAT were used. We did not investigate the type of errors while performing the NAT as we especially wanted to compare the AUC of the total score with the indices of the a- and i-ADL tools. However, it might be interesting to correlate the number and type of errors in the NAT with the indices of the a- and i-ADL tools in the groups of HCs, MCI, and AD in order to investigate further an effective evaluation of functional decline in ADLs.
In conclusion, this study showed that the ability of the report-based a- and i-ADL tools and the performance-based NAT to make a diagnostic differentiation between HCs, MCI, and AD is not significantly different. Both methods have strong concurrent and convergent validity, and appear to be valid and reliable assessments of ADLs, with similar discriminatory power in diagnosing cognitive disorders in older persons.
Ethical statement
The Ethical Committee of the Universitair Ziekenhuis Brussel approved this study (B.U.N. 143201421290). All data were collected in accordance with the ICH-GCP guidance and the Helsinki Declaration. All participants and informants gave written informed consent.
Conflict of interest
None.
Description of authors’ roles
E. Cornelis designed the study, collected the data, carried out the statistical analysis, and wrote the paper. E. Gorus designed the study, was responsible for the statistical design of the study, supervised the statistical analysis, and assisted writing the paper. K. Van Weverbergh collected the data and assisted in analyzing the data. I. Beyer supervised collecting the data, designing the study, and analyzing the data. P. De Vriendt designed the study, was responsible for the statistical design of the study, supervised the statistical analysis, and assisted writing the paper.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1041610218000327
