Introduction
Addenbrooke's Cognitive Examination (ACE) has been validated in several languages and possesses good diagnostic and psychometric properties for diagnosing Alzheimer's disease (AD) and other types of dementia (Mathuranath et al., Reference Mathuranath, Nestor, Berrios, Rakowicz and Hodges2000). The third version of this test, ACE-III, was recently developed and validated in English and in Spanish (Hsieh et al., Reference Hsieh, Schubert, Hoon, Mioshi and Hodges2013; Matías-Guiu et al., Reference Matías-Guiu2015; Matías-Guiu et al., 2016a). Each of these validation studies has reported a good diagnostic ability to discriminate between healthy people and patients with dementia.
However, we need cognitive tools that are able to detect neurodegenerative diseases, and especially AD, at earlier stages. Relatively few studies have evaluated the utility of ACE for diagnosing mild cognitive impairment (MCI) (Crawford et al., Reference Crawford, Whitnall, Robertson and Evans2012; Larner and Mitchell, Reference Larner and Mitchell2014; Menon et al., Reference Menon, Lekha, Justus, Sarma and Mathuranath2014). The ACE-III total score and scores of its five cognitive domains (attention, memory, language, verbal fluency, and visuospatial abilities) make it potentially useful for detecting neurodegenerative diseases in early stages and establishing a cognitive profile that may aid in differential diagnosis (Matías-Guiu et al., 2016b).
Our hypothesis was that ACE-III may be useful to establish a cognitive profile and for the assessment of early stages of AD. The purpose of the present study is two-fold. First, we assessed the correlation between ACE-III domains and the neuropsychological tests specific to each cognitive function in order to establish the ability of the ACE-III to detect cognitive deficit affecting specific domains. Second, we studied the diagnostic ability of the ACE-III in patients with MCI and mild AD.
Methods
Study design
We conducted a prospective cross-sectional study validating the ability of the ACE-III to detect deficits in the cognitive domains of attention, memory, language, and visuospatial abilities. This research project was approved by the ethics committee at our hospital.
Study sample
The study included 175 consecutively recruited patients who visited our center between May 2014 and November 2015 due to cognitive symptoms. Patients included in this study were different from those that participated in the previous work of validation of ACE-III in Spain (Matías-Guiu et al., Reference Matías-Guiu2015).
The inclusion criteria were being aged 50 years or older and having sufficient physical capability and auditory and visual acuity for the neuropsychological assessment.
Patients were classified in four groups according to the results from the diagnostic study:
1. Individuals with subjective memory complaints but with no objective cognitive impairment. We included patients reporting symptoms of memory loss but whose neuropsychological tests did not indicate cognitive impairment. Thus, age- and education-adjusted scores were above the percentile 11–18 (scaled score 7 or higher). Other cognitive or behavioral symptoms were not allowed in this group.
2 and 3. Patients with AD. We included all patients meeting diagnostic criteria for amnestic mild cognitive impairment (aMCI; global Clinical Dementia Rating: 0.5) (group 2) and mild AD (global Clinical Dementia Rating: 1) (group 3) (Morris, Reference Morris1993; McKhann et al., Reference McKhann2011).
4. Patients with other causes of cognitive decline (other types of dementia). This group included patients with other neurodegenerative diseases (behavioral variant of frontotemporal dementia, primary progressive aphasia, and atypical parkinsonism) (Litvan et al., Reference Litvan1996; McKeith et al., Reference McKeith2005; Gorno-Tempini et al., Reference Gorno-Tempini2011; Rascovsky et al., Reference Rascovsky2011).
Furthermore, a group of 25 healthy controls (group 5) was recruited among healthy volunteers and patients’ relatives. These were participants presenting no neurological or systemic diseases potentially affecting cognitive function, no current psychiatric disorders, and no history of abusing alcohol or other substances. Healthy controls had no cognitive and no functional impairment, according to the Mini-Mental State Examination (MMSE) adjusted for age and education > 24 (Blesa et al., Reference Blesa2001), a global Clinical Dementia Rating equal to 0, a Functional Activities Questionnaire (FAQ) of 0 (Pfeffer et al. Reference Pfeffer, Kurosaki, Harrah, Chance and Filos1982; Morris, 1993), absence of depression according to Hamilton Depression scale (Hamilton, Reference Hamilton1960), and absence of subjective memory or cognitive complaints.
Neuropsychological assessment
All patients were evaluated by two neurologists specialized in cognitive neurology and by structural neuroimaging. Neuropsychologists administered the MMSE and ACE-III (Spanish version, available from www.neura.edu.au/frontier/research/test-downloads).
Patients also completed a neuropsychological assessment protocol including several tests from NEURONORMA, a normative data project conducted in Spain (Peña-Casanova et al., Reference Peña-Casanova2009). This project is similar to Mayo's Older Americans Normative Studies (MOANS) in the USA. The tests included in our study were as follows: memory span (forward and backward digit spans), the Corsi block-tapping test, Trail Making Test (TMT), Symbol Digit Modalities Test (SDMT), Boston Naming Test (BNT), Visual Object and Space Perception Battery (VOSP; subtests for object decision, progressive silhouettes, position discrimination, and number location), Judgement of Line Orientation (JLO), Tower of London-Drexel University, Free and Cued Selective Reminding Test (FCSRT), Rey–Osterrieth Complex Figure Test (copy and memory), and Stroop Color–Word Interference Test. Following the NEURONORMA authors’ recommendations, a scaled score ≤ 5 (percentile ≤ 5) was considered deficient. We also assessed participants’ functional status using the Functional Activities Questionnaire (Pfeffer et al., Reference Pfeffer, Kurosaki, Harrah, Chance and Filos1982) and the Clinical Dementia Rating scale (Morris, Reference Morris1993). The diagnostic groups were created based on results from functional and neurological assessments and scores on the battery of tests listed above.
Statistical analysis
The statistical analysis was conducted using SPSS Statistics 20.0 by IBM® and MedCalc® 16.4.3. Kolmogorov–Smirnov test was used to determine normality. We calculated the Pearson correlation coefficient between ACE-III total and cognitive domain scores and specific neuropsychological test results. All participants were included in the correlation analyses. The α level of significance was set at 0.05; p values < 0.05 were considered statistically significant and led to rejection of the null hypothesis. These values are indicated with an asterisk (*). p values lower than 0.01 are marked with two asterisks (**).
We also estimated receiver operating characteristic curves (ROC) for discriminating between groups. The gold standard was the neuropsychological study conducted by applying the NEURONORMA test battery and comparing results to the normative data obtained by that project (Peña-Casanova et al., Reference Peña-Casanova2009). In the ROC curve analysis, the control group and the group with subjective memory complaints were combined in order to estimate the area under the curve (AUC) used to discriminate non-pathological from pathological groups, and only participants aged 65 years or more were selected to avoid differences in age between groups. The ROC curves were compared using the method proposed by DeLong et al. (Reference DeLong, DeLong and Clarke-Pearson1988).
Results
Mean age was 71.94 ± 8.67, 116 of the participants were women (58%), and the mean duration of formal schooling was 9.77 ± 5.02 years. Of all participants included in the study, 25 were healthy controls, 48 had subjective memory complaints, 47 had aMCI, 47 had AD, and 33 had other types of dementia (12 with primary progressive aphasia, 11 with the behavioral variant of frontotemporal dementia, and 10 with atypical Parkinsonism). The demographic distribution and ACE-III scores of each group are shown in Table 1. The mean scores of the other cognitive tests are provided in Table 2.
Clinical and demographic characteristics of diagnostic groups
Main neuropsychological scores
Correlation between ACE-III cognitive domains and neuropsychological tests
The ACE-III memory domain was very largely correlated with the FCSRT (r = 0.806 for total recall, p value < 0.0001 and r = 0.806 for total delayed recall, p value < 0.0001) and also correlated with recall at 30 minutes on the Rey–Osterrieth Complex Figure Test (r = 0.554, p value < 0.0001). The language domain was highly correlated with the BNT score (r = 0.744, p value < 0.0001). Visuospatial domain has a large correlation with SDMT (r = 0.549, p value < 0.0001) and VOSP-number location (r = 0.501, p < 0.0001), and a medium correlation with the copy task of the Rey–Osterrieth complex figure (r = 0.461, p value < 0.0001), VOSP-object decision (r = 0.348, p value < 0.0001), VOSP-position discrimination (r = 0.407, p value < 0.0001), and with JLO (r = 0.498, p value < 0.0001). The correlation between ACE-III visuospatial and VOSP-progressive silhouettes was small (r = –0.233, p value = 0.0005).
The ACE-III attention domain showed a medium correlation with the following tests: verbal span backward, visuospatial span forward, visuospatial span backward, TMT part A, TMT part B, SDMT, ROCF copy accuracy, FCSRT, ROCF memory, Stroop test, VOSP-number location, and JLO. The correlation with the Tower of London was small. None of the tests showed a large correlation with the ACE-III attention domain. Regarding the ACE-III fluency domain, it showed a large correlation with SDMT (r = 0.522, p < 0.0001), BNT (r = 0.549, p < 0.0001), FCSRT total free recall (r = 0.550, p < 0.0001), and Stroop part B (r = 0.645, p < 0.0001) and part C (r = 0.543, p < 0.0001), and it showed a medium correlation with verbal span backward, TMT part B, the other scores of the FCSRT, ROCF memory, Stroop part A, and Tower of London total moves score. Table S1 (available as supplementary material attached to the electronic version of this paper at www.journals.cambridge.org/jid_IPG) shows the correlations of ACE-III and its domains with other tests.
Furthermore, the correlation between ACE-III (total score) and MMSE was 0.820 (p < 0.0001). The ACE-III (total score) showed a large correlation mainly with BNT (r = 0.687, p < 0.0001) and with the FCSRT (especially with total recall, r = 0.694, p value < 0.0001, and delayed total recall, r = 0.691, p value < 0.0001). The correlation was lower but still large or medium with verbal span forward, verbal span backward, Corsi's test forward, Corsi's test backward, TMT part A, TMT part B, SDMT, ROCF memory at 3 and 30 minutes, Stroop test, JLO, and VOSP (Table S1).
For the memory domain, the AUC discriminating deficient scores (scaled score ≤ 5) from normal scores (scaled score ≥ 6) on the FCSRT (delayed total recall) was 0.902. The language domain AUC for detecting deficient scores on the BNT was 0.909. The AUC of the attention domain for detecting a TMT-B/TMT-A ratio of at least 3 (suggestive of executive dysfunction) was 0.667, and for detecting impairment in Tower of London (correct moves), the AUC obtained was of 0.799. The AUC of the visuospatial domain was 0.780 to detect impairment in JLO, 0.797 in ROCF (copy), 0.729 in VOSP-discrimination of position, and 0.777 in VOSP-number location.
ACE-III: discriminant ability between groups
We estimated the ROC curves for the ACE-III and domain scores that were able to discriminate between patient groups. The control group included participants from groups 1 and 5 (healthy participants and those with subjective memory complaints, respectively); participants older than 65 years were selected to limit age differences between the control group and the patient groups. In the group of patients with objective cognitive impairment suggestive of AD (aMCI and mild AD), the AUC was 0.891 for ACE-III (total score) and 0.942 for the memory domain, while the AUC for the MMSE was smaller at 0.853. Using the DeLong method, pairwise comparison of ROC curves was performed. There were significant differences between the AUC of memory domain and MMSE (p = 0.007) and ACE-III-total score (p = 0.01), but not between ACE-III-total score and MMSE (p = 0.14).
For discrimination between patients with aMCI (CDR 0.5) and healthy participants with subjective memory complaints, the AUC was 0.817 for ACE-III (total score), 0.906 for the memory domain, and 0.778 for the MMSE (Figure 1a). The optimal cut-off point for ACE-III (total score) was 73/74, with a sensitivity of 76.6% and a specificity of 75%. For the memory domain, the optimal cut-off point was 15/16, with a sensitivity of 87.2% and a specificity of 80%. There were significant differences between the AUC of memory domain and MMSE (p = 0.005) and ACE-total score (p = 0.003), but not between ACE-III (total score) and MMSE (p = 0.30). The AUC discriminating controls from patients with mild AD (CDR 1) was 0.966 for ACE-III (total score), 0.978 for the memory domain, and 0.928 for the MMSE (Figure 1b). The optimal cut-off point for ACE-III (total score) was 63/64 with a sensitivity and a specificity of 93.6% and 94.9%, respectively. For the memory domain, the optimal cut-off point was 13/14, with a sensitivity of 95.7% and a specificity of 89.7%. There were no significant differences among the AUC of ACE-III (total score), the memory domain, and MMSE, although a trend toward statistical significance was observed between the AUC of the MMSE and memory domain (p = 0.06) and between MMSE and ACE-III (total score) (p = 0.09). Regarding discrimination between patients with aMCI (CDR 0.5) and those with mild AD (CDR 1), the AUC was 0.852 for ACE-III (total score), 0.786 for the memory domain, and 0.789 for the MMSE (Figure 1c), with no differences between the AUC (p > 0.05).
(a) ROC curves for discriminating between controls and patients with aMCI (CDR 0.5) using ACE-III, ACE-III cognitive domains, and the MMSE. (b) ROC curves for discriminating between controls and patients with mild AD (CDR 1) using ACE-III, ACE-III cognitive domains, and the MMSE. (c) ROC curves for discriminating between patients with aMCI (CDR 0.5) and those with mild AD (CDR 1) using ACE-III, ACE-III cognitive domains, and the MMSE.
Continue
Continue
Last, we analyzed the tool's ability to discriminate between controls and patients with other neurodegenerative diseases. The AUC was 0.850 for ACE-III (total score), 0.816 for the memory domain, 0.870 for verbal fluency, and 0.805 for the MMSE, although there were no differences in AUC between the different scores.
Discussion
The present study indicates that ACE-III is a useful tool that can assist in the diagnosis of aMCI and mild AD. The correlation coefficients obtained between each ACE-III domain and neuropsychological tests specific to those cognitive domains suggest that ACE-III is a useful screening instrument contributing in detection of alterations in some cognitive functions.
The memory domain was highly correlated with the FCSRT; this correlation confirms the finding of Hsieh et al. (Reference Hsieh, Schubert, Hoon, Mioshi and Hodges2013) who observed a large correlation between the memory domain and the Rey Auditory Verbal Learning Test (0.59). The correlation with the FCSRT was also stronger than that observed between the Montreal Cognitive Assessment and several other memory tests (Lam et al., Reference Lam2013). The high correlation with the FCSRT is very interesting since that test has demonstrated its high sensitivity for diagnosing prodromal AD and for stratifying the risk of progression to dementia (Sarazin et al., Reference Sarazin2007). We also observed a marked correlation between the language domain and the BNT (r = 0.744), which suggests that ACE-III may be useful for cognitive disorders presenting with language impairment, whether isolated or not, such as primary progressive aphasia and AD (Leyton et al., Reference Leyton, Hornberger, Mioshi and Hodges2010; Matías-Guiu et al., Reference Matias-Guiu2014). The attention domain displayed moderate correlations with tests related to attention and executive function: Memory span, the Corsi block test, TMT, SDMT, and Stroop Color–Word Interference Test. These results are similar to those reported by Hsieh et al. (Reference Hsieh, Schubert, Hoon, Mioshi and Hodges2013) who found a correlation of 0.42 between the attention domain score and the Digit Span Test. This correlation is lower than correlations with other domains, a fact that may be due to including orientation within the attention domain as well as the multiple dimensions of the attention and the executive function. Similarly, the correlation between verbal fluency scores and executive tests was moderate, and visuospatial domain scores, VOSP, and JLO were also moderately correlated. There are several reasons that probably explain why this correlation was not stronger. The visuospatial domain involves not only visuospatial tasks, but also constructional praxis, which is influenced by visuospatial abilities as well as other factors, such as executive function. In addition, education has a significant impact on constructional praxis, which may limit the correlation between tests.
One of the most relevant finding of our study was that the ACE-III discriminates well between groups, in fact this test and specifically the memory domain showed greater discriminant ability than the MMSE. The diagnostic ability of ACE-III was also high in early stages of AD, especially in the group with aMCI. The memory domain exhibited the highest scores on the test, showing an AUC of 0.906 for diagnosis of the group with aMCI. This domain also demonstrated the greatest diagnostic ability for the group scoring 1 on the CDR and for patients with suspected AD (CDR 0.5 and 1). Memory domain scores were therefore most sensitive for diagnosis, as their sensitivity and specificity values show. However, the total ACE-III score showed the highest discriminant ability between CDR scores of 0.5 and 1, which suggests that this score is the most appropriate indicator for patient follow-up. This may be explained by the typical course of cognitive decline in AD: During the first stage (CDR 0.5), it presents memory impairment only, while in the second stage (CDR 1) other cognitive functions are affected (language, executive function, etc.). In addition, this finding supports the idea that the usefulness of certain scores or domains depends on their clinical application (screening for dementia, screening in early stages, differential diagnosis, etc.) (Matías-Guiu and Fernández-Bobadilla, Reference Matías-Guiu and Fernández-Bobadilla2014; Hsieh et al., Reference Hsieh2015).
The cut-off point of ACE-III for detecting mild AD (CDR 1) was 63/64, which is similar to the cut-off point established by a previous validation study conducted in our setting (Matías-Guiu et al., Reference Matías-Guiu2015); 73/74 is the cut-off point for aMCI. While these cut-off points are lower than those reported by other validation studies of English-language versions of ACE-R and ACE-III, they resemble those reported for populations culturally closer to our own, such as the Italian population (Pigliautile et al., Reference Pigliautile2011). Differences in cut-off points may also be linked to age and education differences between patients (Jubb and Evans, 2015). Considering the effect of these factors on test scores, normative data may be more useful than cut-off points.
In the group of patients with other neurodegenerative diseases, the verbal fluency domain and total ACE-III score were shown to have the highest discriminant ability from controls. This may have been the case because this group included patients with language impairment (primary progressive aphasia) and/or executive dysfunction (frontotemporal dementia, atypical parkinsonism) but with relatively well-preserved memory. In any case, this group is smaller and, to a certain extent, more heterogeneous; further studies are necessary to evaluate the usefulness of ACE-III in these patients, as well as in differential diagnosis of neurodegenerative diseases (Elamin et al., Reference Elamin, Holloway, Bak and Pal2015).
Our study has some limitations. First, in some analyses, we have combined the groups’ healthy controls and subjective memory loss, as well as aMCI and AD. Although it is a controversial issue, subjective memory complaints might be a very early symptom of a neurodegenerative disorders (Pedro et al., Reference Pedro, Mercedes, Ramón and Borja2016). Furthermore, not all patients with aMCI go on to develop AD. Due to the selection criteria and the comprehensive neuropsychological protocol used in this study (including FCSRT, which has been specifically recommended for the early diagnosis of AD), we think that it is unlikely that patients with subjective memory complaints included were at risk of AD, while most patients with aMCI were, in fact, at high risk of developing AD. However, further studies evaluating the role of ACE-III in the prediction of progression of aMCI and preclinical stages of AD would be very informative. Another limitation was the fact that AD patients were slightly less educated and older than the control group. To reduce the effect of this limitation, we estimated partial correlations, correcting for age and years of education as confounding variables.
In conclusion, our results suggest that ACE-III is a useful tool for early diagnosis of AD and assessment of cognitive function, and they support using ACE-III as both a screening tool and a means of describing cognitive profiles.
Conflict of interest
None.
Description of authors’ role
Jordi A. Matías-Guiu: Study concept and design, interpretation of data, statistical analysis, writing the manuscript, final approval of the manuscript. Ana Cortés-Martínez: Acquisition of data, statistical analysis, writing of the manuscript, final approval of the manuscript. María Valles-Salgado: Acquisition of data, literature search, critical revision of the manuscript for important intellectual content, final approval of the manuscript. Teresa Rognoni: Acquisition of data, literature search, final approval of the manuscript. Marta Fernández-Matarrubia: Interpretation of data, literature search, final approval of the manuscript. Teresa Moreno-Ramos: Acquisition of data, critical revision of the manuscript for important intellectual content, final approval of the manuscript. Jorge Matías-Guiu: Study concept and design, literature search, acquisition of data, interpretation of data, study supervision, final approval of the manuscript.
Funding
None.
Supplementary material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1041610216001496
