Introduction
The concept of Mild Cognitive Impairment (MCI), a transitional state between normal aging and early dementia, has evolved from a focus on the increased risk of incident dementia of undifferentiated MCI (Petersen et al., Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999) to the development of MCI amnestic subtypes (Petersen et al., Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999), of which multiple domain–MCI (md–MCI) is the most prevalent cognitive phenotype (63.5%) in memory clinics (Rasquin et al., Reference Rasquin, Lodder, Visser, Lousberg and Verhey2005). Patients with md–MCI have a higher conversion rate to either Alzheimer's disease (AD) (Forlenza et al., Reference Forlenza, Diniz, Nunes, Memória, Yassuda and Gattaz2009) or vascular dementia (VaD) (Rasquin et al., Reference Rasquin, Lodder, Visser, Lousberg and Verhey2005). Overall, md-MCI was reported recently to have a 11.9% conversion rate as compared to 2.1% for sd-MCI and 0.3% for No Cognitive Impairment (NCI) (Ganguli et al., Reference Ganguli2011). Furthermore, patients with sd-MCI more frequently resume normal cognitive function (21%–53.4%) than md-MCI (6.3%–16.4%) (Forlenza et al., Reference Forlenza, Diniz, Nunes, Memória, Yassuda and Gattaz2009; Ganguli et al., Reference Ganguli2011). Hence, it is important to establish sensitive cognitive screening tools to detect md–MCI in patients so as to facilitate early intervention and focused clinical management.
The Mini–Mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and McHugh1975) is a widely used cognitive screening instrument for detecting dementia. However, it was reportedly poor in detecting MCI due to its inability to detect complex cognitive deficits (Nasreddine et al., Reference Nasreddine2005; Pendlebury et al., Reference Pendlebury, Cuthbertson, Welch, Mehta and Rothwell2010). Consequently, the Montreal Cognitive Assessment (MoCA) was developed to detect MCI in patients as it has more cognitively demanding items than the MMSE such as memory recall and executive function (Nasreddine et al., Reference Nasreddine2005). Several validation studies reported higher sensitivity of the MoCA in detecting MCI compared to the MMSE at memory clinics (Nasreddine et al., Reference Nasreddine2005; Smith et al., Reference Smith, Gildeh and Holmes2007; Lee et al., Reference Lee2008; Fujiwara et al., Reference Fujiwara2010; Larner, Reference Larner2011). However, the discriminant validity of both the MoCA and MMSE in detecting cognitive impairment at higher risk for dementia (i.e. md–MCI) in patients at a memory clinic has not been evaluated. Thus, we aimed to examine the discriminant validity of the MoCA and the MMSE in detecting md–MCI in patients at a memory clinic setting. We hypothesized that the MoCA is superior to the MMSE in discriminating patients with md–MCI at higher risk for incident dementia from the lower risk group (i.e. NCI and sd–MCI).
Methods
Participants
Consecutive new patients attending a memory clinic at the National University Health System in Singapore were recruited between February 2009 and April 2011. This study was approved by the Domain-Specific Review Board (DSRB) of the National Healthcare Group. Patients were excluded if they had physical conditions that could impede cognitive assessment (e.g. vision or hearing impairment) or if they had active psychiatric disease or were moderately to severely depressed [Geriatric Depression Scale (GDS) score > 10] (Sheikh and Yesavage, Reference Sheikh and Yesavage1986).
Cognitive measures
Patients were administered the MMSE, the MoCA, a formal neuropsychological battery (Yeo et al., Reference Yeo, Gabriel, Chen, Lee, Loenneker and Wong1997), and the GDS by trained research psychologists. The MoCA was modified for Singaporeans by a senior psychologist and a specialist neurologist providing dementia care. The adaptations of the test items were made in consultation with the original MoCA test developer and have been reported previously (Dong et al., Reference Dong2010). The translation and backtranslation of the Chinese and Malay versions of the MoCA were undertaken by bilingual psychologists and three equivalent versions (in English, Chinese, and Malay) of the MoCA were established.
The formal neuropsychological battery employed in this study has been validated locally for Singaporean elderly (Yeo et al., Reference Yeo, Gabriel, Chen, Lee, Loenneker and Wong1997) and covers seven cognitive domains, both memory and non-memory. The memory domains include: (1) verbal memory (word list recall and story recall); (2) visual memory (picture recall and Wechsler Memory Scale–Revised (WMS–R) visual reproduction). The non-memory domains consist of: (1) attention (digit span test, visual span test, and auditory detection test); (2) language [modified Boston naming and category fluency (animal and food)]; (3) visuomotor speed (digit cancellation, maze task, and symbol digit modalities test); (4) visuoconstruction (WMS–R subtest visual reproduction copy, clock drawing, and Wechsler Adult Intelligence Scale–Revised block design subtest); and (5) executive function [Frontal Assessment Battery (Dubois and Litvan, Reference Dubois and Litvan2000)]. The education-adjusted cutoff of 1.5 standard deviations below established norms was used for each individual subtest.
Clinical diagnosis
Consensus diagnoses of participants were established by a multidisciplinary team of neurologists, psychologists, and research personnel upon the evaluation of neuroimaging, standard laboratory blood tests, cognitive performance (excluding MoCA and MMSE scores), and clinical assessments. Patients diagnosed with NCI had subjective complaints of cognitive impairment without objective evidence of impairment determined by the formal neuropsychological battery. Standard clinical diagnostic criteria, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) (The American Psychiatric Association, 1994), and the revised Petersen's criteria (Petersen, Reference Petersen2004) were used for diagnosis of dementia and MCI, respectively. Using Petersen's diagnostic algorithm (Petersen, Reference Petersen2004), the MCI group was further categorized into four different subtypes: amnestic MCI–multiple domain, amnestic MCI–single domain, non-amnestic MCI–multiple domain, and non-amnestic MCI–single domain. The sd–MCI group included amnestic MCI–single domain and non-amnestic MCI–single domain. Likewise, the md–MCI consisted of amnestic MCI–multiple domain and non-amnestic MCI–multiple domain. As both NCI and sd-MCI are at lower risk of incident dementia compared to md-MCI (NCI: 0.3% and sd-MCI: 2.1% vs md-MCI: 11.9%) (Ganguli et al., Reference Ganguli2011), we combined both NCI and sd-MCI as the lower risk group for dichotomization purpose in comparison with the higher risk group, md-MCI.
Statistical analyses
Between-group comparisons were conducted using independent-sample t tests or One-way Analysis of Variance (ANOVA) for quantitative variables (age, years of education, and MMSE and MoCA scores) and Pearson χ2 tests for categorical variables (gender and ethnicity). A receiver operating characteristic (ROC) curve analysis with area under the curve (AUC) was used to determine the discriminant indices [sensitivities, specificities, negative predictive value (NPV), positive predictive value (PPV)] of the MoCA and MMSE for the cognitive status groups. The optimal cutoffs for md–MCI were established using the Youden Index (calculated by sensitivity + specificity −1). In addition, the discriminatory ability of the MoCA and MMSE was compared statistically (Hanley and McNeil, Reference Hanley and McNeil1983). The internal reliability and convergent validity of the MoCA were also examined. All statistical analyses were performed using Statistical Package for the Social Sciences, Version 16.0 (SPSS Inc. Chicago, Illinois, US).
Results
Subject characteristics
The flow diagram of study participation as recommended by the Standards for Reporting of Diagnostic Accuracy (STARD) guidelines (Bossuyt et al., Reference Bossuyt2003) is shown in Figure 1. A total of 310 new patients attended the memory clinic and were assessed. Major reasons for exclusion, in order of frequency, were as follows: (1) incomplete investigation (e.g. incomplete neuropsychological assessment or neuroimaging evaluation; n = 60); (2) physical conditions impeding cognitive assessment (visual impairment, n = 12; hearing impairment, n = 1); (3) moderately and severely depressed patients (GDS > 10; n = 7).
Flow diagram of participation.
*MoCA ≤19; MMSE ≤23
*MoCA ≥20; MMSE ≥24
Note: NCI = No Cognitive Impairment; sd–MCI = single domain–Mild Cognitive Impairment; md–MCI = multiple domain–Mild Cognitive Impairment; MoCA = Montreal Cognitive Assessment; MMSE = Mini-Mental State Examination; Reference test: neuropsychological battery; Target condition: md–MCI, n = 36.
The final sample size for analysis consisted of 230 patients. The majority were females (n = 127, 55.2%) and mainly Chinese (n = 189, 82.2%), followed by Malays (n = 20, 8.7%), Indians (n = 15, 6.5%), and other ethnicities (n = 6, 2.6%). The patients had a mean age of 72.7 ± 10.2 years, a mean of 6.1 ± 4.8 years of formal education, and no significant depression (GDS scores 4.5±2.9). Of these 230 patients, 136 (59.1%) were diagnosed with dementia by DSM-IV criteria, 61 (26.5%) were judged to have MCI according to Petersen's criteria, and 33 (14.3%) were given the consensus diagnosis of NCI. Among patients with MCI, higher proportion of them are md-MCI (n = 36, 59%) than sd-MCI (n = 25, 41%).
As shown in Table 1, patients with dementia were significantly older, with a lower level of education than MCI and NCI (age: F (2, 227) = 33.3, p < 0.001; education: F (2, 227) = 17.6, p < 0.001). There were no significant differences in gender and ethnic group distributions across diagnostic groups, χ2 (2, n = 230) = 0.32, p = 0.85 for gender, and χ2 (6, n = 230) = 11.2, p = 0.08 for ethnicity. There were significant differences in the mean MMSE and MoCA scores among patients with dementia, MCI and NCI [MMSE: F (2, 227) = 91.8, p < .001; MoCA: F (2, 227) = 120.6, p < 0.001]. Among the subtypes of MCI, patients with md–MCI were significantly older, with less education, as well as lower MMSE and MoCA scores than patients at lower risk for incident dementia (NCI and sd–MCI) [age: t (92) = –2.8, p = 0.006; education: t (92) = 2.9, p = 0.005; MMSE: t (92) = 6.2, p < 0.001; MoCA: t (92) = 9.6, p < 0.001].
Population characteristics according to the diagnostic groups
NCI = No Cognitive Impairment; sd–MCI = single domain–Mild Cognitive Impairment; md–MCI = multiple domain–Mild Cognitive Impairment; MoCA = Montreal Cognitive Assessment; MMSE = Mini-Mental State Examination.
* ANOVA significant at p < 0.001 across NCI, MCI, and Dementia.
‡ Post-hoc Tukey significantly different from NCI (p < 0.05).
§ Post-hoc Tukey significantly different from MCI (p < 0.05).
# Significantly different from multiple domain–MCI (p < 0.05).
Reliability and convergent validity
The internal consistency of the MoCA scale was high at α = 0.96. The total MMSE and total MoCA scores had a high positive correlation at r (230) = 0.93, p < 0.01, indicating good convergent validity.
Comparison of the discriminant validity of the MMSE and MoCA
The MoCA had significantly larger AUC than the MMSE in differentiating md–MCI from the lower risk group for incident dementia (NCI and sd–MCI) [MoCA 0.92 (95% CI, 0.86–0.98) vs. MMSE 0.84 (95% CI, 0.75–0.92), p = 0.02]. Moreover, the MoCA also had significantly larger AUC than the MMSE in differentiating md-MCI from sd–MCI [MoCA 0.88 (95% CI, 0.79–0.96) vs. MMSE 0.78 (95% CI, 0.66–0.89), p = 0.03]. In addition, there was a trend toward a larger AUC for the MoCA compared to the MMSE in differentiating md-MCI from NCI [MMSE 0.88 (95% CI, 0.80–0.96) vs. MoCA 0.95 (95% CI, 0.91–1.00), p = 0.054]. However, there was no significant difference in AUCs between MoCA and MMSE for other comparisons.
Detection of md–MCI
The MoCA demonstrated superior discriminatory ability compared to the MMSE at their optimal cutoffs in classifying NCI (50 vs. 48) and the target condition group, md-MCI (30 vs. 26) (Figure 1). The discriminant indices of the MoCA and MMSE at different cut-off values for the detection of md-MCI group are shown in Table 2. The optimal cut-off points were 19/20 for MoCA and 23/24 for MMSE in differentiating the higher risk group (md–MCI) for incident dementia from the lower risk group (NCI and sd–MCI) [sensitivity: 0.83 vs. 0.72; specificity: 0.86 vs. 0.83; PPV: 0.79 vs. 0.72; NPV: 0.89 vs. 0.83; correctly classified: 85.1% vs. 78.7%].
Discriminant indices of the MMSE and MoCA cut-off scores in discriminating md–MCI patients from NCI and sd–MCI
MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; Sen = Sensitivity; Spec = Specificity; PPV = Positive Predictive Value; NPV = Negative Predictive Value; NCI = No Cognitive Impairment; MCI = Mild Cognitive Impairment; sd–MCI = single domain–Mild Cognitive Impairment; md–MCI = multiple domain–Mild Cognitive Impairment.
Moreover, the MoCA (19/20) also demonstrated superior discriminatory ability compared to the MMSE (23/24) in detecting md–MCI from sd–MCI [sensitivity: 0.83 vs. 0.72; specificity: 0.72 vs. 0.68; PPV: 0.81 vs. 0.77; NPV: 0.75 vs. 0.63; correctly classified: 78.7% vs. 70.5%].
Discussion
The principal finding of this study is that the MoCA is superior to the MMSE in discriminating those at higher risk for incident dementia (md–MCI) from the lower risk group (NCI and sd–MCI). This is the first study comparing the discriminant validity of the MoCA and MMSE and reporting their optimal cut-off points for detecting md–MCI [predominantly (97%) of the amnestic MCI–multiple domain subtype] in a memory clinic setting. We focused on establishing the discriminant validity of the MoCA in detecting md–MCI in patients in particular, as md–MCI is the most common (85%) cognitive phenotype immediately prior to dementia conversion, while MCI with milder deficits such as sd–MCI usually demonstrates a heterogeneous pattern of transition: decline or partial-to-full recovery to normal cognitive state (Forlenza et al., Reference Forlenza, Diniz, Nunes, Memória, Yassuda and Gattaz2009).
The majority (75%) of patients with md–MCI had impairment in at least four cognitive domains, of which the most frequently affected were visual memory (94.4%), visuomotor speed (94.4%), verbal memory (77.8%), and visuoconstruction (75%), followed by slightly more than half (55.6%) with impaired executive function, while less than a third of md–MCI patients were impaired in language and attention. Therefore, the MoCA may be more advantageous than the MMSE in screening deficits in these cognitive domains as it has more visuoexecutive function subtests (Trails B, Copy Cube, Draw a Clock) and more cognitively demanding recall items (five-word recall in MoCA vs. three-word recall in MMSE) (Dong et al., Reference Nasreddine2010).
Our findings, that both the MoCA and MMSE were of equivalent discriminatory ability in detecting undifferentiated MCI, differ from previous comparative studies (Nasreddine et al., Reference Nasreddine2005; Smith et al., Reference Smith, Gildeh and Holmes2007; Lee et al., Reference Lee2008; Fujiwara et al., Reference Fujiwara2010; Larner, Reference Larner2011). This could be attributed to the following reasons: (1) different formal neuropsychological batteries to define cognitive status: the previous studies used short neuropsychological batteries or isolated memory functioning subtests, while we employed a more comprehensive neuropsychological evaluation which covers a wider range of cognitive domains; (2) different patient population: the previous studies carefully selected patients without medical conditions such as stroke, while we have included all patients attending the memory clinic; (3) different comparison methods of the discriminant validity between the MoCA and MMSE: the previous studies did not employ inferential statistics to compare the overall AUCs of the MoCA and MMSE, as well as the AUCs at the respective cut-off points of both tests.
The similar AUCs of both screening instruments in detecting MCI and dementia in our study were consistent with that found in a study on patients with Parkinson's disease (Hoops et al., Reference Hoops2009). In addition, the equivalent and moderate sensitivity of both the MoCA and MMSE in detecting MCI and its amnestic subtype from NCI is in keeping with a recent comparison study of these two tests in detecting acute poststroke cognitive impairment (Godefroy et al., Reference Godefroy2011). This may be due to the use of a comprehensive neuropsychological battery assessing a wider range of cognitive performance in these two studies and our study.
Our finding that the MoCA can reliably detect md–MCI is comparable to a recent study where the MoCA was reported to have good sensitivity (83.3%) and suboptimal specificity (52.0%) at a cutoff point of 24 in detecting md–MCI (McLennan et al., Reference McLennan, Mathias, Brennan and Stewart2010). However, the sample size of that study was small (n = 110), with only 12 md–MCI patients. Moreover, the study involved patients attending hospital outpatient clinics for cardiovascular disease rather than memory clinic patients.
The limitations of our study include the following: (1) our results may not be generalizable to the community, as the majority of our memory clinic patients were demented, compared to a higher proportion of patients with MCI in the community setting (Graham et al., Reference Graham1997); (2) we did not examine the discriminant validity of age- and education-adjusted MoCA scores and compared it to the adjusted MMSE scores due to the small sample size of md–MCI patients (n = 36). Hence, future studies should establish the age- and education-adjusted MoCA scores in detecting md–MCI in a larger community sample.
In conclusion, the MoCA is superior to the MMSE and should be considered for the detection of patients with cognitive impairment at higher risk for incident dementia. Patients with impaired MoCA scores should receive a more comprehensive formal neuropsychological evaluation.
Conflict of interest
None.
Description of authors’ roles
Y. Dong designed the study, supervised the data collection and statistical analysis, as well as wrote the paper. W. Y. Lee collected the data and assisted with writing the article. N. A. Basri was responsible for carrying out the statistical analysis. S. L. Collinson provided inputs for the writing of the paper. R. A. Merchant assisted with data collection and provided inputs for the writing of the paper. N. Venketasubramanian assisted with data collection and provided inputs for the writing of the paper. C. L-H. Chen designed the study, provided overall supervision of study conduct, and provided inputs for critical revision.
Acknowledgments
The authors thank all participants for their involvement. They also thank the research team from the National University Health System (NUHS) Memory Ageing and Cognition Centre for data collection and the NUHS Medical Publications Support Unit, Dean's Office, Yong Loo Lin School of Medicine, National University of Singapore, for its editorial support. This study is funded by a Center Grant from the National Medical Research Council (NMRC/CG/NUHS/2010). Y. Dong was a recipient of the NMRC Fellowship Training Award during this study.
