Introduction
Mild cognitive impairment (MCI) is considered a pre-clinical stage in which an individual is at an increased risk of developing dementia (Petersen et al., Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999). MCI is further classified into subtypes based on whether the impairment includes amnestic/non-amnestic and single/multiple domains (Petersen, Reference Petersen2004). Among these subtypes, the prevalence of multiple-domain mild cognitive impairment (md-MCI), which includes both amnestic and non-amnestic md-MCI, is high with a range of 9%–21% (Lopez et al., Reference Lopez2003; Manly et al., Reference Manly, Bell-McGinty, Tang, Schupf, Stern and Mayeux2005; Das et al., Reference Das2007; Brodaty et al., Reference Brodaty2013). Moreover, md-MCI has a higher conversion rate (8%–12%) to dementia, in comparison to single-domain MCI (sd-MCI), for which a range of 2.1% to 3% has been reported (Mitchell et al., Reference Mitchell, Arnold, Dawson, Nestor and Hodges2009; Ganguli et al., Reference Ganguli2011; Brodaty et al., Reference Brodaty2013). Hence, there is a need to establish simple and effective ways to detect elderly people with md-MCI.
The most widely used cognitive screening instrument, the Mini-Mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and McHugh1975), has been reported to be insensitive in detecting MCI primarily due to its ceiling effect and inability to detect subtle cognitive impairment (Pendlebury et al., Reference Pendlebury, Cuthbertson, Welch, Mehta and Rothwell2010). The Montreal Cognitive Assessment (MoCA) was developed to better detect MCI or mild Alzheimer’ disease (AD) with additional memory, executive, attention, and abstraction tasks (Nasreddine et al., Reference Nasreddine2005). Several studies have shown that the MoCA was superior to the MMSE in detecting MCI (Lee et al., Reference Lee2008; Luis et al., Reference Luis, Keegan and Mullan2009; Fujiwara et al., Reference Fujiwara2010; Lu et al., Reference Lu2012). However, these studies did not specifically focus on the detection of md-MCI. To date, there have been only two clinic-based studies that examined the discriminant validity of cognitive screening tests in detecting md-MCI (McLennan et al., Reference McLennan, Mathias, Brennan and Stewart2010; Dong et al., Reference Dong2012). Both studies reported good sensitivity of the MoCA (0.83) in detecting md-MCI in patients attending memory clinics and cardiovascular outpatient clinics. However, the detection of md-MCI in the community using these tests has not been reported. In view of the increasing public health challenge of dementia, it is important to establish a sensitive cognitive screening instrument to detect community-dwelling elderly people with cognitive impairment at higher risk for incident dementia. Therefore, we examined the discriminant validity of the MoCA and the MMSE in detecting md-MCI in a Chinese sub-sample drawn from a population-based study.
Methods
Study population
Participants for the Epidemiology of Dementia in Singapore (EDIS) study were drawn from the population-based Singapore Chinese Eye Study (SCES) (aged 40–85 years; Lavanya et al., Reference Lavanya2009). In order to use the limited resources (funding, availability MRI, and neuropsychological assessment) in an efficient way, it was decided at the start of the EDIS study to focus on those subjects who were most likely to have some cognitive problems. Details of the EDIS study have been published previously (Hilal et al., Reference Hilal2013).
Briefly, all participants who were 60 years or older, and were screen positive on the locally validated Abbreviated Mental Test (AMT; Sahadevan et al., Reference Sahadevan, Lim, Tan and Chan2000) and/or the self-reported Progressive Forgetfulness Questionnaire (PFQ) (Chong et al., Reference Chong2006) (n = 612), were invited for additional examination including extensive neuropsychological testing and neuro-imaging. A total of 300 Chinese subjects underwent these additional examinations. This study was approved by the Singapore Eye Research Institute (SERI) and National Healthcare Group Domain-Specific Review Board (DSRB), and was conducted in conformity with the Declaration of Helsinki. Written informed consent was obtained from all participants and/or legally acceptable representatives.
Cognitive evaluation
All participants received the MoCA and MMSE independent of a comprehensive evaluation using a locally validated formal neuropsychological battery (Yeo et al., Reference Yeo, Gabriel, Chen, Lee, Loenneker and Wong1997), the Neuropsychiatric Inventory (NPI; Cummings, Reference Cummings1997), and the Geriatric Depression Scale (GDS; Sheikh and Yesavage, Reference Sheikh and Yesavage1986), Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE; Jorm, Reference Jorm1994), and AD8 (Galvin et al., Reference Galvin2005) in English or Chinese dialects according to participants’ habitual language. The MMSE is a brief dementia screening instrument which takes less than ten minutes for administration and has a total score of 30. It was initially developed for the differential diagnosis of psychiatric patients in a hospital setting (Folstein et al., Reference Folstein, Folstein and McHugh1975) and has been modified for use in Singaporeans (Feng et al., Reference Feng2012). Although it is the most widely used cognitive screening instrument in elderly people, the MMSE has been reported to be insensitive in detecting subtle cognitive impairment such as MCI due to its ceiling effect (Pendlebury et al., Reference Pendlebury, Cuthbertson, Welch, Mehta and Rothwell2010). The MoCA was developed to better detect MCI or mild AD with additional memory, executive, attention, and abstraction tasks (Nasreddine et al., Reference Nasreddine2005). It takes approximately 12 minutes for test administration (Wong et al., Reference Wong2009) and has been reported to be superior to the MMSE in detecting MCI (Lee et al., Reference Lee2008; Luis et al., Reference Luis, Keegan and Mullan2009; Fujiwara et al., Reference Fujiwara2010; Lu et al., Reference Lu2012). The MoCA was modified for Singaporeans previously in consultation with the original MoCA test developer (Dong et al., Reference Dong2010). The translation and back translation of the Chinese and Malay versions of the MoCA were undertaken by bilingual psychologists and three equivalent versions (in English, Chinese, and Malay) of the MoCA have been established. The formal neuropsychological battery assessed seven cognitive domains namely, executive function, attention, language, verbal memory, visual memory, visuoconstruction, and visuomotor speed with the detailed subtests described previously (Dong et al., Reference Dong2012).
Consensus diagnosis
A multidisciplinary team of neurologists, psychologists, and research personnel established consensus diagnoses of all participants using all available data including neuro-imaging, blood tests, cognitive performance (excluding MoCA and the MMSE scores), and clinical assessments. Diagnosis of dementia and MCI was made according to standard clinical diagnostic criteria, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM–IV; The American Psychiatric Association, 1994) and the revised Petersen's criteria (Petersen, Reference Petersen2004), respectively. Using Petersen's diagnostic algorithm (Petersen, Reference Petersen2004), the MCI group was further categorized into sd-MCI and md-MCI (both amnestic and non-amnestic subtype). Finally, diagnosis of no cognitive impairment (NCI) was made based on: (1) absence of subjective cognitive complaint (n = 34) at formal cognitive evaluation using the AD8 or IQCODE; (2) absence of objective evidence of cognitive impairment defined by the formal neuropsychological battery.
Statistical analyses
Between-group differences were examined using independent sample t-test for quantitative variables and Pearson's χ2 test for categorical variables. The influence of demographic variables on performance in the MoCA and the MMSE was examined using the Pearson correlations. Receiver operating characteristic (ROC) curve analysis with areas under the curve (AUC) was employed to compare the discriminant validity of the MoCA and the MMSE. Optimal cut-off points for discrimination were established based on the largest Youden Index criterion (calculated by sensitivity + specificity − 1). In addition, the discriminatory abilities of the MoCA and the MMSE for md-MCI were compared using Hanley and McNeil's method (Hanley and McNeil, Reference Hanley and McNeil1983). The reliability and convergent validity of the MoCA were also examined using Cronbach's α and Pearson correlation, respectively. All statistical analyses were performed using Statistical Package for the Social Sciences, Version 20.0 (SPSS Inc. Chicago, Illinois, US).
Results
Population characteristics
The flow diagram of study participation is shown in Figure 1. In the period from August 2010 to February 2012, a total of 300 participants were recruited in the study. The reasons for exclusion for analysis, in order of frequency, were as follows: (1) absence of subjective cognitive complaint (n = 34); (2) moderately or severely depressed participants (GDS > 10; n = 1); (3) dementia (n = 7).
Flow diagram of participation. *Montreal Cognitive Assessment (MoCA) abnormal result ≤ 19; Mini-Mental State Examination (MMSE) abnormal result ≤ 25. ^MoCA normal result ≥ 20; MMSE normal result ≥ 26. #Reference test: neuropsychological battery. §Target condition: md-MCI (n = 83).
The final sample of analysis consisted of 258 participants. Included participants had a mean age of 70.2 ± 6.1 years, 5.8 ± 5.0 years of formal education with the mean MoCA score of 20.5 ± 5.1, and the mean MMSE score of 24.9 ± 3.3. Of these participants, 128 (49.6%) were diagnosed as NCI and 130 (50.4%) as MCI. The majority of the MCI participants diagnosed were md-MCI (n = 83; 63.8%), predominantly of the amnestic subtype (n = 76; 91.6%). Important baseline characteristics of our study population were in order of frequency hypertension (58.1%), hyperlipidaemia (49.6%), and history of smoking (31.0%). Participants with md-MCI were significantly older, with less education and lower scores in MoCA and MMSE (age: t (209) = −9.7; years of education: t (209) = 8.2; MMSE: t (209) = 12.0; MoCA: t (209) = 14.9; all p's < 0.001). Besides, there were also significant differences in gender distribution (χ2 (1, n = 211) = 6.88, p = 0.009) as well as GDS scores (t (209) = −2.8, p = 0.006) between NCI and md-MCI (Table 1). Both age and years of education were significantly correlated with MoCA and MMSE scores, respectively (age: r = −0.43, p < 0.001; education: r = 0.64, p < 0.001) versus (age: r = −0.42; education: r = 0.54, p < 0.001).
Study population characteristics according to the diagnostic groups
NOTE: NCI = no cognitive impairment; md-MCI = multiple-domain mild cognitive impairment; MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; NPI = Neuropsychiatric Inventory; GDS = Geriatric Depression scale.
Comparison of the discriminant validity of the MoCA and the MMSE
Using raw scores, the AUC for MoCA was significantly larger compared to the MMSE in differentiating md-MCI from NCI (0.94 (95% CI = 0.91–0.97) vs. 0.91 (95% CI = 0.86–0.95), p = 0.04) (Figure 2). However, at the optimal cut-off points, the MoCA (19/20) was equivalent to the MMSE (25/26) (sensitivity: 0.80 vs. 0.87, specificity: 0.92 vs. 0.80; positive predictive value (PPV): 0.87 vs. 0.74; negative predictive value (NPV): 0.87 vs. 0.90, p = 0.44) in detecting md-MCI (Table 2). Additionally, we compared the discriminant validity of MoCA and MMSE scores adjusted for age and education separately. Using age-adjusted scores, the AUC for MoCA was similar to the MMSE in differentiating md-MCI from NCI (0.97 (95% CI = 0.95–0.99) vs. 0.95 (95% CI = 0.92–0.98), p = 0.16). Similarly, using education-adjusted scores, the AUC for MoCA was similar to the MMSE in differentiating md-MCI from NCI (0.95 (95% CI = 0.92–0.97) vs. 0.91 (95% CI = 0.87–0.95), p = 0.08). Both MoCA and MMSE were comparable in performance even when they are adjusted by age alone or education alone, which is consistent with our findings on the comparison of the MMSE and the MoCA using the standard method of adjustment (Wong et al., Reference Wong2009).
Discriminant indices of the MMSE and the MoCA cut-off scores in discriminating md-MCI from NCI
NOTE: MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; Sen = sensitivity; Spec = specificity; PPV = positive predictive value; NPV = negative predictive value.
*Optimal cut-off score.
Receiver operating characteristics curve analysis. Comparison of area under the receiver operating characteristics curves between the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA) in discriminating participants with multiple-domain mild cognitive impairment from those with no cognitive impairment.
Discussion
The principal finding of this study is that both MoCA and MMSE have good discriminatory validity (sensitivity and specificity ≥ 0.80 at their respective optimal cut-off points) in detecting md-MCI in a Chinese sub-sample drawn from a population-based study. Therefore, the choice of screening instrument can be made based on local availability and validation.
The novelty of this study is that we have appraised the discriminatory abilities of the MoCA and the MMSE in detecting md-MCI in an elderly sample drawn from the community. To date, there are only two studies which reported the discriminant validity of the MoCA in detecting md-MCI in clinical samples (McLennan et al., Reference McLennan, Mathias, Brennan and Stewart2010; Dong et al., Reference Dong2012). Of these two studies, only one study compared the discriminant validity of the MoCA with the MMSE in detecting md-MCI (Dong et al., Reference Dong2012). The cut-off point established in this study for the MoCA is equivalent to the memory clinic study (cut-off point: 19/20; sensitivity: 0.80 vs. 0.83; specificity: 0.92 vs. 0.86; PPV: 0.87 vs. 0.79; NPV: 0.87 vs. 0.89). By comparison, the cut-off point established for the MMSE (25/26) is two points higher than in the memory clinic study (sensitivity: 0.87 vs. 0.72; specificity: 0.80 vs. 0.83; PPV: 0.74 vs. 0.72; NPV: 0.90 vs. 0.83). In view of the similar educational level in elderly people from the community and the memory clinic (5.8 ± 5.0 years vs. 6.1 ± 4.8 years), the higher cut-offs of the MMSE for the detection of md-MCI elderly in the community could be attributed to a younger age in this sample compared to the memory clinic sample (age 70.2 ± 6.1 years vs. 72.7 ± 10.2 years).
However, our MoCA cut-offs are four points lower than the previous study of a cardiovascular disease population (McLennan et al., Reference McLennan, Mathias, Brennan and Stewart2010) (cut-off point: 19/20 vs. 23/24; sensitivity: 0.80 vs. 0.83; specificity: 0.92 vs. 0.52). These differences could be explained by the differences in age and education in the study populations. Our community-dwelling elderly people were older with lower level of education, while the participants from the cardiovascular disease clinic were younger and better educated (age: 70.2 ± 6.1 years vs. 67.9 ± 11.7 years; education: 5.8 ± 5.0 years vs. 10.5 ± 3.2 years). This emphasizes the need to conduct studies in different populations to determine the most appropriate cut-offs.
There are several limitations in our study. First, although drawn from a population-based Chinese study, the participants from EDIS study included in this analysis did not completely reflect the general population. Second, we did not establish age- and education-adjusted cut-off points for the MoCA and the MMSE due to the small sample size in older participants with less education (only three md-MCI participants aged ≤ 70 years with education > six years). In addition, the skewed distribution of education in participants with md-MCI compared to a normal distribution of education in NCI hindered the development of adjusted cut-off points in this sample (skewness: 1.3 vs. 0.4 in md-MCI and NCI, respectively). Future studies should establish both age- and education-adjusted cut-offs or develop population-based norms (Rossetti et al., Reference Rossetti, Lacritz, Cullum and Weiner2011). Third, we excluded one participant with moderate–severe depression due to its confounding effect (Tam and Lam, Reference Tam and Lam2012). In accordance with the internationally accepted criteria for MCI, we also excluded participants with no subjective memory complaints (Petersen, Reference Petersen2004).
In addition, we compared the discriminant validity of the MoCA and the MMSE in the Singaporean Chinese cohort only, as the recruitment in the ethnic groups of Malays and Indians is still ongoing. Ethnic non-equivalence in the mean scores of MMSE amongst three major ethnic groups (Chinese, Malays, and Indians) in Singapore has been observed, even after adjustment for educational levels (Ng et al., Reference Ng, Niti, Chiam and Kua2007). Future studies should examine whether such ethnic differences exist in the performance of the MoCA and develop ethnicity-relevant cut-off points.
In conclusion, both MoCA and MMSE had equivalent and good discriminant validity, at their optimal cut-off points, in detecting md-MCI in a sub-sample of Chinese participants drawn from a population-based study. Either screening instrument can correctly identify more than 80% Chinese elderly cohort in the community. Hence, both instruments can be implemented for screening for md-MCI in community-dwelling Chinese elderly people.
Conflict of interest
None.
Description of authors’ roles
Y. Dong designed the study, collected the data, supervised statistical analysis and interpreted data, and drafted the paper. W. Y. Lee collected the data, carried out statistical analysis, interpreted data, and drafted the paper; S. Hilal collected the data and revised the paper. M. Saini collected the data and revised the paper. T. Y. Wong was responsible for the study design and concept, and revising the paper. C. L. Chen was responsible for the study design and concept, obtain funding, supervision, and critical revision of the paper. N.Venketasubramanian was responsible for the study design and concept, obtain funding, supervision, and revising the paper. M. K. Ikram was responsible for the study design and concept, supervision, and critical revision of the paper.
Acknowledgments
The authors thank all study participants for their participation. They also thank the research team from the National University Health System (NUHS) Memory Ageing and Cognition Centre for data collection. This study was funded by the National Medical Research Council, grant no. R-184-006-184-511. Y. Dong has received research support by a NMRC fellowship training award.
