Introduction
MCI is one of the most important clinical risk factors for dementia, which, in turn, is one of the most common neuropsychiatric disorders of the elderly (Petersen, Reference Petersen2011). The diagnosis of MCI and its differentiation from dementia is frequently challenging in clinical practice (Albert et al., Reference Albert2011; Petersen, Reference Petersen2011). The well-known MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975) lacks sensitivity to detect minor cognitive changes and the use of more complex psychometric batteries (i.e. the CAMCOG, ADAS-Cog, etc.) requires more time and specific training. Also, the gold standard neuropsychological evaluation, which should ideally subsidize the diagnosis of MCI, is costly and not widely available.
Since 2005, the MoCA has become a popular brief cognitive instrument in both clinical and research environments (Nasreddine et al., Reference Nasreddine2005), as it has a high accuracy to detect MCI (Ismail et al., Reference Ismail, Rajji and Shulman2010). Its popularity is due to factors such as its easy and fast application and the capacity to assess several cognitive functions. When compared to the MMSE, the MoCA presents a higher episodic memory demand (five words instead of three) and it assesses more aspects of executive functioning (clock drawing, verbal fluency, trail B). It is still not well understood how the MoCA subtests correlate to specific neuropsychological tests (Lam et al., Reference Lam2013; Moafmashhadi and Koski, Reference Moafmashhadi and Koski2013).
The MoCA test also has some limitations. Different cut-off scores have been proposed to adjust the test for different populations, age groups, and educational background in order to maintain its discriminative power between MCI participants and those with dementia or normal cognition (Smith et al., Reference Smith, Gildeh and Holmes2007; Luis et al., Reference Luis, Keegan and Mullan2009; Coen et al., Reference Coen, Cahill and Lawlor2011). Additionally, the test by itself cannot completely discriminate between healthy participants and those with cognitive impairment in the absence of clinical or scale-based functional evaluations (Nasreddine et al., Reference Nasreddine2005; Coen et al., Reference Coen, Cahill and Lawlor2011). Finally and most important, its subtests have not been extensively evaluated to explore its diagnostic performance to detect MCI independently from functional performance (Damian et al., Reference Damian2011; Rossetti et al., Reference Rossetti, Lacritz, Cullum and Weiner2011). In other words, it is unclear to this date whether all the MoCA subtests are needed to identify cognitive impairment and if there are redundancies that could cost time and human resources in the context of dementia screening.
Given the importance of the MoCA test for the diagnostic process of MCI and dementia, this study aimed to evaluate which MoCA subtests can best discriminate participants with intact cognition from those with MCI and AD. Our second goal was to test the diagnostic accuracy of a reduced version of the MoCA including the subtests that yielded the most significant diagnostic discrimination power among the three groups.
Methods
A cross-sectional study was conducted between January 2010 and December 2013 at the outpatient facility of the Geriatrics Division of the Faculty of Medicine of Jundiaí. The city of Jundiaí, state of São Paulo, is located at the Southeastern region of Brazil. This city has 397.965 inhabitants, and is ranked as forth in human development and as eighth in economy according to the State of São Paulo Human Development Index. A total of 136 community-dwelling elderly participants were recruited. After a thorough assessment, participants were classified into three diagnostic groups: HC group (HC, n = 39), MCI (n = 44), and mild AD (AD, n = 53). The sample involved participants from both sexes, 60 years or older and at least five years of formal education, except for one participant of the MCI group with four years of education. Both groups had similar monthly income varying around 982 US dollars (equivalent to three minimum wages in Brazil).
HC participants fulfilled the following inclusion criteria: sought medical attention due to general geriatric care; did not have any memory or behavioral complaints; no previous or present diagnosis of neuropsychiatric disease; and were totally independent in both basic and instrumental activities of daily living. AD participants were diagnosed following the DSM-IV for dementia (American Psychiatric Association, 1994) and the NIA-AA criteria for AD (McKhann et al., Reference McKhann2011). MCI diagnostic procedure was in agreement with those recommended by the NIA-AA (Albert et al., Reference Albert2011). The study included only amnestic MCI participants. We excluded AD participants with moderate and severe cognitive decline (CDR 2 and 3, respectively). We also excluded participants with major depression, any limb plegia or paresis, important tremor, functional impairment in hands, severe visual or auditory impairment, and patients who refused to complete any of the tests or did not signed an informed consent. All procedures were done in accord with the ethical standards of the Committee on Human Experimentation of the Institution and in accordance with the Helsinki Declaration. The study design and procedures were designed and written by the authors and were approved by the local institutional ethical committee (identified at the Institution as CEP 54/11).
Procedures
Each participant underwent a comprehensive clinical, neurological, and psychiatric evaluation. Caregivers and family members were also interviewed to provide further information regarding participants’ cognitive and functional status. Laboratory (such as hemogram, renal and hepatic function, electrolytes, vitamin B12 and folate, TSH, free T4, HIV, and VDRL) and imaging exams were carried out in all patients to exclude potentially treatable causes for cognitive impairment and identify other dementia etiologies other than AD. Magnetic resonance of the brain was the gold standard imaging exam. Computed tomography was performed if the participant presented with any obstacle to realize the former. Positron emission tomography was done for complimentary diagnostic purposes in selected patients of MCI and mild AD.
The neurocognitive assessment took place in a separate visit. All cognitive tests were applied in one single session, which lasted 140 minutes. Diagnoses were determined afterwards in a consensus meeting. The neuropsychologist who applied the cognitive tests did not participate in the diagnostic process and was blind to clinical information regarding the patient. The assessment encompassed the following tests: the MMSE, the MoCA, the CAMCOG, the GDS, and the FAQ. The MoCA test was not used to determine clinical diagnosis.
The MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975; Brucki et al., Reference Brucki, Nitrini, Caramelli, Bertolucci and Okamoto2003) and the CAMCOG (Roth et al., Reference Roth1986; Aprahamian et al., Reference Aprahamian, Martinelli, Cecato, Izbicki and Yassuda2011) were used to evaluate cognition and contributed to clinical diagnosis. The CAMCOG has 67 items and assesses the following cognitive functions: memory, attention, concentration, orientation, language, abstract thinking, calculation, praxis, and perception. The GDS (Yesavage et al., Reference Yesavage1983) was used to further assess depression disorder, while the FAQ (Pfeffer et al., Reference Pfeffer, Kurosaki, Harrah, Chance and Filos1982) was used to evaluate the performance in daily living activities. The MoCA test was translated into Brazilian Portuguese version and has recently been validated (Memória et al., Reference Memória, Yassuda, Nakano and Forlenza2013).
Statistical analysis
The statistical package SPSS-20 for Windows was used for data analyses. Descriptive statistical analyses were performed regarding the distribution of age, sex, and educational level. Analyses of variance (ANOVA) were performed to assess mean differences among diagnostic groups (HC, MCI, AD) for continuous data that followed normal distribution according to Shapiro–Wilk test. Post hoc analyses were based on Tukey's tests. Comparisons between groups for non-normal variables were performed using Kruskal–Wallis, with post hoc analyses via Mann–Whitney tests using Bonferroni's correction. χ 2 tests were done to assess differences in the distribution of categorical variables. Receiver operating characteristic (ROC) analyses were performed to assess sensitivity, specificity, and areas under the curve (AUCs). DeLong's test was used to access significant statistical differences between the AUCs for the different versions of the MoCA that were tested. In order to organize the reduced versions of the MoCA, we performed a backward conditional logistic regression analysis with all MoCA items used as independent variables and diagnosis (MCI vs. AD; MCI vs. HC) as a dependent variable. The probability to be removed from the model was set at 0.1. The variables kept in the model were used to compose the reduced versions of the MoCA.
Results
The sample consisted of 136 participants classified according to clinical diagnosis in three groups: 53 with AD (38.9%), 44 with MCI (32.3%), and 39 HC (28.6%). There was a pre-dominance of women (n = 89, 65% of the sample) and the mean age was 75.74 years ± 7.38 (60–92 years). AD and MCI participants were significantly older than the HC (Table 1). The majority of the sample had nine years of formal education (80 participants or 58.8%). There were no statistical differences regarding sex or education between the groups (Table 1).
Characteristics of the sample regarding age, sex, and education
Note: *Analysis of variance (All Tukey post hoc analysis resulted in p < 0.001, except: MCI vs. AD: p = 0.639). **χ 2 Test.
As reported in Table 2, participants with AD had lower cognitive and functional scores than participants with MCI and HC, and MCI participants had lower parameters than the HC. As to the GDS, the groups had equivalent scores.
Median and inter-quartile scores for the MMSE, CAMCOG, GDS, FAQ of the three groups
Note: *Kruskal–Wallis Test. (All post hoc Mann–Whitney tests for significant p-values corrected by Bonferroni resulted in p < 0.001). All groups were different when p < 0.001.
The groups presented significantly different median scores in the MoCA (Table 3). The cube drawing, lion, and camel naming, and letters did not differentiate the groups. All the remaining subtests (except for the contour of the clock) helped to discriminate between the AD and HC groups. The following tests were able to discriminate between AD and MCI participants: clock numbers and hands, rhino naming, serial 7, word recall, and orientation (Table 3). Finally, word repetition, inverse digits, serial 7, phrases, verbal fluency, abstraction, and word recall discriminated between MCI and HC participants.
Total and subtest MoCA scores for the three diagnostic groups: median, inter-quartile interval, and percentages of correct answers
Note: C-Contour, C-Numbers, C-Hands = Clock Drawing contour, number and hands. Words 1st and 2nd = first and second series of word repetition respectively. I-Digits = numbers backward repetition. VF = verbal fluency. *χ 2 for binary variables; Kruskal–Wallis test for ordinal variables. All post hoc Mann–Whitney/χ 2 tests for significant p-values corrected by Bonferroni resulted in p < 0.001, except MCI (vs.) AD: Trails p = 0.071; Contour p = 0.091; Words 1st p = 1.00; Words 2nd p = 0.181; Digits p = 0.045; I-Digits p = 1.00; Phrase 1 p = 1.00; Phrase 2 p = 1.00; VF p = 0.121; VF points p = 0.094 and Abstraction p = 0.301; MCI (vs.) NC: Trails p = 0.03; Contour p = 1.00; Numbers p = 1.00; Hands p = 0.412; Rhino p = 0.241; Digits p = 0.037; and Orientation p = 0.042; AD (vs.) NC: Contour p = 0.150.
We tested the performance of a “reduced version” of the MoCA test involving the subtests that could effectively discriminate between MCI and AD groups, and between MCI and HC groups. Due to significant age differences between the groups, we ran a paired analysis for age and education before comparisons. The backward conditional logistic regression removed some of the sub-items that were significant according to Bonferroni-corrected Mann–Whitney tests. This is possible because they contain redundant information (i.e. they are highly correlated), and therefore are not jointly important in differentiating the groups. Following these statistical analyses, only the clock drawing, the rhino naming, word recall, and orientation discriminated between MCI and AD, after correction by Bonferroni. We tested the diagnostic performance of the MoCA reduced version compared with the original MoCA and the FAQ, as the latter has been found to discriminate between AD and MCI participants when they present similar MoCA scores (Table 4). MoCA reduced version showed a similar diagnostic performance when compared to the original MoCA, according to measured sensitivity, specificity, and the AUCs. The AUC of the MoCA reduced version was superior to that of the FAQ, according to DeLong's test (p = 0.025).
Receiver operating characteristic curves area, cut-off, sensitivity, specificity, and accuracy for the FAQ, MoCA, and MoCA reduced to differentiate between MCI and AD groups
Note: MoCA-R = MoCA Reduced: clock drawing, rhino, delayed recall, orientation. *DeLong's test for ROC curves yields p = 0.119 for MoCA (vs.) MoCA Reduced; p = 0.214 for MoCA (vs.) FAQ, and p = 0.025 for MoCA Reduced (vs.) FAQ.
We further tested the MoCA reduced version to discriminate MCI and HC groups (Table 5). This reduced version consisted of the rhino naming, letters, phrases 1 and 2, verbal fluency, abstraction, delayed recall, and orientation. The performance of the original and the reduced version of the MoCA were similar (p = 0.148). According to DeLong's test, the original MoCA was superior to the FAQ showing higher sensitivity and specificity (p = 0.025). The reduced version was not superior to the FAQ (p = 0.144).
Receiver operating characteristic curves area, cut-off, sensitivity, specificity, and accuracy for the MoCA and MoCA reduced to differentiate between MCI and HC groups
Note: MoCA-R = MoCA Reduced: rhino, letters, phrases 1 and 2, verbal fluency, abstraction, delayed recall, and orientation. *DeLong's test for ROC curves yields p = 0.148 for MoCA (vs.) MoCA Reduced; p = 0.025 for MoCA (vs.) FAQ, and p = 0.144 for MoCA Reduced (vs.) FAQ.
Discussion
The MoCA test is a useful instrument to detect mild impairments in cognition and it has become a common tool in the diagnostic process of both MCI and dementia. It is necessary to continue to explore the psychometric characteristics of key cognitive tests for the early diagnosis of pathological cognitive changes (Albert et al., Reference Albert2011; Petersen, Reference Petersen2011). In this study, we compared both the MoCA total score and its subtests among participants with normal cognition, MCI, and dementia of the AD type. To the best of our knowledge, this is the first study to explore the contribution of individual subtests to discriminate between diagnostic groups and to examine the accuracy of a smaller set of MoCA subtests in the diagnosis of MCI and AD. Although the total score could separate the groups, we found that the cube drawing, lion, and camel naming, and letters did not differentiate the groups. We identified a subset of tests that were most sensitive to differentiate HC from MCI (mild impairment) and MCI from AD (more severe impairment).
MoCA proposed cut-off score to identify MCI has been under intense debate (Rossetti et al., Reference Rossetti, Lacritz, Cullum and Weiner2011). The median total score of the MoCA in our sample was 23 points. This value is lower than the one observed in the original study (i.e. 26 points) (Nasreddine et al., Reference Nasreddine2005). Since this study was published, several others have questioned the best suitable score to discriminate between participants with cognitive impairment from healthy participants. Previously, the original 26 points cut-off score yielded lower specificity in a sample from a memory clinic (Smith et al., Reference Smith, Gildeh and Holmes2007). Three later studies found that a score of 23 showed higher accuracy to detect MCI (Luis et al., Reference Luis, Keegan and Mullan2009; Coen et al., Reference Coen, Cahill and Lawlor2011; Tsai et al., Reference Tsai, Lee, Wang, Shia, Nasreddine and Fuh2011). Additionally, a normative study found that a cut-off score around 23 points would be more realistic to identify MCI in a community sample (Rossetti et al., Reference Rossetti, Lacritz, Cullum and Weiner2011). It appears that even lower scores (i.e. 20 points) could generate a higher diagnostic agreement rate with neuropsychological instruments (Waldron-Perrine and Axelrod, Reference Waldron-Perrine and Axelrod2012).
In the present sample, when we tested the accuracy of the MoCA to discriminate between MCI and healthy participants using ROC curves, the best cut-off score was 24 points, with good sensitivity and specificity (92% and 82%, respectively). The accuracy found in our study was quite close to that reported in the original study (sensitivity of 90% and specificity of 87%) (Nasreddine et al., Reference Nasreddine2005). A former Brazilian study found a lower accuracy (sensitivity of 81%, specificity of 77% to detect MCI from controls) compared to our findings, with a 25-point cut-off score (Memória et al., Reference Memória, Yassuda, Nakano and Forlenza2013). The sample of this study was different from ours, as it was handled in a memory clinic and not at a general geriatric outpatient facility. It is reasonable to assume that lower MoCA cut-off scores may be more proper to be used in both general and specialty settings. As previously noticed, using lower cut-off scores appears to maximize specificity, without compromising sensitivity (Luis et al., Reference Luis, Keegan and Mullan2009). A previous study showed that applying even lower MoCA scores to detect MCI participants generated stable specificity values (Kaya et al., Reference Kaya, Aki, Can, Derle, Kibaro and Barak2014).
Several studies have shown a especial interest in finding the earliest possible signs of cognitive impairment in the elderly (Dixon et al., Reference Dixon, Garrett, Lentz, MacDonald, Strauss and Hultsch2007; Gamaldo et al., Reference Gamaldo, An, Allaire, Kitner-Triolo and Zonderman2012). A recent study found that altered attention and executive functions were early cognitive markers of impairment (Gamaldo et al., Reference Gamaldo, An, Allaire, Kitner-Triolo and Zonderman2012). In our study, we found several markers of early impairment in MCI participants. Measures of executive functions and attention, such as inverse digits, serial 7, phrases, verbal fluency, abstraction, and of episodic memory (word recall) discriminated participants with MCI from HC. MoCA subtests scores for clock numbers and hands, rhino naming, serial 7, word recall, and orientation were markers of more severe impairment as they discriminated participants with MCI from AD.
A reduced version of the MoCA was accurate to detect MCI from AD participants. The reduced MoCA presented higher specificity (87% compared to 76% of the original MoCA) and accuracy (86% compared to 81%), and was superior in diagnostic accuracy to a functional questionnaire (FAQ). This reduced version may prove to be useful when clinicians are concerned about determining dementia diagnosis as they follow-up patients with MCI. On the other hand, a second proposed reduced version was not more useful to identify persons with MCI from HC than the MoCA original version, as its diagnostic accuracy was equivalent to that observed for the FAQ and the MoCA.
This appears to be the first study to propose a reduced version for the MoCA. An alternate form of the MoCA was previously tested using two German versions of the instrument (Costa et al., Reference Costa2012). The study analyzed its correlation with the original version as well as the accuracy performance of the instruments in a memory clinic sample. The changes or adaptations made to the original MoCA version were not clearly explicated in this study. The three versions of the MoCA (original and two alternates) yielded lower specificity when using the 26-point cut-off for MCI.
Some limitations should be emphasized. Our sample consisted of community elderly participants that searched for general geriatric medical attention. Therefore, our findings could not be generalized to samples from memory clinics that could have more significant cognitive impairment and concerns. Second, we did not use standard neuropsychological tests or cerebrospinal fluid biomarkers to evaluate amnestic MCI participants. However, our sample underwent a comprehensive evaluation process followed-up with consensus meetings for diagnosis. Current MCI diagnostic criteria were followed, characterizing a realistic sample of MCI and AD participants (Albert et al., Reference Albert2011).
Conclusion
In conclusion, in some clinical samples MoCA cut-off score can be reduced to detect MCI participants without compromising the accuracy performance of the test. Some MoCA subtests can have higher predictive diagnostic value in detecting participants with cognitive impairment. However, the reduced versions of the MoCA tested in the present study did not have higher accuracy to identify MCI patients among HC or AD samples. Yet, the reduced versions proved to be as accurate as the original test, and they may be used when time constraints apply.
Conflict of interest
None.
Description of authors’ roles
All authors designed the study. J. F. Cecato and J. E. Martinelli collected the data. J. F. Cecato and I. Aprahamian wrote the paper. J. E. Martinelli and I. Aprahamian supervised and revised the data collection. R. Izbicki developed the statistical analysis. I. Aprahamian revised the paper. M. Yassuda reviewed the study and discussed the analysis.
