Introduction
The Montreal Cognitive Assessment (MoCA; Nasreddine et al., Reference Nasreddine2005) is a brief (10–15 minute) 30-point cognitive screening instrument which has proved useful in identifying patients with dementia and – unlike the Mini-Mental State Examination (MMSE; Folstein et al., Reference Folstein, Folstein and McHugh1975) – patients with mild cognitive impairment (MCI). MoCA is easily accessed and available in multiple languages (www.mocatest.org). As well as its utility in cognitive clinic populations (Nasreddine et al., Reference Nasreddine2005; Smith et al., Reference Smith, Gildeh and Holmes2007; Damian et al., Reference Damian2011), MoCA has also been reported to identify cognitive impairment in specific clinical conditions, including Parkinson's disease (Zadikoff et al., Reference Zadikoff2008; Gill et al., Reference Gill, Freshman, Blender and Ravina2008; Nazem et al., Reference Nazem2009), REM sleep behavior disorder (Gagnon et al., Reference Gagnon, Postuma, Joncas, Desjardins and Latreille2010), cerebrovascular disease (Wong et al., Reference Wong2009; Pendlebury et al., Reference Pendlebury, Cuthbertson, Welch, Mehta and Rothwell2010; Dong et al., Reference Dong2010), Huntington's disease (Videnovic et al., Reference Videnovic, Bernard, Fan, Jaglin, Leurgans and Shannon2010), and brain metastases (Olson et al., Reference Olson, Chhanabhai and McKenzie2008).
Few data have hitherto been reported on the use of MoCA in memory clinic populations (Nasreddine et al., Reference Nasreddine2005; Smith et al., Reference Smith, Gildeh and Holmes2007; Damian et al., Reference Damian2011), and some of these studies have selected patients based on diagnosis (Alzheimer's disease (AD,) MCI, +/− normal controls), a method alien to the workings of most clinics where patient diagnoses are unknown at the time of testing, clinical heterogeneity is high, and there are no normal controls. For this reason, pragmatic studies of newly developed cognitive screening tests in relatively unselected clinic populations are necessary. Such studies have shown that cut-offs appropriate to index studies may need to be revised in the clinic setting (Larner, Reference Larner2007; Reference Hancock and Larner2009; Hancock and Larner, Reference Hancock and Larner2011).
Combining screening tests to try to improve diagnostic utility in the memory clinic has been attempted on occasion (Flicker et al., Reference Flicker, Logiudice, Carlin and Ames1997; Hancock and Larner, Reference Hancock and Larner2009). Since MoCA is acknowledged to be a highly sensitive but not very specific test, whereas the MMSE is highly specific but not very sensitive, combining MoCA with the MMSE might be anticipated to improve overall diagnostic utility. Nasreddine et al. (Reference Nasreddine2005) advocated sequential use of the MMSE and MoCA in patients with cognitive complaints and functional impairment if MMSE score was normal (≥26/30).
A study was therefore undertaken to examine the screening utility of MoCA in an unselected cognitive disorders clinic population. The study aims were to ascertain whether MoCA was useful in the diagnosis of dementia and MCI in a memory clinic population, and whether its diagnostic utility is improved by combining MoCA with the MMSE.
Methods
Consecutive new patient referrals were recruited prospectively (September 2009–March 2011) from a Cognitive Function Clinic based in a regional neuroscience center. Standard clinical diagnostic criteria (DSM-IV) were used for the diagnosis of dementia (American Psychiatric Association, 2000) and for MCI (Petersen et al., Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999), as in previous studies (Hancock and Larner, Reference Hancock and Larner2009; Reference Hancock and Larner2011). MoCA administration was performed independent of, but on the same day as, patient clinical and neuropsychological assessment (the latter included some or all of the following elements: Wechsler Adult Intelligence Scale Revised, National Adult Reading Test, Wechsler Memory Scale-III, Graded Naming Test, Rey-Osterreith Complex Figure, Stroop color-word test, and verbal fluency tests). The MoCA result was not used in the diagnostic judgment of dementia/MCI (i.e. cognitive impairment) vs. no cognitive impairment in order to minimize review bias (Gifford and Cummings, Reference Gifford and Cummings1999). DSM-IV criteria were operationalized pragmatically as the demonstration of multiple cognitive deficits that included memory impairment, sufficiently severe to cause impairment in occupational or social functioning; Petersen MCI criteria were operationalized as the demonstration of cognitive impairment in one or more domain insufficient to mandate a diagnosis of dementia with essentially normal functional activities.
As this was a pragmatic study, patients were not selected according to diagnosis but simply as they presented to the clinic. Dementia subtypes were not specifically examined in this study.
Standard summary measures of diagnostic utility were generated (sensitivity, specificity, positive and negative predictive values (PPV, NPV), diagnostic odds ratio (DOR), likelihood ratios, ROC curve) with 95% confidence intervals (CI). Utility indices were calculated as per Mitchell et al. (Reference Mitchell, McGlinchey, Young, Chelminski and Zimmerman2009). Correlations and the test of agreement (κ statistic; Cohen, Reference Cohen1960) were calculated between MoCA and MMSE. The STARD guidelines on reporting diagnostic test accuracy were observed (Bossuyt et al., Reference Bossuyt2003).
Diagnostic utility measures were also derived for combined use of MoCA and MMSE using the cut-points (≥26/30) defined in the index study of Nasreddine et al. (Reference Nasreddine2005). Tests were applied both in series (i.e. both tests required to be positive before a diagnosis of cognitive impairment made: the “And” rule) or in parallel (either test positive sufficient for a diagnosis of cognitive impairment to be made: the “Or” rule) (Flicker et al., Reference Flicker, Logiudice, Carlin and Ames1997; Hancock and Larner, Reference Hancock and Larner2009).
As administration of MoCA and MMSE was already routine in this clinic, the study was an audit of established practice, so institutional ethical review and specific consent procedures were not indicated.
Results
One-hundred-and-fifty patients were assessed (M: F ratio = 93: 57, 62% male; age range 20–87 years, median 61 years). Of these, 36 (24%) were judged to have dementia by DSM-IV criteria, 29 (19%) had MCI by Petersen criteria, and 85 (57%) had no dementia.
MoCA proved easy to use, being completed in all cases. MoCA scores ranged from 5/30 to 30/30 (Figure 1a), whereas MMSE scores ranged from 11/30 to 30/30 (Figure 1b). There were weak negative correlations between age and MoCA score (r = −0.38; t = 4.94, df = 148, p < 0.001) and between age and MMSE score (r = −0.26; t = 3.27, df = 146, p < 0.001). MoCA and MMSE scores were highly correlated (r = 0.85; t = 19.2, df = 146, p < 0.001) but the test of diagnostic agreement was not so marked (κ = 0.39, 95% CI 0.26-0.53; where κ = 1 is perfect agreement between tests and κ = 0 is agreement purely due to chance alone).
MoCA scores vs diagnosis.
MMSE scores vs. diagnosis.
In the cognitively impaired group the mean (+/−SD) MoCA score was 18.3 +/− 4.5, and in the non-impaired group it was 25.2 +/− 3.2. The mean MoCA scores differed significantly between the two groups (t = 12.0, df = 148, p < 0.001). Mean MoCA scores in the dementia and MCI groups were 16.6 +/− 4.4 and 20.4 +/− 3.8 respectively and differed significantly between the two groups (t = 3.19, df = 63, p < 0.01).
In the cognitively impaired group the mean MMSE score was 23.6 +/− 3.8, and in the non-impaired group 27.7 +/− 2.1. The mean MMSE scores differed significantly between the two groups (t = 6.62, df = 148, p < 0.001). Mean MMSE scores in the dementia and MCI groups were 22.2 +/− 3.9 and 25.3 +/− 3.1 respectively and differed significantly between the two groups (t = 2.02, df = 63, p < 0.05).
At the specified MoCA and MMSE cut-offs of ≥26/30 used in the index paper for the diagnosis of dementia and MCI (Nasreddine et al., Reference Nasreddine2005), MoCA was very sensitive (0.97) but with only modest specificity (0.60) whereas MMSE was very specific (0.89) but not very sensitive (0.65) (Table 1). In other words, MoCA identified almost all the clinically diagnosed cases of dementia and MCI (63/65) but with a large number of false positives (34), whereas the MMSE missed a significant proportion of the dementia and MCI cases (22 false negatives = 35% of cases) but with many fewer false positives (9). Comparing just the MCI cases (n = 29) against cognitively unimpaired cases, a key differential in the identification of early cognitive impairment, results were little different (sensitivity = 0.93; specificity = 0.60), accounted for by two cases of MCI with MoCA score of 26/30. Screening utility between dementia and MCI was not calculated because of the relatively small numbers (36 vs 29) and because this differential is perhaps less crucial than MCI vs. no cognitive impairment in terms of clinical relevance.
Diagnostic parameters for MoCA and MMSE (with 95% Confidence intervals)
The overall accuracy of MoCA was examined at all cut-off values. The best sensitivity for a diagnosis of dementia or MCI was 1.00 at cut-offs ≥27/30 (with specificity no better than 0.42); the best specificity achieved was 1.00 at a cut-off of <16/30 (with sensitivity = 0.25). Optimal MoCA cutoff, defined by the maximal test accuracy for the differential diagnosis of dementia or MCI/not dementia (= 0.81) was ≥20/30. When compared to the specified cut-off (Table 1) at this MoCA cut-off there was improved specificity (0.95) and PPV (0.91) but poorer sensitivity (0.63) and NPV (0.77).
Areas under the ROC curves, measures of diagnostic accuracy, were 0.91 (95% CI = 0.86–0.95) for MoCA, and 0.83 (95% CI = 0.77–0.90) for MMSE (Figure 2; note that an AUC of 0.5, a straight line, indicates a test providing no added information; an AUC of 1 indicates a test providing perfect discrimination; and an AUC of > 0.75 is thought desirable for diagnostic tests).
ROC curve for MoCA.
In patients who were administered both MoCA and MMSE (n = 148), combining the tests in series (“And” rule; Table 2, left-hand column) gave results almost identical to those using MMSE alone (Table 1, right-hand column), whilst combining tests in parallel (“Or” rule; Table 2, right-hand column) gave results almost identical to those using MoCA alone (Table 1, left-hand column). In other words, MoCA “and” MMSE missed a significant proportion of the dementia and MCI cases (22 false negatives, or 35% of cases) but with few false positives (7), whereas MoCA “or” MMSE identified almost all the cases of dementia and MCI but with a large number of false positives (35).
Diagnostic parameters combining MoCA and MMSE in series (“AND”) and in parallel (“OR”) (with 95% confidence intervals)
Discussion
Study limitations included the selective nature of the population, inevitable in any hospital-based study, although less selective than in some previous MoCA studies (consecutive rather than by diagnosis), and the numbers studied. Diagnoses were clinical, without pathological confirmation or long-term follow-up. Dementia severity was not measured as the purpose of the study was to examine allcomers to the clinic irrespective of severity, thus reflecting day-to-day clinical practice. Nevertheless, the study confirmed the high sensitivity of MoCA for identifying cases of cognitive impairment (dementia and MCI), as previously reported in diagnostically selected populations (Nasreddine et al., Reference Nasreddine2005; Smith et al., Reference Smith, Gildeh and Holmes2007). The values for sensitivity (0.97), specificity (0.60) and area under the ROC curve (0.91) were very similar to those observed in the largest series reported to date (n = 135; 0.98, 0.52, 0.90, respectively; Damian et al., Reference Damian2011), despite the differences in case mix and the prevalence of cognitive impairment in the cohorts (43% vs. 28%).
MoCA compared favorably with the diagnostic parameters of other cognitive screening instruments examined in previous cohorts from this clinic, such as the Addenbrooke's Cognitive Examination (ACE; Larner, Reference Larner2007) and the Addenbrooke's Cognitive Examination-Revised (ACE-R; Larner, Reference Larner, Fisher and Hanin2009), despite dementia prevalence being higher in these cohorts and utility for MCI diagnosis not being examined. Certainly MoCA performed better than MMSE and its derivative Mini-Mental Parkinson (MMP) for MCI diagnosis (Larner, in press).
As in previous pragmatic studies examining cognitive screening instruments (Larner, Reference Larner2007; Reference Hancock and Larner2009; Hancock and Larner, Reference Hancock and Larner2011), a lower cut-off (≥20/30) than that suggested in the index paper (≥26/30; Nasreddine et al., Reference Nasreddine2005) was found to give greater overall test accuracy (0.81 vs 0.76) with improved test specificity (0.95 vs 0.60) for some loss in sensitivity (0.63 vs 0.97). Other studies have also suggested the possibility of using different MoCA cut-offs: in a study of community-dwelling older adults, Luis et al. (Reference Luis, Keegan and Mullan2009) suggested 23/30 might be superior, while Damian et al. (Reference Damian2011) found that a threshold score of 24 would have superior predictive value in a memory disorders clinic where the prior probability of cognitive impairment is high. This study endorses the latter finding, and may be explained in part by the absence of normal controls as used in the index study (Nasreddine et al., Reference Nasreddine2005).
Combining screening tests may sometimes improve overall diagnostic utility (e.g Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) and MMSE, Flicker et al., Reference Flicker, Logiudice, Carlin and Ames1997; IQCODE and ACE-R, IQCODE and MMSE, Hancock and Larner, Reference Hancock and Larner2009). Despite their apparent complementarity in terms of high sensitivity (MoCA) and high specificity (MMSE), combining the MoCA and MMSE in this study was found to add no diagnostic gain. This finding is concordant with an item analysis of the MoCA and MMSE (Damian et al., Reference Damian2011), which indicated that not all subtests were of equal predictive value and that a selection of MoCA and MMSE items with high predictive value might engender a more useful hybrid test.
The recommendation of Nasreddine et al. (Reference Nasreddine2005) that patients with cognitive complaints but without functional impairment be administered MoCA first, because MMSE is likely to produce a normal score, finds some support in the results of the current study.
Conflict of interest
None.
Acknowledgment
I thank Kathleen Storton for help with cognitive assessments.
