Introduction
In recent years, the nosologic concept of vascular cognitive impairment and dementia has generated a huge controversy and disagreement regarding diagnostic criteria. The existence of many cases of cognitive deficit resulting from cerebrovascular disease but not fulfilling the criteria for dementia has led to the recent emergence of a broader concept named Vascular Cognitive Impairment (VCI; Hachinski & Bowler Reference Hachinski and Bowler1993; O'Brien, Reisberg, & Erkinjuntti, Reference O'Brien, Reisberg and Erkinjuntti2003). In this context, three diagnostic categories are mainly proposed: Vascular Mild Cognitive Impairment (vMCI, also known as Vascular Cognitive Impairment No Dementia, VCIND), Vascular Dementia (VaD), and mixed dementia (Moorhouse & Rockwood, Reference Moorhouse and Rockwood2008; O'Brien et al., Reference O'Brien, Reisberg and Erkinjuntti2003).
The concept of VaD, which is classified as a VCI subtype, was introduced in the 1990s (Román et al., Reference Román, Tatemichi, Erkinjuntti, Cummings, Masdeu, Garcia and Scheinberg1993), replacing the previous concepts of Cerebral Arteriosclerosis and Multi-infarct dementia (Hachinski, Lassen, & Marshall, Reference Hachinski, Lassen and Marshall1974). This concept's evolution was associated with the proposal of new diagnostic criteria for Vascular Dementia by the National Institute of Neurological Disorders and Stroke – Association Internationale pour la Recherche et l'Enseignement en Neurosciences (NINDS-AIREN; Román et al., Reference Román, Tatemichi, Erkinjuntti, Cummings, Masdeu, Garcia and Scheinberg1993). These criteria were organized and further operationalized by van Straaten and collaborators (Reference van Straaten, Scheltens, Knol, van Buchem, van Dijk, Hofman and Barkhof2003), considering three fundamental axes of diagnosis: (a) presence of dementia, confirmed by neuropsychological assessment; (b) clinical and imaging evidence of cerebrovascular disease; and (c) establishment of a causal relationship between the vascular lesions and the emergence or aggravation of the cognitive deficits. The harmonization and coverage of clinical, neuroimaging, and anatomic-pathologic data makes these diagnostic criteria much more objective and selective than previous ones, namely the old Hachinski ischemic score (Hachinski et al., Reference Hachinski, Iliff, Zihlka, Du Boulay, McAllister, Marshall and Symon1975), explaining why they have gathered more consensus in clinical practice, becoming the most widely used in research (Santana, Reference Santana2006).
Epidemiological studies, although not consistent, have generally reported that VaD is the second most common cause of dementia, after Alzheimer's disease (AD) (Canadian Study of Health and Aging Working Group, 1994; Lobo, Martinez-Lage et al., Reference Lobo, Launer, Fratiglioni, Andersen, Di Carlo, Breteler and Hofman2000). In Europe, the prevalence rate is approximately 16/1000 in people over 65 years old, representing approximately 20% of all dementia cases (Ikeda et al., Reference Ikeda, Hokoishi, Maki, Nebu, Tachibana, Komori and Tanable2001; Lobo, Launer et al., Reference Lobo, Martinez-Lage, Soininen and Hofman2000). Moreover, in some Asian countries, VaD appears to be more prevalent than AD (Ikeda et al., Reference Ikeda, Hokoishi, Maki, Nebu, Tachibana, Komori and Tanable2001; Ueda, Kawano, Hasuo, & Fujishima, Reference Ueda, Kawano, Hasuo and Fujishima1992). Both prevalence and incidence of VaD and AD increase exponentially with age, and even though the increase in VaD seems to be less steep than for AD, the prevalence rate for individuals over 100 years old surpasses 52/1000 (Jorm & O'Brien, Reference Jorm and O'Brien2004; Lobo, Launer et al., Reference Lobo, Launer, Fratiglioni, Andersen, Di Carlo, Breteler and Hofman2000).
This high prevalence of VaD, highlight the importance of an accurate clinical identification of this disease. Moreover, this purpose is generally acknowledged to be difficult, largely because of the clinical features of VaD (Garrett, Paul, Libon & Cohen, Reference Garrett, Paul, Libon and Cohen2004; Reed et al., Reference Reed, Mungas, Kramer, Betz, Ellis, Vinters and Chui2004). The clinical features of the vascular cognitive impairment reflect the location, number, and extent of underlying ischemic, hemorrhagic, or hypoperfusion lesions (Desmond, Reference Desmond2004; Ueda et al., Reference Ueda, Kawano, Hasuo and Fujishima1992). As a result of this heterogeneity in mechanisms, vascular territory involvement and location, cognitive impairment in VaD is highly variable and potentially affects all cognitive domains. Despite this, the preponderant neuropsychological features of VaD patients point to an overall profile of cognitive dysfunction that is frontally located in the brain, due to the early involvement of executive cortico-subcortical circuits (O'Brien et al., Reference O'Brien, Reisberg and Erkinjuntti2003; Román, Reference Román2003; Román & Royall, Reference Román and Royall1999). The dysexecutive syndrome has been recurrently associated with the vascular clinical conditions (Desmond, Reference Desmond2004; Lamar, Price, Giovannetti, Swenson, & Libon, Reference Lamar, Price, Giovannetti, Swenson and Libon2010). Additionally, cortical functions such as language, calculation, and orientation tend to be relatively preserved, in contrast with AD patients (Looi & Sachdev, Reference Looi and Sachdev1999; Román & Royall, Reference Román and Royall1999). Deficits in recent episodic memory are usually less severe in VaD patients than in degenerative dementias, while concentration difficulties, working memory dysfunction, and psychomotor slowing are early and prominent symptoms in VaD (Looi & Sachdev, Reference Looi and Sachdev1999; O'Brien et al., Reference O'Brien, Reisberg and Erkinjuntti2003; Román, Reference Román2003).
An additional factor for this difficult to diagnose VaD refers to the insufficient breadth of representation of the cognitive abilities relevant to cognitive impairment associated with the disease in cognitive assessment tools, an essential condition for the accurate cognitive recognition and assessment (Mungas, Reed, & Kramer, Reference Mungas, Reed and Kramer2003). In fact, traditional screening tools frequently do not include measures of important domains initially impaired in VaD, mainly the executive functions, or the evaluation is very superficial using low complex tasks. This is the case of the Mini Mental State Examination (MMSE; Folstein, Folstein, & McHugh, Reference Folstein, Folstein and McHugh1975), which is the most widely used cognitive screening instrument for dementia, despite its well-known limitations; namely, it has low sensitivity to milder forms of cognitive impairment and does not evaluate executive functioning (Freitas, Santana, & Simões, Reference Freitas, Santana and Simões2010; Ihl, Frölich, Martin, & Maurer, Reference Ihl, Frölich, Martin and Maurer1992; Tombaugh & McIntyre, Reference Tombaugh and McIntyre1992; Wind et al., Reference Wind, Schellevis, van Staveren, Scholten, Jonker and van Eijk1997). These issues, according to several experts, are especially relevant for dementias such as VaD that present prominent executive impairment (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black and Leblanc2006; Martinić-Popović, Šerić, & Demarin, Reference Martinić-Popović, Šerić and Demarin2006; Pendlebury, Cuthbertson, Welch, Mehta, & Rothwell, Reference Pendlebury, Cuthbertson, Welch, Mehta and Rothwell2010; Pirscoveanu et al., Reference Pirscoveanu, Tudorica, Matcau, Zaharia, Matcau and Ene2009; Román, Reference Román2003).
The Montreal Cognitive Assessment (MoCA; Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin and Chertkow2005) is a cognitive screening test specifically developed for the screening of milder forms of cognitive impairment. Surpassing the limitations of the MMSE and proving be a more useful, sensitive and accurate measure, the MoCA is now recognized as one of the best cognitive screening tests (Gauthier et al., Reference Gauthier, Patterson, Gordon, Soucy, Schubert and Leuzy2011; Ismail, Rajji, & Shulman, Reference Ismail, Rajji and Shulman2009; Jacova, Kertesz, Blair, Fisk, & Feldman, Reference Jacova, Kertesz, Blair, Fisk and Feldman2007; Lonie, Tierney, & Ebmeier, Reference Lonie, Tierney and Ebmeier2009). Currently, the MoCA is an extensively validated screening tool for many disorders; it displays good overall psychometric properties and improved sensitivity to cognitive decline, which contributed to its rapid international recognition. This improved sensitivity in relation to other measures is related to the MoCA's more comprehensive assessment of major cognitive domains, including executive function, short-term memory, languages abilities, and visuospatial processing. Considering the qualities and adequacy of the MoCA, the neuropsychology working group of the NINDS-CSN VCI Harmonization Standards criteria (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black and Leblanc2006), recommends using the abbreviated version of the MoCA (12 items—hereinafter referred as MoCA short version) for brief cognitive screening in vascular patients in special settings such as primary health care, large epidemiological studies or clinical trials. This proposed 5-min protocol may be administered by telephone and is composed of three subtests: a five-word immediate and delayed recall test, a six-item orientation task, and a phonemic fluency test.
Several studies have been conducted to investigate cerebrovascular and cardiovascular diseases using the MoCA full version (Aggarwal & Kean, Reference Aggarwal and Kean2010; Cumming, Bernhardt, & Linden, Reference Cumming, Bernhardt and Linden2011; Dong et al., Reference Dong, Sharma, Chan, Venketasubramanian, Teoh, Seet and Chen2010; Ihara, Okamoto, & Takahashi, Reference Ihara, Okamoto and Takahashi2012; Pendlebury et al., Reference Pendlebury, Cuthbertson, Welch, Mehta and Rothwell2010; Pirscoveanu et al., Reference Pirscoveanu, Tudorica, Matcau, Zaharia, Matcau and Ene2009; Wong et al., Reference Wong, Kwan, Chan, Lam, Nyenhuis, Nasreddine and Wong2008; Reference Wong, Xiong, Kwan, Chan, Lam, Wang and Mok2009). All of these studies indicate that the MoCA is a psychometrically valid, reliable and sensitive cognitive screening test for the mentioned clinical conditions. Thus far, however, there are few previous studies using a homogeneous group of VaD patients to confirm those results and no studies of validation of the MoCA short version.
The aim of the present study was to investigate the validity of the MoCA (Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin and Chertkow2005; Simões et al., Reference Simões, Freitas, Santana, Firmino, Martins, Nasreddine and Vilar2008) for cognitive screening of VaD patients. The psychometric properties and diagnostic accuracy of the MoCA was analyzed and the optimal cutoff points to detect VaD patients were established. In parallel, the same analysis was conducted for the short version of the MoCA proposed by NINDS-CSN VCI Harmonization Standards (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black and Leblanc2006). Furthermore, we also aimed to investigate the cognitive performance of the VaD patients on the MoCA's cognitive domains and analyze the differences with comparison to AD patients.
Methods
Participants and Procedures
The study sample was composed of 102 participants distributed between three subgroups: (I) a VaD group with 34 patients, (II) an AD group with 34 patients, and (III) a Control group with 34 cognitively healthy adults. The demographic data of the participants in each group are summarized in Table 1.
Demographic and clinical characteristics of the subgroups
Note. Gender is characterized by female's n and respective percentage (%). Data of other variables are presented as mean ± standard deviation. VaD = group of patients with Vascular Dementia; AD = group of patients with Alzheimer′s disease; Control = group of cognitively healthy adults; MoCA = Montreal Cognitive Assessment (maximum score = 30); MMSE = Mini Mental State Examination (maximum score = 30).
The VaD patients were recruited between January 2009 and June 2011 at the Dementia Clinic of the Neurology Department of Coimbra University Hospital (Coimbra, Portugal) and included only patients with a diagnosis of probable VaD, based on international criteria (Román et al., Reference Román, Tatemichi, Erkinjuntti, Cummings, Masdeu, Garcia and Scheinberg1993). Patients with Vascular Mild Cognitive Impairment (vMCI), also called Vascular Cognitive Impairment no Dementia (VCIND), or with mixed dementia were not included. Eligibility was based on a comprehensive clinical evaluation, which included a neuropsychological and imaging study, and the final diagnosis was established by a multidisciplinary team consensus composed by two neurologists, two neuropsychologists and two neuroradiologists with expertise in dementia investigation.
The AD group included the patients with a diagnosis of probable AD, as previously established by a multidisciplinary team consensus using international criteria (American Psychiatric Association, 2000; McKhann et al., Reference McKhann, Drachman, Folstein, Katzman, Price and Stadlan1984). This diagnosis subgroup excluded patients with significant vascular disease, using either clinical or imaging evaluations. This group was selected from among the AD patients of the Dementia Clinic of the Neurology Department of Coimbra University Hospital (Coimbra, Portugal) to match VaD patients on gender, age, educational level, and severity of cognitive decline, as assessed by the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975).
All patients were clinically examined by a neurologist (I.S.), and a standard investigation was always performed, including routine laboratory exams and analyses (Apolipoprotein E [APOE] genotyping and imaging studies: structural (computed tomography [CT] and/or magnetic resonance imaging [MRI]) and functional (single photon emission CT [SPECT]). PET and cerebrospinal fluid analyses were carried out more restrictively, although they were considered in younger patients and whenever the suspicion of an inflammatory, infectious, or neoplastic brain involvement was raised. Structural imaging studies, namely CT or MRI, were fundamental in the differential diagnosis of dementia patients in this study. In accordance with the inclusion criteria for VaD and the exclusion criteria for AD, the following findings were relevant: (1) evidence of large cortico-subcortical infarcts; (2) extensive periventricular (PVL) or subcortical white matter lesions (SCWML), affecting more than 25% of the volume of the supra-tentorial white matter (WML), based on a CT or MRI volumetric qualitative analysis; (3) uni- or bilateral thalamic strokes; (4) lacune in the head of the caudate nucleus or in the inferior genu of the internal capsule; (5) more than two lacunes. At the outset of this study, the following were also considered exclusion criteria for patients’ selection: (1) an unstable clinical condition, with significant comorbidities; (2) high dementia severity (only including patients with CDR ≤ 2 and MMSE ≥ 12 points); (3) recent pharmacotherapy changes (patients on stable medication (> 6 months) with cholinesterase inhibitors or psychotropic drugs were allowed to enter the study, but recent dose changes of these drugs, as well as introduction of medication with anti-cholinergic effect were considered an exclusion criteria); (4) recent psychiatric comorbidity (clinically diagnosed in the 6 months before the current neuropsychological evaluation); and (5) significant motor, visual or auditory deficits, all of which may influence neuropsychological assessment. For each patient considered suitable for the study at the time of data collection, a diagnosis was recorded in the clinical file by the neurologist.
All patients also underwent a comprehensive neuropsychological assessment battery including the following instruments: the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975), the Alzheimer's Disease Assessment Scale (ADAS; Rosen, Mohs, & Davis, Reference Rosen, Mohs and Davis1984), the Clinical Dementia Rating scale (CDR; Hughes, Berg, Danziger, Coben, & Martin, Reference Hughes, Berg, Danziger, Coben and Martin1982), the Irregular Word Reading Test (TeLPI; Alves, Simões, & Martins, Reference Alves, Simões and Martins2009), the Subjective Memory Complaints scale (SMC; Schmand, Jonker, Hooijer, & Lindeboom, Reference Schmand, Jonker, Hooijer and Lindeboom1996), and the Geriatric Depression Scale (GDS-30; Yesavage et al., Reference Yesavage, Brink, Rose, Lum, Huang, Adey and Leirer1983). The MoCA was never used for diagnostic purposes.
The control group was composed of cognitively healthy members of the community, living in Portugal. This group was selected from the database of the MoCA's normative study for the Portuguese population to match each patient on variables that were found to be predictive of the MoCA's performance (educational level and age; Freitas, Simões, Alves, & Santana, Reference Freitas, Simões, Alves and Santana2011), and additionally on gender. Details of the controls’ recruitment procedure, inclusion and exclusion criteria, and neuropsychological assessment have been described in a previous study (Freitas et al., Reference Freitas, Simões, Alves and Santana2011).
An informed consent was obtained from all the participants after the research aims, procedures, and confidentiality requirements were fully disclosed by a member of the research team. For the patients who were not capable of providing the informed consent, a legal representative fulfilled that requirement on their behalf. The present research complied with the ethical guidelines for human experimentation stated in the Declaration of Helsinki and was approved by the Ethics Board of Coimbra University Hospital, by the “Fundação para a Ciência e Tecnologia” [Portuguese Foundation for Science and Technology] and by the Faculty of Psychology and Educational Sciences Scientific Committee.
Material and Neuropsychological Testing
Each participant was assessed in a single session by a neuropsychologist experienced in evaluating patients with dementia. In the clinical interview, the demographic and clinical data were collected through a complete sociodemographic questionnaire, an inventory of past habits and of the current clinical health status as well as of the medical history. Following this, the same neuropsychologist administered the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975; Guerreiro, Reference Guerreiro1998) and the MoCA (Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin and Chertkow2005; Simões et al., Reference Simões, Freitas, Santana, Firmino, Martins, Nasreddine and Vilar2008), in that fixed order, to all subjects. Both the MMSE and the MoCA are in paper-and-pencil format and are scored out of a possible total of 30 points, with higher scores indicating better cognitive performance. The MoCA was developed to screen milder forms of cognitive impairment through the assessment of six cognitive domains: (1) executive functions; (2) visuospatial abilities; (3) short-term memory; (4) language; (5) attention, concentration, and working memory; and (6) temporal and spatial orientation (Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin and Chertkow2005). In the current study, the MoCA's total score refers to the raw score without any correction points for education effects. Nasreddine and collaborators (2005) suggest a correction point for individuals with 12 or less years of education. However, the Portuguese population has a considerable lower education level in comparison with the original study population. Due to this major difference, the correction point is not entirely applicable to the Portuguese population. The norms for the Portuguese population were calculated and stratified according to different educational levels (Freitas et al., Reference Freitas, Simões, Alves and Santana2011).
Statistical Analysis
Statistical analyses of the data were carried out using the Statistical Package for the Social Sciences (SPSS, version 19.0) (IBM SPSS, Chicago, IL). Descriptive statistics were used for the sample's characterization. The χ 2 test, the two-sample t test, the one-way between-groups analysis of variance (ANOVA), and the Tukey HSD and Bonferroni post hoc test were conducted to explore group differences. The Cronbach's alpha was considered as an index of internal consistency. The Pearson correlation coefficients were used for analyses of convergent validity, and to explore the correlations between items, cognitive domains and total scores.
The diagnostic accuracy of the MoCA and the MMSE for the prediction of the clinical diagnosis of VaD was assessed through the receiver operating characteristics (ROC) curve analysis, implemented in MedCalc (version 11.6) (MedCalc Software, Mariakerke), with larger areas under the curve (AUC) indicating better diagnostic accuracy. The ROC curves were compared according to the AUC comparison method of Hanley and McNeil (Reference Hanley and McNeil1983). The optimal cutoff points for each screening instrument that yielded the highest Youden index were selected, with higher Youden indices indicating maximum sensitivity and specificity. To analyze the predictive value of the tests, for each cutoff point, we calculated the sensitivity (the probability of subjects with cognitive impairment having a positive result), specificity (the probability of subjects without cognitive impairment having a negative test), positive predictive value (PPV, the probability of disease in subjects with a positive result), negative predictive value (NPV, probability of a “lack of disease” classification in subjects with a negative result), and classification accuracy (probability of correct classification of subjects, with or without cognitive impairment).
Results
Sample Characterization
The characteristics of the study sample, and in more detail of all subgroups, are presented in Table 1. For this description, were considered: sample size, gender, age, educational level, MMSE score, and MoCA score.
As mentioned above, the participants in the three groups were matched based on gender, age, and educational level. Additionally, the VaD and AD patients were also matched based on their MMSE scores. In fact, as seen on Table 1, there are no gender differences between the three groups. Likewise, no statistically significant differences were found based on age (F(2,99) = .095; p = .910) or educational level (F(2,99) = .021; p = .979). Furthermore, statistically significant differences in MMSE scores were not observed between clinical groups (t(66) = .826; p = .412), which suggests similarities in terms of cognitive decline severity.
Psychometric Properties
Cronbach's alpha of the MoCA as an index of internal consistency was 0.908 for the total study sample, confirming the overall reliability of the scale when used to examine Portuguese subjects. Regarding the analysis of MoCA items that could be eliminated to increase consistency, the results indicated that none should be excluded. This reliability coefficient was also computed for each clinical group: α (VaD) = .825 and α (AD) = .806. The respective values for the MMSE were: α (total sample) = .807, α (VaD) = .771, and α (AD) = .804.
The correlations between the MoCA scores and the MMSE scores were significant and positive in the total sample (r = .741; p < .001) and in the VaD group (r = .782; p < .001), which is indicative of convergent validity. Regarding to the MoCA cognitive domains, we found significant and positive correlations between each cognitive domain score and the total score of the scale for all three groups (Table 2), which is suggestive of construct-related validity. Additionally, regarding the correlation coefficients of each item within the cognitive domains, overall items showed a significant and positive correlation with the respective domain. The exceptions are the camel, second subtraction, place, and city items, which showed no significant correlation with any domain due to the lower variance of the results (hit rates: camel = 79.4; second subtraction = 23.5; place = 97.1; city = 100). Furthermore, the cube's copy showed a significant correlation with the visuospatial domain (r = .452; p < .001) and with the executive function domain (r = .491; p < .001).
Correlations of the scores on cognitive domains and the MoCA total score
Note. VaD = group of patients with Vascular Dementia; AD = group of patients with Alzheimer′s disease; Control = group of cognitively healthy adults; MoCA = Montreal Cognitive Assessment.
**p < .01.
* p < .05.
Group Differences
Comparing the MoCA total scores of the three groups, statistically significant differences can be observed (F(2,99) = 78.121; p < .001). However, according to post hoc tests, these differences occurred between the control group and both clinical groups, with higher performance seen in the healthy participants. No differences were found between VaD and AD groups, which were previously matched based on their MMSE scores. As expected, a more detailed analysis regarding sub-scores of the MoCA cognitive domains revealed that the healthy participants had higher performances in all cognitive domains than both clinical groups. The scores on the cognitive domains for all three groups are presented on Table 3. There were no statistically significant differences between the VaD and AD patients at the level of the cognitive domains of the MoCA, nor at the level of some tasks’ sub-scores, such as clock drawing, naming tasks, subtraction, and abstraction tasks. Additionally, no differences in the number of the words stated in the phonemic fluency task were found.
Scores on the cognitive domains
Note. VaD = group of patients with Vascular dementia; AD = group of patients with Alzheimer′s disease; Control = group of cognitively healthy adults.
Data are presented as mean ± standard deviation.
Considering the abbreviated version of the MoCA proposed by the NINDS-CSN VCI Harmonization Standards criteria (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black and Leblanc2006) as a 5-min protocol (total score = 12), no statistically significant differences were found between VaD (5.50 ± 1.93) and AD patients (4.65 ± 1.89). Nevertheless, the short version of the MoCA discriminates control subjects (9.32 ± 1.408) from both clinical groups, according to the post hoc tests (F(2,99) = 68.295; p < .001).
Cutoff Points
The receiver operating characteristics (ROC) curve analysis and the predictive values were computed to evaluate the diagnostic accuracy of MoCA to discriminate VaD patients from cognitively healthy adults. First, the diagnostic accuracy of the MoCA full version was compared with the diagnostic accuracy of the MMSE; second, the MoCA short version was compared with the MMSE; and then, the MoCA full version was compared with the short version (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black and Leblanc2006).
The MoCA full version discriminant potential for VaD was excellent, with an AUC of .950 (95% confidence interval = .868–.988), while the MMSE demonstrated an AUC of .860 (95% confidence interval = .754–.932). These AUCs are significantly different (Z = 2.142; p = .0322), according to the AUC comparison method of Hanley and McNeil (Reference Hanley and McNeil1983), indicating different classificatory accuracy of the instruments. Graphic representations of the ROC curves are provided in Figure 1. Similarly, the MoCA short version also reveals an excellent discriminant potential for VaD with an AUC of .936 (95% confidence interval = .879–.992), although not significantly different from the MMSE results (Z = 1.774; p = .076), according to the AUC comparison method of Hanley and McNeil (Reference Hanley and McNeil1983). The respective graphic representations of the ROC curves are provided in Figure 2. Nevertheless, in both comparisons it can be observed that the MoCA's ROC curves (full and short versions) fully include the curve for the MMSE; this is a clear indication that there is always a cutoff for the MoCA, with higher sensitivity and specificity, for any cutoff chosen for the MMSE. Comparing the AUCs of the MoCA full and short versions (AUC = .936; 95% confidence interval = .849–.981), no statistically significant differences were found (Z = .488; p = .6256), according to the AUC comparison method of Hanley and McNeil (Reference Hanley and McNeil1983). Graphic representations of the respective ROC curves are provided in Figure 3. The optimal cutoff points for maximum accuracy (Youden index) and the respective values of sensitivity, specificity, PPV, NPV, and classification accuracy are described for both instruments in Table 4.
Receiver operating characteristics (ROC) curve analysis of the MoCA full version (continuous line) and MMSE (dashed line) to detect VaD patients. MoCA = Montreal Cognitive Assessment; MMSE = Mini Mental State Examination; VaD = Vascular Dementia.
Receiver operating characteristics (ROC) curve analysis of the MoCA short version (continuous line) and MMSE (dashed line) to detect VaD patients. MoCA = Montreal Cognitive Assessment; MMSE = Mini Mental State Examination; VaD = Vascular Dementia.
Receiver operating characteristics (ROC) curve analysis of the MoCA full version (continuous line) and MoCA short version (dashed line) to detect VaD patients. MoCA = Montreal Cognitive Assessment; MMSE = Mini Mental State Examination; VaD = Vascular Dementia.
Diagnostic classification accuracy
Note. Sensitivity, Specificity, PPV, NPV, and Classification Accuracy values were expressed in percentage. Cutoff values indicate the minimum score required for absence of signal.
MoCA = Montreal Cognitive Assessment (maximum score = 30); MoCA Short Version = abbreviated version proposed by the NINDS-CSN VCI Harmonization Standards criteria (maximum score = 12); MMSE = Mini Mental State Examination (maximum score = 30); AUC = area under the operating characteristic curve; PPV = positive predictive value; NPV = negative predictive value.
Discussion
Accurate screening of the early stages of VaD assumes extreme importance, considering the high prevalence rates of vascular cognitive impairment and the available primary and secondary prevention strategies. The MMSE reveals limitations in effective cognitive assessment of patients with VaD because it does not include tasks for the evaluation of the main area of compromise: executive dysfunction. Thus, this study aims to validate the MoCA, as well as the short version of the MoCA proposed by NINDS-CSN VCI Harmonization Standards, as a 5-min protocol for cognitive screening of VaD patients.
The results of the present study demonstrate that the MoCA is a psychometrically valid and reliable instrument for the cognitive screening of VaD patients. The internal consistency was high, confirming the overall reliability of the scale when used to examine Portuguese patients. Furthermore, the significant and positive correlations between the MoCA scores and the MMSE scores are suggestive of convergent validity. On the other hand, the significant and positive correlations between each MoCA's cognitive domain and the MoCA total score are suggestive of construct-related validity. Finally, all items showed a significant correlation with their respective domains, with the exception of camel, second subtraction, place, and city items; these showed no significant correlation with any domain, possibly due to the lower variance of the items in this sample. Furthermore, we hypothesize that the significant correlation found between the cube's copy item and executive function domain, beyond its correlation with the visuospatial domain, is explained by the three-dimensional nature of the drawing; it involves the use of planning, sequencing and attentional skills.
For the MoCA's discriminant capacity analysis, participants in the three groups (VaD, AD, and controls) were matched for gender, age, and educational level; the two clinical groups were also matched based on MMSE scores to ensure comparable levels of severity of cognitive decline, which we consider be essential for a reliable comparative analysis, although it may have a minor effect on the correlation coefficient between the two tests. The results showed that both the full and short versions of the MoCA efficiently discriminate the cognitively healthy adults from both clinical groups. The performance of the healthy participants in all MoCA cognitive domains was also significantly higher than that of both clinical groups. However, no statistically significant differences were found between VaD and AD patients using both MoCA versions. Additionally, no differences were found between VaD and AD patients’ performances in the different cognitive domains of the MoCA, or at the level of some tasks’ sub-scores (clock drawing, naming tasks, subtraction, and abstraction tasks). These findings were convergent with the results of several prior studies, which have failed to identify significant differences in neuropsychological performance between VaD and AD patients (Braaten, Parsons, McCue, Sellers, & Burns, Reference Braaten, Parsons, McCue, Sellers and Burns2006; Desmond, Reference Desmond2004). Given the characteristics of the MoCA (namely the more comprehensive assessment of major cognitive domains as executive function, short-term memory, languages abilities, and visuospatial processing) and the findings of previous studies from our group, we hypothesized that it could capture any difference in the cognitive profile of these clinical groups. In the three validation studies developed by our research group (Freitas, Simões, Marôco, Alves, & Santana, Reference Freitas, Simões, Marôco, Alves and Santana2012; Freitas, Simões, Alves, Duro, & Santana, manuscript submitted for publication; Freitas, Simões, Alves, & Santana, Reference Freitas, Simões, Alves and Santana2012), the MoCA's sensitivity and discriminant capacity at level of cognitive domains were well-demonstrated. Thus, we believe that the undifferentiated performance founded in this study may reflect the very close cognitive profile between VaD and the AD patients more so than the instrument's inability to capture cognitive profiles.
The MoCA scores displayed excellent diagnostic accuracy in the discrimination of VaD symptoms from age-related changes in cognitively healthy elderly adults. The MoCA full version scores exhibits an AUC significantly higher than that of the MMSE's AUC; this confirms the comparative superior classificatory accuracy of the MoCA full version. The optimal cutoff point on the MoCA for VaD patients, allowing maximum sensitivity and specificity, was below 17 points. This cutoff point is equivalent to the optimal cutoff for AD, as established by our previous validation study (Freitas, et al., Reference Freitas, Simões, Alves and Santana2012), as well as for Frontotemporal Dementia—behavioral-variant patients (Reference Freitas, Simões, Alves, Duro and SantanaFreitas et al., submitted). With this cutoff point, the MoCA showed good sensitivity (77%), excellent specificity (97%) and PPV (96%), and good NPV (81%) and classification accuracy (87%). With an optimal cutoff point of below 26 points, the MMSE had consistently lower respective values, namely in sensitivity (62%) and classificatory accuracy (78%); this suggests that the MMSE is not a good option as cognitive screening instrument for VaD, as some authors have claimed (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black and Leblanc2006; Moorhouse & Rockwood, Reference Moorhouse and Rockwood2008; Pendlebury et al., Reference Pendlebury, Cuthbertson, Welch, Mehta and Rothwell2010; Román, Reference Román2003). The main reasons for the MoCA's higher results at this level were the inclusion of the executive function assessment and the consideration of more complex tasks to measure short-term memory, language, attention, concentration, working memory, and visuospatial skills.
The MoCA's short version, proposed by NINDS-CSN VCI Harmonization Standards (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black and Leblanc2006) as a 5-min protocol, also demonstrated excellent diagnostic accuracy in discriminating VaD patients from the control participants. The AUCs of the MoCA short and full versions scores revealed no statistically significant differences, which indicate that the MoCA short version is a valid abbreviated option, useful in clinical settings with time constraints. However, contrary to what occurred with the MoCA full version, no significant differences were found in comparison with the MMSE, although the MoCA short version's ROC curve fully include the curve for the MMSE, which indicate that there is always a cutoff for the MoCA short version, with higher sensitivity and specificity, for any cutoff chosen for the MMSE. With an optimal cutoff point of below 8 points, the MoCA short version showed high sensitivity (85%), specificity (88%), PPV (88%), NPV (86%), and classification accuracy (87%).
Due to the clinical heterogeneity of the VCI conditions, this investigation was carried out in a homogeneous sample of VaD patients. Patients who did not meet the criteria for dementia or with mixed dementia classification were excluded. The rigorous procedures adopted by the multidisciplinary team ensure the correct diagnosis of patients according to the international criteria for VaD (Román et al., Reference Román, Tatemichi, Erkinjuntti, Cummings, Masdeu, Garcia and Scheinberg1993). Major methodological strengths are the comprehensive clinical evaluation with routine laboratory exams, strict and rigorous imaging inclusion criteria, a comprehensive neuropsychological assessment battery, and several exclusion criteria for patients’ selection at the outset. Although some studies conducted in cerebrovascular and cardiovascular patients support the validity, reliability and sensitivity of the MoCA, there are few previous studies using a homogeneous group of VaD patients to confirm those results and no studies of validation of the MoCA short version.
The main strengths of this study are: (1) the homogeneity of the VaD group; (2) the homogeneity of the samples in terms of group size, gender, age, educational level, and severity of cognitive decline in the clinical groups, which allowed a more clear analysis and minimized the influence of individual and methodological variables; (3) the above-mentioned rigorous procedures adopted to ensure a probable diagnosis of VaD and AD; (4) the well-characterized control group, composed of cognitively healthy adult members of the community; and (5) the reduction of inter-rater variability because all participants were assessed by one of two expert neuropsychologists.
Some limitations of this study must be addressed, however. First, as we intended to include and obtain results from VaD patients, the generalization of these results to other VCI subtypes should be performed with caution. Another point to mention is that the scores of the MoCA short version were computed after the complete administration of the MoCA, rather than resulting from a second separate administration. Despite there are no guidelines for the administration order of short version items, in fact we did not perform a separate administration of both versions. So although we should to admit this procedure as a limitation, we believe that the impact is limited because all additional items are very brief tasks, with short and immediate answers and the full MoCA is itself a short test. Another issue that needs to be pointed out is the use of the MMSE as both a cutoff for severity level's restriction and a variable of interest, making the results incorporating the MMSE somewhat circular in nature. However, unfortunately it was not possible to consider a third instrument for restricting the severity level on the study due to constraints in the department's functioning and neuropsychological assessment protocol currently used. The severity level's restriction was adopted because the MoCA was designed as a test for mild to moderate cognitive decline. The complexity of the tasks that compose the test is not adequate for patients with a more pronounced cognitive decline. Finally, because this is the first study that (i) validated the MoCA short version for VaD patients, (ii) compared a shortened form to the original form of the MoCA, and (iii) computed the cutoff points for both MoCA short and full versions for the screening of VaD patients, a comparative analysis of this results with other studies cannot be performed.
Some future considerations: the need for (i) more studies conducted in different cultural contexts other than the Portuguese, (ii) studies with separate administrations of the two versions of the MoCA; and (iii) studies with larger sample sizes, which allow that the each clinical group is divided into two subgroups, high and low scorers on the MMSE, to further confirm the present findings.
In conclusion, the MoCA full and short versions proved to be valid, reliable, sensitive and accurate measures of cognitive impairment in VaD. Compared with the widely used MMSE, this study produced evidence of the overall superiority of both MoCA versions in terms of diagnostic accuracy, confirming its great potential and usefulness for the brief cognitive assessment of VaD patients.
Acknowledgments
This work was supported by the Fundação para a Ciência e Tecnologia [Portuguese Foundation for Science and Technology] through of a PhD fellowship (SFRH/BD/38019/2007) and PIC/IC/83206/2007. All authors of this study declare that there are no conflicts of interest.
