Introduction
Post-stroke vascular cognitive impairment (VCI) is a syndrome that includes a spectrum of cognitive severity ranging from post-stroke vascular mild cognitive impairment (VaMCI) to dementia (Hachinski et al., Reference Hachinski2006). Post-stroke VaMCI is defined by impairment in at least one cognitive domain with intact or mildly impaired instrumental activities of living (Gorelick et al., Reference Gorelick2011). More than half of patients with VCI (57%) are VaMCI (Rockwood et al., Reference Rockwood, Wentzel, Hachinski, Hogan, MacKnight and McDowell2000), while 40% with non-disabling ischemic stroke had VaMCI (Tham et al., Reference Tham2002). Post-stroke VaMCI can be further classified into four subtypes in line with the MCI diagnostic algorithm proposed by Petersen (Reference Petersen2004): (i) amnestic single-domain (amnestic sd-MCI); (ii) non-amnestic single-domain (non-amnestic sd-MCI); (iii) amnestic multiple-domain (amnestic md-MCI); and (iv) non-amnestic multiple-domain (non-amnestic md-MCI). Of these, patients with multiple cognitive domain deficits (amnestic md-MCI) were at elevated risk (hazard ratio: 5.7) for incident dementia (Narasimhalu et al., Reference Narasimhalu2009) compared to patients with other MCI subtypes or patients without cognitive impairment.
Efforts have been made to validate a brief cognitive instrument for the detection of VCI, either the recently developed Montreal cognitive assessment (MoCA) (Nasreddine et al., Reference Nasreddine2005) or the well-established widely used Mini-Mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and McHugh1975). The validation studies to date have identified three issues. First, there have been inconsistent findings as to whether the discriminatory ability of the MoCA is superior to the MMSE in detecting VCI, either in the acute phase (Godefroy et al., Reference Godefroy2011; Dong et al., Reference Dong2012) or at one year after stroke (Pendlebury et al., Reference Pendlebury, Mariz, Bull, Mehta and Rothwell2012). Second, the discriminatory ability of the MoCA compared to the MMSE in detecting VCI has not been reported at the early convalescent phase of stroke, i.e. three to six months after stroke when VCI is conventionally diagnosed and at an opportune time for early and focused management (Gorelick et al., Reference Gorelick2011). Finally, although impairment in visuomotor processing speed has been reported as an independent predictor for dependency following stroke (Narasimhalu, et al. Reference Narasimhalu2011), both the MMSE and MoCA lack a visuomotor processing speed measure that may have resulted in less optimal sensitivity in screening for VCI. This drawback has been suggested as the reason for suboptimal sensitivity of the MoCA in detecting single-domain non-amnestic impairment in stroke patients (Pendlebury et al., Reference Pendlebury, Mariz, Bull, Mehta and Rothwell2012).
Therefore, we aimed to: (i) compare the MoCA and MMSE in screening for VCI three to six months after stroke; and (ii) examine the widely used visuomotor processing speed measure, the symbol digit modalities test (SDMT) (Smith, Reference Smith1973), independently and as a supplement to the MMSE or MoCA for VCI screening three to six months after stroke. It is hypothesized that the MoCA and MMSE would be equivalent in detecting VCI at three to six months after stroke. It is also hypothesized that the addition of the SDMT will improve sensitivity of the MMSE and MoCA in detecting VCI, especially in single-domain non-amnestic impairment.
Methods
Participants
Methods in this study have been described previously (Dong et al., Reference Dong2012). Briefly, 400 clinically stable non-aphasic patients (≥21 years old) with a recent ischemic stroke or transient ischemic attack (TIA) (≤14 days) were recruited during inpatient admission at the National University Health System in Singapore. Demographic characteristics, the MoCA and MMSE, stroke severity and disability measures (National Institutes of Health Stroke Scales (NIHSS) and the modified Rankin Scale (mRS)) were collected at the subacute stroke phase. These patients were followed up three to six months later to receive the MoCA, MMSE, and a formal neuropsychological evaluation. This study was approved by the local ethics committee and conducted in conformity with the Declaration of Helsinki. Written informed consent was obtained from all patients or their legally acceptable representative.
Cognitive evaluation and diagnosis
Patients were assessed with the MoCA, MMSE, and a formal neuropsychological battery locally validated for Singaporean elderly at three to six months after stroke (Yeo et al., Reference Yeo, Gabriel, Chen, Lee, Loenneker and Wong1997; Narasimhalu et al., Reference Narasimhalu2009). The translation and back-translation of the cognitive tests (MoCA, MMSE, and the formal neuropsychological battery) were undertaken by bilingual research psychologists and three equivalent versions (in English, Chinese, and Malay) of these cognitive tests were established. The modification of the MoCA for Singaporean and its validation has been described previously (Dong et al., Reference Dong2010; Dong et al., Reference Dong2012). The cognitive testing was conducted in patients’ language of choice to ensure test fairness. The formal neuropsychological battery (Supplementary Table S2, available as supplementary material attached to the electronic version of this paper at www.journals.cambridge.org/jid_IPG), covering the domains of executive function, attention, language, visuoconstruction, visuomotor speed, verbal and visual memory, was administered by trained research psychologists blinded to the MMSE and MoCA scores. Age and education-adjusted cut-offs of 1.5 standard deviations (SD) below the established norms were used on individual tests. A result of at least half the tests failing in a domain constituted a failure in that domain. VCI was defined by the presence of any domain impairment on the neuropsychological battery, while post-stroke VaMCI was defined according to the recently proposed criteria (Gorelick et al., Reference Gorelick2011), i.e. the presence of at least one objective cognitive domain impairment on neuropsychological testing with preserved functioning measured by the Bayer-Activities of Daily Living Scale (Hindmarch et al., Reference Hindmarch1998). VaMCI was further classified into four subtypes: (i) amnestic sd-MCI, (ii) non-amnestic sd-MCI, (iii) amnestic md-MCI, and (iv) non-amnestic md-MCI (Petersen, Reference Petersen2004). Diagnoses of dementia were made according to the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV; American Psychiatric Association, 1994).
The SDMT test is one of three tests in the visuomotor processing speed domain of the formal neuropsychological battery. The remaining two tests in this domain included a digit cancellation test (Diller et al., Reference Diller1974) and a maze task (Porteus, Reference Porteus1959). These two tests were not chosen to represent visuomotor processing speed functioning because the SDMT takes approximately five minutes and is a widely used visuomotor processing speed test (Strauss et al., Reference Strauss, Sherman and Spreen2006), while digit cancellation is designed for screening neglect (Diller et al., Reference Diller1974) and the maze task is considered primarily an executive function measure (Strauss et al., Reference Strauss, Sherman and Spreen2006) despite the component of processing speed in both tests.
Statistical analyses
Between-group comparisons were conducted using independent-sample t-tests or Mann–Whitney U tests for continuous variables and a Pearson χ2 test for categorical variables. A receiver operating characteristic (ROC) curve analysis was used to examine the discriminatory ability of the MoCA and MMSE raw scores for VCI using the standard method (Wong et al., Reference Wong2009) of adjustment for the MoCA or age and education adjustment using regression analysis for the MoCA and MMSE. The AUCs of the MoCA, MMSE and SDMT, independently or in combination, were compared using established inferential statistical methods (Hanley and McNeil, Reference Hanley and McNeil1983). Optimal cut-off points for discrimination were established based on the largest Youden index criterion (calculated by sensitivity + specificity –1). Recommended cut-off points were chosen over optimal cut-off points based on higher sensitivity and acceptable specificity. All statistical analyses were performed with IBM SPSS 20.0 for Windows.
Results
Population characteristics
The majority of patients recruited were Chinese (70.3%) and males (69.8%) with a mean age of 59.8 years old (SD = 11.6) and a mean of 7.7 years of formal education (SD = 4.3). Most patients (79.8%) had a mild ischemic stroke and TIA, as evidenced by a low level of disability (median NIHSS score = 2 and median mRS score = 2) at the subacute stroke phase (≤14 days after the vascular event). At three to six months following the index event, 327 patients (81.8%) received MMSE, MoCA, and a formal neuropsychological assessment; 57 (14.2%) declined testing; and 16 (4%) were lost to follow-up (three deceased, one comatose and 12 not contactable). The majority language used for testing was Chinese (47.1%, n = 154), followed by English (41.3%, n = 135), and Malay (11.6%, n = 38). There was no interaction between language and the MoCA or MMSE scores. Patients who did not undertake MMSE, MoCA, and a formal neuropsychological assessment had significantly fewer neurological deficits at baseline than those who completed the assessment (median NIHSS score 1 vs. 2; median mRS score 1 vs. 2 respectively). There were no significant differences in other baseline characteristics between these two groups. A majority had VaMCI (55.0%, n = 180) consisting of amnestic md-MCI (29.4%, n = 96), non-amnestic md-MCI (4.6%, n = 15), amnestic sd-MCI (10.1%, n = 33), and non-amnestic sd-MCI (11.0%, n = 36), followed by a small minority with dementia (2.8%, n = 9). Population characteristics are shown in supplementary Table S1 (available online).
Comparison of the discriminatory ability of the MoCA, MMSE, and SDMT
The MoCA and MMSE had similar AUCs in discriminating VCI from no cognitive impairment (NCI) using the raw scores and age- and education-adjusted scores (raw scores: MoCA 0.87 (95% CI 0.83–0.91) and MMSE 0.84 (95% CI 0.80–0.88), p = 0.13; adjusted scores: MoCA 0.89 (95% CI 0.86–0.93) and MMSE 0.88 (95% CI 0.84–0.92), p = 0.52). In addition, they were similar in detecting post-stroke VaMCI as a whole, as well as VaMCI subtypes using raw scores except for non-amnestic md-MCI where MoCA raw scores had a significantly larger AUC than the MMSE (MoCA 0.94 (95% CI 0.86–1.00) vs. MMSE 0.83 (95% CI 0.71–0.94), p = 0.04). When analysis was repeated using age- and education-adjusted scores, it was found that the adjust scores of the MoCA and MMSE were equivalent in detecting post-stroke VaMCI as a whole, as well as its subtypes.
The SDMT was equivalent to the MoCA in detecting VCI, VaMCI, and its subtypes (AUC: ranging from 0.75 to 0.97 vs. 0.74 to 0.94). Similarly, the SDMT was equivalent to the MMSE in detecting VCI, VaMCI, and its subtypes (AUC: ranging from 0.75 to 0.94 vs. 0.74 to 0.93) except the non-amnestic md-MCI subtype, where the SDMT was superior to the MMSE (AUC (95% CI): 0.97 (0.94–1.00) vs. 0.83 (0.71–0.94), p = 0.01). However, the SDMT did not have statistically larger AUCs compared to the combined measure of MMSE or MoCA with SDMT. By comparison, the combined measure of MMSE and SDMT had significantly larger AUCs in detecting VCI, VaMCI, and its subtypes, except for amnestic sd-MCI compared to MMSE scores alone. Similarly, the combined measure of MoCA and SDMT had significantly larger AUCs in detecting VCI and VaMCI compared to MoCA alone. However, the MoCA combined with SDMT only showed statistically larger AUCs in detecting amnestic md-MCI compared to MoCA alone (supplementary Table S3, available online).
Comparison of discriminant indices of cut-off points
At the recommended cut-off points, the raw scores of MoCA (cut-off at 23/24) were equivalent to the raw scores of MMSE (cut-off at 26/27) in discriminating VCI from NCI (sensitivity: 0.78 vs. 0.71; specificity: 0.80 vs. 0.82; PPV: 0.84 vs. 0.84; NPV: 0.72 vs. 0.67; correctly classified 78.6% vs. 75.5%; p = 0.42) (supplementary Table S4, available online). The MoCA demonstrated similar discriminatory ability compared to the MMSE at their optimal cut-offs in classifying NCI (0.84 vs. 0.80) and VCI (0.72 vs. 0.67). These cut-off points were suited for detecting post-stroke VaMCI as a whole, as well as amnestic md-MCI with similar discriminant indices for MoCA (23/24) and MMSE (26/27) (VaMCI: sensitivity 0.77 vs. 0.69, specificity 0.80 vs. 0.82, PPV 0.83 vs. 0.83, NPV 0.72 vs. 0.67, correctly classified 78.0% vs. 74.8%; p = 0.40; amnestic md-MCI: sensitivity 0.88 vs. 0.83, specificity 0.80 vs. 0.82, PPV 0.75 vs. 0.76, NPV 0.90 vs. 0.88, correctly classified 82.9% vs. 82.5%; p = 0.77, respectively). In addition, the MMSE cut-off point of ≤26 was suitable for detecting non-amnestic sd-MCI while the optimal cut-off point for MoCA in detecting non-amnestic sd-MCI was ≤24. However, the optimal cut-off points for the MMSE and MoCA for the detection of non-amnestic md-MCI were different from VaMCI, and were established at a score of ≤27 and ≤22 respectively (Tables 1 and 2).
Discriminant indices of the MMSE, MoCA, SDMT cut-off scores, and the combined measure (MMSE or MoCA with SDMT) in discriminating VCI and amnestic md-MCI from NCI
MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; NCI = no cognitive impairment; VCI = vascular cognitive impairment; md-MCI = multiple domain- mild cognitive impairment; PPV = Positive Predictive Value; SDMT = symbol digit modalities test; Sens = Sensitivity; Spec = Specificity; NPV = Negative Predictive Value; LR+ = Positive likelihood ratios.
*SDMT impairment cut-off score (SDMT ≤ 13 for education ≤ 6 years, SDMT ≤ 32 for education >6 years).
Discriminant indices of the MMSE, MoCA, SDMT cut-off scores, and the combined measure (MMSE or MoCA with SDMT) in discriminating non-amnestic sd-MCI and non-amnestic md-MCI from NCI
MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; NCI = no cognitive impairment; VCI = vascular cognitive impairment; sd-MCI = single domain-mild cognitive impairment; md-MCI = multiple domain- mild cognitive impairment; PPV = Positive Predictive Value; SDMT = symbol digit modalities test; Sens = Sensitivity; Spec = Specificity; NPV = Negative Predictive Value; LR+ = Positive likelihood ratios.
*SDMT impairment cut-off score (SDMT ≤ 13 for education ≤ 6 years, SDMT ≤ 32 for education >6 years).
The discriminant indices of the SDMT for detecting VCI, amnestic md-MCI, non-amnestic sd-MCI, and non-amnestic md-MCI were shown in Tables 1 and 2. The SDMT showed acceptable sensitivity in detecting amnestic md-MCI with good specificity (sensitivity: 0.77 and specificity 0.96, respectively), and a suboptimal sensitivity and good specificity in detecting VCI, non-amnestic sd-MCI and non-amnestic md-MCI. We further compared the discriminant indices of the MMSE, MoCA, and SDMT cut-off scores, as well as the combined measure of the MMSE or MoCA with SDMT in detecting VCI and VaMCI. The discriminant indices of amnestic sd-MCI were not compared due to equivalent AUCs in the combined measure (MMSE or MoCA with SDMT) and the MMSE or MoCA alone. We established the discriminant indices of the combined measure (MMSE or MoCA with SDMT) based on MMSE or MoCA scores below their respective cut-off points for VCI or VaMCI subtypes and/or SDMT scores below 1.5 standard deviations (SD) of the normative data established for Singaporean elderly (Yeo et al., Reference Yeo, Gabriel, Chen, Lee, Loenneker and Wong1997). As shown in Tables 1 and 2, the combined measure (MMSE or MoCA with SDMT) improved the sensitivity of the MMSE or MoCA substantially in detecting VCI and VaMCI subtypes (except amnestic sd-MCI) with a drop of positive likelihood ratios (LR+).
Discussion
Both the MoCA and MMSE are equal and moderately sensitive in detecting VCI at three to six months after stroke. However, both tests can be supplemented with SDMT to improve their accuracy in VCI screening.
The major contribution of our study is that we examined the utility of a widely used visuomotor processing speed measure, the SDMT, independently and as a supplement to the MMSE or MoCA for VCI screening. We also addressed the limitations identified in previous studies (Godefroy et al., Reference Godefroy2011; Pendlebury et al., Reference Pendlebury, Mariz, Bull, Mehta and Rothwell2012) by attempting to improve the suboptimal sensitivity of the MMSE and MoCA in screening for VCI (<0.80 in the present study and in the study by Pendlebury et al., Reference Pendlebury, Mariz, Bull, Mehta and Rothwell2012) with the addition of the SDMT. We demonstrated that the supplementation of the SDMT improved sensitivity of the MMSE and MoCA substantially in detecting VCI. This may be due to visuomotor processing speed impairment being the most prevalent (61.4%) cognitive domain impairment in patients with VCI in our study. Contrary to previous suggestions (Pendlebury et al., Reference Pendlebury, Mariz, Bull, Mehta and Rothwell2012), the MMSE or MoCA combined with SDMT improved sensitivity in detecting amnestic md-MCI and non-amnestic md-MCI in addition to non-amnestic sd-MCI. This again could be explained by the highly prevalent visuomotor processing speed impairment in patients with amnestic md-MCI (80.2%), non-amnestic md-MCI (86.7%), and non-amnestic sd-MCI (47.2%) in our sample. Additionally, the SDMT measure was found to be equivalent to the MoCA and MMSE in detecting VCI, VaMCI, and its subtypes, except the non-amnestic md-MCI subtype where the SDMT is superior to the MMSE. Moreover, the SDMT could detect amnestic md-MCI independently with acceptable sensitivity and good specificity compared to the MMSE and MoCA. Acceptable discriminant validity, good classification rate, and less than five minutes administration time appeared to qualify the SDMT as a promising VCI screening instrument. However, it could not detect patients without impairment in processing speed, such as isolated memory deficits in amnestic sd-MCI (10.1% in the present study), and therefore not practical for use as a principal VCI screening instrument. By comparison, the MoCA and MMSE possess items covering a wide range of cognitive functions, and have been recognized as global cognitive screening instruments. These two instruments supplemented by the SDMT were better at detecting VCI compared to an independent application of the MoCA, MMSE, or SDMT.
In addition, our finding that the MoCA and MMSE are equivalent in detecting post-stroke VCI is consistent with a previous study conducted by Godefroy et al. at the acute stroke phase (Reference Gorelick2011). However, our finding that the MoCA is equivalent to the MMSE in detecting post-stroke VaMCI and amnestic or non-amnestic sd-MCI differs from an earlier study (Pendlebury et al., Reference Pendlebury, Mariz, Bull, Mehta and Rothwell2012). This may be attributed to a larger sample of 172 patients with post-stroke VaMCI in the present study compared to the previous small sample of 39, which allows for more reliable VaMCI subtypes analysis. For comparison, the present study has a sample size of 107 md-MCI and 65 sd-MCI, while there were only 19 md-MCI and 20 sd-MCI (amnestic sd-MCI = 1) in the previous study.
There are several limitations to the current study. First, we did not establish any cut-off of the MoCA and MMSE stratified by age and education due to the small sample size of the subgroups (e.g. only 15 NCI subjects were age ≥60 and had education ≤6 years). Furthermore, we were unable to establish the scoring adjustment method, as suggested by a previous study (Godefroy et al., Reference Godefroy2011) due to lack of a control group. However, it is practical and relevant to derive the cut-offs using the standard method of education adjustment for the MoCA and raw scores for the MMSE in view of the available sample size in this study. The scores of the MoCA and MMSE have been reported to decline with age and increase with higher levels of education in population-based normative studies (Crum et al., Reference Crum, Anthony, Bassett and Folstein1993; Rossetti et al., Reference Rossetti, Lacritz, Cullum and Weiner2011). Therefore, future studies should derive age and education stratified cut-offs of the MoCA and MMSE in screening for VCI in a larger sample of stroke patients. Second, there is a possible circularity effect of using the SDMT in both the formal neuropsychological battery and the processing speed measure under study. An ideal approach to eliminate the circularity effect would be to employ a processing speed measure external to the formal neuropsychological battery. However, we chose to employ the SDMT because of non-available norms on alternative processing speed measures such as the Digit-Symbol test or Trails Making Test Part A for older adults in Singapore. The diagnosis of VCI in the present study is based on any objective cognitive impairment as defined by a locally validated neuropsychological battery that had been used in previous therapeutic drug trials (Narasimhalu et al., Reference Narasimhalu2009) and clinical studies in a stroke population (Dong et al., Reference Dong2012). In addition to the SDMT, we used two other tests (digit cancellation and maze task) to define the domain of visuomotor processing speed. Therefore, the circularity effect of using the SDMT is mitigated. This is confirmed by the higher frequency of patients who failed at least half of tests in the visuomotor processing speed domain excluding the SDMT, compared to patients who failed the SDMT test (49.3% vs. 36.1%). Third, there were significant mean score increment of both the MoCA and MMSE from the subacute stroke phase to three to six months later. In addition, there were significant correlations between the MoCA and MMSE scores over these two time points. This could be attributed to a practice effect or spontaneous recovery after stroke/TIA over time. A study that includes a control group of stroke-free cognitively normal participants may shed light on the independent contribution of practice effects and spontaneous recovery over time in stroke/TIA patients. Finally, 8.3% of patients did not complete the formal neuropsychological evaluation. However, these patients had milder stroke, therefore, our sample was not skewed towards less impaired patients.
In conclusion, both the MoCA and MMSE are equivalent and moderately sensitive in screening for VCI at three to six months after stroke. However, both tests can be supplemented with SDMT to improve their accuracy in screening for post-stroke VCI.
Conflict of interest
None.
Description of authors’ roles
Y.D. designed the study, performed statistical analysis, interpreted the data, and drafted the papers. M.J.S. designed the study, interpreted the data, and provided critical revision of the paper. B.P-L.C., N.V., V.K.S., and S.L.C. interpreted the data and revised the paper. P.S.S. interpreted the data and provided critical revision of the paper. C.L-H.C. interpreted the data and provided critical revision of the paper.
Acknowledgments
The authors are indebted to the research team from NUHS Memory Ageing and Cognition Center for data collection and the NUHS Medical Publication Support Unit for its editorial support. The authors also thank Ms Angie Russell from CHeBA for her editorial assistance. This study is funded by a Center Grant from Singapore National Medical Research Council (NMRC/CG/NUHS/2010). Y. Dong was a recipient of the NMRC fellowship training award during this study.
