Introduction
Stroke is a major global issue, and it is now the biggest health burden in China (Yang et al., Reference Yang2013). Cognitive sequela of stroke was frequent in around 2/3 patients (Yu et al., Reference Yu2013), exposing a large population to the need of cognitive assessment (Dong et al., Reference Dong2014). Early screening was recommended because post-stroke cognitive impairment was a persistent long-term condition (Douiri et al., Reference Douiri, Rudd and Wolfe2013), which compromised functional recovery and doubled the risk of recurrent stroke (Sibolt et al., Reference Sibolt2013). Unfortunately, however, a significantly hidden dysfunction remains undetected in stroke settings.
Therefore, the efficiency of available cognitive assessment tools was challenged concerning their feasibility and validity (Velayudhan et al., Reference Velayudhan2014). Standard assessment required a comprehensive face-to-face examination using a battery of validated cognitive tests. However, this procedure required much time and effort, may have geographic limits, and it resulted in higher dropout rates in large follow-up studies (Castanho et al., Reference Castanho, Amorim, Zihl, Palha, Sousa and Santos2014). These limitations gave alternatives to telephone-based screening, which was a quicker, more accessible and cost-effective instrument to identify cognitive impairment among potential subjects (e.g. stroke patients).
The neuropsychological battery recommended by the NINDS-CSN was valid and reliable for cognitive assessment in Chinese stroke patients (Hachinski et al., Reference Hachinski2006; Wong et al., Reference Wong2013). In the battery, the 5-minute protocol had been demonstrated to be a valid screening method to detect cognitive impairment in at-risk populations (Kennedy et al., Reference Kennedy2014). The telephone administration of the 5-minute protocol was first tested in a recent UK study, confirming it as a feasible and reliable telephone test of cognition in patients more than one year after they had a transient ischemic attack or stroke (Pendlebury et al., Reference Pendlebury, Welch, Cuthbertson, Mariz, Mehta and Rothwell2013); however, there had not been any published validations in Chinese settings. The SIS was a brief cognitive test administered by telephone or face-to-face interview (Callahan et al., Reference Callahan, Unverzagt, Hui, Perkins and Hendrie2002), it was an effective instrument for dementia detection (Chen et al., Reference Chen, Guo, Cao, Hong and Liu2010), and it had been used to track cognitive status in a US stroke cohort (Wadley et al., Reference Wadley2011). Its Chinese version was validated previously in the outpatient department (Chen et al., Reference Chen, Guo, Cao, Hong and Liu2010), but the efficiency of telephone-based assessment still needed to be explored.
To investigate the feasibility and validity of the telephone-based 5-minute NINDS-CSN protocol and SIS, we tested stroke patients who had recently taken face-to-face Mini-Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA) and the 60-minute NINDS-CSN neuropsychological battery. We hypothesized that both telephone-based cognitive screening instruments were feasible and valid for stroke patients in China.
Methods
Participants
Participants were consecutive hospital-based patients. Between August 2013 and June 2014, patients discharged with a diagnosis of acute ischemic stroke more than three months ago, were invited to take a set of face-to-face neuropsychological assessments after they gave their consent to participation. Exclusion criteria were severe motor or language disabilities, visual or hearing impairments, a pre-existing diagnosis of dementia according to the Informant Questionnaire on Cognitive Decline in the Elderly, scoring 3.4 or more (Fuh et al., Reference Fuh1995), use of cognitive enhancers, Parkinson's disease, hemorrhagic stroke, and patients undergoing stenting or balloon angioplasty. Study recruitment is depicted in Figure 1. The institutional review board at Jinling Hospital approved the study and all the participants provided signed informed consents.
Patients recruitment of the study.
Neuropsychological assessment
The face-to-face assessment included the Chinese version of MMSE, Beijing version of MoCA, and the 60-minute NINDS-CSN neuropsychological battery, which tested the following four cognitive domains: executive/activation (animal naming (Isaacs and Kennie, Reference Isaacs and Kennie1973), WAIS-III Digit symbol-coding (Wechsler, Reference Wechsler1997), trail-making test, parts A and B (Reitan, Reference Reitan1958)), language (modified Boston-naming test (Franzen et al., Reference Franzen, Haut, Rankin and Keefover1995), of which two items with low frequency in Chinese (“octopus” and “beaver”) were replaced with Chinese alternatives “sea horse” and “mouse”), visuoconstruction (Rey-Osterrieth Complex Figure Test (RCFT)-copy trial (Corwin and Bylsma, Reference Corwin and Bylsma1993)), and memory (the Hopkins verbal learning test (Benedict et al., Reference Benedict, Schretlen, Groninger and Brandt1998) revised with one of the items “opal” uncommon to Chinese changed to “diamond,” and RCFT-delayed recall (Corwin and Bylsma, Reference Corwin and Bylsma1993)).
Scores were converted to standardized z scores for each test, and trail-making time scores were multiplied by −1 to maintain consistency of impairment. A summary z score was calculated as the average z scores of tests in executive/activation, language, visuoconstruction, and memory domains. Patients were considered to be cognitively impaired if they had a summary z score below −0.014. This was based on the performances of the 60-minute NINDS-CSN neuropsychological battery in 50 stroke patients and 50 age and sex-matched stroke-free controls in a previous study (Chen et al., Reference Chen2015). By comparison, stroke patients had significantly poorer scores than controls on all individual tests. The 60-minute battery had an AUC value of 0.88, and the cut-off value was optimal with a summary z score of −0.014, reaching a sensitivity of 74% and a specificity of 90%.
Telephone-based cognitive test
All the patients who completed the face-to-face assessment were invited to complete the telephone interview at least one month later. The 5-minute NINDS-CSN protocol was a 30-point scale consisting of five subtests from the Beijing version of MoCA (Wong et al., Reference Wong2013). SIS was composed of date, month, year, three-word-delayed recall from MMSE (Chen et al., Reference Chen, Guo, Cao, Hong and Liu2010). Both tests were administered by X.C. who was blinded to face-to-face findings, and administration time was recorded for each test.
Statistical analysis
Group comparisons between patients with and without cognitive impairment were performed using the χ2 test, independent sample t test, or Mann–Whitney U test, as appropriate. Differences between test scores by telephone versus face-to-face were compared using the Wilcoxon-signed rank test. Receiver operating characteristic (ROC) curve analyses with AUC were used to assess the validity of the 5-minute NINDS-CSN protocol and SIS. An AUC ≥0.7 was considered adequately accurate for classification (Grzybowski and Younger, Reference Grzybowski and Younger1997). Comparisons of AUCs were based on the non-parametric method described by Hanley and McNeil (Reference Hanley and McNeil1982). P < 0.05 indicated statistical significance. All analyses were performed using the SPSS Statistics for Windows, version 17.0.
Results
Eighty-nine stroke patients were recruited, with mean age of 62.9 ± 8.6 years, education level of 9.2 ± 4.2 years, a male percentage of 65.2%, and a median National Institutes of Health Stroke Scale (NIHSS) score of 2 (interquartile range (IQR]) 0.8–5). These patients received a face-to-face assessment after a median of 7.3 (IQR, 4.8–9.4) months post-onset, and 80 of them had a telephone interview at a median delay of 2.8 (IQR, 1.7–7.0) months after the face-to-face interview. MoCA subtests of delayed and recognition memory were better (both P < 0.001) by telephone than face-to-face, while other items showed no difference between telephone and face-to-face administration.
Of the 80 patients who completed both face-to-face and telephone assessment, 39 (48.8%) had cognitive impairment. Compared with those who were cognitively normal, patients with impaired cognition were more likely to be female (51.3% vs. 17.1%, P = 0.001) and were less educated (7.2 ± 4.1 vs. 10.8 ± 3.7, P < 0.001), but no differences were observed regarding age (64.2 ± 9.8 vs. 61.0 ± 7.1, P = 0.092), NIHSS score (3.6 ± 3.5 vs. 2.8 ± 2.9, P = 0.251), and vascular risk factors. In both face-to-face and telephone assessments, after controlling for sex and years of education, patients who were cognitively impaired showed significantly poorer test scores than those who were cognitively normal (Table 1).
Comparison of neuropsychological assessment
*Model controlled for sex and years of education.
Abbreviations: ANT: Animal naming test; DSCT: Digit symbol-coding test; TMT: Trail-making test; BNT: Boston naming test; RCFT: Rey-Osterrieth Complex Figure Test; HVLT-R: Revised Hopkins verbal learning test; NPI-Q: Neuropsychiatric Inventory questionnaire; GDS: Geriatric Depression Scale; MMSE: Mini-Mental State Examination; MoCA: Montreal Cognitive Assessment; NINDS-CSN, the National Institute of Neurological Disorders and Stroke and Canadian Stroke Network; SIS, Six-item screener.
The time required to administer the 5-minute NINDS-CSN protocol was 4.3 ± 1.0 minutes, and SIS 57.3 ± 17.7 seconds. Validity of the telephone-based assessments by AUC was 0.86 (95% CI, 0.78–0.94) for 5-minute NINDS-CSN protocol, and 0.74 (95% CI, 0.63–0.85) for SIS; while for face-to-face MoCA and MMSE was 0.93 (95% CI, 0.88–0.98) and 0.90 (95% CI, 0.83–0.97), respectively (Figure 2). Comparisons between AUCs showed that the 5-minute NINDS-CSN protocol had better diagnostic accuracy than SIS (P = 0.033); while borderline difference was observed between MoCA and the 5-minute NINDS-CSN protocol (P = 0.064), and significant difference between MMSE and SIS (P = 0.007). Sensitivity and specificity were optimal with the cut-off values of 23.5/24 for the 5-minute NINDS-CSN protocol, and 4/5 for SIS (Table 2).
Sensitivities, specificities, positive and negative predictive values for patients with cognitive impairment by SIS and 5-minute NINDS-CSN protocol
Abbreviations: SIS, six-item screener; NINDS-CSN, the National Institute of Neurological Disorders and Stroke and Canadian Stroke Network; PPV, positive predictive value; NPV, negative predictive value.
ROC curves of the face-to-face MoCA, MMSE, 5-minute NINDS-CSN protocol and SIS in detecting patients with cognitive impairment.
Discussion
In this study, we showed that the telephone-based 5-minute NINDS-CSN protocol and SIS were feasible and valid in screening cognitive impairment after stroke in China.
Valid telephone screening of cognitive impairment was lacking in stroke settings, especially for the Chinese population. A recent UK community study first tested the 5-minute NINDS-CSN protocol by telephone in patients with transient ischemic attack or mild stroke, and demonstrated sufficient reliability to detect mild cognitive impairment (Pendlebury et al., Reference Pendlebury, Welch, Cuthbertson, Mariz, Mehta and Rothwell2013). Our study extended their finding by verifying this telephone-based test as well as SIS in Chinese patients, based on the previously validated face-to-face versions (Chen et al., Reference Chen, Guo, Cao, Hong and Liu2010; Wong et al., Reference Wong2013). Note that we observed a better performance for certain MoCA subtests (delayed and recognition memory) by telephone than face-to-face, possibly because of the practice effect with repeated MoCA and MMSE items, or that patients might ask for external aids when performing these tests (Castanho et al., Reference Castanho, Amorim, Zihl, Palha, Sousa and Santos2014); besides, a reversible cognitive impairment over time could also contribute to improved performance, but this cognitive recovery was usually observed in patients within three months of stroke onset (Sivakumar et al., Reference Sivakumar, Kate, Jeerakathil, Camicioli, Buck and Butcher2014), while in our study, patients were assessed after a median of 7.3 months post-stroke, when there was less cognitive fluctuation related to perfusion abnormalities (Koga et al., Reference Koga2005) and the prevalence of cognitive impairment was constant up to five years after stroke (Douiri et al., Reference Douiri, Rudd and Wolfe2013).
Our study was probably one of the first to show the feasibility and validity of telephone-based cognitive screening tests in Chinese stroke patients. We found that both the 5-minute NINDS-CSN protocol and SIS were well accepted by Chinese stroke patients, with administration time of around five minutes and one minute, respectively. Moreover, both tests proved to have adequate discriminating ability to detect patients with cognitive impairment, which could then be applied as effective alternatives for cognitive screening, and in that way contribute to increasing sample size and facilitating follow-up in future studies.
There were several limitations in our study. First, most patients (66/89, 74.2%) suffered a relatively mild stroke, and those with more severe stroke might encounter difficulty when taking tests. Second, our sample size was small, and further large studies would be urged to confirm our findings. Third, we did not evaluate the test-retest reliability of the instruments, as both tests had well-established face-to-face versions (Chen et al., Reference Chen, Guo, Cao, Hong and Liu2010; Wong et al., Reference Wong2013). Fourth, the two telephone tests were not presented in a counterbalanced order, which would possibly build up proactive inhibition (Wilson, Reference Wilson1984), that is, the three words to remember in SIS might interfere with the retention of new learning of five words in the 5-minute NINDS-CSN protocol; however, memory contents of the two tests shared few overlapping features, so impairment in recall would be reduced (Lange and Oberauer, Reference Lange and Oberauer2005). Finally, our study was cross-sectional, hence the validity in longitudinal assessment still needed to be investigated, and future studies on the sensitivity for cognitive progression would be warranted.
In conclusion, the telephone-based instruments of both the 5-minute NINDS-CSN protocol and SIS were brief and valid screening tools for cognitive impairment in Chinese stroke patients. The telephone assessment was inadequate for diagnosis and in that case, face-to-face evaluation was essential together with comprehensive clinical examinations.
Conflict of interest
None.
Description of authors’ roles
Xinfeng Liu designed the study and supervised the data collection. Xiangliang Chen initiated the study and drafted the manuscript. Xinying Fan preformed the statistical analysis and gave critical comments on the manuscript. Lingling Zhao and Lihui Duan were involved in data collection and interpretation. Zhaojun Wang assisted with collecting the data. Yunfei Han critically revised the manuscript. Qiliang Dai was responsible for the statistical design of the study. Dezhi Liu and Xiaohao Zhang assisted with writing the paper. Zhonglun Chen supervised the data collection. Yunyun Xiong contributed to the study design.
Acknowledgments
We thank all the patients involved in the assessment. The project was supported by National Natural Science Foundation of China (Grant Number: 81070922, 81201078, and 81300244).
