Introduction
In China, the focus on dementia, which is an aging-related condition, has been on the increase; this disease has adverse effects on the quality of life of the patient and is also associated with socioeconomic problems (Wang et al., Reference Wang2008). MCI is an intermediate clinical state between normal cognitive aging and dementia, and it precedes and leads to increased risk for dementia (Petersen et al., Reference Petersen2001), particularly Alzheimer's disease (AD) (Morris et al., Reference Morris2001). Therefore, methods of detecting and treating MCI effectively are important to keep a check on the development of dementia among older adults (Petersen et al., Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999). The MoCA is a brief cognitive test that has been developed specifically for screening patients with MCI (Nasreddine et al., Reference Nasreddine2005). It has demonstrated excellent reliability and validity for differentiating patients with MCI from healthy individuals worldwide (Lee et al., Reference Lee2008; Luis et al., Reference Luis, Keegan and Mullan2009; Fujiwara et al., Reference Fujiwara2010; Memoria et al., Reference Memoria, Yassuda, Nakano and Forlenza2013).
There are five Chinese versions of the MoCA, including the Beijing version, the Cantonese version, the Changsha version, the Hong Kong version and the Taiwan version (available at the MoCA official website: http://www.mocatest.org/). Because the instructions in the MoCA-BJ are in Mandarin, it is the most popular version in mainland China. Similar to many other linguistic versions of the MoCA, the Beijing version has been shown to have high sensitivity and specificity for screening patients with MCI among older participants (Lu et al., Reference Lu2011; Yu et al., Reference Yu, Li and Huang2012; Hu et al., Reference Hu2013). Furthermore, in Chinese hospitals, it is used to detect cognitive impartment in different neurologic diseases and multiple conditions (Chen et al., Reference Chen2011; Nie et al., Reference Nie2012; Zhao et al., Reference Zhao, Tang, Sun, Wang, Chen and Fu2012).
According to the sixth National Population Census, the Chinese aging population has been growing rapidly, and as of 2010, there were approximately 120 million older adults (over 65 years) in China. Among the different aging-related diseases, dementia in particular has shown a rapid increase in incidence in the country (Nie et al., Reference Nie2011). Therefore, there is a need for more regular cognitive screening using efficient methods. Screening of elderly adults with MCI and dementia is not the responsibility of hospitals alone: community clinics need to play a role too. However, similar to many other developing countries, in the Chinese community clinical setting, the main focus is on screening elderly patients for common chronic diseases such as hypertension, diabetes, respiratory tract infections and lumbago. In the community clinical setting, there is a need for more professionals and skilled personnel in the neuropsychological field. For example, the usage of instructions during cognition assessment has not been standardized among primary care physicians in China. Moreover, because the original version of the MoCA is a paper-pencil test, the presentation of information and data output is inconvenient. A computer software that could improve the technique for conducting the MoCA would be valuable.
The goals of the present study are to develop a computerized tool for the Chinese version of the MoCA (MoCA-CC) and to evaluate its effectiveness. The optimal cut-off score for screening MCI was also determined.
Materials and methods
Development of MoCA-CC
The research team maintained the original manipulation and scoring system of MoCA-BJ, but changed the testing pattern from one-on-one interaction between the patient and physician to person-machine interaction. The team comprised three neurologists who are proficient with the MoCA and one computer engineer who has considerable experience with cognitive assessment software. One of the neurologists was appointed as the head of the team, whose responsibility was to organize and manage the process during the development of the tool. The detailed process is as follows. Two neurologists converted business requirements for each MoCA-BJ item into computerized operation tasks. The leader audited each conversion and created the final list of tasks. The engineer wrote the programs for the designed tasks and set up the preliminary software. The software was tested in clinical practice on a small sample, and feedback was collected from physicians and patients. On the basis of the feedback, the research team adjusted and modified the items to optimize the software. This procedure was repeated until the outcome was satisfactory.
The Chinese MoCA software developed here comprises an input system, test system, output system, and management system, all of which can be made available for different Microsoft Windows operating systems. The system was set up for three different types of users: “Testing,” “Doctor” and “Administrator.” When MoCA-CC begins, the user can hear the instructions in the standard pronunciation. At the same time, the test process is video recorded on the computer. When the test is over, the cognitive assessment report can be printed directly, and the database can be imported to an Excel spreadsheet for storage and analysis. Figure 1 illustrates the structure and function of MoCA-CC.
Schematic representation of the testing process using the Chinese computerized tool for the Montreal Cognitive Assessment.
Participants
We selected 181 participants (aged 65 years and over) who were divided into two groups: normal control (NC) group (n = 85) and the MCI group (n = 96). The patients with MCI were recruited from the outpatients registered at the Neurology Department of Chengdu Military General Hospital, and the NC participants were selected from the elderly who periodically underwent physical examination at the Department of Geriatrics of the same hospital. A common inclusion criteria for two groups were as follows: (i) fluency in Chinese, (ii) more than four years of education, and (iii) no history of psychiatric or neurological disease. The review board of our institution approved the study protocol, and all the participants or their caregivers provided their informed consent.
The NC participants could independently perform their daily living activities, and had no memory or cognitive complaints. They had no active disease and performed normally in a Chinese version of the Mini-Mental State Examination (MMSE) (Katzman et al., Reference Katzman1988) and the Clinical Dementia Rating (CDR) (Morris, Reference Morris1993).
MCI was diagnosed based on the criteria established by Petersen et al. (Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999; Reference Petersen2001): (i) subjective cognitive complaint, preferably corroborated by a reliable informant; (ii) objective memory impairment based on the age and education; (iii) preserved general cognitive function; (iv) mainly intact ability to perform activities of daily living, and (v) no dementia (CDR score < 0.5). MCI were classified as amnestic subtype (single or multiple domains) or non-amnestic subtype (single or multiple domains). In our study, the patients with MCI were composed of 25 amnestic MCI/single domain, 39 amnestic MCI/multiple domains, 9 non-amnestic MCI/single domain and 23 non-amnestic MCI/multiple domain.
Instruments
The instruments used in this study included MoCA-BJ and MoCA-CC. The MoCA-BJ applied in the current study has been translated by Wei Wang and Hengge Xie, and published on the MoCA official website (http://www.mocatest.org/). Except for a few slight modifications because of cultural and linguistic differences, MoCA-BJ is otherwise true to the original English version of the MoCA. It is a ten-minute test that covers eight cognitive domains: attention and concentration, executive function, memory, language, visuoconstructional skills, conceptual thinking, calculation, and orientation. According to the administration and scoring instructions, the total possible score is 30 points: a score of 26 or above is considered normal. To better adjust the MoCA for participants with lower education, two points should be added to the total MoCA score for participants with 4–9 years of education and one point should be added for participants with 10–12 years of education. MoCA-CC is a computerized version that is true to MoCA-BJ.
Procedures
Above all, a battery of neuropsychological tests (including MMSE and CDR), the neurological examination and brain magnetic resonance imaging were conducted using standard procedures. Then, based on the clinical data, the diagnosis was made by a team comprising a neurologist, a geriatrician and a psychiatrist. After that, the participants were administered MoCA-BJ and MoCA-CC by two trained neurologists: one was responsible for MoCA-BJ and another administered MoCA-CC. To avoid any experimental bias caused by repeated assessment, the subgroups were established randomly and MoCA-CC and MoCA-BJ were also administered in a different sequence in the subgroups. The participants of NC subgroup I and MCI subgroup I were evaluated by MoCA-CC first and by MoCA-BJ after six weeks. The participants of NC subgroup II and MCI subgroup II were administered the same two tests but in the opposite sequence. The participants of NC subgroup III and MCI subgroup III were evaluated twice with MoCA-CC at a six-week interval. Figure 2 illustrates the subgroup establishment and the procedures used in the present study.
Procedures used for administering the cognitive test in the present study. MoCA-BJ = Beijing version of the Montreal Cognitive Assessment. MoCA-CC = Chinese computerized tool for the Montreal Cognitive Assessment. MCI = Mild Cognitive Impairment. NC = Normal Controls.
Statistical analyses
All the statistical analyses were conducted using the Statistical Package for the Social Sciences version 13.0 (SPSS, Chicago, IL, USA). Gender distribution was compared using the χ2 test. Mean values of age, education, and MoCA-BJ and MoCA-CC scores between the two groups were compared by the independent-sample t-test. To measure the internal consistency of MoCA-CC, Cronbach's α was calculated. Test–retest reliability was assessed using intraclass correlation coefficients (ICCs) for the baseline and follow-up MoCA-CC scores. Pearson correlation coefficients between the MoCA-CC and MoCA-BJ scores were used to index criterion-related validity. In order to assess the predictive value of MoCA-CC, the area under the curve (AUC) in the receiver-operator curve (ROC) analysis was calculated to determine the sensitivity and specificity of the method for detecting MCI. A larger AUC value indicated better diagnostic performance. P < 0.05 was considered to indicate statistical significance.
Results
Completion rate for MoCA-CC
A total of 176 (97.24%) participants were evaluated successfully using MoCA-CC. Only five participants (one from NC subgroup I, one from NC subgroup III, two from MCI subgroup II, and one from MCI subgroup III) failed to complete the test due to lack of experience and skill with the computer. Only two NC subgroup III members were lost to follow-up. The data of these seven participants were eliminated from subsequent statistical analyses.
Demographic findings and MoCA scores
Because the 56 participants of NC subgroup III and MCI subgroup III were evaluated twice with MoCA-CC to determine its test−retest reliability, the demographic data and MoCA-CC scores of these participants are not discussed below.
The demographic characteristics of subgroup I and subgroup II are displayed in Table 1. Group differences were not found in age (t = 1.49, P = 0.14), sex ratio (χ2 = 0.37, P = 0.68) and mean education (t = 0.06, P = 0.96).
Comparison of demographic data and mean scores (mean ± standard errors) for MoCA-BJ and MoCA-CC between the MCI and NC groups
MoCA-BJ = Beijing version of the Montreal Cognitive Assessment. MoCA-CC = Chinese computerized tool for the Montreal Cognitive Assessment.MCI = Mild cognitive impairment. NC = normal controls.
Table 1 also shows descriptive statistics for the MoCA-BJ and MoCA-CC. MoCA-BJ total score (t = 17.35, P < 0.001) and individual item scores (t = 2.26–13.02, P < 0.05) were significantly different between the NC and MCI groups, with the exception of the item “Naming,” (t = 1.05, P = 0.81). Similarly, there were significant differences between the two groups with regard to both MoCA-CC total score (t = 9.38, P < 0.001) and individual item scores (t = 2.18–8.62, P < 0.05), the only exception was the score for the item “Naming” (t = 0.24, P = 0.81).
Internal consistency and test–retest reliability
The Cronbach's α value for MoCA-CC, which was used as an index of internal consistency, was 0.72. The 26 NC subgroup III participants and 30 MCI subgroup III participants were evaluated by MoCA-CC at a 6-week interval for determining its test–retest reliability: the intraclass correlation coefficient between the baseline and follow-up scores was 0.82 (P < 0.001).
Concurrent validity
Concurrent validity data were collected for a subsample of 63 MCI participants (32 from MCI subgroup I and 31 from MCI subgroup II). MoCA-CC total scores were found to be highly and positively associated with the MoCA-BJ total scores (r = 0.93, P < 0.001).
Sensitivity and specificity
Figure 3 shows the predictive accuracy of MoCA-CC for screening for MCI. The AUC of the MoCA-C scores was 0.97 (95% CI = 0.95–1.00), which shows good diagnostic validity.
Receiver operating characteristic (ROC) curve showing the sensitivity and specificity of the Chinese computerized tool for the Montreal Cognitive Assessment (MoCA-CC) for distinguishing between patients with mild cognitive impairment and normal Chinese elderly. The area under the ROC curve was 0.97 (95% confidence interval = 0.95–1.00), which indicated good diagnostic validity.
Table 2 demonstrates the sensitivity and specificity of MoCA-CC for detecting MCI according to different cut-off points. In the present study, the best cut-off point was 25/26: the sensitivity and specificity at this cut-off point were 95.8% and 87.1% respectively.
Sensitivity, specificity, positive predictive value and negative predictive value of MoCA-CC for the detection of mild cognitive impairment
MoCA-CC = Chinese computerized tool for the Montreal Cognitive Assessment. SE = Sensitivity; SP = Specificity; PPV = Positive predictive value; NPV = Negative predictive value
Discussion
The purpose of this study was to develop an efficient and practical computer tool for the Chinese version of MoCA (MoCA-CC). There are two reasons for choosing MoCA-BJ as a model to develop MOCA-CC. For one, MoCA-BJ is the most widely used version of MoCA in China. Second, although a few modifications were made to the original version of MoCA to create MoCA-BJ, it has still been proved to be suitable for screening cognitive decline in elderly Chinese individuals (Lu et al., Reference Lu2011; Zhao et al., Reference Zhao2011; Hu et al., Reference Hu2013). It is important for a cognitive assessment tool to have a high acceptance rate and completion rate; moreover, it should have adequate reliability and validity. We therefore tested MoCA-CC on 181 participants in order to evaluate this new tool. MoCA-CC demonstrated a satisfactory test completion rate, excellent internal consistency and high test–retest reliability. Moreover, similar to the performance of MoCA-BJ in the study, it showed adequate concurrent validity and outstanding ability to differentiate between NC and MCI participants. The results also indicated that the sensitivity and specificity of MoCA-CC were satisfactory with regard to screening MCI patients.
Since MOCA-CC is a computerized version of MoCA-BJ, the test results need to be compared between these two versions of MoCA. First, the mean MoCA-CC scores for the MCI participants were similar to the MoCA-BJ scores. Second, the internal consistency of MoCA-CC was close to that reported by studies on MoCA-BJ (Lu et al., Reference Lu2011; Zhao et al., Reference Zhao2011; Yu et al., Reference Yu, Li and Huang2012). Third, the optimal cut-off score for detecting MCI was equal to that calculated for MoCA-BJ in a previous study (Hu et al., Reference Hu2013). A notable finding was that the scores of the item “Naming” did not differ significantly between the NC and MCI groups in the participants evaluated by MoCA-CC; however, we believe that this does not pose an issue because similar results were found for MoCA-BJ. A possible reason for this is that the word “rhinoceros” could not be recognized by many participants in both the NC group and MCI group. Similar results have been reported by other MoCA-BJ-related studies on the Chinese elderly (Nie et al., Reference Nie2012; Wu et al., Reference Wu, Wang, Ren and Xu2013). It is likely that the naming item is not culturally appropriate for use with Chinese elderly.
The purpose of developing MoCA-CC is mainly to assist community physicians in MCI screening and to establish a database for cognitive function. However, this tool does offer some other advantages. First, tests can be replayed as a result of the video recording function, which was not possible with the traditional method; therefore, in case of any confusion, physicians can re-evaluate the test and confirm the findings. Second, the software is easy to transport, set up and synchronize, so it is possible to perform group assessments. Third, for most people, the computer test is more engaging, and it can improve the motivation and concentration of the participants and reduce stress during the assessment.
Some limitations of this study should be mentioned. First, the sample only represented Sichuan province and was small. Second, the MCI group sample was heterogeneous in our study, MoCA-CC may have different assessment effect on each subtype of MCI. Third, we could not apply MoCA-CC to participants with less than four years of education, because a basic education level is required for the completion of a few MoCA tasks. Finally, participants with MCI in the study were recruited from a hospital, so it was not a community study and data from other samples are required.
In conclusion, as a computerized cognitive assessment tool derived from MoCA-BJ, MoCA-CC has high validity for the screening of cognitive impairment. It also has certain obvious advantages compared to the traditional “paper-pencil” technique. More studies using this computerized cognitive assessment tool should be performed, especially in different population groups, in order to further validate it.
Conflict of interest
None have financial or other conflicts of interest in relation to this research and its publication. We promise that this study was only conducted for research purposes, and not for any commercial purposes.
Description of authors’ roles
Ke Yu designed the study, preformed the statistical analysis, and wrote the first draft. Shangang Zhang and Qingsong Wang assisted with formulating the research question, designing the study, and writing the paper. Xiaofei Wang contributed to develop the computerized tool of the MoCA. Yang Qin, Jian Wang and Congyang Li were responsible for diagnosing MCI. Yuxian Wu and Weiwen Wang performed the Cognitive evaluation with MoCA-BJ and MoCA-CC respectively. Hang Lin was involved in the participant recruitment. All authors read and approved the final paper.
Acknowledgments
This study was supported by a research grant to K. Yu from the Medical Scientific Research Foundation of Chengdu Military General Hospital (No. 2011YG-B37), and grants to Q. Wang from the Twelfth Five-Year Plan for Medical Projects of Chengdu Military Command (No.C12045) and the Key Medical Project of Chengdu Military General Hospital (No.2013YG-A006). We thank Dr. Z. Nasreddine for the Montreal Cognitive Assessment.
