Introduction
The objective characterization of cognitive impairment is required for the clinical diagnosis of dementia. In most cases such evidence can be time-effectively provided by the expert use of cognitive screening tests (McKhann et al., Reference McKhann2011). However, this approach may not be sufficient to portray the cognitive syndrome that arises at an incipient stage of dementia. The condition recently referred to as mild cognitive impairment (MCI) due to Alzheimer's disease (AD; Albert et al., Reference Albert2011) is one such example. In such cases, subtle evidence of cognitive impairment may be depicted by formal neuropsychological testing in one or more cognitive domains – mostly episodic memory and executive dysfunction (Forlenza et al., Reference Forlenza, Diniz, Nunes, Memoria, Yassuda and Gattaz2009) __ in spite of preserved global cognitive and functional status. Therefore, the clinical diagnosis of incipient and pre-dementia AD is acutely dependent on a comprehensive neuropsychological assessment.
The clinical diagnosis of MCI can be particularly difficult because its cognitive manifestations may overlap with benign changes that occur as a consequence of normal aging. Although formal neuropsychological assessment is the gold-standard approach to identify mild forms of cognitive dysfunction, it is costly, time-consuming, and requires the intervention of an expert professional. The use of computerized screening protocols is an efficient strategy in this diagnostic work-up. In addition to covering a wide range of cognitive abilities and accurately recording latency and speed of response, the computerized format of test sessions renders it less prone to administration bias (Wild et al., Reference Wild, Howieson, Webbe, Seelye and Kaye2008). On the other hand, computerized testing does not account for qualitative information that an experienced neuropsychologist can depict when conducting the assessment.
The adaptation of cognitive test batteries into computer-based formats has been attempted by different groups in the recent past. Most such batteries were initially devised to identify differences between normal controls (NC) and patients at early stages of dementia, e.g., the Cambridge Neuropsychological Test Automated Battery (CANTAB; Robbins et al., Reference Robbins, James, Owen, Sahakian, McInnes and Rabbitt1994). For instance, Égerházi et al. (Reference Égerházi, Berecz, Bartók and Degrell2007) compared the performance of 40 cognitively impaired elders with MCI and AD on the CANTAB schedule, indicating (as expected) a worse performance of AD patients in all tests, but particularly in the paired associate learning subtest (which assesses episodic memory), in which both patient groups had lower scores as compared with age-corrected norms. In Brazil, Charchat et al. (Reference Charchat, Nitrini, Caramelli and Sameshima2001) investigated the contribution of the Computerized Neuropsychological Test Battery (CNTB) to identify neuropsychological markers of AD. They showed that episodic memory, short-term memory, and choice reaction time were sensitive and specific to discriminate cases of AD from normal controls.
The Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI; Tornatore et al., Reference Tornatore, Hill, Laboff and McGann2005) was designed to be sensitive to the subtle changes that arise in the MCI–AD continuum to differentiate mildly impaired individuals from normal controls. The CANS-MCI assesses multiple cognitive domains based on a three-factor model of cognition (memory, language/spatial fluency, and executive function/mental control). The test schedule addresses cognitive abilities that are allegedly impaired in incipient AD (Tornatore et al., Reference Tornatore, Hill, Laboff and McGann2005), namely executive inhibitory functions, episodic memory, spatial relations, and language recognition and retrieval. In addition to the accuracy of answers, the test also provides measures of reaction/response time. In the CANS-MCI, the participants see and hear instructions to increase comprehension and reduce the possibility of errors from sensory impairment. The CANS-MCI was originally developed in US English, and it has been translated and/or culturally adapted into other languages such as UK English, Dutch, Portuguese, French, and Spanish (www.screen-inc.com). Preliminary studies have shown that CANS-MCI is sensitive to the changes that occur in the transition from MCI to incipient dementia because of AD (Tornatore et al., Reference Tornatore, Hill, Laboff and McGann2005).
To date, only two studies have addressed the diagnostic accuracy of the CANS-MCI as compared with standard or other computerized cognitive assessment batteries. In a sample of highly educated individuals, Snyder et al. (Reference Snyder2011) found that the CANS-MCI more accurately identified cases of MCI than other comparison tests, namely Computer Assessment of Memory and Cognitive Impairment (CAMCI; Saxton et al., Reference Saxton, Morrow, Eschman, Archer, Luther and Zuccolotto2009), CogState (Hammers et al., Reference Hammers2012), and Mindstreams (Dwolatzky et al., Reference Dwolatzky2003). Ahmed et al. (Reference Ahmed, de Jager and Wilcock2012) compared the diagnostic accuracy of three cognitive screening tests in a sample of older adults; the CANS-MCI and the Montreal Cognitive Assessment (MoCA) more accurately differentiated cases of MCI from normal controls as compared with the Revised Addenbrook's Cognitive Examination (ACE-R).
The aims of the present study were to examine the psychometric characteristics of the Brazilian version of CANS-MCI, and to determine its accuracy in diagnostic screening for MCI in a sample of older adults with varying degrees of cognitive impairment (ranging from normal controls to mild dementia) as compared with the Brazilian version of the MoCA test.
Material and Methods
Participants
Ninety-seven older adults (age range: 65 to 88 years) with moderate to high education (4 to 20 years of formal education) and predominantly female (71%) were assessed at a university-based memory clinic. All participants were volunteers in a prospective study on cognitive aging, currently in course at the Institute of Psychiatry, University of São Paulo. This cohort includes cognitively unimpaired elders with or without memory complaints, in addition to patients with MCI and dementia, who were recruited from community sources (largely from the hospital catchment area), reinforced by institutional media advertisements. In a minority of cases, patients were referred from other clinics within the hospital complex for the assessment of suspected cognitive decline.
Participants were invited to participate if they were minimum 65 years of age and had at least four years of schooling. Eligible participants whose cognitive symptoms might be secondary to other neurological disorders (including non-AD dementia), or clinically relevant medical conditions with impact on cognition and/or not adequately compensated (e.g., cardiovascular diseases, diabetes mellitus, hypertension), were excluded from the present analysis. Additional exclusion criteria were severe sensory limitations, important tremor, or other movement disorders (i.e., conditions that might impair the completion of tests) and concomitant psychiatric illnesses, such as schizophrenia, bipolar disorder, severe or recurrent depression, and dependence on alcohol and drugs.
Participants were allocated into diagnostic groups according to their global cognitive state (normal controls, n = 41; MCI, n = 35; AD, n = 21). These diagnoses were ascertained clinically at multidisciplinary consensus sessions, taking into account medical, neuropsychological, laboratorial, and neuroimaging data. In the AD group, only mild cases were included, as indicated by the Mini-Mental State Examination (MMSE) scores higher than 19. The diagnosis of MCI was made according to the Mayo Clinic criteria (Petersen et al., Reference Petersen1999; Petersen, Reference Petersen2004): (1) Subjective cognitive complaint, preferably corroborated by a reliable informant; (2) objective cognitive impairment in neuropsychological assessment; (3) preserved global intellectual function; (4) preserved or minimal impairment in activities of daily living, and (5) non-demented. For cognitive testing, a complete neuropsychological battery was applied (for a description of the implementation of MCI and AD criteria, the reader is referred to Yassuda et al., Reference Yassuda2009). Patients with MCI can be classified as amnestic (single or multiple domains) or non-amnestic (single or multiple domains). Our sample comprised both types of MCI (26 amnestic and nine non-amnestic). Diagnosis of probable AD was based on the National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer's Disease and Related Disorders Association criteria (NINCDS–ADRDA; McKhann et al., Reference McKhann, Drachman, Folstein, Katzman, Price and Stadlan1984).
The present study was approved by the local ethical committee and was performed in accordance with the Helsinki Declaration. All participants signed the informed consent form, which provided information on the nature of the present study. In brief, they were explained that additional data would be collected in an effort to investigate the psychometric characteristics of a new, computer-based cognitive screening test (CANS-MCI) as compared with other neuropsychological instruments.
Procedures and instruments
The following conventional neuropsychological tests were used to determine the concurrent validity of the CANS-MCI: MoCA (Nasreddine et al., Reference Nasreddine2005; Memória et al., Reference Memória, Yassuda, Nakano and Forlenza2013), Rey Auditory Verbal Learning Test (RAVLT; Rey, Reference Rey1964; Malloy-Diniz et al., Reference Malloy-Diniz, Lasmar, Gazinelli, Fuentes and Salgado2007), semantic verbal fluency (VF; Caramelli et al., Reference Caramelli, Carthery-Goulart, Porto, Charchat-Fichman and Nitrini2007), and Digit Symbol and Digit Span (Weschler Adult Intelligence Scale; Nascimento et al., 2004; Wechsler, Reference Wechsler2004).
Participants were tested with the CANS-MCI at baseline and retested three months later to assess temporal stability. The retest sample comprised 25 participants recruited from the three diagnostic groups and was comparable with the study sample in terms of socio-demographic data and cognitive performance (data not shown). Testing sessions were guided by recorded instructions, with minimal interference from the examiner. It required the utilization of a touch screen monitor by the examinee. Test instructions were repeated as required by the examinee, or automatically reinforced by the computer program when the pattern of errors suggested that instructions were misunderstood. The CANS-MCI test sessions are supposed to last for approximately half an hour in cognitively unimpaired (or minimally impaired) individuals; longer sessions may be required in the presence of more severe cognitive impairment. Since in the present study we included a subsample of patients with mild dementia, testing sessions had a mean duration of 50 minutes.
The CANS-MCI test addresses three major cognitive domains, namely executive function (EF), language fluency (LF), and memory. The EF domain assesses mental control and spatial ability. The former is measured by an adaptation of the Stroop test (the examinee is asked to tell the color of the ink with which a given word is written, rather than the reading of the word itself), and the latter comprises general reaction time subtests with minimal cognitive complexity (touching ascending numbers presented on a jumbled display; design–letter matching; word-to-picture matching; and the clock hand placement test by touching hour and minute positions for a series of designated times). Memory for the names of 20 common objects is assessed with an immediate and a delayed recognition test. Language fluency is tested with a picture naming test (pictures are presented along with four two-letter word beginnings and the participant has to choose the correct name).
Translation and cross-cultural adaptation of the CANS-MCI
Permissions to adapt the CANS-MCI to Brazilian Portuguese and to use it for research purposes were provided by the authors. A multidisciplinary panel comprising bilingual psychiatrists and neuropsychologists specialized in cognitive assessment of older adults was engaged in the translation and cultural adaptation of the test. A few pictures were replaced for the sake of cultural and local adequacy, e.g., “baseball bat” was replaced by “boxing gloves,” and “Statue of Liberty” was replaced by “Christ the Redeemer” (see Appendix A for a complete list of the changes made to the original test). The CANS-MCI has a version for drivers and another for non-drivers; the latter form of the test was used in the present study because the majority of participants were non-drivers.
Pilot testing of the Brazilian Portuguese version of the CANS-MCI was carried out in a group of eight randomly selected patients from our cohort, and no relevant problems related to the comprehension of test instructions were identified.
Statistical analyses
Statistical analyses were conducted with the aid of SPSS for Windows 14.0. χ 2 tests were used to analyze the distribution of categorical variables across the three diagnostic groups (e.g., gender). ANOVAs followed by Tukey's post-hoc tests were used to compare mean values of continuous variables (e.g., age and education in years). Internal consistency was calculated using Cronbach's α. Intraclass correlation coefficients (ICCs) were used to assess stability over time. Z-scores obtained for distinct CANS-MCI tests and factors were compared across diagnostic groups by means of ANOVAs with Tukey's post-hoc tests. Pearson's correlations were used to test the association between conventional neuropsychological test scores and the CANS-MCI sub-domains.
Exploratory Factor Analysis was used to arrange the variables in domains using principal components extraction. Kaiser criterion was used to choose the number of factors, and the result was rotated under Varimax procedure to ease interpretation. For confirmatory factor analysis, AMOS 16 was used to evaluate the structure previously found using the maximum likelihood method. The goodness-of-fit was assessed with the Bentler Comparative Fit Index and the Root Mean Squared Error Approximation.
Receiver Operating Characteristic (ROC) curves were used to estimate the best CANS-MCI cut-off scores to discriminate diagnostic groups, considering consensus diagnosis as the gold standard. Power analyses considering 90% of power were carried out and suggested that each diagnostic group should contain at least 21 individuals to guarantee that one standard deviation difference would be statistically significant. All tests considered bilateral hypotheses and a 5% level of significance.
Results
Table 1 summarizes demographic and cognitive information across diagnostic groups. Normal controls were significantly younger than the AD patients. There were no significant differences in the educational level among diagnostic groups. The proportion of men in the AD group was significantly higher than in the normal controls and MCI groups. The CANS-MCI total score and its factor sub-scores discriminated the three diagnostic groups (p < 0.001). ANCOVAs were conducted with age as a covariate, and group differences were maintained (p < 0.001). As to the influence of socio-demographic factors, the total CANS-MCI score was negatively correlated with age (r = −0.409; p < 0.001) and positively correlated with schooling (r = 0.245; p = 0.033).
Demographic and cognitive information of the sample (n = 97)
Notes: NC = normal controls; MCI = Mild Cognitive Impairment; AD = Alzheimer's disease.
aSignificantly different from NC.
bSignificantly different from MCI.
cSignificantly different from AD.
# χ 2 test.
*ANOVA with Tukey's post-hoc tests.
The internal consistency of each CANS-MCI subtest was high (Cronbach's α > 0.70), as can be seen in Table 2. The internal consistency of the battery was also high (Cronbach's α = 0.77). The test-retest reliability data were collected for a subsample after three months. The intra-class correlation coefficient between the scores at baseline and the retest was 0.875 (p < 0.001), considering the total score. Total scores in the two evaluations did not differ significantly (p = 0.715), suggesting stability over time.
Internal consistency of the CANS-MCI subtests
Note. NA = Since all participants did not answer the same number of items, reliability analyses could not be performed.
There was a high correlation between the Brazilian version of MoCA test (MoCA-BR) and the CANS-MCI total score (r = 0.76, p < 0.001). Concurrent validity of the CANS-MCI was also evaluated by examining the correlations of the CANS-MCI subtests with the score on the Digit Symbol, RAVLT scores, and verbal fluency. The correlations between each CANS-MCI subtest and its corresponding domain-specific conventional neuropsychological tests are listed in Table 3. The coefficients ranged from 0.37 to 0.61 and all of these were highly significant (p < 0.005).
Correlations between conventional tests with the CANS-MCI subtests
Note: All tests were administered within the same testing session.
†Based on z-score.
*Correlation is significant at 0.01 level (2-tailed).
To evaluate the diagnostic accuracy of the Brazilian version of the CANS-MCI, ROC curve analyses were performed to compare pairs of diagnostic groups (NC × MCI and NC × AD). The CANS-MCI total score showed 81% of sensitivity and 73% of specificity (Area Under Curve (AUC) = 0.80 ± 0.05, p < 0.001) to discriminate MCI from normal controls (Table 4). The CANS-MCI had excellent accuracy to discriminate normal controls from AD, with 100% of sensitivity and 100% of specificity (AUC = 0.98 ± 0.02, p < 0.001).
Summary of ROC analyses with cut-off scores
The three factors identified in the Brazilian version of the CANS-MCI were highly correlated between each other (EF and memory = 0.762; EF and language = 0.889, and memory and language = 0.605). The current factor analysis did not identify the exact three factors identified in the original scale (Table 5). The Free Recognition I and II and the Guided Recognition Errors tests were loaded in the memory factor, as in the original study (Tornatore et al., Reference Tornatore, Hill, Laboff and McGann2005), and the Stroop test was also loaded in this factor in the current analyses. In the present study, two tests scores made up the language/spatial fluency factor: clock hand placement and picture naming. The executive function/mental control factor comprised general reaction time, design matching, word-to-picture matching (latency), and picture naming (latency). In the original study, picture naming (latency) was loaded in the language/spatial fluency factor.
Results for factor analysis
Note: Highlighted items revealed factor loadings that differed from the original study.
Discussion
The objective of the present study was to determine the psychometric properties of the Brazilian version of CANS-MCI and its accuracy in the diagnostic screening of MCI. To our knowledge, this is the first study to present data on CANS-MCI in a sample of older adults outside the United States. The present results suggest that the translated and culturally adapted version of the battery maintains its original psychometric characteristics. In spite of linguistic and cultural differences, the internal consistency reported for the Brazilian version of CANS-MCI was quite similar to the one reported in the original study (ranging from 0.72 to 0.98 for the subtests), and the test was stable over time. In addition, we found significant correlations between the CANS-MCI subtests and other neuropsychological tests, namely Digit Symbol, RAVLT (learning curve and the last delayed recall), and semantic verbal fluency. Concurrent validity findings are in agreement with the original validation study (Tornatore et al., Reference Tornatore, Hill, Laboff and McGann2005).
We also found a high correlation between the total scores of CANS-MCI and MoCA-BR (r = 0.76), which has been recently validated in this country (Memória et al., Reference Memória, Yassuda, Nakano and Forlenza2013). As both tests aim to screen for MCI, despite using different testing paradigms, such results may also be regarded as evidence for concurrent validity. They suggest that clinicians may choose to use the conventional paper-and-pencil test or the computerized battery. It is noteworthy that the CANS-MCI may require more time to be administered; yet, its automated format does not require the participation of expert staff.
Interestingly though, MCI patients required a significantly longer time to complete CANS-MCI as compared with controls. This result is in agreement with previous studies (Gorus et al., Reference Gorus, De Raedt, Lambert, Lemper and Mets2008; Saxton et al., Reference Saxton, Morrow, Eschman, Archer, Luther and Zuccolotto2009) as they also reported lower speed of processing among MCI patients. The WAIS-III Digit Symbol subtest, which more closely parallels the speed of processing aspect of CANS-MCI, displayed a moderate negative correlation with the total CANS-MCI score. This result supports the hypothesis that speed of processing is useful in differentiating the three diagnostic groups. Therefore, one of the contributions of the present study is to suggest that time to complete cognitive tasks ought to be considered in MCI diagnosis, in addition to test scores.
In the current sample, the memory subtest scores did not discriminate the MCI patients from controls. This may be due to the fact that in CANS-MCI the memory tests are highly dependent on recognition tasks, i.e., not requiring the free recall of learned items. Grönholm-Nyman et al. (Reference Grönholm-Nyman, Rinne and Laine2010), for instance, found that the MCI group performed equally well as controls in incidental learning and recognition memory tests, whereas the AD group showed impairment in such tasks. According to Schlegel and Gilliland (Reference Schlegel and Gilliland2007), episodic memory assessment represents the most important limitation of computerized batteries, as they usually involve recognition paradigms only, disregarding the free recall of learned information, which is a more sensitive measure of pathological episodic memory decline. To overcome this limitation, clinicians may need to implement an additional free recall of memory stimuli before the recognition test is implemented.
The subtests addressing executive functions were the ones that best discriminated MCI from normal controls in the current sample. This may be due to the fact that most of the tasks in this domain are timed. Previous studies demonstrated that executive dysfunction is common among patients with amnestic MCI (Ready et al., Reference Ready, Ott, Grace and Cahn-Weiner2003). As noted by other investigators (Bozoki et al., Reference Bozoki, Giordani, Heidebrick, Berent and Foster2001; Forlenza et al., Reference Forlenza, Diniz, Nunes, Memoria, Yassuda and Gattaz2009; Albert et al., Reference Albert2011), tests of learning, attention, perceptual speed, category fluency, and executive function may be particularly suitable to identify possible MCI cases. In the current study, scores for the MCI group were usually in an intermediate position between normal controls and AD, supporting the notion of MCI as a transitional stage between healthy and pathological cognitive aging.
High specificity is essential for an MCI screening test as older individuals may be misclassified as cognitively unimpaired due to the transitional nature of MCI. The Brazilian version of the CANS-MCI battery demonstrated its robust capacity to discriminate MCI from normal controls (81% of sensitivity and 73% of specificity) and almost perfect accuracy to identify AD (100% of sensitivity and 97% of specificity) even in a sample comprising early AD cases. These results were similar to the ones presented in the original validation study (Tornatore et al., Reference Tornatore, Hill, Laboff and McGann2005), and they are quite encouraging.
Finally, the exploratory factor analysis confirmed the existence of three factors in the computerized battery which supposedly assess aspects of memory, language, and executive functioning. Interestingly, Stroop and picture naming did not load respectively in the executive function/mental control and language/spatial fluency domains, as expected. This finding may be related to sample size and sample characteristics. The factor structure of CANS-MCI will need to be confirmed in future studies.
Although standard paper-and-pencil tests are available for MCI screening (Nasreddine et al., Reference Nasreddine2005; Memória et al., Reference Memória, Yassuda, Nakano and Forlenza2013), computerized testing is self-administered and reports are generated automatically, saving staff time. The CANS-MCI offers a score that indicates the probability that an MCI diagnosis will be confirmed by comprehensive neuropsychological evaluation. In addition, computerized testing seems more ecological as test stimuli include colored pictures of daily life objects. It may be more appealing to adults who interact with technology on a daily basis.
Some limitations of this study should be mentioned, such as the small number of participants, and the exclusion of patients with schooling lower than four years. Another limitation refers to the possibility that older adults with limited experience using computers may have been reluctant to get involved with the test and this may have affected test scores. However, all individuals completed the test and offered positive feedback. An additional limitation refers to the fact that the MCI group was heterogeneous as different MCI subtypes were included in the sample. The cross-sectional design of the present study limits some of our conclusions. As the main goal of MCI screening is to identify people with higher risk for conversion to dementia, longitudinal studies should test the CANS-MCI ability to predict future cognitive decline.
In conclusion, brief computerized cognitive schedules, such as the CANS-MCI, represent promising tools for low-cost identification of early stage cognitive decline. Such instruments may help identify patients who may need more detailed assessments as they may be at risk for dementia. The evaluation of psychometric properties of CANS-MCI-BR yielded favorable results and support its use among Brazilian older adults.
Conflict of interest
None.
Description of authors’ roles
C. M. Memória collected the data and wrote the paper. M. S. Yassuda participated in study design, supervised data analyses, and revised the paper. E.Y. Nakano participated in statistical analysis. O. V. Forlenza helped formulate the research questions and revised the paper.
Acknowledgments
Dr.Helio Elkis for permission to use the program AMOS to make the Confirmatory Factor Analysis. Financial sources: The Laboratory of Neuroscience is supported by grants from the Associação Beneficente Alzira Denise Hertzog da Silva (ABADHS) and FAPESP (projects Nos. 02/12633-7 and 2009/52825-8).
Appendix A
In the Word-to-Picture Matching subtest:
-
1. Hockey puck was replaced by badminton shuttlecock.
-
2. American football was replaced by soccer ball.
-
3. Grapefruit was replaced by orange.
In the Immediate Recall subtest:
-
1. Baseball bat was replaced by boxing gloves.
-
2. The character Sylvester was replaced by Donald Duck.
-
3. American coins nickel (5 cents) and dime (10 cents) were replaced by Brazilian coins (one real and 25 cents).
-
4. The landmark Statue of Liberty was replaced by the statue of Christ the Redeemer.
In the Pictures Naming subtest:
-
1. Donut (donut) was replaced by cake.
-
2. Wagon was replaced by bicycle.
-
3. The person skiing was replaced by the person swimming.
-
4. The person mowing the lawn was replaced by the person sawing.
