Introduction
The recent publication of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5, American Psychiatric Association, 2013) introduced diagnostic criteria for major and mild NCD. The manual specifies etiologic subtypes and level of diagnostic certainty (e.g. due to probable or possible AD), and discusses accompanying behavioral disturbances as well as severity. Classification according to all specifications delineates increasingly homogeneous subgroups (Ganguli et al., Reference Ganguli2011). Ganguli (Reference Ganguli2013) points out that the core criteria could be further divided into inclusion criteria (features that must be present) and exclusion criteria (features that must be absent), although classification is not explicitly arranged in this way.
Evidence of cognitive decline from a previous level of performance in one or more cognitive domains (e.g. complex attention, learning and memory, language) constitutes the basic inclusion criteria for a diagnosis of major or mild NCD. The two main NCD categories differ primarily in the degree of impairment in cognitive performance (substantial in major NCD, modest in mild NCD), as well as in the impact that this impairment has on everyday functioning (severe interference in major NCD, preserved independence or subtle interference in mild NCD). According to the recommendation of the DSM-5, formal neuropsychological testing should be used for quantitative assessment of all relevant domains. When such testing is unavailable or not feasible, cognitive screening tests, whether domain-specific or global, could be used for brief quantitative assessment (American Psychiatric Association, 2013; Ganguli, Reference Ganguli2013). No specific tests are named and hence no specific thresholds are provided. However, the DSM-5 manual suggests that performance be defined in terms of standard deviations or percentiles relative to appropriate norms. Performance that exceeds two standard deviations below the norm (under percentile 3) reflects major NCD, whereas performance that falls between 1 and 2 standard deviations below the norm (or between the 3–16 percentiles) reflects mild NCD. This diagnostic procedure implies a departure from a dichotomous medical model, in which a cognitive deficit is either present or absent, toward a psychological model in which cognitive functioning falls along a continuum (Ritchie, Reference Ritchie, Jacoby, Oppenheimer, Dening and Thomas2008).
At present, the selection of cognitive tests is based on various considerations, including basic purpose (e.g. screening vs. differential diagnosis), domains assessed (e.g. memory, language), administration time, setting (e.g. primary, secondary, tertiary care), patient population, or the professional specialization required for administration and interpretation (e.g. Jacova et al., Reference Jacova, Kertesz, Blair, Fisk and Feldman2007; Lonie et al., Reference Lonie, Tierney and Ebmeier2009; Appels and Scherder, Reference Appels and Scherder2010; Velayudhan et al., Reference Velayudhan2014). The psychometric properties of the selected tools also matter, as not all tests have appropriate norms for the specific patient population. However, there is currently no consensus regarding the individual test that best suits the diagnosis of NCD. Thus far, only the Montreal cognitive assessment (MoCA) has been examined according to the DSM-5 classification of major and mild NCD (Liew et al., Reference Liew, Feng, Gao, Ng and Yapin press).
The Kingston standardized cognitive assessment (KSCA) has proven to be an effective and efficient tool in the assessment of cognitive functioning in individuals with dementia, and its more advanced revision, the KSCA-R, was found sensitive to very early dementia (Hopkins et al., Reference Hopkins, Kilik, Day, Rows and Hamilton2004, see also http://www.kingstonscales.org). The KSCA-R consists of 19 subtests that assess memory, visuo-spatial processing, and language. Recently, Hopkins and Kilik (Reference Hopkins and Kilik2013) introduced a shorter version of this test, the mini-KSCAr, which could serve as a screening tool that is neither too brief nor too extensive. The mini-KSCAr (total score 55) contains a memory task of immediate and delayed word recall and recognition (max = 30 points), an orientation task (max = 10 points), an executive task of abstract thinking (max = 8 points), and a spatial task of clock drawing (max = 7 points). Administration takes about 7–10 minutes, and the total score can be transformed into percentiles. Hopkins and Kilik (Reference Hopkins and Kilik2013) compared mini-KSCAr total scores as well as subtest scores of individuals with AD and healthy controls, demonstrating significant group differences. The test was found to have high inter-rater and test-retest reliability and good correlations with the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975), with the Wechsler adult intelligence scale-III (WAIS-III; Wechsler, Reference Wechsler1998), and with the Wechsler memory scale-III (WMS-III; Wechsler, Reference Wechsler1997). The authors did not encourage the use of a single cutoff score and instead suggested that a total score as well as individual subtest scores (assessed through percentiles) could offer the best way to interpret results.
The purpose of the current study was to investigate the usefulness of the mini-KSCAr in classifying individuals according to the DSM-5 criteria. Toward this aim we recalculated mini-KSCAr scores for individuals who had been tested with the full KSCA-R (Hopkins et al., Reference Hopkins, Kilik, Day, Rows and Hamilton2004) and compared scores in groups defined independently according to the DSM-5 criteria. We then examined the sensitivity, specificity, accuracy, and LRs of the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975), the CDT (Heinik et al., Reference Heinik2004), and the mini-KSCAr. Because the mini-KSCAr has a larger component of memory assessment than the other two screening tools, we hypothesized that it would be more useful for the classification of NCD due to AD. Lastly, we investigated how well each of these screening tools predicted formal neuropsychological memory test performance, again hypothesizing that the mini-KSCAr would show an advantage over the MMSE and the CDT.
Methods
Participants
The sample included 85 individuals who were assessed with the Hebrew version of the full KSCA-R (Hopkins et al., Reference Hopkins, Kilik, Day, Rows and Hamilton2004) between October 2004 and October 2007. Administration of the KSCA-R was part of the clinical procedure at a psychogeriatric outpatient clinic of a large tertiary hospital in Tel Aviv. Participants went through a comprehensive evaluation by a multi-disciplinary team that included an assessment by a psychiatrist and a nurse in the first visit (also consisting of cognitive screening tests), an assessment by a geriatric specialist and a social worker in the second visit, administration of the KSCA-R followed by the CAMCOG-R in the third visit, and neuropsychological testing in the fourth visit. Medical history was taken with an emphasis on physical and psychiatric status, basic and instrumental activities of daily living, and cognition. The entire procedure was completed within 8–10 weeks. In all cases, the functional assessment was independent of the cognitive assessment (see Heinik et al., Reference Heinik2004 as well as Heinik and Shaikewitz, Reference Heinik and Shaikewitz2009 for details of the full procedure). Basic requirements for KSCA-R and CAMCOG-R administration consisted of Hebrew proficiency, no significant sensory (sight, hearing) or motor difficulties, as well as no severe psychiatric or physical conditions that could interfere with testing. Files were selected for review in the current study out of 157 files with KSCA-R scores as well as other information required for diagnosis. Preliminary exclusion criteria included abrupt onset of cognitive decline, fluctuating or reversible course of impairment, history of delirium, presence of cerebrovascular disease, or any other neurological condition that would prevent a diagnosis of NCD due to AD (based on report, examination, laboratory tests, and imaging). There were no statistically significant differences in age, sex, education, and MMSE scores between the 85 individuals who were included in the study and the 72 individuals who were not included in the study but had KSCA-R scores. The study design and protocol were approved by the local Helsinki committee. Participant characteristics are presented in Table 1.
Sample characteristics, by group
Note: * p < 0.001; standard deviations are presented in parentheses. Presence or absence of (minor) psychiatric disturbances was assessed through a clinical interview.
Assessment tools
In addition to the mini-KSCAr (Hopkins and Kilik, Reference Hopkins and Kilik2013), which was adapted from the KSCA-R (Hopkins et al., Reference Hopkins, Kilik, Day, Rows and Hamilton2004), three types of tests were administered: (i) tools used for diagnosis; (ii) screening tools; (iii) neuropsychological memory tasks.
-
I. Tools used for diagnosis
To determine whether participants had major NCD, mild NCD, or no cognitive decline, we used the CAMGOG-R (Roth et al., Reference Roth, Huppert, Mountjoy and Tym1998; Hebrew version by Heinik et al., Reference Heinik, Werner, Mendel, Raikher and Bleich1999) as well as the CDR scale (Huges et al., Reference Hughes, Berg, Danziger, Coben and Martin1982).
-
II. Screening tools
To determine whether the mini-KSCAr was more diagnostically useful than other screening tools, we used the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975; Hebrew version by Werner et al., Reference Werner, Heinik, Mendel, Raikher and Bleich1999) as well as the CDT (Heinik et al., Reference Heinik2004).
-
III. Neuropsychological memory tasks
To determine whether the mini-KSCAr correctly predicts memory differences among the diagnostic groups, we used the Rey auditory learning test (RAVLT; Hebrew version by Vakil and Blachstein, Reference Vakil and Blachstein1997), the logical memory subtest of the WMS-III (Wechsler, Reference Wechsler1997), as well as the Rey-Osterrieth complex figure test (ROCF). Only a subset of participants (N = 28) underwent the extensive memory testing, depending on clinical considerations at the time of their assessment. For the purpose of the current analysis, we calculated a composite memory score that summed all information units recalled after delay in the three neuropsychological tasks.
Operational diagnostic criteria for major and mild NCD due to possible AD
-
1. All participants expressed subjective concern about their cognitive status, which led them to approach clinical advice.
-
2. History of insidious onset and progressive, gradual decline in cognition, especially memory.
-
3. Objective impairment in cognitive performance, especially memory, as determined with the use of CAMCOG-R scores that were measured against appropriate norms (Williams et al., Reference Williams, Huppert, Matthews and Nickson2003; Stephan et al., Reference Stephan, Matthews, McKeith, Bond and Brayne2007). A total score below the 25th percentile and a subtest (memory: remote, recent, learning; orientation) score below the 16th percentile were considered impaired. We used the less stringent values of below-average rather than completely impaired performance on the basis of our clinical experience. This procedure allowed us to include participants with milder levels of cognitive impairment rather than only those with severe difficulties. We focused on the memory and orientation subtest scores because poor performance in these domains is the hallmark of possible AD. A diagnosis of major NCD was given to individuals with impairment on both the total score and the scores on one or more memory or orientation subtests; a diagnosis of mild NCD was given to individuals with impairment in either the total score or one or more subtest scores; no cognitive decline was determined when both the total score and the subtest scores were unimpaired.
-
4. Interference with independence in everyday activities was assessed through clinical interview with reference to basic and instrumental activities of daily living, especially handling of medications and finances. Functioning was also assessed with the CDR (Hughes et al., Reference Hughes, Berg, Danziger, Coben and Martin1982). Major NCD was diagnosed in individuals who were impaired both in handling medications and in handling finances and received a CDR score of 0.5 or 1; mild NCD was diagnosed in individuals who were either fully responsible for both medications and finances or had subtle interference with one of these functions, receiving a CDR score of 0 or 0.5; no cognitive decline was diagnosed in individuals who were fully independent on both functions and received a CDR score of 0.
-
5. Etiology was assessed on the basis of the medical file, with specific attention to the exclusion of individuals with cerebrovascular disease, substance abuse, or any psychiatric, neurological, or systemic disorder that could account for cognitive decline.
-
6. Individuals with no decline fulfilled no NCD criteria except for expressing subjective concerns.
Analytic plan
Group differences in demographic characteristics were examined either with a χ 2 for categorical variables or with a one-way analysis of variance (ANOVA) for continuous variables. Since age differed significantly across groups, it was held as a covariate in the analysis of test scores. We examined sensitivity, specificity, accuracy, and LRs along different cutoff points. Pearson correlations were used to study the associations among screening tools. A stepwise regression analysis was conducted to predict the composite delayed memory score. Analyses were performed with the SPSS software, version 16.
Results
As can be seen in Table 1, groups did not differ significantly in mean number of years of education, in the proportion of women, or in the presence of minor psychiatric disturbances. However, there was a statistically significant group difference in age, F (2, 82) = 4.187, p < 0.05. A post-hoc Scheffe test revealed that the major NCD group was significantly older than was the group with no cognitive decline, p < 0.05. No other group comparisons were significant. In accordance with our classification criteria, groups differed significantly in CAMCOG-R scores, F (2, 82) = 87.347, p < 0.001, as well as in CDR scores, F (2, 82) = 106.547, p < 0.001. Post-hoc Scheffe tests revealed that all group comparisons in both CAMCOG-R and CDR were significant, p < 0.01.
A one-way ANOVA (with and without age as a control variable) yielded significant differences in MMSE, CDT, and mini-KSCAr scores. Despite the fact that all tests differed significantly across groups, the effect size of the mini-KSCAr was greatest, suggesting that it reflected group differences most accurately (see scores and statistics in Table 2). A post-hoc Scheffe test revealed that the major NCD group had significantly lower MMSE scores than did the other two groups, with no significant difference between the mild NCD group and the no-decline group, p < 0.01. A post-hoc Scheffe test revealed no significant differences in CDT scores between the major and mild NCD groups, but both groups differed significantly from the no-decline group, p < 0.05. A post-hoc Scheffe test revealed that all group comparisons in mini-KSCAr scores were significant, p < 0.01.
Mean scores on the MMSE, the CDT, and the mini-KSCAr, by group
Note: * p < 0.001; standard deviations are presented in parentheses.
Sensitivity, specificity, and LRs (positive + and negative –) were studied for different cutoff points of the MMSE, CDT, and the mini-KSCAr, considering the entire sample as well as the diagnostic groups. We report only cutoff points that had sensitivity or specificity values higher than 80%, LR+ no less than 5 (with LR+ > 10 considered good and LR+ between 5–10 considered moderate), LR– no more than 0.2 (with LR– < 0.1 considered good and LR– between 0.1 and 0.2 considered moderate, see Appels and Scherder, Reference Appels and Scherder2010). When analyzing the MMSE, we found that a cutoff score of 27 identified individuals with major NCD relative to individuals with no cognitive decline with sensitivity of 84%, specificity of 86%, accuracy of 84%, LR+ of 5.86, and LR– of 0.19. The same cutoff (27) identified individuals with major NCD relative to individuals with mild NCD with sensitivity of 84%, specificity of 89%, accuracy of 86%, LR+ of 7.54, and LR– of 0.18. No useful sensitivity, specificity, accuracy, or LR were found for the CDT in any of the analyses. The best results were obtained for the mini-KSCAr. A cutoff of 37 yielded sensitivity of 92%, specificity of 95%, accuracy of 94%, LR+ of 19.36, and LR– of 0.08 in identifying individuals with no cognitive decline relative to the other two groups. A cutoff of 36 and 37 identified individuals with major NCD relative to individuals with no cognitive decline with sensitivity of 100%, specificity of 95%, accuracy of 98%, LR of 21+, and LR– of 0. A cutoff of 37 identified individuals with mild NCD relative to individuals with no cognitive decline with sensitivity of 81%, specificity of 95%, accuracy of 88%, LR+ of 17.11, and LR– of 0.19.
As can be seen in Table 3, scores on the mini-KSCAr were highly correlated with scores on other screening and diagnostic tests. All correlations were significant and remained significant after controlling for age. Finally, a stepwise regression analysis was conducted to predict the composite memory score of the delayed RAVLT, WMS-III, and ROCF, entering scores on the MMSE, the CDT, and the mini-KSCAr as predictors. This analysis showed that the MMSE and CDT scores had no significant contribution to the prediction of the composite memory score, whereas the mini-KSCAr accounted for 47.1% of the variance in memory, F (1, 22) = 19.566, p < 0.001.
Pearson correlations among cognitive screening tests
Note: * p < 0.001.
Discussion
We studied the efficiency of the mini-KSCAr in classifying patients according to the DSM-5 diagnoses of major and mild NCD due to possible AD. The mini-KSCAr was found to be a useful screening tool in distinguishing the two patient groups from a group of cognitively healthy individuals. Although group differences also emerged for the MMSE and the CTD, the effect size of the differences in mini-KSCAr scores was largest. Post-hoc tests revealed that MMSE scores did not differentiate between the mild NCD group and the control group. CDT scores differentiated between patients and controls but not between the two types of NCD groups. Only the mini-KSCAr distinguished among all three groups. Similarly, although a cutoff score of 27 on the MMSE was found to distinguish patients from healthy controls, sensitivity, specificity, accuracy, and LRs were lower for this test than they were for the mini-KSCAr. The CDT demonstrated no sensitivity, specificity, accuracy, or LRs worth reporting.
Previous studies found that the MMSE distinguished between healthy individuals and people with mild AD, and correctly classified 89% of the sample with a sensitivity of 92% and a specificity of 86% (Kalbe et al., Reference Kalbe2004). Other studies have also emphasized the utility of the 27 MMSE cutoff score in distinguishing between mild cognitive impairment and mild AD as well as between individuals who had these conditions and individuals with no cognitive decline (Custodio et al., Reference Custodio2014). In addition, a meta-analysis found that the MMSE had a pooled sensitivity of 80% in memory clinics (Mitchell, Reference Mitchell2009). Research with the CDT suggested that its role as a single test in the early diagnosis of cognitive impairment is controversial (Heinik et al., Reference Heinik and Shaikewitz2009; Lonie et al., Reference Lonie, Tierney and Ebmeier2009; Velayudhan et al., Reference Velayudhan2014).
Importantly, the mean mini-KSCAr score of our major NCD group resembled the mean score of the Alzheimer group studied by Hopkins and Kilik (Reference Hopkins and Kilik2013). In contrast, the mean score of their healthy individuals (42.35) was slightly higher than the mean score in our healthy group (40.36). We note that age and education were similar in our sample to those characterizing the individuals who participated in the Hopkins and Kilik (Reference Hopkins and Kilik2013) study. However, they recruited their healthy controls from the community while our cognitively healthy group consisted of clinical referrals. Our sample also included individuals with concurrent psychiatric disturbances, and psychiatric morbidity has long been known to be detrimental for cognitive performance. Nevertheless, there were no differences in the presence of psychiatric disturbances across our three groups, suggesting that there were other reasons for the documented cognitive differences. Note that Hopkins and Kilik (Reference Hopkins and Kilik2013) used previously accepted diagnostic criteria, whereas our criteria were based on the DSM-5. Thus, some of the patients included in their Alzheimer group would be classified as mild NCD according to the DSM-5.
Hopkins and Kilik (Reference Hopkins and Kilik2013) reported that the mini-KSCAr was highly correlated with other tests, both with a screening test (MMSE) and with specific measures of immediate and delayed memory (WMS). Our results fit well with these findings, demonstrating significant correlations between scores on the mini-KSCAr and the other screening and diagnostic tools. Yet, the mini-KSCAr predicted memory performance better than did the other screening tools that we used (e.g. MMSE, CDT). This is not surprising because the mini-KSCAr assesses memory in a much more comprehensive way than do the MMSE and the CDT. Given that the NCD syndromes are assumed to differ in the severity of memory decline, the substantial memory component of the mini-KSCAr makes it a good short tool for diagnostic purposes.
Liew et al. (Reference Liew, Feng, Gao, Ng and Yapin press) have recently demonstrated the diagnostic utility of the MoCA, another brief-to-intermediate cognitive test, in classification according to DSM-5 criteria. The MoCA had good sensitivity, specificity, and accuracy in distinguishing between major NCD and community-based cognitively healthy controls. However, its usefulness in detecting mild NCD was much more modest. Our results suggest that the mini-KSCAr might be better at classifying the mild NCD group, though a direct comparison of the MoCA and the mini-KSCAr within the same population should be conducted to determine which test is more useful. Previous studies have indeed shown that a few brief tests (with administration of less than 10 minutes) are efficient in identifying the beginning of AD (e.g. Solomon et al., Reference Solomon1998; Hogervorst et al., Reference Hogervorst, Combrinck, Lapurta, Rue, Swales and Budge2002; Kalbe et al., Reference Kalbe2004) but are less efficient in identifying mild cognitive impairment (e.g. Kalbe et al., Reference Kalbe2004). Custodio et al. (Reference Custodio2014) have recently reported that the memory alteration test proved excellent in differentiating both mild AD and amnestic-mild cognitive impairment from cognitively healthy individuals.
Several limitations of our study should be acknowledged. First, we used only memory and orientation CAMCOG-R subtest scores for patient classification. Although these domains are most often impaired in AD, poor performance on other subtests could also be used within the diagnostic process. We suspect that using the other subtests would lead to similar results. Second, the current investigation calculated the mini-KSCAr scores from the full KSCA-R version. Whereas in the KSCA-R delayed recall is measured approximately 10–15 minutes after immediate recall, the delay is shorter in the mini-KSCAr. Lyness et al. (Reference Lyness, Lee, Zarow, Teng and Chui2014) found that using the modified MMSE with a 10-minute delay between presentation and recall was useful in distinguishing between persons with AD and persons with non-AD pathology, but a 1-minute delay was not useful. However, Zhao et al. (Reference Zhao, Lv, Zhou, Hong and Guo2012) reported that testing memory with both short (3–5 minutes) and long (20 minutes) delays between presentation and recall was equally efficient in distinguishing between healthy individuals and individuals with amnestic MCI or mild AD. Further research is required to examine whether delay time undermines the usefulness of the mini-KSCAr relative to the administration of the KSCA-R. Based on Zhao et al.'s (Reference Zhao, Lv, Zhou, Hong and Guo2012) results, it is safe to assume that the shorter delay in the mini-KSCAr would still be long enough to show distinct patterns of performance across the three groups that we studied here.
Another possible limitation of our study is the fact that current research is a retrospective study with no longitudinal data. A prospective replication study with a larger sample and longitudinal follow-up testing should be conducted to confirm the validity of our findings. It is also possible that the utility of the mini-KSCAr will be different in populations that are more heterogeneous in terms of both socio-demographic characteristics and non-AD etiologies (e.g. vascular). Lastly, as the mini-KSCAr was designed for rapid use in primary care settings, its screening utility should be explored in such a setting.
To conclude, the mini-KSCAr was found to be an efficient instrument for the diagnosis of DSM-5 major and mild NCD due to possible AD in a specialized psychogeriatric setting, and its utility was greater than that of the MMSE and the CDT.
Conflict of interest declaration
None.
Description of authors’ roles
Both authors designed the study, collected the data, carried out statistical analysis, and wrote the article.
