Introduction
The 6CIT is a brief (2–3 minute) CSI which covers a limited number of cognitive domains (Katzman et al., Reference Katzman, Brown, Fuld, Peck, Schechter and Schimmel1983; Table 1).
A number of studies of the clinical utility of 6CIT have been reported since its first description (Jefferies and Gale, Reference Jefferies, Gale and Larner2013). In a study involving patients selected from a hospital database and tested both in community and outpatient settings 6CIT correlated with the MMSE but performed better than MMSE in detecting patients with mild dementia (Brooke and Bullock, Reference Brooke and Bullock1999). In light of this study, 6CIT has been recommended by some authorities as a suitable dementia screening test for use in primary care (Brodaty et al., Reference Brodaty, Low, Gibson and Burns2006), and as a quick and reliable alternative to the MMSE (Woodford and George, Reference Woodford and George2007), although others have questioned its reliability and validity in the primary care setting (Hessler et al., Reference Hessler2014). Despite expert recommendations, the frequency of reported 6CIT use in patients referred from primary care to a neurology-led memory clinic was found to be low (< 10%; Cagliarini et al., Reference Cagliarini, Price, Livemore and Larner2013).
A meta-analysis of multidomain tools for dementia screening and case finding suggested that 6CIT should be considered for use in specialist settings, although this analysis did appear to conflate studies of 6CIT with those on the similarly named but entirely different six-item screener (Mitchell and Malladi, Reference Mitchell and Malladi2010). To our knowledge, only two large diagnostic accuracy studies of 6CIT have been undertaken in secondary care settings. Upadhyaya et al. (Reference Upadhyaya, Rajagopal and Gale2010) compared 6CIT and MMSE in 209 patients referred to an old age psychiatry service and found that 6CIT performed well as a dementia screening instrument (sensitivity and specificity > 80%). Tuijl et al. (Reference Tuijl, Scholte, de Craen and van der Mast2012) administered 6CIT to 253 general hospital patients aged ≥ 70 years and found high diagnostic accuracy (sensitivity and specificity ≥ 90%, area under the receiver operating characteristic (ROC) curve 0.95); it was quicker to administer than MMSE.
We undertook a pragmatic diagnostic accuracy study of 6CIT in a dedicated neurology-led cognitive disorders clinic, a clinical environment wherein 6CIT has hitherto not been examined to our knowledge. The aims of the study were to measure the diagnostic accuracy of 6CIT in cross-sectional use in consecutive patients referred with memory complaints of unknown aetiology, for the diagnosis of both dementia and MCI, and to compare its performance to the MMSE.
Item content and scoring of 6CIT (NB: negative scoring, i.e. higher score = worse performance; maximum score/worst performance = 28)
Methods
Over a one-year period (June 2013–June 2014), new clinic patients referred to a neurology-led memory clinic based in a regional neuroscience center were administered 6CIT, on the same day as clinical history taking and informant interview (including semi-structured assessment of everyday function), other neuropsychological tests and neuroimaging. There were no exclusion criteria, other than those patients already administered the 6CIT in primary care prior to referral. All patients underwent the same cognitive assessment. Standard diagnostic criteria for dementia (DSM-IV), dementia subtypes, and MCI (Petersen et al. Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999) were used, as previously in this clinic (Larner, Reference Larner2014a). In all cases, criterion diagnosis was by judgment of an experienced neurologist based on diagnostic criteria and blind to 6CIT and MMSE scores in order to avoid review bias.
Standard summary measures of diagnostic test accuracy were calculated, including overall test accuracy, sensitivity, specificity, Youden index, positive and negative predictive values, predictive summary index, positive and negative likelihood ratios, diagnostic odds ratio, clinical utility indexes, and area under the ROC curve, as for previous pragmatic diagnostic test accuracy studies (Larner, Reference Larner2014a). The STARDdem guidelines for reporting such studies in dementia were observed (Noel-Storr et al., Reference Noel-Storr2014).
Similar diagnostic test accuracy measures were also calculated for the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975) in those patients for whom data were available. 6CIT and MMSE were compared not only by these summary data but also by means of the test of agreement (kappa statistic; Cohen, Reference Cohen1960) and by weighted comparison performed as per the method of Moons et al. (Reference Moons1997).
\begin{eqnarray*}
{\rm WC} &=& \Delta \;{\rm sens} + [ (1 - \pi /\pi )\\
&& \times {\rm relative}\;{\rm cost}\;({\rm FP}/{\rm TP}) \times \Delta \;{\rm spec} ],
\end{eqnarray*}
where π = prevalence; FP = false positives; and TP = true positives. The relative misclassification cost (FP/TP), a parameter which seeks to define how many false positives a true positive is worth, was set at FP/TP = 0.1, following previous authors (Mallett et al., Reference Mallett, Halligan, Thompson, Collins and Altman2012; Larner, Reference Larner2013, Larner and Hancock, Reference Larner and Hancock2014), and reflecting the clinical desire for high test sensitivity.
Effect sizes were calculated as the difference of the means of diagnostic groups divided by the weighted pooled standard deviations of the groups (Cohen's d; Cohen, Reference Cohen1988).
Results
Of 257 new patient referrals seen, twelve (= 4.7%) had already been administered 6CIT in primary care prior to referral, a frequency similar to that observed in previous cohorts referred to this clinic (Cagliarini et al., Reference Cagliarini, Price, Livemore and Larner2013; Ghadiri-Sani and Larner, Reference Ghadiri-Sani and Larner2014).
Of 245 patients assessed with 6CIT (M:F = 124:121, 50.6% male; age range 16–94 years; median 59 years), 48 were diagnosed with dementia (prevalence 19.6%) and a further 67 (27.3%) had MCI.
Figure 1 illustrates the distribution of 6CIT scores in the dementia and MCI patient groups compared to those without cognitive impairment. 6CIT scores showed only a low correlation with patient's age (r = − 0.32).
6CIT scores versus diagnosis (N = 245).
Table 2 contains statistics summarising the diagnostic accuracy of the 6CIT for the diagnosis of dementia versus no dementia (n = 245), dementia versus MCI (n = 115), and for MCI versus normal cognition/subjective memory impairment (n = 197). At the specified cutoff of ≤ 4 (see Table 1), 6CIT had good sensitivity and specificity for the diagnosis of dementia versus no dementia (0.88 and 0.78, respectively), good sensitivity (0.88) but poor specificity (0.61) for the diagnosis of dementia versus MCI, and parameters for the diagnosis of MCI versus no cognitive impairment (specified cutoff of ≤ 9) were suboptimal (sensitivity and specificity 0.66 and 070 respectively). Area under the ROC curve, a measure of diagnostic accuracy, was 0.90, 0.85, and 0.71 for the diagnosis of dementia versus no dementia, dementia versus MCI, and MCI versus no cognitive impairment, respectively.
Demographic and diagnostic parameters for 6CIT for diagnosis of dementia and of MCI (with 95% confidence intervals)
MMSE scores concurrently recorded in a subgroup of patients tested with 6CIT (n = 150) showed a high negative correlation between 6CIT and MMSE scores (r = −0.73; t = 13.0, df = 148, p < 0.001), suggesting concurrent validity. As for 6CIT scores, MMSE scores showed only a low correlation with patient's age (r = −0.37).
Table 3 shows diagnostic accuracy statistics for MMSE for the diagnosis of dementia versus no dementia (n = 150), dementia versus MCI (n = 65), and for MCI versus normal cognition/subjective memory impairment (n = 128). Since, unlike 6CIT, MMSE cutoffs are not pre-specified, these were chosen for comparable levels of overall test accuracy to 6CIT. At the MMSE cutoff of ≤ 22/30, MMSE was less sensitive (0.59) but marginally more specific (0.85) than the 6CIT for the diagnosis of dementia versus no dementia and less sensitive (0.59) but more specific (0.74) for the diagnosis of dementia versus MCI. At the MMSE cutoff of ≤ 25/30, MMSE was less sensitive (0.51) but marginally more specific (0.75) than the 6CIT for the diagnosis of MCI. Area under the ROC curve was 0.83, 0.74, and 0.69 for the diagnosis of dementia versus no dementia, dementia versus MCI, and MCI versus no cognitive impairment, respectively, all inferior to 6CIT.
Demographic and diagnostic parameters for MMSE for diagnosis of dementia and of MCI (with 95% confidence intervals)
Data for both 6CIT and MMSE were available for 150 patients. The test of agreement (kappa statistic) between 6CIT and MMSE gave κ = 0.47 (95% CI = 0.29–0.63), indicating moderate agreement according to the classification of kappa statistic by Landis and Koch (Reference Landis and Koch1977).
Calculating weighted comparisons according to the equation of Moons et al. (Reference Moons1997) gave positive WC scores indicating a net benefit for 6CIT compared to MMSE for the diagnosis of dementia versus no dementia, dementia versus MCI, and of MCI versus no cognitive impairment (Tables 4–6, respectively). From these WC values it was possible to calculate an equivalent increase in TP patients per 1000 tested, using the equation WC × prevalence × 1000. The results showed that for every 1000 patients tested in a patient cohort with this dementia prevalence around 22 extra cases of dementia versus no dementia, and 55 extra cases of dementia versus MCI, would be detected by 6CIT than by MMSE; and that for every 1000 patients tested in a patient cohort with this MCI prevalence around 14 extra cases of MCI would be detected by 6CIT than by MMSE.
Weighted comparison 6CIT versus MMSE for diagnosis of dementia versus no dementia (N = 150)
Weighted comparison 6CIT versus MMSE for diagnosis of dementia versus MCI (N = 65)
Weighted comparison 6CIT versus MMSE for diagnosis of MCI versus no dementia (N = 128)
Calculation of effect size (Cohen's d) comparing patients with dementia and no dementia was large for both 6CIT (1.89) and MMSE (1.34), according to the classification of effect sizes suggested by Cohen (Reference Cohen1988), but when comparing patients with MCI and no cognitive impairment there was only a medium effect size for both 6CIT (0.65) and MMSE (0.70).
Discussion
Although recommended as a dementia screening test in primary care (Brodaty et al., Reference Brodaty, Low, Gibson and Burns2006), 6CIT has also been assessed in secondary care settings (Upadhyaya et al., Reference Upadhyaya, Rajagopal and Gale2010; Tuijl et al., Reference Tuijl, Scholte, de Craen and van der Mast2012), as well as in this study. Access to detailed neuropsychological testing is far from universal in secondary care settings, and hence brief cognitive screening tests may contribute to patient assessment in this setting, as well as in primary care, if test metrics are satisfactory.
In this pragmatic study, 6CIT proved quick and easy to use, and acceptable to patients: no patient either declined testing or failed to complete the test. Because it is an entirely oral test, no specialized equipment is required, it may be used in visually impaired patients (Rawle and Larner, Reference Rawle and Larner2013), and it could be administered by telephone.
Using the specified cutoffs, 6CIT score ≥ 10 had good sensitivity and specificity in this patient cohort for the diagnosis of dementia versus no dementia (0.88 and 0.78, respectively) whereas 6CIT score > 4 but ≤ 9 showed less good metrics for MCI diagnosis versus no cognitive impairment (0.66 and 0.70, respectively). Diagnostic accuracy was moderate (area under the ROC curve = 0.90 for dementia, 0.71 for MCI). Effect size (Cohen's d) calculations suggested a consistent difference in 6CIT scores between individuals with dementia and without dementia, but less consistency between MCI and without dementia non-MCI patients; effect sizes were comparable to those of other frequently used CSIs examined in this setting (Larner, Reference Larner2014b).
6CIT and MMSE scores were highly (negatively) correlated (−0.73), as in previous reports: −0.91 (Brooke and Bullock, Reference Brooke and Bullock1999), and −0.82 (both Upadhyaya et al., Reference Upadhyaya, Rajagopal and Gale2010 and Tuijl et al., Reference Tuijl, Scholte, de Craen and van der Mast2012). Compared to MMSE, 6CIT had better sensitivity for diagnosis of both dementia and MCI, as reflected also in weighted comparisons, although these measures are dependent on the exact MMSE cutoff chosen, and different methods for cutoff selection may give different results (Larner, Reference Larner2015).
Limitations of the study may include the selected nature of the population encountered in any secondary care setting, although as a pragmatic study no pre-selection for patient diagnosis was made and hence the approach is more reflective of day-to-day clinical practice. The prevalence of dementia was low, unlike the assumption in other secondary care studies, but this reflects changes in patient cohorts referred to this clinic in recent years, perhaps as a consequence of national directives on dementia which have resulted in referral of more individuals with subjective memory impairment (Larner, Reference Larner2014c). Predictive values were calculated using the prevalence rates for dementia and MCI observed in the study group, and not by making any assumptions about prevalence rates.
In summary, 6CIT performance in this study was acceptable, and head-to-head with MMSE it showed no inferiority. Despite its brevity and simplicity, and the recommendation that it be used in primary care (Brodaty et al., Reference Brodaty, Low, Gibson and Burns2006), it remains the case that the majority of patients referred with memory complaints from primary care have not been tested with 6CIT (Cagliarini et al., Reference Cagliarini, Price, Livemore and Larner2013; Ghadiri-Sani and Larner, Reference Ghadiri-Sani and Larner2014). The negative scoring of 6CIT may perhaps be confusing for those more familiar with MMSE, although 6CIT scores are classified to aid test interpretation (Table 1). Hence, 6CIT might be considered an acceptable alternative to MMSE for cross-sectional assessment of cognitive disorders, particularly in the light of current concerns about, and copyright issues surrounding, use of the latter (Carnero-Pardo, Reference Carnero-Pardo2014). Further studies will be required to assess the reliability of 6CIT in secondary care settings.
Conflict of interest
None.
Description of authors’ roles
A. J. Larner conceived and planned the study. Both authors collected the data. A. J. Larner analyzed the data and drafted the manuscript. Both authors revised the manuscript for intellectual content.
