Introduction
The Clock Drawing Test (CDT) is one of the most popular instruments for dementia screening and only the Mini-mental State Examination (MMSE) is more frequently used (Shulman et al., Reference Shulman2006). The CDT is an easy and fast test, especially for the initial diagnostic assessment of patients with Alzheimer's disease (AD). It is a good option particularly for visuospatial, visuoconstructional and executive function evaluation, and it can be used both in non-specialized settings and in clinics specialized in geriatrics, neurology and psychiatry (Mendez et al., Reference Mendez, Ala and Underwood1992; Shulman et al., Reference Shulman, Gold, Cohen and Zucchero1993; Royall et al., Reference Royall, Cordes and Polk1998; Shulman, Reference Shulman2000).
Despite its popularity there is no consensus on the most appropriate way to apply and score the CDT because of the large number of scoring systems available and their varying levels of complexity. This makes it difficult to compare study results (Shulman, Reference Shulman2000; Storey et al., Reference Storey, Rowland, Basic and Conforti2001; Richardson and Glass, Reference Richardson and Glass2002; Shulman et al., Reference Shulman2006). In addition, these scales were originally developed and studied in heterogeneous populations and patients were rarely compared to normal controls. Therefore, there is limited information on CDT performance among normal older adults (Hubbard et al., Reference Hubbard2008).
Previous studies have reported varying degrees of sensitivity and specificity for the CDT when used to detect dementia syndromes. In a meta-analysis, Shulman (Reference Shulman2000) documented 85% sensitivity and specificity in all CDT scoring systems studies between 1983 and 1998. Nevertheless, various authors have been unable to match this accuracy level or the one described in the original CDT study (Storey et al., Reference Storey, Rowland, Basic and Conforti2001; Richardson and Glass, Reference Richardson and Glass2002; Schramm et al., Reference Schramm, Berger, Muller, Kratzsch, Peters and Frölich2002; Seigerschmidt et al., Reference Seigerschmidt, Mosch, Siemen, Förstl and Bickel2002; Scanlan et al., Reference Scanlan, Brush, Quijano and Borson2002; Powlishta et al., Reference Powlishta2002; Connor et al., Reference Connor, Seward, Bauer, Golden and Salmon2005). This may be explained by sampling differences associated with culture, education and recruitment procedures.
There is also variability in reported degrees of agreement between the CDT and other screening tests. The correlation between CDT and MMSE ranges from moderate (0.30) to high (0.77), with a mean of 0.61 (Shulman, Reference Shulman2000), and few studies have compared the CDT with more complex neuropsychiatric batteries. Most of the studies available have compared the CDT with the Cambridge Cognitive Examination (CAMCOG), using only one scoring system, in heterogeneous samples (for example, some included patients with various psychiatric disorders) without the inclusion of normal controls (Bourke et al., Reference Bourke, Castleden, Stephen and Dennis1995; Heinik et al., Reference Heinik, Solomesh, Shein and Becker2002; Reference Heinik, Solomesh, Bleich and Berkman2003; Reference Heinik, Solomesh and Berkman2004; van der Burg et al., Reference van der Burg2004).
Although progress has been made in increasing knowledge about CDT accuracy in different clinical contexts, there are still many unanswered questions about this instrument. Previous studies have highlighted methodological limitations of earlier research (Heinik et al., Reference Heinik, Solomesh, Bleich and Berkman2003; Reference Heinik, Solomesh and Berkman2004). Although the CDT has been shown to be interchangeable with the MMSE and the CAMCOG previously (Heinik et al., Reference Heinik, Solomesh, Bleich and Berkman2003; Reference Heinik, Solomesh and Berkman2004), this finding cannot yet be generalized to the general population due to the lack of the inclusion of a substantial number of normal community dwelling controls in previous samples, the absence of patients with low educational levels, and absence of data from non-specialized settings such as primary care or general geriatric services.
The objective of the present study was to corroborate and expand on previous findings by evaluating the sensitivity and specificity of the CDT in a sample composed of older adults diagnosed with AD and normal controls, with heterogeneous educational levels in an outpatient geriatric facility. We also compared CDT accuracy to that of the MMSE and the CAMCOG. An additional aim was to test whether the association of the MMSE with the CDT would lead to higher or comparable accuracy as that reported for the CAMCOG in this sample.
Methods
The current study used cross-sectional methodology and included 121 community-dwelling outpatients previously diagnosed with AD and 99 normal controls (NC). All subjects were older than 60 years and had heterogeneous levels of education. Both groups consisted of patients who sought general medical attention at a geriatrics outpatient facility.
We excluded AD patients with severe cognitive decline (Clinical Dementia Rating = 3), major depression, plegia or paresis, important tremor, functional impairment in both hands, severe visual or auditive impairment, and patients who refused to complete any of the tests.
Patients included in the NC group did not have memory complaints or mood disorder. They were thoroughly evaluated by two of the authors (I. A. and J. E. M.) using a clinical protocol which included the CAMDEX (see Procedures below). In all patients, medical diseases had to be stable and could not be severe for the patient to be included in the study protocol. If the CAMDEX indicated any mood or cognitive disorder, the patient was excluded.
Procedures
Each patient underwent a comprehensive psychiatric, clinical and neurological examination. Caregivers and family members were interviewed to provide information regarding functional status. Laboratory (hemogram, renal and hepatic function, electrolytes, vitamin B12 and folate, TSH, T4, and VDRL) and imaging exams (computerized tomography and magnetic resonance of the brain, with single photon emission tomography in selected cases of mild AD) were carried out in all patients to exclude potentially treatable causes of cognitive impairment and identify other dementia etiologies other than AD. All subjects with AD met DSM-IV (American Psychiatric Association, 1994) and NINCDS-ADRDA criteria for probable AD (McKhann et al., Reference McKhann, Drachman, Folstein, Katzman, Price and Stadlan1984). Two geriatricians participated in the diagnostic process in order to increase diagnostic reliability. The Clinical Dementia Rating (CDR) was obtained using the scoring rules suggested by Morris et al. (Reference Morris1993).
The Cambridge Examination for Mental Disorder of the Elderly (CAMDEX), validated in Brazil by Bottino et al. (Reference Bottino, Stoppe, Scalco, Ferreira, Hototian and Scalco2001), was completed with each patient. It includes the CAMCOG (Roth et al., Reference Roth1986; Nunes et al., Reference Nunes2008), which yields scores for the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975; Brucki et al., Reference Brucki, Nitrini, Caramelli, Bertolucci and Okamoto2003) and the CDT (Freedman et al., Reference Freedman, Leach, Kaplan, Winocur, Shulman and Delis1994; Shulman, Reference Shulman2000; Fuzikawa et al., Reference Fuzikawa, Lima-Costa, Uchoa, Barreto and Shulman2007). The CAMCOG score ranges from 0 to 107.
All CAMCOG-derived CDTs were scored independently by two of the authors (I. A. and J. E. M.) using the Shulman et al. (Reference Shulman, Gold, Cohen and Zucchero1993), Mendez et al. (Reference Mendez, Ala and Underwood1992) and the Sunderland et al. (Reference Sunderland1989) scoring systems. Possible scores range from 0 to 5, 0 to 20, and 0 to 10 for each scale respectively. These scales were chosen because they are the most frequently used in the literature and they capture both qualitative (Shulman) and quantitative (Mendez and Sunderland) aspects of the CDT. Several studies in the literature have indicated that the scales by Shulman et al. (Reference Shulman, Gold, Cohen and Zucchero1993), Mendez et al. (Reference Mendez, Ala and Underwood1992) and Sunderland et al. (Reference Sunderland1989) are those with higher diagnostic accuracy, and that they tend to show similar results when compared with neuropsychiatric exams, even when used in populations with diverse cultural backgrounds and educational levels (Brodaty and Moore, Reference Brodaty and Moore1997; Shulman, Reference Shulman2000; Scanlan et al., Reference Scanlan, Brush, Quijano and Borson2002; Richardson and Glass, Reference Richardson and Glass2002). A pre-drawn circle was not used because circle drawing may reveal visual constructional impairment. The hand setting time was established at 11:10, because unimpaired performance requires overcoming stimulus bound errors such as drawing the minute hand at number 10 (Freedman et al., Reference Freedman, Leach, Kaplan, Winocur, Shulman and Delis1994). The clocks were scored on each system blind to the diagnosis and the CAMCOG score.
The study protocol was designed and written by the lead author and approved by the local institutional ethical committee.
Statistical analysis
The statistical package SPSS-14 for Windows was used for data analysis. Differences between diagnostic groups (AD × NC) on continuous variables were assessed using independent samples t test when data followed normal distribution. Nonparametric data were analyzed using the Mann-Whitney U test. Pearson's χ2 statistics were calculated to compare sociodemographic data for the two groups. An intra-class correlation coefficient was used to evaluate inter-rater reliability for the three clock drawing scoring systems. Correlations between CDTs and cognitive scores (MMSE and CAMCOG) were examined using Spearman's correlation coefficient. Receiver operating characteristic (ROC) curve analyses were carried out to compare the sensitivity and specificity of each CDT scoring system. ROC curves for CAMCOG, MMSE and CDT, and for CDT and MMSE combined were constructed, and the area under the curve was calculated. Logistic regression analysis was used to determine whether the CDT combined with the MMSE provided better prediction for dementia diagnosis. The equations used in logistic regression for the three CDT scoring systems were: 1.18–0.55*MMSE-0.23*Mendez, 16.36–0.52*MMSE-0.76*Shulman, 17.59–0.54*MMSE-0.43*Sunderland, respectively for each CDT scale. Areas under the curve (AUC) for each instrument and combinations were also compared according to Hanley and McNeil's method. To examine the effect of education on CDT accuracy, ROC curve analyses were carried out with the sample divided into two educational strata: 1 to 4 years of education, and 5 years and more. All statistical tests were two-tailed and regarded as significant when p < 0.05.
Results
Both groups had similar mean ages: 77.3 (±7.52) years in the control group and 77.9 (±7.30) years in the AD group (p = 0.553). Gender and education levels of the subjects are shown in Table 1. There was a non-significant predominance of women in the AD group (p = 0.088). Educational level was significantly different between the two groups. In the AD group, there were more subjects with low education (0–4 years) (p = 0.001).
Characterization of the sample regarding gender and education
*Pearson's χ2; AD = Alzheiner's disease.
There was a high inter-rater correlation coefficient for the three CDT scales, as can be seen in Table 2 (p < 0.001). The highest correlation was with Mendez's scale and the lowest with Shulman's.
Inter-rater correlation coefficient for the three CDT scales
CC = correlation coefficient; CI = confidence interval.
Table 3 presents the results of the neuropsychiatric tests. There was a significant difference between the groups for the MMSE, CAMCOG and CDT (p < 0.001). For all tests, median values were smaller in the AD group. Non-parametric tests were used because the distribution was not normal for cognitive variables.
Comparison between the MMSE, CAMCOG and CDT (Mendez, Shulman and Sunderland) in the control and AD groups
*Mann-Whitney.
AD = Alzheimer's disease; IQI = inter-quartile interval; MMSE = Mini-mental State Examination; CAMCOG = Cambridge Cognitive Examination; CDT = Clock Drawing Test.
The association between the CDT scales of Mendez, Shulman and Sunderland and the MMSE and the CAMCOG were significant and robust (p < 0.001) (Table 4). The relationship between the CDT and the MMSE, and the CAMCOG was slightly higher with the Shulman scale, when compared with the other two CDT scales.
Correlation between the CDT and the MMSE and the CAMCOG
*Spearman's correlation.
CDT = Clock Drawing Test; MMSE = Mini-mental State Examination; CAMCOG = Cambridge Cognitive Examination.
Sensitivity and specificity for the CAMCOG, MMSE and each CDT scoring system were obtained in isolation, and for the CDT combined with the MMSE, using ROC analysis (Table 5). Sensitivity of the CAMCOG and the MMSE were similar (89.2% and 88.3%, respectively), with increased specificity for the CAMCOG (81.8%). The cut-off scores for the three CDT scoring systems were high. The accuracy for the three CDT scoring systems varied, with sensitivity between 74.2% and 85.8%, and specificity between 66.7% and 89.9%. The highest sensitivity and specificity were obtained for Shulman's scale (85.8%) and Sunderland's scale (89.9%), respectively. The combination of the CDT and the MMSE enhanced the sensitivity to 89.2% and 90%, equivalent or slightly better than the CAMCOG and the MMSE separately. Specificity was also enhanced in the combination of Mendez's CDT scale and the MMSE, but was lower in the other two CDT scoring systems when combined with the MMSE.
Receiver operating characteristic curves area, sensitivity and specificity for the MMSE, CAMCOG, CDT and for the CDT and MMSE combined considering both diagnostic groups
SE = sensitivity; SP = specificity; CI = confidence interval.
MMSE = Mini-mental State Examination; CAMCOG = Cambridge Cognitive Examination; CDT = Clock Drawing Test.
Table 5 also includes ROC curve results for the sample stratified in two educational levels (1–4 years, and ≥ 5). Results suggested that for the lower educational level, sensitivity and specificity were adequate and high. For the higher educational group, there was reduced specificity for the Mendez scale and reduced sensitivity for the Sunderland scale. The Shulman CDT displayed the best balance between sensitivity and specificity for this subgroup.
The areas under the curve showed high and significant values for all instruments. The highest values for the calculated areas were obtained for the CAMCOG (area = 0.914; 95% confidence interval 0.871–0.955), followed by the MMSE (area = 0.911; 95% CI 0.871–0.951). All CDT scoring systems showed smaller areas (between 0.840 and 0.860) than the CAMCOG and the MMSE, but when combined with the MMSE, the areas were slightly better than the CAMCOG.
Statistical comparisons of the AUCs, by means of Hanley and McNeil's method, showed significant differences between the MMSE and the Mendez and Sunderland CDT scales separately (Table 6). The same result was found for the CAMCOG, indicating superior accuracy for the MMSE and the CAMCOG. MMSE and CAMCOG separately did not show significantly superior accuracy when compared to Shulman's scale. Combination of the CDT scales compared to the MMSE and the CAMCOG did not generate significantly higher accuracy.
Comparison of AUCs for the MMSE, CAMCOG, CDT and for the CDT and MMSE combined considering both diagnostic groups
a Information about the first instrument or combination; b Information about the second instrument. Bold indicates significant differences.
AUC = area under the curve; MMSE = Mini-mental State Examination; CAMCOG = Cambridge Cognitive Examination; CDT = Clock Drawing Test.
Discussion
To our knowledge, this is the first CDT study to conduct a more controlled comparison between elderly people diagnosed with AD and normal controls, using three formal CDT scoring systems, the MMSE and the CAMCOG. Previous studies with similar objectives presented methodological limitations such as: (i) they used only one or two formal CDT scoring systems; (ii) they included mixed dementia etiologies; (iii) they did not include normal controls in the initial study protocol for direct comparison; or (iv) they included subjects with mood disorders; raising questions about the utility of the CDT in non-specialized settings (Bourke et al., Reference Bourke, Castleden, Stephen and Dennis1995; Heinik et al., Reference Heinik, Solomesh, Shein and Becker2002; Reference Heinik, Solomesh, Bleich and Berkman2003; Reference Heinik, Solomesh and Berkman2004; van der Burg et al., Reference van der Burg2004).
Present results indicate that the three CDT scales performed with similar accuracy to AD screening instruments, and had a high association with the MMSE and the CAMCOG. The sensitivity and specificity of the three scoring systems studied were different from the ones reported in their original studies (Sunderland et al., Reference Sunderland1989; Mendez et al., Reference Mendez, Ala and Underwood1992; Shulman et al., Reference Shulman, Gold, Cohen and Zucchero1993). In fact, several studies have shown different CDT outcomes when attempting to replicate the original accuracy by comparing various scoring protocols (Storey et al., Reference Storey, Rowland, Basic and Conforti2001; Richardson and Glass, Reference Richardson and Glass2002; Schramm et al., Reference Schramm, Berger, Muller, Kratzsch, Peters and Frölich2002; Seigerschmidt et al., Reference Seigerschmidt, Mosch, Siemen, Förstl and Bickel2002; Scanlan et al., Reference Scanlan, Brush, Quijano and Borson2002; Powlishta et al., Reference Powlishta2002; Connor et al., Reference Connor, Seward, Bauer, Golden and Salmon2005). Therefore, our findings (sensitivity between 74.2% and 85.8%, and specificity between 66.7% and 89.9%) were consistent with the literature. In contrast to van der Burg et al. (Reference van der Burg2004), inter-rater reliability was high for all CDT scales in the current study.
In this study, CDT scales had high correlations with the MMSE and the combination between the two tests enhanced diagnostic accuracy, being slightly better than the CAMCOG alone. The highest correlations were found for the scales of Shulman et al. (Reference Shulman, Gold, Cohen and Zucchero1993), Mendez et al. (Reference Mendez, Ala and Underwood1992) and the CLOX scale (Royall et al., Reference Royall, Cordes and Polk1998). Brodaty and Moore (Reference Brodaty and Moore1997) showed that the CDT can be better than the MMSE in a memory clinic. Previous studies have shown that there is also a potential advantage when both tests are applied concomitantly (Ferrucci et al., Reference Ferrucci1996). Juby (Reference Juby1999) conducted a study with 150 elderly outpatients at a general clinic comparing the MMSE and three interpretation methods of the CDT. All CDT scores were significantly associated with the MMSE results, with high to moderate correlations. The association between the MMSE score and the various CDTs was also high in the study by Schramm et al. (Reference Schramm, Berger, Muller, Kratzsch, Peters and Frölich2002) at a memory clinic, and the combination between the two tests enhanced the accuracy. Present results corroborate these previous findings.
The three CDT scoring systems were highly correlated with the CAMCOG in our study. The CAMCOG sensitivity and specificity were lower than previously reported (Roth et al., Reference Roth1986; Hendrie et al., Reference Hendrie1988; Neri et al., Reference Neri1994). This may be related to the heterogeneous educational level of the population in this study, as most international studies have included older adults with an average of 12 years of education.
Only a few studies have compared the CDT with neuropsychometric batteries of higher diagnostic accuracy in dementia (Bourke et al., Reference Bourke, Castleden, Stephen and Dennis1995; Heinik et al., Reference Heinik, Solomesh, Shein and Becker2002; Reference Heinik, Solomesh, Bleich and Berkman2003; Reference Heinik, Solomesh and Berkman2004; van der Burg et al., Reference van der Burg2004). Bourke et al. (Reference Bourke, Castleden, Stephen and Dennis1995) compared the CDT to the CAMCOG in 77 patients with AD, using the Shulman and Mendez scales and found a strong association between both CDT scales and the CAMCOG. The study by Heinik et al. (Reference Heinik, Solomesh, Shein and Becker2002) aimed to compare the scales by Shulman et al. (Reference Shulman, Gold, Cohen and Zucchero1993) and Freedman et al. (Reference Freedman, Leach, Kaplan, Winocur, Shulman and Delis1994) to the MMSE and the CAMCOG in 49 elderly individuals with mild to moderate AD. Both scales had a high association with the CAMCOG in both stages of dementia and with the MMSE only in the mild stage. The scale by Freedman et al. (Reference Freedman, Leach, Kaplan, Winocur, Shulman and Delis1994) had poorer performance among patients in the moderate stage when compared with patients with mild AD. No difference was found with the Shulman scale (1993). The same authors later selected 56 patients with AD, 36 with vascular dementia and 26 controls with psychiatric disorders at a psychogeriatrics outpatient clinic (Heinik et al., Reference Heinik, Solomesh, Bleich and Berkman2003). The CAMCOG was compared with the MMSE and the CDT, interpreted according to Freedman et al. (Reference Freedman, Leach, Kaplan, Winocur, Shulman and Delis1994). The CDT showed high correlations with the MMSE (0.73) and with the CAMCOG (0.80). The relationship between the MMSE and the CAMCOG was also high (0.93). Later, Heinik et al. (Reference Heinik, Solomesh and Berkman2004) analyzed three scales for CDT interpretation in the same group of patients as the previous study. They added the application of the Shulman and the CAMCOG CDT scales. The authors found significant correlations between the three scales with the CAMCOG score, as well as with the MMSE.
In the present study, in agreement with Heinik et al. (Reference Heinik, Solomesh, Bleich and Berkman2003; Reference Heinik, Solomesh and Berkman2004), we also found strong associations between CDT scales and the MMSE and the CAMCOG. In the present sample we found lower AUCs than in these two previous studies, which may be due to differences in sample characteristics. We also found better accuracy when CDT scales were combined with the MMSE, similar to CAMCOG accuracy. Yet, statistical tests comparing ROC curves did not show significant differences between the combined instruments and the CAMCOG. Therefore, present results replicate previous findings in samples with distinct characteristics (education, ethnicity, sampling context).
A previous study conducted in Brazil (Nunes et al., Reference Nunes2008) investigated the usefulness of the CDT as a screening tool among elderly people with eight or more years of education. They compared the CAMCOG with the MMSE, the verbal fluency test and the CDT in a sample of normal controls, people with mild cognitive impairment and dementia patients with mean educational levels of 14.2 ± 3.8 years. The CDT was analyzed according to the Sunderland scale. The CDT did not prove as efficient as the former tests in the differentiation of any of the diagnostic groups. The CDT showed sensitivity of 83% and specificity of 66%. In agreement with our study, CDT accuracy was not as high as in other previously published studies, when used with elderly people with higher educational levels. In the present study, CDT proved to be a better screening instrument for elderly people with fewer years of education, in contrast to previous findings (Shulman, Reference Shulman2000; Pinto and Peters, Reference Pinto and Peters2009).
In fact, this is the second study to show good screening accuracy for the CDT in non-English speaking and low educational level AD patients compared with normal controls (Leung et al., Reference Leung, Lui and Lam2005). Brodaty and Moore (Reference Brodaty and Moore1997) obtained heterogeneous results when correlating CDT scores with education, while Borson et al. (Reference Borson1999) found lower CDT scores for poorly educated non-English AD patients versus normal community participants. The CDT displayed the best balance between specificity and sensitivity in the less educated group. Our study showed worse CDT performance in highly educated patients. These results agree with Leung et al. (Reference Leung, Lui and Lam2005) who showed a low cut-off point for the highest education group suggesting that the CDT may be less useful for the screening of AD in this population due to the simplicity of the test.
The results presented here expand the existing literature on the CDT as a dementia screening instrument as this was the first study to compare AD patients to normal controls, including three CDT scoring systems, the MMSE and the CAMCOG. Results suggest that the CDT is an accurate screening instrument for AD, when used in an elderly population with heterogeneous backgrounds, although with lower accuracy for elderly people with five or more years of education. Results also suggest that the CDT and the MMSE combined may replace the CAMCOG, with similar accuracy, when time for screening is limited.
Conflict of interest
None.
Description of authors’ roles
I. Aprahamian designed the study, collected the data and wrote the paper. J. E. Martinelli collected the data and supervised the data collection. A. L. Neri participated in study design and statistical analysis. M. S. Yassuda participated in study design and statistical analysis, and revised the paper.
