Introduction
Dementia is one of the most common and burdensome diseases in the elderly and, with the increase in life expectancy, the number of people with dementia will steadily increase (Wancata et al., Reference Wancata, Musalek, Alexandrowicz and Krautgartner2003; Ferri et al., Reference Ferri2005). With regard to costs of medical care (Quentin et al., Reference Quentin, Riedel-Heller, Luppa, Rudolph and Konig2010; Hotte et al., Reference Hotte, Lankers, Kissler, Freyberger and Schroder2010) and the expected ongoing shift from family to inpatient and outpatient care, a diagnosis of dementia has an enormous impact on society and its healthcare systems. Furthermore, the diagnosis of dementia is a burden not only for the patients themselves but also for their caregivers. Currently dementia cannot be cured, but several promising methods of treatment exist to improve the patient's well-being and to delay disease onset (Middleton and Yaffe, Reference Middleton and Yaffe2009; Qiu et al., Reference Qiu, Kivipelto and von Strauss2009). Current research aims to predict developing dementia reliably (Pentzek et al., Reference Pentzek, Fuchs, Wiese and Abholz2010) in order to enable the effective use of treatment options.
Individuals diagnosed with mild cognitive impairment (MCI) constitute a high-risk population for the development of dementia and are thus of major clinical interest with regard to a reliable early diagnosis of the disease (Busse et al., Reference Busse, Hensel, Guhne, Angermeyer and Riedel-Heller2006; Zanetti et al., Reference Zanetti, Ballabio, Abbate, Cutaia, Vergani and Bergamaschini2006). Consensus criteria for MCI were provided in 2004 for the first time by an international working group on mild cognitive impairment (Winblad et al., Reference Winblad2004). In recent years, research has also shown a strong association of MCI with dementia-specific brain changes, biomarkers, genetic and social risk factors (Jackson and Snyder, Reference Jackson and Snyder2008; Petersen and Negash, Reference Petersen and Negash2008; Ries et al., Reference Ries2008). However, obtaining a diagnosis of MCI is time-consuming due to the need for a comprehensive neuropsychological assessment. Hence, there is a strong need for quick and easy screening instruments for MCI as a pre-stage of dementia.
The clock drawing test (CDT) is a well-established and validated dementia screening tool (Shulman, Reference Shulman2000) because of its easy, quick and simple administration and good acceptance among elderly patients. Moreover, the CDT requires different cognitive abilities including auditory and visual comprehension, concentration, visuospatial abilities, abstract conceptualization, and executive control (Shulman, Reference Shulman2000). However, the findings concerning the CDT's ability to differentiate between subjects with and without MCI are rather inconsistent (Lee et al., Reference Lee, Kim, Hong, Lee, Oh and Cheong2008; Pinto and Peters, Reference Pinto and Peters2009; Ehreke et al., Reference Ehreke, Luppa, König and Riedel-Heller2010). For example, Yamamoto et al. (Reference Yamamoto2004) found that the CDT has utility to screen for MCI, whereas Lee et al. (Reference Lee, Kim, Hong, Lee, Oh and Cheong2008) did not recommend the use of the CDT as a screening instrument for MCI. The inconsistent results might particularly be due to the variety of versions of application and scoring. Thus, the aim of the study is to compare and evaluate the diagnostic utility in screening for MCI of different scoring systems of the CDT using a representative German population-based sample aged 75 years and older.
Methods
Sample
The data were derived from the Leipzig Longitudinal Study of the Aged (LEILA 75+), a population-based study on the epidemiology of dementia and mild cognitive impairment. At baseline a total of 1692 participants aged 75 years and over were included in the sample: 1500 of these were identified by systematic random sampling from an age-ordered list from the local registry office. Additionally, institutionalized patients were included in the study by proportion (n = 192), using systematic random sampling from an age-ordered list provided by the four institutions in the study area. The baseline assessment was conducted between January 1997 and June 1998. The study design of the LEILA 75+ study is described in detail elsewhere (Riedel-Heller et al., Reference Riedel-Heller, Schork, Matschinger and Angermeyer2000; Reference Riedel-Heller, Busse, Aurich, Matschinger and Angermeyer2001). All participants who were assessed at baseline were requested to take part in up to five follow-up assessments, which were conducted between July 1998 and April 2005, on average every 1.4 years.
The sample consists of all study participants interviewed face to face at the second follow-up wave of the LEILA 75+ study, which was conducted between January 2000 and July 2001. We started with the second follow-up, because it was at this stage that the CDT was applied for the first time in the course of the study. Of the 741 study participants interviewed face to face at this time-point, data for 313 study participants were excluded from the following analyses: 138 participants suffered from dementia and 175 had incomplete assessments. Thus, as shown in Figure 1, the results of the present study are based on the data of the remaining 428 study participants.
Sampling flowchart of the study.
Data collection
Participants were interviewed in their home environment by trained psychologists and physicians using structured clinical interviews.
Data on sociodemographic variables (age, gender, education and marital status), on subjective memory impairment (“Do you feel like your memory has gotten worse”) and on comorbidity (diabetes mellitus, stroke, myocardial infarction, rheumatism, impairments in hearing and vision) were collected based on a standardized questionnaire.
The main instrument employed was the Structured Interview for the Diagnosis of Dementia of the Alzheimer Type, Multi-infarct Dementia and Dementias of other Etiology according to ICD-10 and DSM-IV (SIDAM; Zaudig et al., Reference Zaudig1991; Zaudig and Hiller, Reference Zaudig and Hiller1996). The SIDAM consists of a neuropsychological test battery including the 30 items of the Mini-Mental State Examination (MMSE; Folstein et al., Reference Folstein, Folstein and McHugh1975), third party information on psychosocial impairment, and a section for clinical judgement including severity rating of dementia. The neuropsychological test battery based on 55 items covers six areas of cognitive functioning: orientation, memory, intellectual abilities, verbal abilities and calculation, constructional abilities (visual-spatial) as well as aphasia and apraxia. The section on psychosocial impairment includes a scale for the assessment of activities of daily living with 14 items (SIDAM-ADL scale). Clinical evaluation is based on information from the participant and, if necessary, from a third-party concerning sociodemographic characteristics, potential risk factors for cognitive impairment and possible changes in personality.
If it was not possible to administer the SIDAM (e.g. because of death or severe weakness or because relatives refused the administration of the SIDAM on behalf of the elderly person in their care), we offered a fully structured proxy interview in order to obtain information on cognitive and psychosocial functioning as well as subjective memory impairment.
Clock Drawing Test
A sheet of paper with a pre-drawn circle of 9 cm diameter was presented and the following instruction was given: “This is supposed to be a clock. Please put the missing numbers on the clock. Then set the time at 10 after 11.” No additional prompts were given.
Within this study, we scored each clock according to different scoring systems blinded to the results of the rest of the assessment. We chose six scoring systems of semi-quantitative and quantitative character (Table 1). The two semi-quantitative scoring systems (Sunderland et al., Reference Sunderland1989; Shulman et al., Reference Shulman, Gold, Cohen and Zucchero1993) focused on scoring the whole clock, while the four quantitative scoring systems (Clock Drawing Interpretation Scale by Mendez et al., Reference Mendez, Ala and Underwood1992; Rouleau et al., Reference Rouleau, Salmon, Butters, Kennedy and Mcguire1992; Babins et al., Reference Babins, Slater, Whitehead and Chertkow2008; and the Three-item Clock Drawing Test of Lin et al., Reference Lin2003) focused on different aspects of the clocks (such as clock face, numbers or hands) and scored them separately. Furthermore, the scoring systems differ regarding direction of scoring. Most of the scoring systems which were applied in our study have been constructed by giving higher scores for better clock drawing. Only one scoring system (Shulman et al., Reference Shulman, Gold, Cohen and Zucchero1993) has been constructed with a reversed scoring direction (the better the clock drawing, the lower the score). In order to test the inter-rater reliability, half of the clocks (n = 232) were scored by two independent raters.
Scoring systems
Note: ↓ the lower the score, the better the cognitive abilities; ↑ the higher the score, the better the cognitive abilities.
Diagnosis of MCI
MCI was diagnosed according to the criteria proposed by the International Working Group on MCI (Winblad et al., Reference Winblad2004). These consensus criteria include: (a) absence of dementia according to DSM-IV or ICD-10; (b) evidence of cognitive decline: subjective cognitive impairment (measured by self-rating or informant report) and impairment on objective cognitive tasks and/ or evidence of decline over time on objective cognitive tasks; and (c) preserved baseline activities of daily living or only minimal impairment in complex instrumental functions.
These MCI criteria were operationalized as follows:
• dementia according to DSM-IV was diagnosed with the SIDAM;
• subjective cognitive impairment was fulfilled when the question on subjective memory impairment was positively answered;
• the impairment on objective cognitive tasks was diagnosed with the SIDAM neuropsychological test battery; impairment in all six cognitive domains was defined as test performance of more than 1 standard deviation below the main value for age- and education specific norms (Luck et al., Reference Luck, Zaudig, Wiese and Riedel-Heller2007b);
• the activities of daily living were assessed with the SIDAM-ADL scale; and subjects with only one impairment or with no impairment in the 14 items of the SIDAM-ADL scale were regarded as functionally unimpaired or only minimal impaired.
According to Winblad et al. (Reference Winblad2004) four subtypes of MCI were classified: (i) participants having an objective deficit in memory but not in any other domain of cognitive functioning received a diagnosis of Single Amnestic MCI; (ii) Single Non-amnestic MCI was diagnosed only if a single domain other than memory was impaired; (iii) if at least two cognitive domains other than memory showed an objective impairment, participants received a diagnosis of Multidomain Non-amnestic MCI; and (iv) Multidomain Amnestic MCI was diagnosed if memory and at least one other cognitive domain were impaired.
Analysis
All statistical analyses were carried out using PASW Statistics 18 for Windows and Microsoft Office Excel (including the Analyse-it tool). The significance level was set at 0.05 for all analyses.
Differences in sociodemographic characteristics and CDT scores between subjects with and without MCI were investigated using the two-sided t-test, Mann-Whitney-U-test and corrected χ2 test as appropriate. In order to assess the ability of the CDT to differentiate participants with and without MCI, the receiver operating characteristic (ROC) including area under the curve (AUC), sensitivity, specificity and Youden-index (Youden, Reference Youden1950) were calculated. The ROC curves were compared according to DeLong et al. (Reference DeLong, DeLong and Clarke-Pearson1988). The inter-rater reliability was tested using the inter-rater correlation coefficients Spearman-Rho (rs) and Kendall-Tau-b (τB).
Results
Sample characteristics
Fifty-eight (13.6%) of the 428 participants were diagnosed as having MCI. In terms of the subtypes, prevalence was highest for Single Non-amnestic MCI (5.4%), followed by Multidomain Amnestic MCI (5.1%), Single Amnestic MCI (1.9%), and Multidomain Non-amnestic MCI (1.2%). Owing to the small number of MCI cases, however, no subtype-specific analyses could be included. The characteristics of the 58 study subjects diagnosed with MCI and the 370 subjects without MCI are presented in Table 2. The subjects diagnosed with MCI did not significantly differ from the subjects without MCI in level of education and gender. The subjects having a diagnosis of MCI, however, were somewhat older than subjects without.
Sample characteristics (n = 428)
a Based on the revised version of the international new Comparative Analysis of Social Mobility in Industrial Nations (CASMIN) educational classification (Brauns and Steinmann, Reference Brauns and Steinmann1999).
SD = standard deviation; MCI = mild cognitive impairment
CDT scores of subjects with and without MCI
CDT scores of subjects with and without MCI are shown in Table 3. The CDT scores differ significantly between the two groups according to all semi-quantitative and quantitative scoring systems applied. Examining the scoring system in detail revealed significant differences in the scoring of the clock's numbers and hands. However, no statistical differences could be found for scoring the clock face, center or gestalt of the clock.
Differences in mean CDT scores by different scoring systems (n = 428)
Receiver operating characteristic of the CDT scoring systems for screening for MCI
Calculating the ROC of the CDT scoring systems for screening for MCI yielded a significant AUC for all scoring systems applied: 0.676 (p < 0.001) for Shulman et al. (Reference Shulman, Gold, Cohen and Zucchero1993); 0.663 (p < 0.001) for Sunderland et al. (Reference Sunderland1989) and Ihl et al. (Reference Ihl2000); 0.678 (p < 0.001) for Rouleau et al. (Reference Rouleau, Salmon, Butters, Kennedy and Mcguire1992); 0.694 (p < 0.001) for Babins et al. (Reference Babins, Slater, Whitehead and Chertkow2008); 0.689 (p < 0.001) for the Clock Drawing Interpretation Scale by Mendez et al. (Reference Mendez, Ala and Underwood1992); and 0.642 (p = 0.001) for the three-item Clock Drawing Test of Lin et al. (Reference Lin2003). Hence, there is a 64–69% probability of MCI patients achieving different CDT scores to cognitively healthy individuals. None of the different subscales of the single scoring systems revealed a higher predictive value for MCI than the overall score of the whole scoring system (data not shown).
Moreover, no statistical difference could be found between the ROC curves of the different scoring systems (Table 4).
Comparison of the ROC of the different scoring systems (n = 428)
The best cut-off point for screening MCI for the CDT scoring system of Shulman et al. (Reference Shulman, Gold, Cohen and Zucchero1993) was a score of ≥2 points, according to a maximum Youden index (Youden, Reference Youden1950) of 0.335, a 76% sensitivity and a 58% specificity. For Sunderland et al. (Reference Sunderland1989) and Ihl et al. (Reference Ihl2000) the best cut-off point was a score of ≤ 9 with a Youden index of 0.322, a 69% sensitivity and a 63% specificity. For Rouleau et al. (Reference Rouleau, Salmon, Butters, Kennedy and Mcguire1992), the best cut-off point was a score of ≤ 7 (0.275; 48%; 79%), for Babins et al. (Reference Babins, Slater, Whitehead and Chertkow2008) a score of ≤ 15 (0.306; 60%; 70%) for Mendez et al. (Reference Mendez, Ala and Underwood1992) a score of ≤ 18 (0.338; 64%; 70%) and for Lin et al. (Reference Lin2003) a score of ≤ 2 (0.251; 76%; 49%) respectively.
Inter-rater reliability
The calculations on inter-rater reliability were based on more than half of the sample (n = 232). Table 5 presents the inter-rater correlation coefficients of the two raters for each scoring system. Correlation coefficients above 0.5 (indicating good values) could be found for all scoring systems used. The best inter-rater correlations were found for the scoring system of Sunderland et al. (Reference Sunderland1989) and Ihl et al. (Reference Ihl2000) with rs = 0.790 and τB = 0.729; the worst inter-rater correlations were for the scoring system of Lin et al. (Reference Lin2003) with rs = 0.583 and τB = 0.570.
Inter-rater correlations of the different scoring systems (n = 232)
Discussion
The aim of this study was to compare the utility of different CDT scoring systems for screening for MCI (Winblad et al., Reference Winblad2004). Significant differences in CDT scores between participants with and without MCI have been found for all scoring systems applied. Moreover, a significant probability of correct differentiation between subjects with and without MCI could be shown for all scoring systems by ROC analyses (a 64–69% probability of MCI subjects achieving a different CDT score from subjects without MCI). However, no statistical differences could be found between the ROC curves of the different scoring systems. Furthermore, an indepth examination of screening utility indicators (sensitivity and specificity) showed that none of the screening systems was able to screen reliably for MCI, due to the fact that no cut-off point in any scoring system yielded values of sensitivity higher than 80% and values of specificity higher than 60% (according to the recommendation of Blake et al., Reference Blake, McKinney, Treece, Lee and Lincoln2002). The CDT scoring system of Shulman et al. (Reference Shulman, Gold, Cohen and Zucchero1993), however, almost met these values with a 76% sensitivity and a 58% specificity.
Although our analyses revealed that no scoring system is suited to screen for MCI, some suggestions for improvement of the CDT screening utility could be made. First, the detailed examination of the significant differences in the utility of CDT scores to screen for MCI showed that the subscales evaluating the “clock's hands” and “clock's numbers” in particular distinguished significantly between the MCI and non-MCI subjects. By contrast, no significant differences could be found for the rather basic subscales “gestalt”, “clock face” and “center”. These findings are in line with other authors who proposed to set a high value on the drawn hands when scoring the clock: the hands items – in particular, the placing of the minute hand (Leyhe et al., Reference Leyhe, Milian, Muller, Eschweiler and Saur2009) – were found to have a high predictive value for early stages of dementia/ MCI (Esteban-Santillan et al., Reference Esteban-Santillan, Praditsuwan, Ueda and Geldmacher1998; Babins et al., Reference Babins, Slater, Whitehead and Chertkow2008). Greater focus should therefore be placed on the clock's hands and numbers to increase the CDT's value when screening for MCI.
Secondly, when analyzing the differences in the ROC curves, it becomes clear that scoring systems which have a wider score range seem to enhance the diagnostic value of the CDT. For example, the CDT scoring system of Babins et al. (Reference Babins, Slater, Whitehead and Chertkow2008) and Mendez et al. (Reference Mendez, Ala and Underwood1992), using a wider score range (0–18 and 0–20 points, respectively), appeared to be the best among the quantitative systems, even though the differences between the scoring systems were not statistically significant. These more detailed scoring systems should be able to survey properly the numbers and the hands of the clock by using more items for grading. As mentioned above, the items relating to the clock's hands are of good predictive value for cognitive decline.
Thirdly, because the semi-quantitative scoring systems yielded higher inter-rater correlations compared to those of the quantitative scoring systems, it seems to be more advantageous for the rater if pictures of clocks for each score are provided. In the clinical setting in particular, these more user-friendly scoring systems seem to be even more appropriate as they require less time to score. This is in line with Lee et al. (Reference Lee, Kim, Hong, Lee, Oh and Cheong2008), who also pointed out that multiple scales with quantitative and qualitative features are needed in order to increase the clinical utility of the CDT.
Moreover, an increase in the CDT's utility in screening for MCI might be achieved by improving the formal administration of the CDT. In particular, having the participant draw the clock's face on his/ her own (in the current study a pre-drawn circle was used) is supposed to add diagnostic information, since early stage Alzheimer disease patients tend to draw smaller clock faces (Chiu et al., Reference Chiu, Li, Lin, Chiu and Liu2008). In addition, the impact of the specific time (in the current study the time 11.10 was used), which should be illustrated by setting the hands, could also be proved with regard to CDT's screening utility.
Suggestions for improvement of the CDT's screening utility for MCI might be applied to the scoring system of Shulman et al. (Reference Shulman, Gold, Cohen and Zucchero1993), as this scoring system – already in the current form – almost met the requirements of a clinically useful screening instrument (Blake et al., Reference Blake, McKinney, Treece, Lee and Lincoln2002).
Limitations
The results of our study are subject to some limitations. First, a representative sample of a large population-based study was investigated. However, a substantial number of subjects had to be excluded from our analyses of CDT's screening utility for MCI. Thus, a selection bias cannot be excluded and it is possible that the more highly motivated and healthy subjects remained in the sample during the study course. Thus cognitive impairment might be underestimated. Moreover, the limited sample size of 428 could negatively affect the statistical power of the study. Secondly, owing to the small study sample, only 58 MCI cases could be included in our analysis. We therefore could not examine MCI in depth, especially the different severity and subtypes of MCI. Thirdly, even though the SIDAM, which was used to determine objective cognitive impairment in this study, represents a well-established assessment for diagnosing MCI in epidemiological studies (Luck et al., Reference Luck2007a), it is a brief instrument and as such may lack some sensitivity and specificity for MCI compared to an extensive neuropsychological test battery. Thus, limitations caused by the operationalization of MCI with SIDAM (Zaudig et al., Reference Zaudig1991; Zaudig and Hiller, Reference Zaudig and Hiller1996) could also affect the results. In sum, generalization of the results should be treated with caution.
Conclusions
We found evidence that the CDT is able to differentiate significantly between subjects with and without MCI, regardless of the applied scoring system. However, the screening utility indicators do not suggest that the CDT, as currently administered, is a good screening instrument for MCI. In order to increase the CDT's clinical utility in screening for MCI, our results indicate that the CDT could be improved in terms of its scoring system (in particular, by being semi-quantitative, having a wider score range, and focusing on the clock's hands and numbers in more detail). Moreover, clinical utility might also be increased with regard to the formal administration of the CDT (e.g. measuring the size of the clock).
Improving the CDT's clinical utility as described above might be a worthwhile way of providing a cognitive screening instrument that enables early identification of a higher number of subjects with initial dementia in the stage of MCI (Lonie et al., Reference Lonie, Tierney and Ebmeier2009). Compared to instruments like the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975) and the Mattis Dementia Rating Scale (DRS; Mattis, Reference Mattis1988), an improved CDT might be a more viable cognitive screening measure for MCI due to several advantages. First, compared with the MMSE and DRS which take 15 to 30 minutes to administer, the CDT is a short and easy-to-administer instrument (less than 2 minutes). Secondly, the CDT can be assessed without specialist knowledge and training, which is required for most other comprehensive cognitive assessments. Thirdly, the CDT provides a standardized procedure in terms of its application and scoring. Finally, it has the advantage of being more independent of culture, language and education compared to other assessments.
Conflict of interest
None.
Description of authors' roles
L. Ehreke, T. Luck, M. Luppa, H.-H. König, A. Villringer and S.G. Riedel-Heller formulated the research questions; L. Ehreke, T. Luck, M. Luppa, S.G. Riedel-Heller designed the study; L. Ehreke, T. Luck and M. Luppa carried out the study; L. Ehreke and T. Luck analyzed the data; and L. Ehreke wrote the paper.
Acknowledgments
This work was supported by the junior research grant from the Medical Faculty, University of Leipzig. This paper was also supported by LIFE – Leipzig Research Center for Civilization Diseases, Universität Leipzig. This project was funded by the European Social Fund and the Free State of Saxony.
