Introduction
Drawing tests have long been used in neuropsychological assessment as measures of visuospatial functions and planning in addition to their use as screening measures (Robins-Whalin et al., Reference Robins-Whalin, Backman, Wahlin and Winblad1993; Lezak et al., Reference Lezak, Howieson and Loring2004; Smith, Reference Smith2009) Cognitive screening of older adults is common in contexts in which time is not available for longer, more comprehensive assessments, or in which there is insufficient evidence of the need for more detailed assessment (Cullen et al., Reference Cullen, O'Neill, Evans, Coen and Lawlor2007; Lonie et al., Reference Lonie, Tierney and Ebmeier2009; Pachana et al., Reference Pachana, Helmes, Byrne, Edelstein, Konnert and Pot2010). Drawing tests are frequently used to evaluate visuospatial functions (Lezak et al., Reference Lezak, Howieson and Loring2004) and are used as screening measures in their own right (Pachana et al., Reference Pachana, Helmes, Byrne, Edelstein, Konnert and Pot2010). Popular content for drawing tasks include simple geometric figures (Sivan, Reference Sivan1992), complex figures such as that designed by A. Rey (Rey, Reference Rey1964; Meyers and Meyers, Reference Meyers and Meyers1995), human figures (Goodenough, Reference Goodenough1926; Ericsson et al., Reference Ericsson, Winblad and Nilsson2001), and clock faces (Pam et al., Reference Pam, Stern, Sano and Mayeux1989; Tuokko et al., Reference Tuokko, Hadjistavropoulos, Miller and Beattie1992). Among the simpler geometric figures that have been used for assessment purposes is that of two pentagons overlapping one another such that a four-sided figure is formed by the intersecting corners of the pentagons. The interlocking pentagons have a long history as a method of cognitive screening. The first popular use was in the set of nine Bender Gestalt figures (Bender, Reference Bender1938). Since then, the pentagons, as with other figures from the Bender, have been incorporated into other measures, notably the Mini-Mental State Examination (MMSE; Folstein et al., Reference Folstein, Folstein and McHugh1975) and the later Modified Mini-Mental State Examination (3MS; Teng and Chui, Reference Teng and Chui1987). The 3MS added four items and expanded the range of scores of items such that the possible scores ranged from 0 to 100. Among the changes was the increase to ten points for the correct reproduction of the pentagons from one point on the MMSE.
Scores on the MMSE are known to be influenced by age and education (Bravo and Hébert, Reference Bravo and Hébert1997) and the 3MS likely is as well, although perhaps to a lesser extent. It would be prudent to incorporate age and education in evaluations of such screening measures and their items. While scores on the MMSE are commonly used to reflect the severity of cognitive impairment, less is known as to which items primarily contribute to such differentiation. One attempt at doing so is provided by Fillenbaum et al. (Reference Fillenbaum, Wilkinson, Welsh and Mohs1994), who evaluated the ability of four-item subsets of the MMSE to distinguish older adults who were aging normally from those with dementia and between levels of the severity of dementia. Pentagon copying was not among those four-item subsets that were formed on the basis of their sensitivity to group differences. Ashford et al. (Reference Ashford, Kolm, Colliver, Bekian and Hsu1989) reported pentagon drawing to be intermediate in difficulty in item characteristic curve analyses. Galasko et al. (Reference Galasko, Klauber, Hofstetter, Salmon, Lasker and Thal1990) reported that pentagon drawing was the third-most sensitive MMSE item in differentiating Alzheimer's disease from normal aging, behind items for memory and orientation to place. Ala et al. (Reference Ala, Hughes, Kyrouac, Ghobrial and Elble2001) note that drawing tests can also be helpful in discriminating some forms of dementia. These findings suggest that pentagon copying may function as a screening measure in itself.
In many settings, screening measures such the MMSE are increasingly used as demands on clinicians’ time increase, and measures even shorter than the MMSE are studied and used in practice. Clock drawing is one example of a brief test that has been proposed as a screening measure for conditions such as Alzheimer's disease (Kirk and Kertesz, Reference Kirk and Kertesz1991; O'Rourke et al., Reference O'Rourke, Tuokko, Hayden and Beattie1997) and mild cognitive impairment (MCI; Matteau et al., Reference Matteau, Jean, Nicolas and Simard2009), and for evaluation of focal injuries (Tranel et al., Reference Tranel, Rudrauf, Vianna and Damasio2008), although its utility for screening has been disputed (Powlishta et al., Reference Powlishta, Von Dras, Stanford, Carr, Tsering and Miller2002). Such uses of screening tests for the diagnosis of dementia and rating of the severity of dementia raise the expectations of a screening measure significantly. There appears to be general support for the conclusion that drawing skills remain intact in the early stages of dementia or of MCI (Albert, Reference Albert, Craik and Salthouse2008), although there is some decline in visuospatial functions with increasing age among normal older adults as well (Robins-Whalin et al., Reference Robins-Whalin, Backman, Wahlin and Winblad1993). However, those with moderate dementia generally show signs of impaired drawing (Kurylo et al., Reference Kurylo, Corkin and Growdon1994; Rouleau et al., Reference Rouleau, Salmon and Butters1996).
Pentagon drawing has been used as a screening measure independently of its use in the MMSE and 3MS (Cormack et al., Reference Cormack, Aarsland, Ballard and Tovée2004). Cormack et al. found the pentagon task to illustrate differences in cognitive processes in different forms of dementia. Freeman et al. (Reference Freeman2000) also note the presence of visuoconstructive problems in dementia, supporting the use of drawing measures for screening purposes.
Performance on screening measures, as for all psychological tests, requires psychometric evaluation to determine the extent to which a test performs as advertised. From a psychometric perspective, it stands to reason that an expanded scoring of the pentagon-copying item by the 3MS would be superior to the traditional binary scoring used by the MMSE, but there is no evidence to date as to the magnitude of such an improvement. The binary scoring is retained on the revised MMSE-2 (Folstein et al., Reference Folstein, Folstein, White and Messer2010), so presumably there was no urgency from the perspective of the team responsible for the revisions.
One of the objectives of the current study then is to compare the binary and ten-point scoring systems for the pentagon-copying item in terms of discriminative power in the first stage of data collection of a large community sample of older adults. A second goal is to report on scorer reliability of the binary and ten-point scoring systems. The primary hypothesis was that the ten-point scoring system would be more sensitive to group differences than the traditional two-point system. The second hypothesis was that the ten-point system would be superior to the two-point system in differentiating cases with cognitive impairment from those with no cognitive impairment.
Methods
Participants
The Canadian Study of Health and Aging (CSHA-1; Eastwood et al., Reference Eastwood, Nobbs, Lindsay and McDowell1992; McDowell et al., Reference McDowell, Xi, Lindsay and Tuokko2004) was a national epidemiological study of the health of those Canadians 65 years of age and over. The study involved 18 study centers and over 30 areas of all Canadian provinces. The second follow-up study was about 5 years later and involved an assessment of health status and psychosocial functioning. The methodology was similar in contacting those who still consented to participate. The third wave was five years after the second and focused on those with cognitive impairment in contrast to the focus on those with dementia in the first two waves (Lindsay et al., Reference Lindsay, Sykes, McDowell, Verreault and Laurin2004). Screening assessments were conducted at all three waves in addition to the more detailed clinical assessments. Details of the assessment process of the first wave were provided by the Canadian Study of Health and Aging Working Group (1994). The first wave of the CSHA comprised 10,263 participants. Of these, the current sample of 8,702 completed the Modified Mini-Mental State Examination (Teng and Chui, Reference Teng and Chui1987). Those who underwent the CSHA screening assessment only completed it once, whereas those who also completed the clinical assessment were administered it again. Both administrations were performed by the clinical nurse members of the assessment teams, who were specifically trained in the administration of the 3MS. All participants who scored under 78 on the screening assessment plus a random sample of those scoring above 77 proceeded to the clinical assessment at which the screening assessment was repeated. Because of the different goals and need for a broad range of 3MS scores, the sample size here differs from that used by Bravo and Hébert (Reference Bravo and Hébert1997) in being less selective. There were more women (59.3%) than men in this subsample from CSHA-1, with a mean age of 75.5 years (SD = 6.99, range 64–99) and a mean of 10.1 years of education (SD = 3.89).
Procedure
The basic scoring system for the figure of interlocking pentagons is binary (0 or 1) on the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975), with a positive score only for two, five-sided figures, with a four-sided figure formed by the intersection of the two pentagons. The same system is continued in the MMSE-2 (Folstein et al., Reference Folstein, Folstein, White and Messer2010). The 3MS uses a ten-point system, with up to four points for each pentagon and up to two points for the intersecting quadrilateral (see Tombaugh et al. (Reference Tombaugh, McDowell, Kristjansson and Hubley1996) for the scoring system for the 3MS and reference to a detailed scoring guide). The pentagon item in the 3MS thus accounts for 10% of the final score, as opposed to 3% in the MMSE. The 3MS scoring system was converted to the equivalent all-or-none MMSE scoring by taking only a 3MS score of ten to equal a score of 1 for the binary scoring, with values of 0 through 9 being given a score of 0. Both scores were analyzed during the course of the CSHA (Tombaugh et al., Reference Tombaugh, McDowell, Kristjansson and Hubley1996).
Coefficient α was calculated for the 3MS scoring system, along with means and standard deviations. Means were compared for the CSHA-1 diagnostic groups of dementia (DEM), no cognitive impairment (NCI), and cognitive impairment not dementia (CIND; Canadian Study of Health and Aging Working Group, 1994) that were based upon the revised third edition of the Diagnostic and Statistical Manual (DSM-III-R; American Psychiatric Association, 1987). Standard analysis of variance was used to determine the extent to which pentagon drawing from the 3MS administered at the CSHA-1 clinical assessment could differentiate the three broad diagnostic groups. A related analysis of variance examined the ability of the binary and ten-point pentagon scores to differentiate degrees of severity of dementia. DSM-III-R criteria were used by the CSHA assessment teams to rate the severity of the disorder among those 1,132 people diagnosed with dementia as mild, moderate, or severe. Bonferroni corrections were used to compensate for post hoc comparisons. Bravo and Hébert (Reference Bravo and Hébert1997) provide reference values for the MMSE and 3MS, taking age and education into account. Because these two factors are well known to be associated with MMSE scores, adjusting for their influence was felt to be essential in these comparisons.
Scorer reliability was determined by a correlation of overall and component scores for the 3MS pentagons from the CSHA-1 screening assessment and the nurse-administered 3MS during the clinical assessment. Similar calculations were done for the screening 3MS and nurse assessment from CSHA-2 five years later, but the single administration used at CSHA-3, which was five years later again (Lindsay et al., Reference Lindsay, Sykes, McDowell, Verreault and Laurin2004), did not facilitate a calculation for scorer reliability. Note that all surviving participants who completed the clinical assessment at CSHA-1 were also assessed at the later two waves. In this sense, the waves were not independent, but the five-year gap between assessments would minimize any recall of content. More might be remembered between screening and clinical assessments, but as this was a matter of weeks in many cases, memory of content again was likely modest, as would be the likelihood of sizeable practice effects.
Standard analysis of variance was used to determine the extent to which pentagon drawing from the 3MS could differentiate the three broad diagnostic groups. A related analysis of variance examined the ability of the binary and ten-point pentagon scores to differentiate degrees of severity of dementia. DSM-III-R criteria were used by the CSHA assessment teams to rate the severity of the disorder among those 1,132 people diagnosed with dementia as mild, moderate, or severe.
Logistic regression was used to contrast the 921 individuals determined not to be cognitively impaired at the clinical assessment with the 861 found to have some cognitive impairment (CIND), and with the 1,132 determined to have some form of dementia in separate analyses, with gender, age at study onset, years of education, number of drugs prescribed, and number of depression symptoms as additional independent variables. Many forms of medication are known to influence cognitive processes, so the number of prescription medications up to 12 was used as a covariate. Depression may also influence cognition, and so the number of DSM-III-R symptoms of depression up to 13 was also used as a covariate. Analyses were carried out both for the binary pentagon score and the 3MS version, leading to four separate analyses.
All analyses were carried out using SYSTAT 12 (2007).
Results
Table 1 reports means, standard deviations, and coefficients α for the five administrations of the 3MS during the CSHA, with the three diagnostic groups formed by the diagnostic process of the clinical assessment separated out. These figures are also reported for the three screening assessments and for the CSHA-2 clinical assessment. Table 2 reports the scorer reliability for traditional binary pentagon scoring and the 3MS ten-point scoring across the various waves using Pearson correlations, Cohen's κ, and the related 95% confidence interval. Note that the value of κ for the ten-item overall score is lower than that of its components because of the increase in the proportion of off-diagonal scores in comparison to the diagonal agreements. The equivalent correlation is higher for the sum than for the sub-elements.
Means, standard deviations, and coefficients α for CSHA administrations of the3MS pentagon copying task
CIND = cognitive impairment not dementia; CSHA = Canadian Study of Health and Aging.
Scorer agreement between CSHA-1 screening and clinical assessments (n = 1461)
CSHA = Canadian Study of Health and Aging.
The analysis of variance for diagnostic groups for the binary scoring system was significant (F = 112.94, 2, 2358 df, p < 0.001, R 2 = 0.146), and also for the ten-point scoring system (F = 379.43, 2, 2358 df, p < 0.001, R 2 = 0.285), with both showing the expected significant change in scores from NCI to CIND to the DEM group, but more substantial differences for the ten-point system than for the binary system. In both cases, the three groups differed significantly using the Bonferroni correction with p < 0.001 in all cases. Age and education were significant covariates for the binary scoring system (age: F = 5.04, 1, 2358 df, p = 0.025; education: F = 118.08, 1,2358 df, p < 0.001). For the ten-point system, only education as a covariate was significant (F = 84.64, 1,2358 df, p < 0.001).
In the comparison of degrees of severity of dementia, the effect for group differences for the binary scoring system was significant (F = 19.59, 2.748 df, p < 0.001, R 2 = 0.072), as was the effect for the ten-point system (F = 132.49, 2, 748 df, p < 0.001, R 2 = 0.267), with all the three groups differing from one another by Bonferroni corrected post hoc comparisons for both scoring systems. For the analysis of both the binary and the ten-point pentagon scores, age was not statistically significant. Education was significant in both cases: binary scoring (F = 2.44, 1, 748 df, p < 0.001) and ten-point scoring (F = 11.41, 1, 748 df, p < 0.001). As expected, the ten-point system accounted for more variance (R2 = 0.267 vs. 0.072 for the binary scoring system).
The logistic regressions reported in Table 3 were all significant for both pentagon scoring systems, age, and the number of depression symptoms. Gender was significant only for the NCI versus dementia comparison for the ten-point scoring system. Education was significant for all but the NCI versus CIND comparison for the binary scoring system. The number of drugs was significant for the NCI versus dementia comparisons, but not for the two NCI versus CIND comparisons. The area under the receiver operating characteristic curve (AUC) was 0.738 for the binary system and the NCI versus dementia comparisons, with the corresponding figure for the ten-point system being higher at 0.789. The AUC in the case of the binary scoring system for the NCI versus CIND comparison was 0.668. Using the ten-point pentagon scoring from the 3MS, the analysis showed somewhat better results than with the binary scoring (AUC = 0.682).
Logistic regression results for prediction of pentagon copying scores
CIND = cognitive impairment not dementia; CSHA = Canadian Study of Health and Aging; NCI = no cognitive impairment,
Discussion
The ten-point scoring system for the pentagons from the 3MS showed several advantages over the traditional scoring system used on the MMSE. While both scoring systems differentiated the three broad diagnostic groups of the CSHA, the ten-point system did so more effectively, as hypothesized. The same was true for the discrimination of the degree of severity of dementia, with the ten-point scoring system accounting for more variance in scores than the original binary scoring. The ten-point scoring system also had larger AUCs in the discrimination of the NCI group from both the CIND and dementia groups than the binary scoring system.
The scale analysis showed questionable to acceptable levels of internal consistency reliability for the 3MS pentagon-copying item. It was of course not possible to calculate an equivalent figure for the corresponding binary MMSE item scoring system. It is well known that more items lead to higher values of coefficient α (Allen and Yen, Reference Allen and Yen1979), and such short scales rarely have very high figures. Higher reliability was observed in those cases where more variation exists, as reflected in the size of the variance for the cognitively impaired groups in Table 1 with values of 0.74 and higher. Values of κ and of the correlation coefficients for the agreement between the scorers of the screening assessment and the clinical assessment are statistically greater than zero, but are not within the ranges one would generally regard as acceptable. At the same time, it must be remembered that these are values for a single item, for which any reliability figure is apt to be lower than for a longer composite. This is one of the costs of short, single-item screening measures. The 3MS scoring system for this item reflects its nature as a very short, three-item measure.
In support of its use for screening purposes, pentagon copying scored as in the 3MS shows good differentiation of the three broad diagnostic groups and of the degrees of severity among older adults with dementia. In the associated analyses, the 3MS ten-point scoring for the pentagon-copying item also showed better power of discrimination in logistic regressions over the same analyses conducted with the binary scored pentagon item, supporting the relevant hypothesis. In contrast, Tombaugh et al. (Reference Tombaugh, McDowell, Kristjansson and Hubley1996) report AUC values of 0.90 and higher for both the full MMSE and 3MS, with somewhat higher values for the 3MS, but of course figures for a single item are not comparable to those for the full scale. The ten-point scoring system showed better accuracy for the comparison of the NCI group with the group with dementia than for the NCI versus CIND comparison, but significant differences remained. This finding differs from the case of drawing a clock face in which differentiation of degrees of severity of dementia was not able to be made (Powlishta et al., Reference Powlishta, Von Dras, Stanford, Carr, Tsering and Miller2002).
The results suggest that with the 3MS ten-point scoring system, pentagon copying is viable as a screening test. Bourke et al. (Reference Bourke, Castleden, Stephen and Dennis1995) reported that pentagon drawing was performed equivalently to drawing a clock in a group of people with Alzheimer's disease. Only future empirical research that compares the performance of the pentagon copying with drawing of a clock (or a related task (Bodner et al., Reference Bodner, Delazer, Kemmler, Gurka, Marksteiner and Fleischhacker2004), a box or cube (Gaestel et al., Reference Gaestel, Amieva, Letenneur, Dartigues and Fabrigoule2005; Paganini-Hill and Clark, Reference Paganini-Hill and Clark2007), or a bicycle (Schmitt et al., Reference Schmitt, Livingston, Galusha and Davis2009) can provide the evidence as to its relative utility. With Alzheimer's disease, the most common form of dementia, memory tests will remain among the most effective screening measures for dementia. Drawing tests, with their complex demands for the involvement of multiple brain systems, will also have value in this context, and only more data will tell us whether drawing one object is superior to drawing another.
Conflict of interest
None.
Acknowledgments
Core funding for Phases 1 and 2 of the CSHA was provided by the Seniors’ Independence Research Program, through Health Canada's National Health Research & Development Program (NHRDP). Funding for analysis of the caregiver component was provided by the Medical Research Council. Additional funding was provided by Pfizer Canada Inc. through the Medical Research Council of Canada/Pharmaceutical Manufacturers Association of Canada (MRC/PMAC), NHRDP, by Bayer Inc., and by the B.C. Health Research Foundation. Core funding for Phase 3 was provided by the Canadian Institutes of Health Research (CIHR). Supplementary funding for the caregiver component was obtained from CIHR. Additional funding was provided by Merck-Frosst and by Jansen-Ortho. CSHA was coordinated through the University of Ottawa and Health Canada.
An earlier version of this report was presented at the 4th Pacific Rim Conference of the International Neuropsychological Society and the Australian Society for the Study of Brain Impairment, Auckland, July, 2011. I thank Dr. Truls Østbye for his provision of the CSHA data and his continued support and the anonymous reviewers for their comments.
