Introduction
Given the high prevalence of dementia worldwide and that there is no therapeutic intervention to cure (e.g. Jalbert et al., Reference Jalbert, Daeillo and Lapane2008), an early detection of Alzheimer disease (AD) for older adults is important for them to take proper treatments to slow down the progress of the disease (e.g. Maurer et al., Reference Maurer, Ihl, Dierks and Frölich1997). It has been documented that older adults with lower education tend to get lower scores in some dementia screening tasks (e.g. Jones and Gallo, Reference Jones and Gallo2001; Anderson et al., Reference Anderson, Sachdev, Brodaty, Trollor and Andrews2007). Even in popular tests like Mini-Mental State Examination (MMSE, Folstein et al., Reference Folstein, Folstein and McHugh1975), higher scores could be attributed to better test-taking skills or cognitive reserve (e.g. Jones and Gallo, Reference Jones and Gallo2002). In fact, various cut-off scores for the MMSE have been proposed according to the education level of specific populations (e.g. Uhlmann and Larson, Reference Uhlmann and Larson1991; Chiu et al., Reference Chiu, Lee, Chung and Kwong1994). For some screening items that involve numeracy or literacy, older adults with lower education could be misclassified as very mildly demented (as defined by 0.5 in Clinical Dementia Rating, CDR) even though they are cognitively unimpaired (as defined by 0 in CDR) (e.g. Anderson et al., Reference Anderson, Sachdev, Brodaty, Trollor and Andrews2007). In this study, we examined whether the tasks and even items within each task were equally efficient in identifying very mild AD for Chinese older adults with low versus high education level in Hong Kong.
Most screening tasks were designed in the developed countries where fewer older adults received low education, relative to those in the developing countries. For example, about 80% of older adults finished their high-school education in the United States (U.S. Department of Health and Human Services, 2011), but only about 31% did so in Hong Kong (Hong Kong Census and Statistics Department, 2011). Tasks may not be suitable for older adults with limited education when they rely on reading and writing skills, e.g. writing-a-sentence in MMSE, and word-recognition and word-recall in Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog). Thus, while these tasks can be used to predict CDR (e.g. 0 vs. 0.5) among educated older adults, they might not be as useful for those with lower education. It may also be true that some tasks may be differentiating for the CDR of older adults with low education, but not for that of educated older adults. In other words, these tasks have different discrimination power for high versus low education groups.
Validated with satisfactory psychometric properties in community-dwelling older adults, the screening tasks in the current study are commonly used in Hong Kong and tap various cognitive abilities, including attention, memory, and language. They included the Cantonese version of MMSE (CMMSE, Chiu et al., Reference Chiu, Lee, Chung and Kwong1994), Chinese version of ADAS-Cog (Chu et al., Reference Chu2000), Verbal Fluency (Chiu et al., Reference Chiu1997), Abstract Thinking, and forward and backward digit/visual span (Wechsler, Reference Wechsler1999) (see Table 1). We examined both total and individual item scores in these screening tasks in order to reveal their potential bias in very mild AD discrimination due to older adults’ education level. Even though the total scores of a task might not be biased in discriminating very mild AD among older adults with different education levels, it is not clear whether the individual item scores within the task would also be unbiased. These item-level analyses have only been done in very few studies (e.g. on performance difference due to education bias in Yassuda et al. (Reference Yassuda2009)), and to our knowledge, has never been reported for Hong Kong Chinese population. As summarized by Walker (2011, pp.364–365), “if the content of some test items were unfamiliar to minority examinees, then these examinees would be more likely to answer incorrectly despite there being no true difference in their overall cognitive ability compared to majority examinees.” Despite often conducted in large-scale educational assessments, but rarely reported in the dementia literature, the item-level analysis is an important part of instrument development and validation (e.g. for Mini-Cog and MMSE; Borson et al., Reference Borson, Scanlan, Watanabe, Tu and Lessig2005).
Description of the screening tasks and their sample sizes
CDR = Clinical Dementia Rating.
Note: N = total sample size. The sample size varied across tasks (and subtests in the ADAS-Cog task) because not all participants finished every single task.
aThe factor structure of CMMSE follows that of the original instrument (Folstein et al., Reference Folstein, Folstein and McHugh1975) and has also been commonly adopted by other studies (e.g. Chiu et al., Reference Chiu, Lee, Chung and Kwong1994; Reference Chiu1998; Jones and Gallo, Reference Jones and Gallo2002; Banos and Franklin, Reference Banos and Franklin2002; Castro-Costa et al., Reference Castro-Costa2009).
Methods
Participants
The data were collected in an epidemiological survey of the prevalence of very mild dementia in Hong Kong (see Chan et al., Reference Chan2008; Lam et al., Reference Lam, Tam, Lui and Chan2008a; Reference Lam2008b, for more details). Phase 1 screening was conducted via the Thematic Household Survey commissioned by the Census and Statistics Department of the Hong Kong SAR Government. A total of 6,100 community-dwelling older adults were screened by trained lay interviewers using various tests (e.g. CMMSE). Participants who were screen-positive and those randomly selected from the screen-negative group were invited to participate in Phase 2 in which diagnosis of dementia was determined using CDR (i.e. 0 – free from dementia or 0.5 – very mild AD). As participants did not necessarily complete all tasks in the research, the sample sizes varied across studies (Lam et al., Reference Lam2008c). Based on the CDR scales, among 788 participants (364 males and 424 females), 405 (51.4%) had very mild dementia (CDR 0.5) and 383 (48.6%) were healthy controls (CDR 0). The mean age was 72.08 years (SD = 7.27, range = 57–96) and the mean years of education was 4.7 (SD = 4.6, range = 0–20, other than 0 years, mode = 6 (14% of participants)).
This study was approved by the research ethics committees of the Chinese University of Hong Kong and the Department of Health of the Hong Kong SAR Government. Informed consents of all participants were obtained before the assessment.
Material and design
Participants’ cognitive functions were assessed via CMMSE, Chinese version of ADAS-Cog, Abstract Thinking, Verbal Fluency, and forward and backward digit/visual span (see Table 1). The severity of dementia was defined by CDR based on a semi-structured clinical interview with the participant and a collateral informant (e.g. spouse) on the current status of their functional abilities on memory, orientation, personal care, judgment/problem solving, home/hobbies, and community affairs. Based on a set of algorithms, the overall CDR is derived by synthesizing ratings in each of the six domains (Morris, Reference Morris1993). The CDR scoring is mainly reliant on the informants’ provided information, rather than the participants who might lack insight to their real impairments in various aspects. The psychiatrists who determined the CDR status were independent and blind to participants’ psychometric performance in the screening tests (e.g. in CMMSE). The reliability and validity of CDR have been established in Hong Kong populations (Chiu et al., Reference Chiu1998).
Statistical analyses
Different methods have been used to identify potential test and item bias in the screening tasks. For example, at the test level, Jones and Gallo (Reference Jones and Gallo2001) concluded that MMSE total score was not educationally biased by showing similar correlations of the MMSE total scores with age in high and low education groups (r = −0.29 vs. −0.27, respectively). At the item level, Jones and Gallo (Reference Jones and Gallo2002) used the differential item function method and showed that some MMSE items, such as naming-season, repeating-object-names, writing-a-sentence, and spelling-“WORLD”-backward, had different discrimination power for high versus low education groups. In the present study, we improved on these previous methods by controlling for other relevant variables with regression analyses (cf. Jones and Gallo, Reference Jones and Gallo2001) and by treating education as a continuous variable and analyzing the interactions involving education (cf. Jones and Gallo, Reference Jones and Gallo2002). The latter approach is preferred because it does not artificially divide education level into high and low groups and thus would not reduce the power of the interaction analyses (Marsh et al., Reference Marsh, Hau, Wen, Nagengast, Morin and Little2013). Also, contrary to the use of the total score of the screening task in Jones and Gallo's differential item function method, CDR was used as an external validation criterion. Specifically, in the logistic regression analyses, the absence of the interaction between older adults’ performance in the screening task and education level in predicting their CDR would suggest that the task could predict the CDR similarly irrespective of their education level. Other methods have been used in previous studies (e.g. multiple indicators multiple causes – MIMIC model in Proitsi et al. (Reference Proitsi2011)) to identify the effect of education using a latent variable approach after controlling for other relevant variables (e.g. depression). However, these models have not been used to assess the differential discrimination power in various tasks. In our logistic regression analyses, differential discrimination power of tasks/items for high/low education older adults were assessed, with all relevant variables (e.g. education and age) being controlled.
Education was measured by the years of education that each participant completed. The severity of dementia was defined by CDR. Logistic regression analyses were performed with participants’ overall CDR and CDR subscale scores (0.5 vs. 0) as dependent variables. The scores in one CMMSE component and one CDR subscale were non-discriminating and discarded due to ceiling effects (99.7% of participants scored 0 in CDR-care; 94.3% scored 3 in CMMSE-registration). We predicted CDR with age, score in screening item, years of education, and their interaction (screening item × education) in the regression models. To avoid the problem of multicollinearity, we mean-centered all predictor variables (Jaccard, Reference Jaccard2001) and computed the product term by multiplying the values in mean-centered education and screening item variables. For a significant interaction, the main effect is the odds ratio of the respective variable (e.g. effect of screening item) at the mean values of the other variables (e.g. education and age). To control for Type I errors in the large number of statistical significance tests, we adopted significant level at p < 0.001. To follow up significant interactions, we divided participants by median split based on their years of education (0–3 (M = 0.96, SD = 1.21) vs. 4–20 (M = 8.37, SD = 3.63)) and conducted similar logistic regression analyses to test the simple main effect of screening items separately for low and high education groups. This division of education into two groups was to test the simple main effect of screening items in participants with higher versus lower level of education (rather than the interaction per se) only when the statistical significance of the interaction was obtained in the logistic regression. The odds ratio of all main effects and interaction are shown in Table 2. For example, the value 0.412 (first row, Screening Item column) is the multiplying factor of CMMSE-time orientation score on CDR. For each unit increase in item score, the odds ratio of 0.5 versus 0 in CDR decreased by 58.8% (i.e. (1 − 0.412) × 100). Significant odds ratios in the Screening Item column suggest the sensitivity of screening items on discriminating very mild dementia from healthy aging. While some screening items were equally sensitive for older adults with high or low education level (i.e. their relations with CDR were similar), a significant interaction suggests that the sensitivity of screening items on CDR changes with years of education (see Interaction (1) × (2) column in Table 2). The results of follow-up analyses on these items (see Screening Item (low-ed or high-ed group)) revealed their differential discriminating powers for the low and high education groups, respectively.
Predicting overall CDR with screening items – differential effects in high and low education groups
Note: *p < .05, **p < .01, ***p < .001.
Results
Consistent with previous studies (e.g. Doraiswamy et al., Reference Doraiswamy1997; Jones and Gallo, Reference Jones and Gallo2002; Hawkins et al., Reference Hawkins, Cromer, Piotrowski and Pearlson2011), older adults with more education performed better in various tasks (r = 0.453 for CMMSE, 0.521 for ADAS-Cog, 0.375 for Abstract Thinking-similarity, 0.212 for Abstract Thinking-difference, 0.409 for Digit Span, 0.373 for Visual Span, 0.325 for Verbal Fluency-30s, and 0.307 for Verbal Fluency-60s; all ps < .001).
Effect of age
As previously reported (e.g. Jones and Gallo, Reference Jones and Gallo2001), the chance of developing very mild dementia increased with age as shown in the simple correlation (r = 0.318, p < 0.001) and the logistic regressions when relevant variables were statistically controlled. To interpret the regression findings (see Table 2), we take the scores in CMMSE-location-orientation (on overall CDR) as an example. One year increase in age resulted in 8.6% (i.e. (1.086 - 1) × 100) increase in the chance of being diagnosed as CDR 0.5 versus 0. As these effects might not be linear, we computed the percentage increase in the odds ratio of 0.5 versus 0 for each 10 years’ increase in age. For example, the odds ratio of CDR (0.5 to 0) increased by 127% (i.e. (2.27 – 1) × 100, see Table 2) for every 10 years’ increase in age. The multiplying factor for each 10-year increase of age, varied across screening items, on CDR ranged from 1.58 to 2.41 (M = 2.09, SD = 0.20). Given the large effect of age on predicting the CDR, it was controlled in all relevant analyses.
Effect of education
The results regarding the effect of education could be interpreted similarly. The multiplying factor for each 10-year increase of education, varied across screening items, on CDR ranged from 0.18 to 1.06 (M = 0.31, SD = 0.15). In line with earlier research (e.g. Hawkins et al., Reference Hawkins, Cromer, Piotrowski and Pearlson2011), these figures showed that the chance being rated as very mild dementia (CDR 0.5 vs. 0) would decrease by 69% (i.e. (1 – 0.31) × 100) for every 10 additional years of education.
Differential power of tasks/items on discriminating individuals with CDR 0 versus 0.5 in high and low education groups
As indicated in Table 2, Interaction (1) × (2) column, none of the education × screening item interaction was significant for the total scores in our screening tasks, suggesting that there was no education bias in these measures when discriminating the older adults with CDR 0.5 versus those with CDR 0 in their overall CDR. However, at the item level, we did obtain significant education × screening item interactions in three cases:
-
(a) CMMSE-language-and-motor was more useful for discriminating older adults’ CDR in the low education group. One unit increase in this item led to a larger decrease in the odds ratio of CDR (0.5 to 0) for low (58.2% (i.e. (1–0.418) × 100)) than high education group (20.8% (i.e. (1–0.792) × 100)).
-
(b) ADAS-Cog-word-recognition was more useful for discriminating older adults’ CDR in the high education group. One unit increase in this item led to a larger decrease in the odds ratio of CDR for high (41.9%) than low education group (22.7%).
-
(c) ADAS-Cog-word-recall was more useful for discriminating older adults’ CDR in the high education group. One unit increase in this item led to a larger decrease in the odds ratio of CDR for high (58.9%) than low education group (50.6%).
We also noted that Sugar-Vinegar in the Abstract Thinking was not useful for screening as it did not yield a main effect of screening item.
Additional analyses on CDR subscales
Despite rarely reported in such details in the literature, we also performed analyses to examine the characteristics of our tasks/items in discriminating older adults with 0.5 versus 0 in each of the CDR subscale scores. We obtained significant education × screening task interactions in some tasks/items (see Appendices A and B). According to the relative sizes of the simple main effects of these screening items for high and low education groups, they could be classified as:
-
(a) Useful for screening high education group only – with a significant education × screening task interaction and a much larger effect of the screening items for the high (vs. low) education group; e.g. CMMSE-location-orientation for CDR-community.
-
(b) Useful for screening low education group only – with a significant education × screening task interaction and a much larger effect of the screening items for the low (vs. high) education group; e.g. CMMSE-language-and-motor for CDR-memory.
-
(c) Not useful for screening high or low education group – with non-significant effect of the screening task and education × screening task interaction, e.g. ADAS-Cog-commands for CDR-community.
-
(d) Useful for screening both high and low education groups—with a significant effect of the screening task and a non-significant education × screening task interaction; e.g. Sugar-Vinegar in Abstract Thinking for CDR-orientation.
As indicated in Appendices A and B, the power of some items/tasks in discriminating older adults with 0 versus 0.5 in various CDR subscales was different for those with high versus low education. While these interaction effects and differential discrimination power of individual tasks/items were illuminating, particularly for the future reconstruction of the tasks for dementia assessment, our following discussion focuses on the results involving the more commonly used overall CDR scores.
Discussion
A number of dementia-screening tasks have been developed in countries where most of older adults attained certain level of schooling. These tasks are susceptible to a potential bias against individuals with low education (e.g. Koepsell et al., Reference Koepsell2008). The current study investigated the differential discrimination power of screening tasks (at the levels of individual item scores and total scores) for very mild AD individuals (CDR 0.5) versus healthy older adults (CDR 0) with low versus high education levels in Hong Kong. The results showed that the total scores of all tasks were equally effective in discriminating very mild AD individuals from healthy older adults, irrespective of their education level. Whereas this was consistent with some studies that also showed no bias for the total scores in some screening tests (e.g. Schmand et al., Reference Schmand1995; Jones and Gallo, Reference Jones and Gallo2001), other studies (e.g. Liu et al., Reference Liu1994; Stern et al., Reference Stern1994) reported that the total scores in some screening tests might be biased. However, it is important to note that some items within certain tasks might discriminate the older adults’ CDR better for high or low education group (see Table 2). For these items/tasks, it is inappropriate to use identical cut-off scores in differentiating the very mild AD and cognitively normal individuals for high and low education groups, especially when the cut-off scores were originally determined by biased samples (e.g. more highly educated older adults in developed countries).
Consistent with previous research (e.g. Jones and Gallo, Reference Jones and Gallo2002), older adults’ CMMSE performance was correlated with education level in the current study. While the discrimination power of CMMSE total scores in CDR 0 versus 0.5 individuals was not different for high versus low education groups, we found that the CMMSE-language-and-motor was only suitable for low education group in discrimination. In this item, participants are tested for their language and motor skills by naming objects, repeating a phrase, following a three-stage command, reading aloud a word, creating and reading aloud a sentence, and copying a figure. These task demands might not be high enough, such that participants with higher education level would be able to achieve that, regardless of their severity of dementia. In contrast, for older adults with, on average, 3 or fewer than 3 years of education (i.e. low education group), very mild AD individuals performed more poorly in this item than healthy older adults. Hence, although previous studies often showed that compared with memory and attention, language and motor skills are relatively preserved during early-stage of AD (e.g. Pettersson et al., Reference Pettersson, Olsson and Wahlund2005), some of these results might be due to their using participants with relatively high education level (e.g. Kosmidis et al., Reference Kosmidis, Zafiri and Polytimou2011).
Regarding the education bias in the ADAS-Cog, some showed that the performance in this task was minimally related to education (e.g. Hannesdóttir and Snaedal, Reference Hannesdóttir and Snaedal2002), others reported the otherwise (e.g. Doraiswamy et al., Reference Doraiswamy1997). We did show a positive correlation between older adults’ education level and their ADAS-Cog total scores. While ADAS-Cog total scores did not show any bias in discrimination for older adults with high versus low education, two items in this battery, word-recognition and word-recall, were more sensitive for older adults with high than low education. There are different potential explanations for these results, including the possibility that these two items may be loaded with skills that are heavily drilled or practiced in schools. This could be addressed by replacing words by pictures in ADAS-Cog (e.g. Lin et al., Reference Lin2002).
There are limitations in the current study. First, the range of participants’ number of years of education may not be wide enough to capture the non-linear effect of education. In the current sample, other than the 28% of the participants with no education, the modal year of education was 6 years (14%). Previous studies showed that while education has a positive impact on cognitive performance at the low end (e.g. from 0 to 6 years of education, Lin et al., Reference Lin2002), the impact becomes smaller when the number of years of education is beyond 12, thus reflecting a ceiling effect of education (e.g. Brucki and Nitrini, Reference Brucki and Nitrini2008). Future studies should test whether the current findings (i.e. differential discrimination power of CDR (0 vs. 0.5) for older adults with high vs. low education in some screening test items) could be generalized for participants with a wider range of education level.
Second, although the CDR has been validated and commonly used in Hong Kong (e.g. Lam et al., Reference Lam, Tam, Lui and Chan2008a), the current study only used this single criterion to define the severity of dementia (CDR 0.5). Future research should incorporate the performance from screening measures that have been used to identify individuals with mild cognitive impairment in Hong Kong (e.g. Category Verbal Fluency Test, see Lam et al., Reference Lam2009) in the analyses.
Third, we did not examine the influence of participants’ socioeconomic status (SES) in the current study. SES could refer to their education level and income at three levels, namely, SES of the participants themselves, their parents, and their children, if any. Higher SES at the three levels may imply a more supportive learning environment and higher education attainment, and probably better nutrition and attention to physical health. Other than participants’ education attainment, SES at all levels would also influence older adults’ performance in various screening tasks. This variable should be investigated in details in the future studies.
While the current study focused on CMMSE, ADAS-Cog, Abstract Thinking, visual and digit span, and Verbal Fluency, there are some other screening tasks that have been developed and validated in Hong Kong. Some have been shown to be equally discriminating for low and high education groups. For example, by evaluating the functions of visuospatial organization, orientation and conceptualization of time, the Clock-face task is a useful screening task for dementia in Chinese elderly (Lam et al., Reference Lam1998). In the Chinese version of Fuld Object Memory Evaluation (Chung and Ho, Reference Chung and Ho2009), encoding and retrieval functions are tested by identifying the objects by touch, vision, and hearing. Note that these tasks involve non-verbal materials and are less susceptible to education bias. This is consistent with the current results that bias in item discrimination due to education more likely occurred in verbal items, e.g. ADAS-Cog-word-recognition.
In conclusion, the present study aimed to test whether the dementia screening tasks were equally effective for Chinese older adults with high versus low education levels in discriminating the very mild AD individuals from healthy older adults in Hong Kong. Unlike most of the previous studies, we tested the discrimination power of the total scores and individual item scores of the tasks. All of the screening tasks’ total scores could discriminate the very mild AD (CDR 0.5) individuals from the healthy older adults (CDR 0) equally well in both high and low education groups. However, some items within the tasks (CMMSE–language-and-motor and ADAS-Cog word-recognition-and-word-recall) showed a clear trend of bias. The bias in discrimination due to education demonstrated in the screening items that involved verbal materials (e.g. ADAS-Cog-word-recognition) suggests that this kind of materials should be avoided in tasks that are used to screen dementia among older adults with different education levels. Overall, more caution should be paid in the interpretation of findings yielded in these tasks. Item-level analyses should be regularly performed to verify whether the findings are contaminated by some items’ bias in very mild AD discrimination due to older adults’ education level. If this is the case, these items should be modified or even eliminated in order to minimize their potential bias.
Conflict of interest
None.
Description of authors’ roles
J. Chang designed the study, conducted the statistical analysis and wrote the paper. C. S. Tse provided guidance on the research idea and helped to write and revise the paper. G. T. Y. Leung and A. W. T. Fung undertook the interviews. K. T. Hau contributed to the design of study and statistical analyses. H. F. K. Chiu and L. C. W. Lam helped the formulation of the study, designed and supervised the collection of data.
Acknowledgments
The project is supported in part by the Mr. Lai Seung Hung and Mrs. Lai Chan Pui Ngong Dementia in Hong Kong Research Fund. We would also like to thank the elderly people for their participation, and the staffs of participating centers for their help.
Predicting CDR subscale scores with screening items - differential effects in high and low education groups
Categories of screening tasks for different education groups
