Introduction
Dementia screening is a public health priority in Taiwan, where the prevalence of dementia is increasing because of an aging population (Wu et al., Reference Wu2013). The Ascertainment of Dementia-8 (AD8) test, which was developed by Washington University in St. Louis, United States of America, with a sensitivity of 74% and a specificity of 0.834, is a widely used screening tool that is quick and easy to administer (Galvin et al., Reference Galvin2005). AD8 was also translated into different languages and used cross-different cultures, such as in Brazil (Correia et al., Reference Correia2011), Japan (Meguro et al., Reference Meguro, Kasai, Nakamura and Kurihara2015), China (Li et al., Reference Li, Wang, Yang, Galvin, Morris and Yu2012), Korea (Ryu et al., Reference Ryu, Kim and Han2009), and Spanish (Carnero Pardo et al., Reference Carnero Pardo2013). In Taiwan, Chinese AD8 is also an effective dementia screening tool. The cut-off value was also 2 in discriminating non-dementia from very mild dementia with an AUC of 0.948, a sensitivity of 95.9%, and a specificity of 78.1% (Yang et al., Reference Yang, Galvin, Morris, Lai, Chou and Liu2011). An analysis of the individuals participating in dementia screening using the AD8 in primary care facilities found that only 1.4% of individuals accepted a follow-up referral at a specialist memory clinic (Dong et al., Reference Dong, Cheng, Tsou, Chan and Chen2014). The reasons for this low rate of acceptance of referrals are unclear. Understanding which factors influence the decision to participate in referrals at specialist memory clinics is essential for designing interventions to improve the uptake of screening.
One factor that might influence the individuals’ decision to accept referrals is the interview with a professional, wherein patients are informed about the risks and benefits of accepting an appointment to be screened for dementia (Fowler et al., Reference Fowler2012). Another factor is the individuals’ awareness of the real possibility of developing dementia. As a self- or informant-report tool to detect dementia (Galvin et al., Reference Galvin2005), the AD8 consists of eight questions that track intra-individual functional decline and help the patients to become aware of their own cognitive decline. It contains items such as “repeating the same questions over and over,” “forgetting the correct month or year,” and so on. Thus, an AD8 score ≥2 is expected to aid the patient's decision to accept a referral to a specialist memory clinic. However, most recommendations for referral are declined after the AD8 is administered (Dong et al., Reference Dong, Cheng, Tsou, Chan and Chen2014).
Since the AD8 relies on the subjective recall of functional decline, in the absence of objective cognitive tests, it is liable to subjective biases in reporting and therefore, may carry the risk of under- or over-reporting. Relatively few studies have been conducted to assess the clinical utility of the AD8 compared to those of other performance-based neuropsychological screening instruments, such as the Mini-Mental Screening Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and McHugh1975) and the Cognitive Abilities Screening Instrument (CASI) (Liu et al., Reference Liu1994), in the general Taiwanese population (Chen et al., Reference Chen, Wang, Ma and Yang2014).
Low levels of education have been suggested to influence the sensitivity and specificity of the MMSE, depending on the cut-off values used. When a cut-off value of 24/25 was used, the MMSE shows specificity in 85.2% of individuals who received six or more years of education, but only a specificity of 57.3% in the group who received less than six years of education (Ng et al., Reference Ng, Niti, Chiam and Kua2007). In Taiwan, the wide range of educational levels in the population makes it an ideal sample set for assessing the impact of education on the sensitivity and specificity of the AD8 questionnaire for screening of patients with dementia. In a previous study, the AD8 was used to assess the prevalence of dementia in Taiwan (Liu et al., Reference Liu1995). However, it assessed neither the impact of education on the screening capacity of the AD8 nor other objective cognitive measures, such as MMSE and CASI.
The objectives of the present study were to (1) examine the feasibility and acceptability of using the AD8 for improving screening outcome; (2) perform a direct comparison between the AD8 and objective neuropsychological tests (such as MMSE and CASI) used for the detection of mild dementias; and (3) use a cut-off value of 1/2 to determine if different levels of education significantly affected the sensitivity and specificity of the AD8.
Methods
Participants
Twenty-four sites of Tzu-Chi branches (http://www.tzuchi.org.tw/) participated in the Memory Screening Project and 1,012 community-dwelling volunteers underwent the screening. All sites agreed to provide data to the Memory Screening Project. Flow of information through the different phases of the Memory Screening Project was shown in Figure 1. A mean of 42 participants were screened at each site.
Flow of information through the different phases of the Memory Screening Project. A total of 1,012 participants attended the Memory Screening Project. A total of 102 participants had an Ascertainment of Dementia-8 (AD8) score ≥2. There were 40 participants (which included 26 participants with an AD8 score ≥2 and 14 participants with an AD8 score <2) who attended the second visit and completed cognitive testing. Of those participants with an AD8 ≥2, 13 participants were diagnosed with dementia, while 13 were diagnosed without dementia. CDR = clinical dementia rating.
Materials
On arrival at the screening site, each participant was given a survey to voluntarily complete anonymously. This questionnaire included questions about demographics and medical history. All sites used the well-validated dementia screening test, the AD8. The face-to-face interview questionnaire was used. Experienced clinicians conducted independent semi-structured interviews with the participant or an informant (usually the spouse or close family member) to capture features suggestive of disorder with dementia. After a qualified healthcare professional (e.g. physician, nurse, social worker, neurologist, or physician's assistant) had administered the screening test they tallied the score, which indicated if there was a need for further assessment.
Individual with AD8 scores ≥2 were advised to visit the clinic again for further evaluation of cognitive performance using three validated dementia tests, i.e. the MMSE, CASI, and the Clinical Dementia Rating (CDR). The psychometric and discriminative ability of AD8, MMSE, and CASI was tested, respectively, to measure and compare each effectiveness in classifying dementia and non-dementia (Galvin et al., Reference Galvin2005). Participants were defined as dementia base on the consensus of panels composed of neurologist, neuropsychologist, and neuroradiologists (Chang et al., Reference Chang2016).
Procedure
A private room or area was set up for each rater, and testing adhered to a standardized format. Each site was responsible for the following: organizing qualified healthcare professionals to administer the test, distribution and collection of all of the required forms, and emphasizing to participants that the screening was confidential and was not a medical diagnosis. The healthcare professionals also advised participants who had screening scores above the cut-off value to accept a referral to a specialist memory clinic, and distributed additional educational materials to them. However, the decision to accept or refuse the referral lay with the patient, regardless of whether the AD8 score was above the cut-off value or not.
Study design
We conducted a retrospective study and its protocol was approved by the Human Ethics Committee of Kaohsiung Chang Gung Memorial Hospital.
Statistical analyses
All data were expressed as mean ± standard deviation (SD). Categorical variables were compared using the χ 2 test or Fisher's exact test. Continuous variables were compared using the Student's t-test. Data that were not normally distributed were logarithmically transformed to achieve normality and then analyzed.
We calculated the area under the curve (AUC) of the receiver-operator characteristics (ROC) curves as a measure of the predictive value of the test. Diagnostic accuracy was assessed by calculating the sensitivity and specificity for the threshold that yielded the highest Youden index [Youden index = sensitivity − (1 − specificity)]. The slope of the ROC curve was used to determine the cut-off values (≤1) on AD8 test to draw comparisons between the patients with non-dementia and patients with dementia.
All of the statistical analyses were conducted using the Statistical Package for Social Sciences software (version 18 for Windows®, SPSS Inc., Chicago, IL), and a p-value <0.05 (two-tailed) was considered statistically significant.
Results
Effect of demographic data on screening outcome
The mean age between participants with suspected dementia (67.2 ± 13.7) and without dementia (64.3 ± 11.1) was significantly different (p = 0.042). Men were found to be significantly more educated than women Odds ratio (OR) 1.915, 95% confidence interval (CI) (1.440–2.546, p <0.001). Patients suspected of having dementia made up 10.1% of the recruited participants. For participants suspected of having dementia, the mean AD8 score was 2.7 ± 1.2. χ 2 tests for associations between an AD8 score ≥2 and gender of the participant showed no significant difference (p = 0.811).
Demographic data from the screened participants are shown in Table 1. The participation rate was 92.9%, indicating that the questionnaire was well accepted and answered. The major reasons for exclusion from the test were as follows: (1) participants or their family members had severe hearing loss and difficulty completing the interview (n = 34), (2) participants were younger than 50 years (n = 41), (3) other (e.g. insufficient language skills, etc.) (n = 2). A total of 102 participants had an AD8 score ≥2. Only 4% of participants attended the referral appointment with a specialist memory clinic, of whom 35% had an AD8 <2 and 25.5% had an AD8 ≥2. A χ 2 test showed no significant differences between the participants who completed a second visit and participants lost at follow-up in terms of age, educational attainment, and gender (p > 0.05).
Demographic characteristics of participants
AD8: Ascertainment of dementia-8. Referral suggestion were only made if AD8 ≧ 2.
There were 40 participants (which included 26 participants with an AD8 score ≥2 and 14 participants with an AD8 score <2) who attended the second visit and completed cognitive testing. The results of the screening test (acceptance/refusal of screening) were analyzed using χ 2 analyses with age, gender, education, and an AD8 score ≧ 2 as predictor variables. Age was initially entered using six levels and education with two levels. χ 2 analyses indicated that the screening outcome did not vary significantly between the six age groups or between genders. It showed significant difference between the two levels of education (p = 0.044) and an AD8 score ≥2 (p <0.001). This suggested that the acceptance or refusal to attend the referral appointment varied significantly between the groups, and were influenced by the level of education and scores above/below the cut-off value in the AD8. Higher level of education and AD8 ≧2 predicted attending referral (p <0.05).
Demographic data of follow-up participants
Table 2 shows the demographic data of follow-up participants. Compared with participants who had an AD8 score <2, participants who had an AD8 score ≥2 had significantly lower scores for short-term memory in the CASI (AD8 score <2: 10.2 ± 1.6; AD8 score ≥2: 8.1 ± 2.9; p = 0.03). There were no significant differences between the group with an AD8 score <2 and the group with an AD8 score ≥2 in terms of age, educational attainment, MMSE score, or the CASI and its subscores for concentration, attention, orientation, long-term memory, abstraction, visual construction, list-generating fluency, or language abilities (AD8 score <2: 9.1 ± 1.9; AD8 score ≥2: 9.0 ± 2.3; p > 0.05).
Demographic data of the follow-up participants
Data are mean ± standard deviation; AD8: Ascertainment of dementia 8.
MMSE: Mini-Mental-Screen Examination; CASI: Cognitive Ability Screening Inventory; *p <0.05 short-term memory scoring compared with AD8 <2 participants. Lower scores on the CASI and MMSE reflect poorer performance.
Comparison of AUC in the MMSE, CASI, and AD8 for detecting dementia
When compared with non-dementia participants, dementia participants had significantly lower scores on the MMSE (CDR = 0: 24.9 ± 4.4; CDR > 0: 20.6 ± 6.1; p = 0.014), and CASI subscores-orientation (CDR = 0: 17.0 ± 1.9; CDR > 0: 14.4 ± 4.5; p = 0.04) and CASI subscores-short-term memory (CDR = 0: 10.1 ± 1.4; CDR > 0: 7.3 ± 3.0; p = 0.002) tests. There were no significant differences between the CDR = 0 and CDR > 0 groups in terms of age, educational attainment, CASI total scores, or the other CASI subscores.
An ROC analysis was used to evaluate the potential of AD8 scores ≥2 to differentiate between patients with non-dementia and patients with dementia, compared to that of the CASI and the MMSE. A summary of the comparison of the AUC for the two education groups is listed in Table 3. For all of the educational levels examined, the optimal cut-off values of 89.9 on the CASI (Youden index = 0.36) and 25/26 on the MMSE (Youden index = 0.40) were not significant in differentiating between the non-dementia and participants with dementia in the ROC analysis (p > 0.05). However, using a cut-off score of 1/2 on the AD8 (Youden index = 0.30) significantly differentiated between the patients with non-dementia and patients with mild dementia (p = 0.03). For participants who had received six or more years of education and those who had received less than six years of education (Figure 2(a)–(c)), the optimal cut-off values on the CASI and MMSE reached significance and could differentiate between the participants with non-dementia and participants with dementia in the ROC analysis (p <0.05). In contrast, the optimal cut-off value of 1/2 on the AD8 was not significantly different between groups and could not differentiate between the participants with non-dementia and participants with dementia in the ROC analysis in the group of participants who had six or more years of education (p > 0.05). Using an optimal cut-off value of 1/2 on the AD8 differentiated between the patients with non-dementia and the patients with mild dementia in the group of participants who had received less than six years of education (p = 0.03).
ROC curves for the CASI, MMSE, and AD8 for the detection of dementia
ROC curve: Receiver Operation Characteristic curve; CASI: Cognitive Ability Screening Inventory; MMSE: Mini-Mental-Screen-Examination; AUC: Area under curve; CI: Confidence interval.
Receiver operator characteristic (ROC) curve comparing the utility of the Ascertainment of Dementia-8 (AD8), Mini-Mental Screening Examination (MMSE), and Cognitive Abilities Screening Instrument (CASI) for discriminating between mild dementia and no dementia in all participants (a), in participants with an educational attainment of less than six years (b), and in participants with an educational attainment of six or more years (c). The area under the ROC curve in the AD8 was 0.625 making it inferior to the MMSE (AUC = 0.769) and the CASI (AUC = 0.785). The AD8 failed to discriminate non-dementia from dementia in those participants with an educational attainment of more than six years. (AUC = area under the curve). ROC curve for actual AD8 score to discriminate between non-dementia and dementia participants in all participants (d), in participants with educational level <6 (e), in participants with educational level ≥6 (f).
Discussion
Main findings
In this study, we tested the feasibility of using the AD8 to screen for dementia in the general Taiwanese population. First, although the AD8 informant interview was efficiently administered and completed in the majority of individuals approached, only a small number of patients attended the referral appointment at a specialist memory clinic. Second, an AD8 score ≥2 and higher educational attainment played a role in an individual's decision to accept the recommended referral appointment. Third, in the sample of participants who attended the referral appointment, we used the MMSE, CASI, and AD8 to detect dementia. Using an optimal cut-off value, which was set according to the educational attainment of individuals, the performance of both the MMSE and CASI was superior to the AD8 in groups who had received six or more years of educations.
Screening rate
Most of the data available regarding the prevalence of AD in Taiwan is limited to certain areas (Wu et al., Reference Wu2013). Thus, this is the first AD8 feasibility study conducted throughout both northern and southern Kaohsiung. Previous studies examining the prevalence of dementia in Taiwan, conducted over the past two decades, showed that its prevalence ranges from 3.7% (Lin et al., Reference Lin, Lai, Tai, Liu, Yen and Howng1998) to 13.6% (Chen et al., Reference Chen, Wang, Ma and Yang2014). The prevalence rates of dementia identified between studies are different because of the fact that the studies were conducted separately in different localized areas of Taiwan (Liu et al., Reference Liu1995; Reference Liu, Lin, Chen, Tai, Yen and Howng1996; Lin et al., Reference Lin, Lai, Tai, Liu, Yen and Howng1998). In the present study, the prevalence rate of dementia was 10% and the majority of participants with dementia recruited were women, with a mean age of 67.2 ± 13.7 years. Such results might occur owing to the fact that women are more inclined to attend the dementia screening facility than men (Chen et al., Reference Chen, Wang, Ma and Yang2014).
Although the AD8 interview was both feasible and acceptable to patients from the general population in Taiwan, only a small number attended the referral appointment at a specialist memory clinic, and among those only 35% had an AD8 score <2.
One previous study from Singapore found that only 18.6% of patients with an AD8 score ≥2 accepted the recommended referral appointment at a specialist memory clinic (Dong et al., Reference Dong, Cheng, Tsou, Chan and Chen2014), suggesting a low attendance rate. In line with that report, we found that only 25.5% of participants with an AD8 score ≥2 attended the referral appointment. The following may be explanations for the poor attendance at referral appointments: (1) low perception about the benefit/harm of dementia screening (Fowler et al., Reference Fowler2012), (2) lack of knowledge about dementia (Martin et al., Reference Martin2015), (3) existing poor health status (Martin et al., Reference Martin2015), and (4) misconception of poor prognosis and the treatment options available for dementia (Martin et al., Reference Martin2015). In addition, the impact of stigma and labeling around dementia, or recognizing the fact that developing dementia is a real possibility, is also associated with the refusal of individuals to attend referral appointments (Fowler et al., Reference Fowler2012; Phillipson et al., Reference Phillipson, Magee, Jones, Reis and Skaldzien2015). In our study, we found that higher educational attainment helped the individuals to decide to accept the recommend referral appointment. This suggests that further studies evaluating the benefits and risks of using the AD8 to screen for dementia in primary care facilities are necessary (Boustani et al., Reference Boustani, Peterson, Hanson, Harris and Lohr2003).
AD8 and memory
Participants with an AD8 score ≥2 had significantly lower short-term memory scores in the CASI. This was similar to a previous study in Taiwan that reported that memory problems occur more frequently in participants with suspected dementia (AD8 score ≥2) (Chen et al., Reference Chen, Wang, Ma and Yang2014). As reduced memory function is a salient manifestation of Alzheimer's disease, the ability of the AD8 to detect changes in memory function also suggests that it might be a useful screening tool for the detection of early dementia (Galvin et al., Reference Galvin, Roe, Xiong and Morris2006).
AD8, MMSE, CASI, and education
Previous studies have shown that the AD8 has a higher rate of discrimination (13%) than the MMSE (4.4%) for differentiating between participants with dementia and non-dementia participants (Liu et al., Reference Liu1995; Chen et al., Reference Chen, Wang, Ma and Yang2014). Consistent with this report, we also found that the AD8 had a better AUC for discriminating between participants with dementia and non-dementia. However, for those participants who had received six or more years of education, the optimal cut-off value in the CASI and the MMSE could significantly differentiate between the participants with non-dementia and participants with dementia in the ROC analysis (p <0.05), whereas an optimal cut-off value of 1/2 in the AD8 did not reach significance and could not be used to differentiate between participants with non-dementia and participants with dementia in the ROC analysis (p > 0.05).
MMSE and education
Four items in the MMSE are reading and writing dependent as follows: (1) reading and following the command “Close your eyes,” (2) writing a sentence, (3) using a pencil to copy a figure, and (4) serial sevens, all of which are negatively influenced by illiteracy (Xu et al., Reference Xu, Meyer, Huang, Du, Chowdhury and Quach2003). Using the Chinese adapted version of the MMSE, in which the literacy-dependent items of reading and writing were modified for participants with less education, the optimal cut-off points of 20/21 achieved high sensitivity and specificity in minimally educated elderly Chinese people. In the present study, we used the original Chinese version of the MMSE, and therefore the three items of reading and writing could not be performed among participants who were illiterate or less educated. Thus, we found that the optimal cut-off value was 17/18 for participants who had received less than six years of education.
CASI and education
As for the CASI, the optimal cut-off scores were 49/50 and 67/68 in participants with an education level of zero years and 1–5 years, respectively (Lin et al., Reference Lin, Wang, Liu, Chen, Lee and Liu2002). We pooled these two groups together and found a cut-off value of 64/65, which is in line with the literature. The cut-off value for the CASI in participants with an educational attainment of six or more years was 89/90, which is higher than previous reports from the Chinese population (79/80). This difference is likely because of the fact that the participants recruited into the previous study were part of a center-based assessment where patients who were older and had a more advanced stage of dementia were followed and treated (Lin et al., Reference Lin, Wang, Liu, Chen, Lee and Liu2002).
Comparison of the impact of educational attainment on the AD8, MMSE, and CASI
As a whole, we found a comparable AUC between the AD8 and the MMSE in participants who had received less than six years of education, while the MMSE and the CASI yielded better sensitivity and specificity than the AD8 in participants who had received six or more years of education. Several studies have shown that educational level could affect individuals’ score on MMSE and CASI (Crum et al., Reference Crum, Anthony, Bassett and Folstein1993; Lin et al., Reference Lin, Wang, Liu and Teng2012). After minimizing the effect of education, MMSE, and CASI are supposed to show larger AUC in discriminating dementia and non-dementia. Thus, AUCs for MMSE and CASI are larger than the AD8 in the subgroups of education <6 and education ≥6.
The impact of educational level on the AD8 has several potential interpretations. First, it is possible that the AD8 relies on the subjective recall of functional decline. Since education provides a cognitive reserve that compensates for functional decline (Meng and D'Arcy, Reference Meng and D'Arcy2012), the AD8 is liable to subjective biases of under-reporting in highly educated participants. Second, while it was not possible to perform some literacy-dependent items in the MMSE and the CASI among illiterate or less-educated participants, these items provide a cognitive assessment which helps to discriminate between participants with non-dementia and participants with dementia in those with a high level of education, which is absent in the AD8.
Limitations
An advantage to this study is the fact that the screening was conducted over a 1-year period throughout both northern and southern Taiwan. However, there are several limitations to the present study. First, only a small number of patients attended the recommended referral appointment at a specialist memory clinic. Thus, the low attendance at referral appointments requires further examination. Further data collection and improvement of referral strategy will make the comparisons more reliable and meaningful. Second, there was low sensitivity and specificity when the AD8 was used to screen highly educated patients; however, we used formal neuropsychological tests and clinical evaluations to make our own cut-off point, which was similar to the established cut-off point for the informant AD8 interview.
Conflict of interest
None.
Decsription of authors’ roles
Research project conception, organization, execution: YTC, SFC, NCC, HDH, WLT. Statistical analysis execution and review and critique: CCC, YTC, MHL. Manuscript preparation: SFC and YTC.
