Introduction
Alzheimer dementia (AD), the most common cause of dementia in the elderly (Geldmacher and Whitehouse, Reference Geldmacher and Whitehouse1997), is a neurodegenerative disorder causing deterioration of cognitive functions that include memory, language, agnosia, attention, apraxia, executive functions, and orientation (Hannay and Lezak, Reference Hannay, Lezak, Lezak, Howieson and Loring2004). Prevalence is approximately 10% among those 65 years and older and increases to about 50% in those over 85. (Evans et al., Reference Evans, Funkenstein, Albert, Scherr and Cook1989; Corrada et al., Reference Corrada, Brookmeyer, Paganini-Hill, Berlau and Kawas2010). In Turkey, the percentage of the population over 65 years is 7.3%, approximately 5.48 million adults (Turkstat, 2011).
Neuropsychological evaluation is essential for screening dementia; however, neuropsychological test batteries are often developed for literate population and thus of course include literacy-dependent items. Turkey has a high rate of illiteracy or limited education among its elderly, especially in rural areas: more than 45% according to Turkstat data (Turkstat, 2011). In this situation, diagnosis presents major difficulties, including misdiagnosis and/or delayed detection of dementia (Yassuda et al., Reference Yassuda, Diniz, Flaks, Viola and Pereira2009). To overcome these difficulties, cognitive impairment scales are needed that are appropriate for populations with little to no education.
Attempts to improved methods for assessing cognitive changes in illiterate individuals have typically followed one of two approaches. The first is standardization of well-known standard neuropsychological batteries developed for literate patients in the west. In doing so, however, inaccuracies arise because of diversity in language, culture, and levels of literacy (Kalaria et al., Reference Kalaria, Maestre, Arizaga, Friedland and Galasko2008). An alternative is to develop new neuropsychological batteries more feasible for patients with heterogeneous educational levels who are suspected of having AD.
The four aims of the present study were (1) to develop a brief screening battery (Cognitive State Test, COST, provided in Appendix) to detect the presence of dementia in both illiterate and literate patients, (2) to identify cut-off points that correlate with cut-off points of the Mini Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA), as well as correlation with the Geriatric Depression Scale (GDS), the Clinical Dementia Rating (CDR), and Basic Activities of Daily Living (BADL) for standardized use in clinical applications, (3) to validate the COST in an independent sample, and (4) to assess its reliability.
Methods
Development and validation of the COST
One of the authors who recently began to work at Bezmialem Vakif University Hospital, Turkey, noticed that forgetful patients who have little to no literacy skills were often misdiagnosed through the neuropsychological tests being used. To remedy the problem, she decided to develop another, completely new diagnostic tool that would be fairly short, easy to administer, and give accurate results.
The first step was to determine the main areas of cognitive functioning that are vital in assessing cognition. Second, before starting a validation study, the administration of the COST was determined in a separate pilot study consisting of 68 volunteers.
COST is presented on two pages and, like MMSE takes about 5 minutes. It consists of 38 items divided into 11 domains each representing a specific cognitive functions: orientation (8 items/4 points), memory (9 items/9 points), attention (7 items/5 points), executive functions (3 items/3 points), language (7 items/5 points), agnosia (1 item/1 point), apraxia (2 items/2 points), and visuospatial function (1 item/1 point). The maximum total score is 30. The higher the score, the better the subject's cognitive function.
Cognitive domains are described as follows:
• Orientation (4 points): This domain consists of three parts: orientation in time, person, and space. The questions are limited to the year, the season, and the time of day. Examples of questions asked to persons are as follows: “How old are you?”; “Where were you born?”; and “What is my job?”Place orientation consists of two (country and city) questions.
• Registration Memory (3 points): This domain contains the three words registration (repeating three words: “melon,” “anchovy,” and “green”). The test administrator says all three words and asks the subject to repeat them. The first repetition determines the subject's score, however, if he cannot repeat them all, the three words are repeated three times until he learns.
• Attention (5 points): The subject is asked to count aloud backward from 10 to 1 and to repeat sequences of 2–3, and 4-digits, respectively.
• Recall Memory (3 points): The subject is asked to recall the three words above (“melon,” “anchovy,” and “green”).
• Abstraction and Judgment (2 points): This domain consists of two parts. In the abstraction part, the subject is instructed to explain the meaning of the well-known proverb “A rolling stone gathers no moss.” For the judgment part, the subject is asked what he should do if he found a two-year-old child who appeared to be lost.
• Verbal/Category Fluency (1 point): For category fluency, the subject is asked to produce as many different animals as possible in 1 minute. Repetitions are not counted.
• Retrograd Memory and General Informations (3 points): Three general knowledge questions are asked including the name of the founder and first president of the Republic of Turkey, the name of the current prime minister, and the number of months in a year.
• Language (5 points): This domain consists of three parts: “Repeat phrase,” “Understanding,” and “Naming.” In the repeat phrase, the subject is to repeat the phrase “Above, beyond and below.” The understanding part consists of a two-stage command for the subject to perform: “Hold your right hand and touch your left ear.” For the naming section, one common-known animal (camel) and three objects (glasses, pen, and watch) are used.
• Agnosia (1 point): We show a key and ask to subject to tell the use for this object.
• Apraxia (2 points): Subjects were asked to perform two commands: “Salute like a soldier” and “Pretend your are holding a toothbrush and brush your teeth.”
• Visuospatial functions (1 point): The subject is asked to copy a circle divided into four equal parts.
Subjects
Enrolled in this study were patients referred to the Neurology and Geriatric outpatient clinic of Bezmialem Vakif University, Medical Hospital, Turkey.
Subjects were excluded if they had a history of pre-existing organic brain disorders causing dementia, stroke, epilepsy, significant aural or visual impairments or major depression, any of which may confound cognitive measurements. In the AD group, patients underwent general physical, neurological, and psychiatric examinations and laboratory tests including thyroid function tests, serum vitamin B12, and folate analyses, and magnetic resonance imaging of the brain.
After applying the exclusion criteria, a total of 72 patients with dementia (the probable Alzheimer disease according to the National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer's Disease and Related Disorders Association criteria – NINCS-ADRDA) (McKhann et al., Reference McKhann, Drachman, Folstein, Katzman, Price and Stadlan1984) were prospectively recruited into this study.
The subjects in the control group had no cognitive complaints and any history of neurological and psychiatric disorders. They also were not taking any medication that might affect their cognitive status and mood.
The subjects were divided into two groups by age: Group A1: <65 years and Group A2: ≥65 years. Subjects’ degree of literacy and their definition of illiteracy were based on self-report. To investigate their effect on scores, educational level was classified into three subgroups (Group E1: illiterate, Group E2: 1–5 years; and Group E3: six or more years of schooling with respect to the Turkey's former education system). Excluded were those who could read and write but had no formal education.
This study was approved by the Internal Ethical Committee of Bezmialem Vakif University, School of Medicine, Turkey. It was conducted between March 2011 and January 2012.
Clinical and cognitive evaluation
Neuropsychological measures included the COST, MMSE, MoCA, GDS, BADL, and the CDR. Each subject took all the tests on the same day. These were administered by a psychologist (Emine Aydemir) and two neurologists (Gulsen Babacan-Yildiz and Emel Ur). All subjects completed the COST, the MMSE, the BADL, the GDS, and the CDR, but only literate subjects completed MoCA.
• The MMSE is a widely used instrument for screening and monitoring global cognitive impairment. It measures orientation, registration, attention, recall, language, and visuospatial functions. Scores range from 0 to 30, with lower scores indicating more severe impairment (Folstein et al., Reference Folstein, Folstein and McHugh1975). The Turkish version of the MMSE was validated for the Turkish population (Gungen et al., Reference Gungen, Ertan, Eker, Yasar and Engin2002). As a screening test to assess cognitive functions in illiterate subjects, we used the MMSE modified version (MMSE-I) in which the items were simplified to the level of patients with no education (Ertan et al., Reference Ertan, Eker and Gungen1999).
• The cognitive domains tested by the MoCA include orientation, language, attention, executive functions, memory, and visuospatial functions. The total scoring (30 points) is similar to that of the MMSE where higher scores indicate better cognitive functioning. In the MoCA validation study, those who had 11 or more years of education were selected (Nasreddine et al., Reference Nasreddine, Philips, Bedirian, Charbonneau and Whitehead2005).
• The GDS short version consists of 15 questions in which subjects respond yes or no in reference to how he felt over the past week (Yesavage et al., Reference Yesavage, Brink, Rose, Lum and Huang1983). The higher the score, the more severe the level of depression; 0–4: normal, 5–8: mild, 9–11: moderate, and 12–15: severe depression.
• The CDR, which was designed to assess AD outpatients, is an informant-based global assessment scale that grades on a scale beginning with 0 (normal) and includes 0.5 (questionable dementia), 1 (mild dementia), 2 (moderate dementia), and 3 (severe dementia) (Hughes et al., Reference Hughes, Berg and Danziger1982).
• Functional status of subjects was assessed through interviews with patients and their caregivers using the Katz BADL. It was designed to measure six basic activities of daily life: self-feeding, dressing and undressing, getting in and out of the bed or a chair, bathing by oneself, toileting and controlling micturition, and defecation. Scores range from 0 (no impairment in BADL) to 12 (highest functional impairment) (Katz et al., Reference Katz, Ford and Moskowitz2000).
Data analysis
Three demographic variables (age, education, and gender) were assessed to analyze their influence on the mean scores of the neuropsychological tests.
To establish concurrent validity with the COST, we correlated COST scores with the MMSE and MoCA. Optimal cut-off scores that were determined by the receiver operating characteristic (ROC) curve were defined as the score that led to the maximal accuracy of classification and for literate and illiterate subjects. Different cut-off points were employed to detect optimal sensitivity and specificity values. The COST scores were also correlated with CDR, BADL, and GDS scores.
Results
Sociodemographic characteristics
This study was conducted on 114 control subjects (64 males and 50 females aged 64.9±9.8 years) and on 74 Alzheimer's patients (19 males and 55 females aged 67.2±13.2 years) and schooling from 0 to 15 years (mean 5.7±4.2 for the control group and 3.3±3.8 years for the AD group). When comparing demographic variables, significant group differences were found for age (p = 0.000), gender (p = 0.000), and education (p < 0.05). Illiterate women outnumbered illiterate men (p < 0.001). Men scored higher than women on both COST and MMSE score but no statistically significant difference was found existed between men and women except for MoCA scores (p < 0.105). Among all subjects, GDS scores were significantly higher in woman (p < 0.001). Table 1 gives the mean scores of tests and demographic data.
The mean score of tests and demographic data
The AD group consisted of 32 patients with mild (CDR:1), 33 patients with moderate (CDR:2), and 9 patient with severe stage dementia (CDR:3). Patients with CDR:0.5 were excluded to ensure a clear separation between AD patients and normal, healthy participants.
Reliability and validity
For each of the three subgroups of educational level, the diagnostic accuracy was determined through the area under ROC curves (AUC). In each education level, ROC analyses revealed the different cut-off scores and sensitivity, specificity and AUC of COST, MMSE, and MoCA for identifying dementia. In the E1, E2, and E3 education groups, the optimal cut-off points of COST chosen for diagnosis of AD were 23/24 (sensitivity: 81%, specificity: 99%, and AUC: 0.94), 24/25 (sensitivity: 75%, specificity: 86%, and AUC: 0.90), and 26/27 (sensitivity: 77%, specificity: 84%, and AUC: 0.86), respectively. When illiterate and literate subjects were then pooled, the optimal cut-off score of COST was 24/25, which yielded a sensitivity of 81%, a specificity of 87%, and AUC of 0.92. The positive and negative predictive values for the COST were good in the pooled group (86% and 83%, respectively). For discriminating between the control and AD groups, the optimal cut-off scores of MMSE and MoCA were given in each group. MMSE-I showed excellent sensitivity (97%) and specificity (99%) in detecting the cognitive impairment of illiterate people with AD (Table 2).
Comparison of COST, MMSE, and MoCA in AD groups. Group E1: illiterate, Group E2: 1–5 years, and Group E3: six or more years of schooling
AUC = Area Under the Curve; PPV = Positive Predictive Value; NPV = Negative Predictive Value.
As shown in Table 3, the COST total score had inverse correlation with age (r = −0.26; p = 0.00), GDS (r = −0.18; p = 0.01), BADL (r = −0.59; p = 0.00), as well as CDR (r = −0.77; p = 0.00). A good positive correlation existed between COST and MMSE (r = 0.84; p = 0.00) and between COST and MoCA (r = 0.69; p = 0.00).
The correlations of COST with AGE, MMSE, MoCA, CDR, BADL, and GDS
Education level correlated significantly and positively with the COST (r = 0.47; p = 0.00), MMSE (r = 0.44; p = 0.00), and MoCA scores (r = 0.27; p = 0.00) and inversely with CDR (r = −0.32; p = 0.00), BADL (r = −0.82; p = 0.00), and GDS (r = −0.15; p < 0.04) (Table 3).
The internal consistency of the COST was good, yielding a Cronbach's coefficient α on the standardized items of 0.86.
Discussion
Diagnosis of AD in patients complaining of forgetfulness depends on neuropsychological evaluation. It is well known that most neuropsychological tests are significantly influenced by education (Lezak, et al., Reference Lezak, Howieson and Loring2004; Tombaugh and McIntyre, Reference Tombaugh and McIntyre1992), but regrettably, only a limited number of tests are designed for AD patients with little to no education. For batteries developed for literate populations in western countries, literacy is naturally a precondition. In the absence of appropriate neuropsychological batteries for the illiterate, neuropsychological evaluation may present major difficulties, including misdiagnosis and/or delayed detection of dementia in these populations and the diagnosis remains often challenging. Illiteracy and low education are associated with an increased risk of not only dementia (Prencipe et al., Reference Prencipe, Casini, Ferretti, Lattanzio and Fiorelli1996; Lee et al., Reference Lee, Chang, Jang, Chang and Suh2008) but also of misdiagnosis (Teng, Reference Teng2002).
A misdiagnosis of AD may have profound emotional implications both for patients and their families, as well as increase their financial burden by unnecessary tests and treatment (Teng, Reference Teng2002). Growing awareness that standardized neuropsychological tests may delay or cause a wrong diagnosis of a patient's condition has lead to research on alternative approaches in order to prevent this.
The need for efficient and effective cognitive screening has become more pronounced particularly in developing countries. To address this problem, the COST was developed. The main goals in developing this neuropsychological test battery were to prevent a misdiagnosis of AD among illiterate or semi-literate patients who are forgetful and to use same battery in both patient groups of our study. The COST evaluates almost all cognitive domains. In addition, the single cut-off score of COST (24/25) serves good sensitivity, and specificity across all subgroups of the various educational levels.
One common neuropsychological test is the MMSE, the most widely used bedside screening instrument to measure cognitive impairments (Folstein et al., Reference Folstein, Folstein and McHugh1975). The original MMSE was designed for literates. Since its value in diagnosing dementia in general practice is limited in illiterate populations (Lezak et al., Reference Lezak, Howieson and Loring2004; Scazufca et al., Reference Scazufca, Almeida, Vallada, Tasse and Menezes2009), a new version of MMSE (MMSE-I) is frequently used among those with low educational levels. (Tombaugh and McIntyre, Reference Tombaugh and McIntyre1992; Kraemer et al., Reference Kraemer, Moritz and Yesavage1998). Still, the accuracy of the MMSE-I remains problematic because of the diversity of language, culture, and levels of literacy among those in developing countries. In addition, MMSE-I has no single sensitivity, specificity, and cut-off scores. In the present study, a version of MMSE-I was used for our illiterate subjects (Ertan et al., Reference Ertan, Eker and Gungen1999). When comparing the COST with the MMSE (p = 0.00), and the MMSE-I (p = 0.00) correlations in both cases were statistically significant. MMSE and MMSE-I's sensitivity and specificity may seem higher than COST's; however, compared with MMSE and MMSE-I, COST has some advantages. For example, in clinical settings, we observed that attention items of the MMSE were difficult for the patients to perform in this population, whether or not they had dementia. For this reason, in order to increase the accuracy, we decided to use simpler items in COST test for the assessment of attention. Another advantage is that COST has more cognitive domains such as retrograd memory, general information, apraxia, agnosia, and executive functions and more detailed assessment of attention and language functions are evaluated. In addition, COST can be used with both illiterate and literate populations and in the differentiation between Alzheimer disease and other types of dementia.
In Turkey, another neuropsychological test validated in populations with varying educational levels is the MoCA (Selekler et al., Reference Selekler, Cangoz and Uluc2010). We found a good correlation between the COST and the MoCA (p = 0.00). The precondition of the MoCA is 11 or more years’ education; however, in the current study, the MoCA was administered to Groups E2, and E3. When these were pooled, the MoCA's sensitivity and specificity were 87% and 75%, respectively, using a cut-off score of 19/20. As expected, this cut-off point was lower than those found by Nasreddine et al. (Nasreddine et al., Reference Nasreddine, Philips, Bedirian, Charbonneau and Whitehead2005) and by Selekler et al. (Selekler et al., Reference Selekler, Cangoz and Uluc2010).
Cognitive impairment is usually accompanied by limitations in basic and instrumental activities of daily living (BADL and IADL), which can cause the family distress. According to almost all diagnostic criteria in AD, cognitive impairment should affect the patient's ability to carry out ADLs ((McKhann et al., Reference McKhann, Drachman, Folstein, Katzman, Price and Stadlan1984; APA, 1994). Although IADLs are affected in early stages of AD, BADL remain preserved until the progression of Alzheimer disease (Galasko et al., Reference Galasko, Bennett, Sano, Ernesto and Thomas1997; Lechowski et al., Reference Lechowski, de Stampa, Denis, Tortrat and Chassagne2008). We found that impairment of BADL is statistically associated with COST score, results consistent with previous studies (Breen et al., Reference Breen, Larson, Reifler, Vitaliano and Lawrence1984; Loewenstein et al., Reference Loewenstein, Amigo, Duara, Gutterman and Hurwitz1989). Dysfunction in IADLs may predict neuropsychological impairment rather than BADL, but in the present study, IADL was not assessed. The reason for excluding IADL was to avoid incorrect assessments since in Turkey most men in this age group eschew some of the activities – particularly housekeeping, food preparation, and doing laundry.
The current study confirmed the commonly quoted association between depressive symptoms and cognitive impairment in AD (Wilson et al., Reference Wilson, Mendes-DeLeon, Bennett, Bienias and Evans2004), as indicated by the negative correlation between GDS and severity of dementia. The COST was also found to have a statistically significant negative correlation with the CDR (p = 0.00), a well-known scale used to assess functional status based on caregiver reports (Loewenstein et al., Reference Loewenstein, Amigo, Duara, Gutterman and Hurwitz1989).
As in the vast majority of the neuropsychological tests, performance on the COST was negatively influenced by advanced age, female gender, and less education (Escobar et al., Reference Escobar, Burnam and Karno1986; Tombaugh and McIntyre, Reference Tombaugh and McIntyre1992).
Limitations of the present study include the relatively small sample size for a study involving illiterate subjects. Second, the construct validity for the subdomains of the COST was not examined. Third, this study excluded those who could read and write but had no formal education, so we have no neuropsychological test scores from them to assess. A fourth limitation is the lack of normative data in the MMSE-I, the GDS-15, and the BADL.
In summary, demand remains for efficient, reliable neuropsychological batteries to measure cognitive impairments in illiterate populations complaining of memory problems. Such batteries should be applicable and be adaptable across a range of educational levels. The COST has proved to be a sensitive and specific cognitive instrument that may be useful for detecting cognitive impairment in illiterate population, while also appropriate in educated populations.
Conflict of interest
None.
Description of authors’ roles
Dr G. Babacan-Yildiz designed the study, collected the data, and wrote the paper. Dr A. T. Isik participated in study design. Dr Emel Ur, Dr Pinar Soysal, Psych., C. Ertas, Psych., E. Aydemir, Dr E. Gursoy, Dr M. Kolukisa, Dr G. Kocaman, and Dr A. Celebi collected the data. Psych. M. Cebi performed the statistical analyses.
Acknowledgment
We wish to thank S. Delacroix for her assistance with English editing and O. Uysal for his statistic support.
