Introduction
According to the United Nations Educational, Scientific and Cultural Organization (UNESCO), the illiteracy rate of adults aged 15 and older is 16% in the world, which represents approximately 776 million adults (UNESCO, 2009). Two-thirds of this group are women, and the illiteracy rates of South and West Asia, Sub-Saharan Africa and Latin America are higher in comparison with the developed Western countries. In South Korea, 1.7% of adults aged 19 and older cannot read and write at all, and 7% of adults (about 2.6 million) have reading and writing skills that are inadequate to cope with the demands of everyday life (National Institute of the Korean Language, 2008). The illiteracy rate is still high among the elderly in South Korea, especially in rural areas. In Yonchon, a Korean rural agricultural area, 26.4% of the elderly aged 65 years and older have been found to be illiterate (Lee et al., Reference Lee2008).
Low education is associated with an increased risk of dementia (Prencipe et al., Reference Prencipe, Casini, Ferretti, Lattanzio, Fiorelli and Culasso1996). According to the cognitive reserve hypothesis, persons with higher education have a greater reserve capacity and thus require greater pathological changes to develop manifest dementia (Ngandu et al., Reference Ngandu2007). Illiteracy has also been associated with a higher risk of Alzheimer's disease (AD) after adjustment for education level (Lee et al., Reference Lee2008).
Illiterate persons usually cannot perform tests that require reading and writing skills or copying a complex figure with a pencil, in spite of having normal cognition. Therefore, illiterate people are usually excluded in dementia drug trials (Choi et al., Reference Choi2008). Some researchers have used informant-based reports such as the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) as a screening tool for dementia, especially in a population including a high concentration of illiterates and low-educated people (Fuh et al., Reference Fuh1995). The IQCODE was found to be superior to the Mini-mental State Examination (MMSE) for dementia screening in elderly people with low education (Fuh et al., Reference Fuh1995). However, the IQCODE is not a neuropsychological test but a symptom questionnaire, and cannot measure cognitive function in each of cognitive domains. The 10/66 Dementia Research Group used the Community Screening Instrument for Dementia (CSID) as a screening tool for dementia in some developing countries, which had been useful in distinguishing people with dementia from those with low education but without dementia (Jacob et al., Reference Jacob, Kumar, Gayathri, Abraham and Prince2007). The CSID covers multiple cognitive domains, including orientation, language, memory, praxis and abstract thinking, but it does not cover some domains of visuospatial function, attention and executive function. The Rowland Universal Dementia Assessment Scale (RUDAS) was developed to assess cognition in culturally diverse populations, but it was not validated in illiterate elderly (Storey et al., Reference Storey, Rowland, Conforti and Dickson2004).
Many studies have examined the neuropsychological characteristics of illiterates. Illiterate subjects were better at naming two-dimensional representations of real objects when they were presented as color photos instead of black and white line drawings (Reis et al., Reference Reis, Faísca, Ingvar and Petersson2006). There was also a significant literacy effect on the digit span task. This effect was thought to result from illiterate subjects having less knowledge of numerical magnitude (Petersson et al., Reference Petersson, Reis and Ingvar2001). Illiterate subjects performed significantly worse on a digit span task with digits >5 compared to digits < 5 (Petersson et al., Reference Petersson, Reis and Ingvar2001). The Stick Design test was found to be more sensitive and acceptable than the Constructional Praxis in participants with low or no education (Baiyewu et al., Reference Baiyewu2005). Larger literacy effect was also reported in phonological verbal fluency compared to semantic verbal fluency (Ostroski-Solis et al., Reference Ostroski-Solis, Ardila and Rosselli1998).
Considering these neuropsychological characteristics of illiterates, we set out to develop a neuropsychological test battery applicable to both illiterate and literate elderly and to assess its validity and reliability for a diagnosis of dementia. We called this battery the Literacy Independent Cognitive Assessment (LICA).
Methods
Development of the LICA
Four neurologists, three psychiatrists, and one neuropsychologist developed the LICA. At first, we identified cognitive domains important in assessment of cognition, which were associated with early cognitive impairment in dementia or essential in classifying the different subtypes of mild cognitive impairment (MCI). These were memory, language, visuoconstruction, executive function, attention and calculation. Secondly, we proposed potential items for each cognitive domain from the previously published and validated items and experiential-based items. Thirdly, the items were initially applied to dozens of illiterate and literate normal elderly participants and patients with dementia, and some items which were inapplicable to the illiterate elderly or were unable to differentiate the dementia patients from the normal elderly were discarded or modified. The same process was repeated three times. Fourthly, the corrected LICA was presented to a panel of dementia experts and was amended once again according to their suggestions. The final version of the LICA was obtained following modifications suggested through a fourth pilot study.
The LICA consists of 13 subtests and total scores range from 0 to 300 (Table 1). Subtest I is an immediate recall test of a story. Subtest II is a ten-item test of stick construction that was modified from the Stick Pattern Reversal Test (Benson and Barton, Reference Benson and Barton1970). Subtest III is an immediate verbal recall test of ten words that were selected from high-frequency Korean words (Seo, Reference Seo1998). Subtest IV is a block-tapping test. The orders of nine blocks on a board and tapping sequences were modified from Corsi's block-tapping task (Milner, Reference Milner1971). Subtest V is the Digit Stroop test. Subtest VI is a 12-item test of money calculation. Subtest VII is a test to recall the story of subtest I. Subtest VIII is a ten-question story recognition test. Subtest IX is a test of visual recognition. Subtest X is a delayed recall test of the words heard in subtest III. Subtest XI is a recognition test about the words from subtest III. Subtest XII is an animal fluency test. Subtest XIII is a 15-item test of confrontation naming and the Color and Object Recognition Test (CORT) that was modified for semantic knowledge tests about visual form and color of objects (Patterson et al., Reference Patterson, Nestor and Rogers2007).
The outline of the Literacy Independent Cognitive Assessment
Subjects
The LICA was administered to 159 patients with dementia, 68 with MCI and 655 normal controls. Thirteen controls, two patients with MCI and seven with dementia did not complete the LICA due to withdrawal from the study. Three illiterate normal elderly could not perform the Digit Stroop test because they were not able to read the numerals “1”, “2”, and “3”. After excluding 25 subjects with incomplete LICA scores, 152 patients with dementia, 66 with MCI and 639 normal controls were included in this study.
Dementia was diagnosed according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (American Psychiatric Association, 1994). There were 109 subjects with probable or possible AD by the criteria of the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association (McKhann et al., Reference McKhann, Drachman, Folstein, Katzman, Price and Stadlan1984), 34 subjects with probable or possible vascular dementia by the criteria of the State of California Alzheimer's Disease Diagnostic and Treatment Centers (Chui et al., Reference Chui, Victoroff, Margolin, Jagust, Shankle and Katzman1992), and nine subjects with dementia from other causes. MCI was diagnosed by the criteria of the International Working Group on MCI (Winblad et al., Reference Winblad2004). Participants with dementia or MCI were subjected to extensive neuropsychological testing including the Seoul Verbal Learning Test (SVLT) of a 12-word list consisting of four household, four flower, and four stationery words, the Short Form of the Korean-Boston Naming Test (K-BNT) consisting of black and white line drawings (Kang and Na, Reference Kang and Na2003), Rey Osterrieth Complex figure (ROCF), digit span forward and backward tests, semantic and phonemic verbal fluency, and Color-Word Stroop test (CWST), brain MRI or CT, and routine biochemical and serological tests that included the thyroid function test, serum vitamin B12 and folate levels, and Venereal Disease Research Laboratory test. The illiterate participants with dementia or MCI could not complete the extensive neuropsychological tests due to illiteracy, excepting the SVLT and Short Form of the K-BNT.
Normal controls were at least 60 years old and did not have subjective memory complaints, any of 28 diseases or a history suggestive of a decrease in cognitive function: stroke or transient ischemic attack, seizures, Parkinson's disease, multiple sclerosis, cerebral palsy, Huntington's disease, encephalitis, meningitis, brain surgery, surgery to clear arteries to the brain, diabetes that requires insulin to control, hypertension that is not well controlled, cancer other than skin cancer diagnosed within the past three years, shortness of breath while sitting still, use of home oxygen, heart attack with changes in memory, walking, or solve problems lasting at least 24 hours afterwards, kidney dialysis, liver disease, hospitalization for mental or emotional problems in past five years, current use of medications for mental or emotional problems, alcohol consumption greater than three drinks each day, abuse of drugs in past five years, treatment for alcohol abuse in past five years, unconsciousness for more than one hour other than during surgery, overnight hospitalization because of a head injury, illness causing a permanent decrease in memory or other mental functions, trouble with vision that prevents reading ordinary print even with glasses on, and difficulty understanding conversations because of hearing even if wearing a hearing aid (Christensen et al., Reference Christensen, Multhaup, Nordstrom and Voss1991). They also had scores that were higher, or at most one standard deviation below, the mean scores of the respective age- and education-matched population on the Korean MMSE (K-MMSE) (Kang, Reference Kang2006) and had an average score of 0.42 or less on the Korean Instrumental Activities of Daily Living (K-IADL) (Kang et al., Reference Kang, Choi, Lee, Kwon, Na and Han2002). This score has been found to discriminate dementia from normal aging. K-IADL was an 11-item questionnaire that included IADLs of shopping, use of transportation, ability to handle finances, housekeeping, food preparation, ability to use telephone, taking medication, recent memory, hobbies, watching television, and household chores. For each item, a three-point scale (0–3) was created ranging from 3 (“unable to do”) to 0 (“independent activity”). The K-IADL was scored by averaging each informant's ratings over all 11 items, and was in the range 0–3. Lower scores mean better function. Patients with dementia or MCI were recruited from the outpatient memory clinics at six university hospitals. Normal controls were spouses of patients or were recruited from the senior welfare centers in three rural areas and four cities.
Subjects were asked to read aloud two sentences: “Young-Hee was thirsty due to physical exercise. She opened a refrigerator.” They were then asked to write about what would happen next. A subject who could read the sentences and write an appropriate response was judged to be literate. Patients with dementia were sorted into the literate or illiterate groups according to caregiver reports of their premorbid reading and writing ability. Subjects who were illiterate due to learning and developmental difficulties were excluded.
Validation study of the LICA
To evaluate test-retest reliability, the LICA was administered twice with a mean interval of 34.7 ± 12.8 days to 62 subjects: 6 illiterate and 31 literate normal elderly subjects, 2 illiterate and 7 literate subjects with MCI, and 3 illiterate and 13 literate subjects with dementia. Inter-rater reliability among 11 raters was obtained using videotaped tests. Raters observed ten videotaped LICA tests and independently gave their ratings. To evaluate the discriminant validity, the LICA total scores were compared among the Clinical Dementia Rating (CDR) scale groups (Morris, Reference Morris1993). The K-MMSE was administered to all the participants to evaluate the concurrent validity with the LICA. The LICA, K-MMSE and CDR scale were administered on the same day. The test administrators were blinded to dementia diagnosis of participants when administering the LICA, K-MMSE and CDR scale. Using a random sampling method, 297 controls were matched with 152 demented patients by age, sex and education. This permitted evaluation of the sensitivity and specificity of diagnosing dementia with Receiver Operating Characteristic (ROC) curve analysis.
The study protocol and informed consent form were approved by the Institutional Review Board. Literate subjects gave their written informed consent to participate in the study, and illiterate subjects gave verbal consent.
Statistical analysis
An analysis of variance and a post-hoc Tukey analysis were used to test for differences in age and education among the control, dementia and MCI groups and to evaluate the differences of the LICA total scores among the CDR groups. Differences in the sex ratio among the three groups were evaluated using the χ2 test. The effects of age, sex, and education on the LICA total score and the subtest scores were tested with multiple linear regression analysis of the “enter” method in the normal controls. To evaluate concurrent validity, LICA total scores were compared with K-MMSE scores using Pearson correlation coefficients. Intraclass correlation coefficients were obtained for the inter-rater reliability and test-retest reliability of the LICA. Internal consistency was measured using Cronbach's coefficient α. Sensitivity and specificity of the LICA to make a diagnosis of dementia were evaluated using ROC analysis. Factor analysis was performed using principal component analysis with varimax rotation to determine the factor structure of the LICA. The statistical package SPSS-18 for Windows was used for data analysis, and PASS 2008 was used for power analysis. p < 0.05 was considered to be significant for all analyses.
Results
Demographic characteristics
This study was conducted between May 2008 and March 2009. The demographic characteristics of the subjects are shown in Table 2. Patients with dementia had a higher mean age (F[2, 854] = 17.9, p < 0.001) and a lower educational level (F[2, 854] = 6.4, p = 0.002) compared to participants with MCI and controls.
Demographic characteristics of the subjects
*p < 0.01 and †p < 0.05 vs. the dementia group by post hoc Tukey analysis.
Reliability
Cronbach's coefficient α was 0.908 in illiterate subjects and 0.912 in literate subjects. Inter-rater reliabilities were high for all the subtests (r = 0.85–1.00, p < 0.001) (Table 3). Test-retest reliabilities of the LICA total score were high (r = 0.92, p < 0.001), and the subtests had moderate to high test-retest reliabilities (r = 0.50–0.86, p < 0.001) (Table 3).
The test-retest reliabilities and interrater reliabilities of the LICA subtests
*p < 0.001 by intraclass correlation coefficients
Validity
The LICA total scores and all the subtests except visual recognition, forward visuospatial span, and Digit Stroop were associated significantly with age, education, and sex. The visual recognition, forward visuospatial span, and Digit Stroop tests were not significantly associated with sex, but were significantly associated with age and education.
The linear correlation between the K-MMSE and LICA total scores was substantial in both the illiterate subjects (r = 0.75, p < 0.001) and the literate subjects (r = 0.76, p < 0.001). The mean LICA total scores were 221.4 ± 20.1, 184.8 ± 27.3, 144.0 ± 34.2, 104.7 ± 59.3, and 34.8±32.3 among literate subjects in the CDR 0, CDR 0.5, CDR 1, CDR 2 and CDR 3 groups, respectively, and 195.6 ± 24.2, 150.5 ± 39.0, 111.3 ± 33.8, and 81.7 ± 36.3 among illiterate subjects in the CDR 0, CDR 0.5, CDR 1, and CDR 2 groups, respectively. LICA total scores among the CDR groups were significantly different in the illiterate (F[3, 167] =119.0, p < 0.001) and literate subjects (F[4, 681] = 325.7, p < 0.001), respectively. According to the additional post-hoc analyses, LICA total scores were significantly different between all CDR groups for either literacy groups.
The linear correlation of the subtests was good for the word immediate recall (r = 0.75; against the immediate recall from the SVLT), the word delayed recall (r = 0.77; against the delayed recall from the SVLT), the story delayed recall (r = 0.63; against the delayed recall from the SVLT) and the naming test (r = 0.70; against the K-BNT). The linear correlation of the subtests was moderate for the word recognition (r = 0.54; against the recognition from the SVLT), the story immediate recall (r = 0.42; against the immediate recall from the SVLT) and the story recognition (r = 0.46; against the recognition from the SVLT).
After normal controls were matched to the demented patients to evaluate the sensitivity and specificity of diagnosing dementia, there were no significant differences in age, education or sex between the patients and controls for either literacy groups (Table 2). The area under the ROC curve (AUC) was 0.985 (95% confidence interval [CI], 0.975–0.994, standard error = 0.005) in the literate subjects, and 0.985 (95% CI, 0.972–0.998, standard error = 0.007) in the illiterate subjects (Figure 1). Sensitivity and specificity of the LICA to make a diagnosis of dementia was 91.9% and 91.8%, respectively, at the cutoff point of 186.0 in the literate subjects, and 96.2% and 91.1%, respectively, at the cutoff point of 154.5 in the illiterate subjects.
The receiver operating characteristic curves of the LICA used to make a diagnosis of dementia in the literate (A) and illiterate (B) subjects.
Factor analysis yielded two factors in the LICA accounting for 62.7% of the variability (Table 4). Factor 1 included seven subtests (stick construction, visual recognition, visuospatial span, Digit Stroop test, animal fluency, CORT and naming, and calculation) and accounted for 34.9% of the total variance. We named this factor “tests for other cognitive domains except verbal memory”. Factor 2 included six subtests (immediate and delayed recall, and recognition of the story and words) and accounted for 27.8% of the total variance. We named this factor “verbal memory tests”.
Factor loading of the two main factors
Application
The LICA subtests were applicable to 108 of the 111 illiterate normal controls (97.3%). The total application time of the LICA was 28.6 ± 6.4 minutes in literate normal controls (n = 78), 30.7 ± 5.8 minutes in illiterate controls (n = 22), 40.7 ± 12.4 minutes in literate subjects with dementia (n = 29), and 40.3 ± 11.9 minutes in illiterate subjects with dementia (n = 16).
Discussion
The LICA was found to be a valid and reliable instrument for detection of dementia in both illiterate and literate elderly persons. Although the IQCODE was already found to be valid for screening of dementia among persons with very low education levels, it is a symptom questionnaire, and cannot measure cognitive function in each cognitive domain (Fuh et al., Reference Fuh1995). The LICA has strength to be able to assess the function of each cognitive domain in illiterate elderly.
It performed well in discriminating participants across CDR stages in both literacy groups. Therefore, the LICA may be an effective tool for monitoring disease progression in illiterate and literate patients and for differentiating MCI with CDR 0.5 from cognitively normal subjects with CDR 0. The concurrent validity of the LICA with the K-MMSE was good in both literate and illiterate participants. It showed excellent internal consistency in both literacy groups and thus is appropriate to use as a unified cognitive test for both literate and illiterate patients. Although we employed 11 different raters for the LICA, the high inter-rater reliabilities for all the subtests showed that our results were very reliable. The LICA showed high sensitivity and specificity for diagnosing dementia in both illiterate and literate participants. However, the cutoff score differed between these groups. The LICA was also meaningfully factored into two clearly separated factors (verbal memory tests and tests for other cognitive domains except verbal memory).
The test-retest reliability of the LICA total score was very high, and the subtests had moderate to high test-retest reliabilities indicating that the performance on the LICA has acceptable stability over time. However, the learning effect could influence the test-retest reliabilities. The mean LICA total score was up 15 points at the retest. The verbal learning test using the words from the LICA showed high test-retest reliabilities, and the story recall tests had moderate to good test-retest reliabilities. The story recall test consisted of only one presentation with a recall of three sentences and 20 memory units. If a subject did not listen carefully to the story, he/she received low scores in the immediate, delayed and recognition tests of the story. This characteristic of the story recall test could result in relatively lower test-retest reliability compared to the word recall test which had three presentations.
We could only evaluate the construct validity of the verbal memory and naming tests from the LICA because the illiterate participants could not perform most of the extant neuropsychological tests due to illiteracy. Compared to the verbal learning test of ten words from the LICA, the story recall test from the LICA had moderate construct validity with the SVLT. Our results are similar to those of a previously reported study that found that scores on the Babcock story recall test were more correlated with the Wechsler Memory Scale-Revised Logical Memory scores than those of the California Verbal Learning Test (Horner et al., Reference Horner, Teichner, Kortte and Harvey2002). The verbal learning test and logical memory test are generally thought to be interchangeable measures of verbal memory, but the cognitive domains used in the two tests may be somewhat different. In another study, executive dysfunction was more closely associated with the verbal learning test compared to the logical memory (Tremont et al., Reference Tremont, Halpert, Javorsky and Stern2000).
We used color photos as a tool in the naming test to reduce the influence of literacy (Reis et al., Reference Reis, Faísca, Ingvar and Petersson2006). The photos were of living things because a more rapid degradation of the memory of living compared to non-living things has been detected in AD (Almor et al., Reference Almor2009). Semantic memory is affected at an early stage in semantic dementia and at a later stage in AD (Rogers et al., Reference Rogers, Patterson and Graham2007). The CORT may be effective for differentiating semantic dementia from early AD and for monitoring semantic memory impairment in AD.
Although we did not calculate the study sample size prior to the study, we calculated the power of our study. The null hypothesis was that the accuracy of the LICA was not different from that of the IQCODE with the AUC of 0.913 (Fuh et al., Reference Fuh1995). To detect a difference of 0.072 between the AUC under the null hypothesis of 0.913 and an AUC under the alternative hypothesis of 0.985 using a two-sided z-test at a significance level of 0.05, a sample of 99 from the positive group and 207 from the negative group achieved 100% power in the literate group, and a sample of 53 from the positive group and 90 from the negative group achieved 84% power in the illiterate group.
Although all the subtests were applicable to most of the illiterate controls (97.3%), some illiterate controls got very low scores in the money calculation test (score 5 or less in 15%) and backward visuospatial span test (2 or less in 19%). The short-form of the LICA consisting of more applicable subtests to illiterate elderly could be considered in future.
The present study has several limitations. It was a cross-sectional investigation, and the patients were a clinic-based patient population. To solve these problems, we suggest that the LICA should be investigated in longitudinal studies and in a community-based cohort study. Another limitation is that we applied the same diagnostic process to both illiterate and literate subjects. However, some of the extant neuropsychological tests were not applicable to the illiterate participants with dementia or MCI. Therefore, there is a possibility of misclassification of the illiterate participants. The illiterate subjects may also be relatively poor and have sociocultural differences with the literate subjects, but we did not investigate or control for socioeconomic and sociocultural effects on the LICA. Finally, depression was not screened in the controls but only in the patients with dementia or MCI. Therefore, the inclusion of the depressed controls might result in lowering the mean score of the LICA. Despite these limitations, this study has shown that the LICA is a valid and reliable instrument for detecting dementia in both illiterate and literate elderly. The LICA may become an alternative test for populations that include a high concentration of illiterate and low-educated people.
Conflict of interest
None.
Description of authors' roles
S. Choi designed the study, collected the data, was responsible for the statistical design of the study and for carrying out the statistical analysis, and wrote the paper. Y. Shim, S.-H. Ryu, D. Lee, and J. Jeong developed part of the LICA and collected the data. J.-Y. Lee designed the study, developed part of the LICA and collected the data. H. Ryu developed part of the LICA, collected the data, and assisted with writing the paper. S.-H. Han designed the study, collected the data, and wrote the paper.
Acknowledgements
This study was supported by a grant (No. 11-1351000-000049-01) from the Ministry of Health and Welfare, and a grant (No. A050079) from the Korea Healthcare Technology R&D Project, Ministry of Health & Welfare, Republic of Korea. The authors thank Tae-You Kim, MD (Department of Neurology, Willis Hospital, Busan) and Kang Soo Lee, MD (Department of Psychiatry, Bundang Cha Medical Center, Seongnam) for helping with the collection of data, and Professor Young Ju Suh and Dae Hyung Lee (Inha University Hospital, Incheon) for advice about the statistical analysis.
