Introduction
Screening tests are essential in the investigation of patients suspected of having dementia. As the population ages and dementia incidence rates increase, such tests have become even more important. Among screening tests are cognitive assessments performed directly on the patient while others are based on information provided by the carer (or family member). Studies have shown that the association of both is an effective method of performing screening tests (Mackinnon and Mulligan, Reference Mackinnon and Mulligan1998; Bustamante et al., Reference Bustamante2003), increasing sensitivity and specificity of the instruments when used independently. However, in some cases the patient is not able to provide information properly, due to severe disease, refusal to cooperate, or because the patient has a very low level of education (Jorm, Reference Jorm2004). In such cases, indirect assessments, applied only to the caregiver, play a major role in clinical practice, especially in developing countries, in which mean educational level is often lower than that seen in developed countries.
Ideally, screening tests have to be brief, easily adapted, with a high reproducibility level between different raters and in different cultures, and without any educational bias. Unfortunately, many of the current tests do not meet these requirements, and those that do are often long and require training prior to their application (Galvin et al., Reference Galvin, Roe, Xiong and Morris2006). In this context, a brief screening test applied to the caregiver, the IQCODE (Jorm and Jacomb, Reference Jorm and Jacomb1989), was evaluated to verify the efficacy of its long and short versions adapted for use in Brazil.
The IQCODE was originally developed as an interview with the informant by Jorm and Korten (Reference Jorm and Korten1988). A total of 39 initial items were reduced to 26, making the test easier to apply. There are now versions available in several languages including Chinese, German, Finnish, French, Canadian French, Dutch, Italian, Japanese, Korean, Norwegian, Polish, Spanish, and Thai. Many short versions have been evaluated; for example, 16-item versions in English (Jorm, Reference Jorm1994) and German (de Jonghe et al., Reference de Jonghe1997), and a 17-item version in Spanish (Morales et al., Reference Morales, Gonzalez-Montalvo, Bermejo and Del-Ser1995).
This study aims to evaluate the performance of long (L) and 16-item short (S) Brazilian versions of the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) and compare them with the Mini-mental State Examination (MMSE) in the screening of dementia among elderly people with low education. Evaluating the results retrospectively, we also added a comparison with a new version, the IQCODE 15-item short Brazialian version (SBr). We hypothesized that instruments based on the informant report can be as useful as a brief cognitive test for the screening of dementia in elderly people with low education.
Method
A cross-sectional, case-control study was performed on patients diagnosed with dementia at the Old Age Research Group (PROTER) Clinic of the Institute of Psychiatry, Hospital das Clínicas, Faculty of Medicine, University of São Paulo (IPq-HCFMUSP). The control population comprised patients in the Geriatrics Service of HCFMUSP, and elderly relatives of patients in the PROTER Clinic.
Inclusion criteria were (i) older than 60 years and (ii) definitive diagnosis of mild to moderate dementia, performed by psychiatrists at PROTER Clinic of IPq-HCFMUSP, in accordance with the ICD-10 criteria (World Health Organization, 1993) and DSM-III-R (American Psychiatric Association, 1987). Exclusion criteria were (i) presence of other psychiatric disorders; (ii) severe stage of dementia according to DSM IIIR criteria; and (iii) lack of any relatives who could provide reliable information on the patients' current and previous health status. For the controls, the exclusion criteria were (i) having a psychiatric disorder and (ii) absence of relatives who could provide reliable information on the controls' current and previous health status.
Both the patients and controls were classified into three groups, according to their socioeconomic and educational level, in accordance with the criteria set down by the Brazilian Association of Market Research Institutes (ABIPEME) (Associação Brasileira de Anunciantes, 1989).
The instruments used were the Mini-mental State Evaluation (MMSE) (Brucki et al., Reference Brucki, Nitrini, Caramelli, Bertolucci and Okamoto2003); Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE), long (L) (26 items), short (S) (16 items) (Jorm et al., Reference Jorm2004), and a new short version containing 15 items (IQCODE-SBr), developed based on the results obtained with the present sample; a scale to assess psychiatric symptoms, Psychiatry Screening Questionnaire (SRQ-20) (Mari and Williams, Reference Mari and Williams1986); a brief cognitive test, Abbreviated Mental Test Score (AMTS) (Qureshi and Hodkinson, Reference Qureshi and Hodkinson1974); and the Criteria for Socioeconomic Classification by ABIPEME (Associação Brasileira de Anunciantes, 1989). All the instruments were used in their Brazilian Portuguese versions. The IQCODE versions used in this study were translated from their original version in English by a geriatric psychiatrist (M.A. Lopes) and by a psychologist and English teacher (P. Zukauskas).
The study was performed in two stages. The first stage was the selection of controls, which used the scales AMTS and SRQ-20 to assess psychiatric symptoms and cognitive performance. Most of the controls had been carefully assessed at the geriatric clinic in our hospital, and all of them (including the informants selected from within our service) were interviewed to rule out significant cognitive complaints and impairment in daily living activities, before being evaluated with the AMTS and SRQ20. The second study stage was the application of screening instruments for dementia in cases and in controls, which was performed at the Clinic of the Psychiatry Institute at HCFMUSP by examiners blinded to the individuals' clinical status.
Thirty-four patients diagnosed with mild to moderate dementia according to ICD-10 criteria were assessed, as well as 58 elderly controls, who underwent the SRQ-20 and AMTS. Individuals were divided into three groups, according to their socioeconomic status and educational level (high, middle and low). The patients were assessed using the MMSE, and the informants were interviewed using the 26 items of the IQCODE by examiners blinded to the individuals' previous clinical diagnosis. The scoring of the IQCODE in the present study involved summing the items and dividing them by the number of completed items, because there were some missing items (Fuh et al., Reference Fuh1995; Lim et al., Reference Lim, Lim, Anthony, Yeo and Sahadevan2003; Jorm, Reference Jorm2004).
This study was approved by the Ethics Committee of the Faculty of Medicine, University of São Paulo, and all the patients and informants agreed to participate in the study after signing a consent form.
Data analysis was performed using the statistical package SPSS for Windows version 14.0 and the MedCalc version 9.5, with descriptive analysis, analysis of receiver operating characteristics (ROC) curves and logistic regression. Diagnosis of dementia was the dependent variable, and the MMSE and IQCODE versions were the independent variables.
Results
Table 1 shows the sociodemographic characteristics of both groups. Note that the patient group had a higher mean age, and the control group had a higher proportion of women.
Sociodemographic characteristics of the sample
χ2 = Pearson χ2; t = Student t test; p = p-value.
There was no difference in mean IQCODE L, S and SBr (15 or 16 items) according to gender as shown in Table 2. There was a significant correlation between educational level and the MMSE in the control group (r = 0.272; p = 0.039), but not in the patient group (r = 0.303; p = 0.082). As the correlation coeficients in patients and controls were comparable, the observed significance was probably due to low statiscal power in the patient group. On the other hand, the IQCODE versions were not significantly correlated with age and educational level (Table 2).
Instruments' characteristics by gender and correlation with age and education
t = Student t test; p = p-value; r = Pearson correlation coefficient; * = significant (p < 0.05).
The performance of all four instruments evaluated by the areas under the ROC curves was very similar (see Figure 1): MMSE = 0.94 (95% CI: 0.90–0.98), IQCODE-L = 0.94 (95% CI: 0.88–1.00), IQCODE-S = 0.96 (95% CI: 0.90–1.01), IQCODE-SBr = 0.96 (95% CI: 0.90–1.01) (p ≤ 0.001). The ROC curves were also used to separate the sample (patients + controls) into two social classes (upper vs. middle + low social classes) and also into three social classes (upper, middle, and low). By dividing the sample into two or three social classes, the areas under the ROC curves for all the instruments (MMSE, IQCODE, IQCODE-S, IQCODE-SBr) ranged from 0.92 to 0.99 and remained statistically significant (p < 0.001).
ROC curves comparison.
The various IQCODE versions had good internal consistency (Cronbach's α: IQCODE-L = 0.976, IQCODE-S = 0.975, IQCODE-SBr = 0.977). Table 3 shows that the items with poorer performance, according to the ROC curves, were items 15 (area = 0.46) and 16 (area = 0.58) of the IQCODE. Item 15 is “Remembering things that happened to him/her when he/she was young”; and item 16 is “Remembering things he/she learned when he/she was young”. On the other hand, the three items with the largest area under the curve were 14, 10 and 5, which deal respectively with learning new things in general, remembering where to find things, and recalling conversations a few days later. In the current sample, item 20 (composing a letter to friends or for business purposes) of the IQCODE-L had the lowest response frequency (n = 74 responses) (Table 3), whereas the other items were responded to by 86–92 carers. Based on the analysis of areas under the ROC curves (Table 3) of individual items, new versions in Portuguese were tested, obtaining a 15-item version (IQCODE-SBr) with area under the curve equal to the original 16-item IQCODE-S (0.96). There were no more than five missing values for IQCODE-L and no more than two missing values for IQCODE S and SBr for each subject. For the IQCODE-L, 66.3% of the sample answered all the questions, 31.5% had fewer than four missing responses and 2.2% had four or five missing values. For the IQCODE-S, 88% of the sample answered all the questions, 8.7% had one missing value and 3.3% had two missing values. For the IQCODE-SBr, 90.2% of the sample answered all the questions, 6.5% had one missing value and 3.3% had two missing values.
ROC curve areas for each individual IQCODE item
a = IQCODE-S: 16 items; b = IQCODE-SBr: 15 items.
Table 4 uses the ROC curve to present sensitivity, specificity and positive and negative predictive values corrected by the prevalence of dementia in the community for a few cut-off points. The dementia prevalence estimated in the study published by Bottino et al. (Reference Bottino2008), conducted in the same city as the present study, was used as a reference (12.9%), despite being higher than previously published results on dementia prevalence. The cut-off point that showed the best balance between sensitivity and specificity for the MMSE was ≤ 26 (sensitivity = 94.12% and specificity = 77.97%). The cut-off points obtained for the three versions of the IQCODE that had the best balance between sensitivity and specificity were IQCODE-L 3.38 (sensitivity = 88.2%; specificity = 93.1%); IQCODE-S 3.53 (sensitivity = 85.3%, specificity = 100%); IQCODE-SBr 3.6 (sensitivity = 91.2%; specificity = 94.8%).
Instruments comparison
(a) 95% confidence interval.
(b) PPV and NPV corrected for a community dementia prevalence of 12.9%.
Using logistic regression, including age and gender, the MMSE correctly classified 87% of the individuals (sensitivity = 76.5%, specificity = 93.1%). On the other hand, the IQCODE-L version correctly classified 98.3% of the individuals (sensitivity = 82.4%, specificity = 98.3%), the IQCODE-S version 95.7% (sensitivity = 91.2%, specificity = 98.3%) and the IQCODE-SBr version 94.6% (sensitivity = 91.2%, specificity = 96.6%).
Discussion
In this study the comparison between IQCODE-L, IQCODE-S, IQCODE-SBr and MMSE in elderly people with low levels of education showed the superiority of IQCODE versions over the MMSE in terms of sensitivity and accuracy. It was observed that educational level had an influence on the MMSE results (as expected), but there was no significant correlation between educational level and the IQCODE, both in the long and short versions in Portuguese. Evaluation of each item alone showed that the items excluded from the long version did not compromise the instrument's performance. The data suggest that items 15 and 16, because they dealt with the past, may have compromised the IQCODE-L efficacy, with a little smaller area under the curve than the S and SBr versions, which did not include these items. This sample had a lower response frequency in item 20, probably because it evaluates a task that is much influenced by low educational level. This same item was found to have a 97% non-response rate by Tang et al. (Reference Tang2003). This finding could also explain why the 15-item version had a similar performance to longer versions that maintained item 20 (see Table 3). Thus, the 15-item version (IQCODE-SBr), which had a similar performance and shorter application time than the other IQCODE versions tested, can be considered the most appropriate for application in a population such as that assessed in the present study.
Educational level in the sample was homogeneously low, with means of 5.44 years of schooling for the patients, and 4.78 years for the controls. Therefore, it differs from studies such as those conducted in Italy (Isella et al., Reference Isella, Villa, Frattola and Appollonio2002), in a sample that had a higher level of education (7 years), and in China (Fuh et al., Reference Fuh1995), in which mean educational level was low (controls = 1.6 years, patients = 6.6 years). However, in the study performed in China (Fuh et al., Reference Fuh1995), there was a large variation between minimum and maximum educational level, especially in cases (controls = 0–15 years; cases = 0–19 years), and 63% of the sample comprised illiterate individuals. In the study by Fuh et al. (Reference Fuh1995) on the 26-item version of the IQCODE, only nine items could be excluded with no significant impact on sensitivity and specificity, and both the long (26 items) and short (17 items) versions had quite similar diagnostic accuracies. The authors proposed that, in population samples with a low level of education, items assessing writing and reading skills could be excluded without impairment to test performance. Two items alone had excellent accuracy as a screening test, namely, recalling a conversation a few days later, and handling financial matters, e.g. one's pension, dealing with the bank. The Chinese IQCODE did not show any association with educational level or gender and had a weak association with age; therefore, it can be useful in populations with large variations of educational level (Fuh et al., Reference Fuh1995). The sample assessed by Senanarong et al. (Reference Senanarong2001) had a mean educational level of 4 years, very similar to the sample used in this study, and the diagnosis was performed independently according to DSM-IV criteria. Logistic regression showed that only three items contributed significantly to the diagnosis: remembering what day and month it was; learning to use a new gadget; and handling everyday arithmetic problems. The IQCODE showed an area under the ROC curve of 0.928 vs. 0.814 for the MMSE, similar to the results obtained in this study.
Using the short IQCODE in its Spanish version, Morales et al. (Reference Morales, Gonzalez-Montalvo, Bermejo and Del-Ser1997) compared the instrument in two different samples, one urban and the other rural. The authors did not report educational level for the rural sample, but 78.2% of the individuals were illiterate. Mean educational level was 4.9 years for the urban sample. The IQCODE was better than the MMSE in both samples and there was no influence of age, educational or cultural level, in contrast to what occurred in the MMSE.
The cut-off points for this study were IQCODE-L = 3.38, IQCODE-S = 3.53, and IQCODE-SBr = 3.6, which are very similar to those suggested in the literature (Jorm, Reference Jorm1994; Jorm et al., Reference Jorm1996; Del-Ser et al., Reference Del-Ser, Morales, Barquero, Canton and Bermejo1997; Harwood et al., Reference Harwood, Hope and Jacoby1997), as can be seen in Table 5.
Review of papers with short versions of the IQCODE
na = not available.
Limitations of the study include the relatively small sample with low mean educational level, but with few illiterate elderly individuals. Data about the educational level of informants were not collected, and this should be addressed in further studies since it may influence the IQCODE results. The control group taken from the Geriatrics Clinic of HCFMUSP did not come from a community-based sample, but they represent the elderly subjects attending our hospital. This sample may comprise informants who are more aware of health changes and who may put too much emphasis on small changes in the elderly, inflating the result of the IQCODE versions used in this study.
In summary, the results suggest that the long and short Brazilian versions of the IQCODE can be used for the screening of mild to moderate cases of dementia. The IQCODE is not influenced by educational level, which makes it an adequate instrument for use in Brazil, a country that has a sizeable proportion of elderly subjects with low educational attainment, and useful for other samples with a similar educational profile. New studies should be conducted to confirm the findings obtained with the new short Brazilian version (IQCODE-SBr) of only 15 items but with a similar performance to that of the longer versions.
Conflict of interest
None.
Description of authors' roles
Dr. Perroco participated in data analysis, and wrote the paper. Dr. Zevallos Bustamante participated in data collection, data analysis and assisted with writing. Ms. Moreno and Drs. Hototian, Lopes and Azevedo participated in data collection. Professor Litvoc participated in data analysis and assisted with writing. Professor Jacob Filho supervised patient recruitment and assisted with writing. Professor Bottino designed the study, obtained funding, supervised the data collection, participated in data analysis and assisted with writing.
Acknowledgements
The authors thank FAPESP (Grant Nr. 01/05959–7) for financial support and Eduardo Yoshio Nakano for statistical assistance.
