Introduction
The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) is a questionnaire which asks an informant about changes in a person's everyday cognitive function over a 10-year period (Jorm and Jacomb, Reference Jorm and Jacomb1989). Studies have suggested that the IQCODE is as good as the Mini-mental State Examination (MMSE) in the diagnosis of dementia (Jorm et al., Reference Jorm, Scott, Cullen and MacKinnon1991), but unlike the MMSE and other cognitive tests it is relatively unaffected by patient education and pre-morbid ability. In light of its performance, the IQCODE has been widely adopted in clinical practice (Jorm, Reference Jorm2004). However, a recent review of screening tests for cognitive impairment (Cullen et al., Reference Cullen, O'Neill, Evans, Coen and Lawlor2007) identified only one study that met the authors' selection criteria which examined the diagnostic utility of the IQCODE specifically in a memory clinic setting (Flicker et al., Reference Flicker, Logiudice, Carlin and Ames1997). In that study, using a cut-off of 3.9 (IQCODE score range = 1–5), test sensitivity was 0.74 and specificity 0.71. Although receiver operating characteristic (ROC) curves were constructed, the area under the curve (AUC), a measure of diagnostic accuracy, was not reported in this paper. In another memory clinic-based study examining the IQCODE, area under the ROC curve was 0.82 but test sensitivities and specificities were not given (Stratford et al., Reference Stratford, LoGiudice, Flicker, Cook, Waltrowicz and Ames2003).
As there are few data reported on the use of the IQCODE in memory clinic-based populations, we undertook a study to examine its diagnostic utility in our clinics, where the prevalence of dementia in new patient referrals is around 50%. We also looked at combining the IQCODE with other tests, specifically the Addenbrooke's Cognitive Examination-Revised (ACE-R) (Mioshi et al., Reference Mioshi, Dawson, Mitchell, Arnold and Hodges2006), a widely adopted test for finding dementia cases, which we have experience of using in our clinics (Larner, Reference Larner2007) and which incorporates the Mini-mental State Examination (MMSE; Folstein et al., Reference Folstein, Folstein and McHugh1975). Correlation of the IQCODE with the Patient Health Questionnaire-9 (PHQ-9), a recently described and validated instrument for measurement of depression severity (Kroenke et al., Reference Kroenke, Spitzer and Williams2001), which has proved useful as a brief screen for depression in patients attending dementia clinics (Hancock and Larner, Reference Hancock and Larner2009), was also examined in a subgroup of patients.
The questions we sought to answer in this study were (i) is the IQCODE useful in the diagnosis of dementia in a memory clinic population? (ii) is its diagnostic utility improved by combination with the ACE-R and MMSE? and (iii) are IQCODE scores correlated with a measure of patient depression (PHQ-9)?
Methods
Consecutive new patient referrals who attended with an informant were recruited prospectively over a 12-month period (July 2007–July 2008) from a memory clinic based in a psychiatric hospital and from a cognitive function clinic based in a regional neuroscience centre. Standard clinical diagnostic criteria (DSM-IV) were used for the diagnosis of dementia (American Psychiatric Association, 2000) as in previous collaborative studies between these two units (Hancock and Larner, Reference Hancock and Larner2007; Reference Hancock and Larner2008a; Reference Hancock and Larner2008b; Reference Hancock and Larner2009). As this was a pragmatic study, patients were not selected according to diagnosis but simply as they presented to the clinics. Neither dementia subtypes nor mild cognitive impairment (MCI) were specifically examined in this study.
Informants were invited to complete the IQCODE; the long form (Jorm and Jacomb, Reference Jorm and Jacomb1989) rather than the short form (Jorm, Reference Jorm1994) was used in order to obtain results comparable with the aforementioned study by Flicker et al. (Reference Flicker, Logiudice, Carlin and Ames1997). IQCODE administration was performed independent of, but on the same day as, patient clinical and neuropsychological assessment but its result was not used in the diagnostic judgment of dementia/no dementia in order to minimize review bias (Gifford and Cummings, Reference Gifford and Cummings1999). ACE-R and/or MMSE was administered to most patients.
Standard summary measures of diagnostic utility were generated (sensitivity, specificity, positive predictive value (PPV), diagnostic odds ratio (DOR), likelihood ratios, ROC curve) with 95% confidence intervals (CI). The STARD guidelines on reporting diagnostic test accuracy were observed (Bossuyt et al., Reference Bossuyt2003).
Correlations and agreement (kappa statistic; Cohen, Reference Cohen1960) were calculated between IQCODE and ACE-R, and IQCODE and MMSE. Diagnostic utility measures were also derived for combined use of the IQCODE, at its optimal cut-point as defined in this study population, with the ACE-R and with the MMSE at their optimal cut-points as previously defined in these clinics, namely 73/100 and 24/30 respectively (Larner, Reference Larner2009) in comparison with clinical diagnoses. Tests were applied both in series (both tests required to be positive before a diagnosis of dementia made: “And” rule) or in parallel (either test positive sufficient for a diagnosis of dementia to be made: “Or” rule) (Flicker et al., Reference Flicker, Logiudice, Carlin and Ames1997).
As administration of the IQCODE, ACE-R, MMSE and PHQ-9 was already routine in our clinics, this study was an audit of established practice, so institutional ethical review and specific consent procedures were not indicated.
Results
Over the study period, 144 patients with informants were assessed (male: female ratio = 70: 74, 51% female; age range 29–94 years, median 67 years). Of these, 85 (59%) were judged to have dementia by DSM-IV criteria and 59 (41%) had no dementia. This dementia prevalence was higher than the approximately 50% recorded in other patient cohorts from these clinics (Hancock and Larner, Reference Hancock and Larner2007; Reference Hancock and Larner2008b; Reference Hancock and Larner2009) but similar to that recorded in a previous study which examined an informant questionnaire (Hancock and Larner, Reference Hancock and Larner2008a), presumably reflecting the fact that individuals with no dementia are more likely to attend these clinics without an informant (Larner, Reference Larner2005).
The IQCODE proved easy to use, being completed in all cases (up to three missing items were permitted).
In the dementia group the mean (+/–SD) IQCODE score was 4.10+/–0.43, and in the no dementia group 3.76+/–0.44. The mean IQCODE scores differed significantly between the two groups (t = 4.52, df = 142, p < 0.001).
The sensitivity and specificity of IQCODE scores were examined at cut-off values between 3.0 and 4.6. Best sensitivity for a diagnosis of dementia was 0.99 at a cut-off of ≥3.0 (with specificity = 0.07); best specificity was 1.0 at a cut-off of ≥4.5 (with sensitivity = 0.21). Optimal IQCODE cut-off, defined by the maximal test accuracy for the differential diagnosis of dementia/not dementia (= 0.67), was ≥ 3.6. At this cut-off, traditional parameters of test diagnostic utility were calculated (Table 1, column 1), showing sensitivity of 0.86, specificity of 0.39, PPV of 0.67, with DOR of 3.89. Diagnostic gain as measured by positive and negative likelihood ratios (LR) was unimportant (LR+ = 1.41) or small (LR– = 0.36). A ROC curve was constructed (Figure 1): AUC, a measure of diagnostic accuracy, was 0.71 (95% CI = 0.62–0.79), where 0.5 indicates a test providing no added information, 1 indicates a test providing perfect discrimination, and a value of >0.75 is thought desirable for a diagnostic test.
Diagnostic parameters for IQCODE, IQCODE + MMSE and IQCODE + ACE-R in series and in parallel (with 95% confidence intervals)
IQCODE = Informant Questionnaire on Cognitive Decline in the Elderly; ACE-R = Addenbrooke's Cognitive Examination-Revised; MMSE = Mini-mental State Examination; PPV = positive predictive value; NPV = negative predictive value; DOR = diagnostic odds ratio; LR = likelihood ratio.
IQCODE receiver operating characteristic (ROC) curve.
Many of the patients were administered the ACE-R (n = 114) and/or the MMSE (n = 132) at the same time as an informant completed the IQCODE. The correlation between IQCODE and MMSE was r = −0.37 (t = 4.49, df = 130, p < 0.001). Using the test of agreement (kappa statistic), which measures the percentage of agreement beyond chance, κ = 0.23 (95% CI = 0.07–0.39), where 1 is perfect agreement between tests and 0 is agreement purely due to chance alone (Cohen, Reference Cohen1960). For IQCODE and ACE-R, correlation was r = −0.46 (t = 5.46, df = 112, p < 0.001) and κ = 0.29 (95% CI = 0.11–0.46).
Results of using the IQCODE in combination with MMSE (n = 132) or ACE-R (n = 114) in series or in parallel (Table 1) showed the expected improvement in specificity in the series (“And” rule) paradigm, with some reduction in sensitivity but with improved overall accuracy, PPV, DOR, and LR+. There was little difference between results combining IQCODE and MMSE versus IQCODE and ACE-R, with a marginal advantage for ACE-R. In the parallel (“Or” rule) paradigm, there was the expected improvement in sensitivity, but no change in accuracy, specificity or PPV.
Administration of the PHQ-9 to patients (n = 58) at the same time as an informant completed the IQCODE allowed correlation between these scales to be calculated (r = 0.31, t = 2.4, p ≈ 0.02). Previous studies have shown a small positive association between the IQCODE and measures of anxiety, depression and general psychological distress (Jorm, Reference Jorm2004).
Discussion
The overall results of this study of diagnostic accuracy of the IQCODE in a memory clinic based population were disappointing: although test sensitivity was better than in the previously reported study of Flicker et al. (Reference Flicker, Logiudice, Carlin and Ames1997) (0.86 vs. 0.74), specificity was inferior (0.39 vs 0.71). AUC was inferior to that in the previously reported study of Stratford et al. (Reference Stratford, LoGiudice, Flicker, Cook, Waltrowicz and Ames2003) (0.71 vs. 0.82). Different composition of clinic populations might have contributed to the different findings; for example, our sample was younger (mean age 67.7 +/−11.4 years) compared to the populations reported on in previous publications (73.4+/−9.3 years in Flicker et al., Reference Flicker, Logiudice, Carlin and Ames1997; 72.9 years in Stratford et al. Reference Stratford, LoGiudice, Flicker, Cook, Waltrowicz and Ames2003), and dementia prevalence was lower (59% vs. 72% and 64%, respectively) suggesting a different case mix. The latter difference may explain the weaker correlation between IQCODE and MMSE than in the previous studies examining memory clinic population (−0.37 vs −0.56 and −0.57). Sampling error, due to the smaller number of patients examined (144 vs. 299 and 577) might also be a contributory factor.
Diagnostic issues may also have influenced the results. Cases of MCI might have contributed to the low specificity, since MCI may be due to dementing disorders which have not yet fulfilled diagnostic criteria for dementia, and hence fall within the no dementia group. The ability of the IQCODE to reflect the differing symptomatology of different causes of dementia – such as Alzheimer's disease, frontotemporal lobar degenerations, and dementia with Lewy bodies – might also contribute to the results. Shortcomings of the IQCODE itself might also play a role, such as the influence of informant characteristics, and the quality of the relationship between informant and subject, which may influence the validity of the data (Jorm, Reference Jorm2004).
Compared with another informant questionnaire, the Cambridge Behavioural Inventory (Wedderburn et al., Reference Wedderburn2008), which has previously been examined in a demographically similar cohort in these clinics (Hancock and Larner, Reference Hancock and Larner2008a), the IQCODE had similar accuracy at the chosen cut-offs, with poorer sensitivity, PPV, DOR, and LR+, but with better specificity, negative predictive value and LR–. Of course, these summary parameters will be dependent on the cut-off chosen: it has been suggested that IQCODE cut-offs should be selected according to the particular application for which the test is being used (Jorm, Reference Jorm2004). In that case, a low IQCODE cut-off (≥3.0) will give excellent sensitivity for the diagnosis of dementia in this population, and a high cut-off (≥4.5) excellent specificity.
It has previously been questioned whether use of the IQCODE in combination with, rather than in isolation from, an education-dependent test such as the MMSE, might add to diagnostic utility since these tests are complementary (Jorm, Reference Jorm2004; Flicker et al., Reference Flicker, Logiudice, Carlin and Ames1997). This combination has also been examined in community samples with the IQCODE short form (Mackinnon et al., Reference Mackinnon, Khalilian, Jorm, Korten, Christensen and Mulligan2003). To our knowledge, this study is the first to examine the ACE-R in combination with the IQCODE. We looked at series (“And” rule) and parallel (“Or” rule) test use, but did not examine logistic regression since this is difficult to implement in clinical practice. Our results suggested that in this memory-clinic based cohort such a combination applied in series, using either the MMSE or the ACE-R, greatly improved specificity with little loss in sensitivity and greater overall accuracy and PPV. Indeed PPV approached the 90% thought desirable for diagnostic tests (Ronald and Nancy Reagan Research Institute of the Alzheimer's Association and the National Institute on Aging Working Group, 1998). Hence, series combination of the IQCODE and the ACE-R/MMSE may be a simple method for identification of dementia in the memory clinic setting.
Conflict of interest
None.
Description of authors' roles
Both authors conceived and planned the study and collected and analyzed the data. AJL drafted the manuscript and PH critically revised the manuscript for important intellectual context.
