Introduction
Diagnostic or screening tools should be able to correctly classify participants into healthy or diseased cases. However, this process is not always dichotomous. Pre-diseased or intermittent stages usually exist in many disease processes as patients progress from the healthy stage to the fully diseased stage. This transitional stage may represent an optimal treatment window before the disease is fully developed.
For the field of dementia, mild cognitive impairment (MCI) represents a transitional zone between cognitively normal (CN) and dementia (Petersen et al., Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999). An increasing number of screening tools have been developed for cognitive dysfunction (Jorm et al., Reference Jorm, Scott, Cullen and MacKinnon1991; Cherbuin et al., Reference Cherbuin, Anstey and Lipnicki2008). A good screening tool requires good discriminating between one stage from the other two stages of cognitive dysfunction. However, most screening tools have been evaluated only for the ability to discriminate dichotomously, such as between CN and dementia, cognitive normal and MCI, or MCI and dementia. In clinical practice, these dichotomous abilities have not been useful to classify multiple stages of cognitive dysfunction simultaneously.
The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) is a commonly used informant questionnaire to screen for cognitive dysfunction (Jorm and Korten, Reference Jorm and Korten1988; Jorm et al., Reference Jorm, Scott, Cullen and MacKinnon1991; Morales et al., Reference Morales, Bermejo, Romero and Del-Ser1997; Jorm, Reference Jorm2004; Isella et al., Reference Isella, Villa, Russo, Regazzoni, Ferrarese and Appollonio2006; Tokuhara et al., Reference Tokuhara, Valcour, Masaki and Blanchette2006; Ehrensperger et al., Reference Ehrensperger, Berres, Taylor and Monsch2010; Sikkes et al., Reference Sikkes2010; Cruz-Orduna et al., Reference Cruz-Orduna2012; Lin et al., Reference Lin, O'Connor, Rossom, Perdue and Eckstrom2013; Quinn et al., Reference Quinn, Fearon, Noel-Storr, Young, McShane and Stott2014), the IQCODE has performed at least as well as cognitive screening tests, such as the Mini-Mental State Examination (MMSE). Moreover, an important advantage of the IQCODE over the MMSE is that it is not affected by the patient's premorbid intelligence or education (Jorm et al., Reference Jorm, Scott, Cullen and MacKinnon1991).
There have been reported many studies evaluated the IQCODE's screening for dementia (Morales et al., Reference Morales, Bermejo, Romero and Del-Ser1997; Jorm, Reference Jorm2004; Tokuhara et al., Reference Tokuhara, Valcour, Masaki and Blanchette2006; Ehrensperger et al., Reference Ehrensperger, Berres, Taylor and Monsch2010; Sikkes et al., Reference Sikkes2010; Cruz-Orduna et al., Reference Cruz-Orduna2012; Lin et al., Reference Lin, O'Connor, Rossom, Perdue and Eckstrom2013) or MCI (Isella et al., Reference Isella, Villa, Russo, Regazzoni, Ferrarese and Appollonio2006; Tokuhara et al., Reference Tokuhara, Valcour, Masaki and Blanchette2006; Ehrensperger et al., Reference Ehrensperger, Berres, Taylor and Monsch2010; Sikkes et al., Reference Sikkes2010; Lin et al., Reference Lin, O'Connor, Rossom, Perdue and Eckstrom2013). However, no previous study has been published concerning the IQCODE's ability to simultaneously discriminate between multiple stages of cognitive dysfunction. This study, therefore, aimed to explore if the IQCODE, as a screening tool, can accurately classify individuals, not only as whether they have or do not have cognitive dysfunction, but also to place them in the correct category, namely, CN, MCI, or dementia.
Materials and methods
Participants
This study utilized a matched samples design. A total of 228 participants were recruited: 76 participants with dementia, 76 participants with MCI, and 76 participants who were CN. All subjects visited a memory clinic at the University Hospital in the Republic of South Korea from January 2008 to October 2016 and were referred for neuropsychological testing. All participants attended by an informant were included. Standard clinical diagnostic criteria were used to diagnose MCI (Winblad et al., Reference Winblad2004) and dementia (APA, 2000). CN individuals did not meet the criteria for MCI or dementia. All subjects from the three diagnostic groups were age-matched by three-year bands, gender, and education by five-year bands. Consensus diagnosis by a neurologist and neuropsychologist was used to determine the clinical status of each individual based on neuropsychological testing results, neurological and physical exams, and interviews with an informant who assessed global cognitive status, functional status, mood, and behavioral status. Neither dementia subtypes nor MCI subtypes were specifically examined in this study. Informants were the spouses or relatives of individuals who lived in the same household and reported no psychiatric or neurological disease themselves. Subjects with pre-existing conditions such as intellectual disabilities, drug and/or substance abuse, and severe psychiatric illness that might affect their performance on cognitive measures were excluded. The Institutional Review Board at our institute approved this study. The need to obtain informed consent was waived by the board.
Procedure
The cognitive screening procedure included medical history, informant-based history, physical and neurological examination, laboratory tests, brain magnetic resonance imaging, and a battery of neuropsychological tests, including the IQCODE (Jorm and Korten, Reference Jorm and Korten1988), the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975) and the Clinical Dementia Rating (CDR) (Morris, Reference Morris1993).
The IQCODE was independently completed by the informant, and the study clinicians were blinded to the scores. The IQCODE is a reliable, validated informant-based instrument with the goal of assessing change in cognitive functioning over ten years and has been widely adopted by clinical researchers across different cultures and languages (Jorm and Korten, Reference Jorm and Korten1988). This study is based on the 26-item Korean version (Lee et al., Reference Lee2005) and the cognitive changes are scored on a five-point scale, with 1 indicating “much improved,” 3 “not much change,” and 5 “much worse.” Higher scores correspond to greater cognitive decline. The ratings are averaged, composing a mean score between 1 and 5. In this study, the IQCODE was not used in establishing the clinical diagnosis.
The MMSE is a widely used instrument used to obtain an estimate of global cognitive function and screen for dementia (Folstein et al., Reference Folstein, Folstein and McHugh1975; Han et al., Reference Han, Jo, Jo, Kim, Park and Kang2008). It consists of 30 items assessing orientation, attention/working memory, memory, language, and visuospatial skills. It has been shown to effectively discriminate participants with dementia from those without dementia and is sensitive to progressive deterioration in dementing patients. The maximum score is 30 points, and a higher score indicates better cognitive functioning.
The CDR was used to determine the presence or absence of cognitive dysfunction and, if present, stage its severity (Morris, Reference Morris1993). The CDR assesses a patient's cognitive and functional performance in the areas of memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care. CDR rating was based upon the subject's performance on neuropsychological testing and his/her history of clinical symptoms as provided by the informant.
Statistical analysis
Clinical characteristics were evaluated using χ 2 tests for the difference between proportions, and one-way analysis of variance (ANOVA) or non-parametric analysis of variance (Kruskal–Wallis) was used to test the difference between continuous scales after checking Levene's test for equality of variance. The Bonferroni correction was used for post-hoc comparisons. For convergent validity, the correlation between the IQCODE and other neuropsychological tests was calculated using Spearman correlation analysis.
The area under the curve (AUC) can be used as a measure of the dichotomous screening ability of the test and for choosing an optimal cut-off point that provides the largest AUC. Screening ability was computed to distinguish between dementia versus non-dementia (MCI or CN) and between MCI versus CN. The corresponding sensitivity, specificity, AUC, and predictive values were subsequently calculated. Differences between the AUCs were analyzed using the Hanley and McNeil correction (Hanley and McNeil, Reference Hanley and McNeil1983).
For the multi-category classification, receiver operating characteristic (ROC) extends these ideas to instances in which there are more than two diagnostic or screening categories and the volume under the ROC surface (VUS), an extension of the AUC, has been proposed (Ferri et al., Reference Ferri, Hernández-Orallo, Salido, Lavrač, Gamberger, Todorovski and Blockeel2003; Xiong et al., Reference Xiong, van Belle, Miller and Morris2006; Murray and McKenzie, Reference Murray and McKenzie2014). Optimal cut-off points can be selected based on either the ROC surface or the Youden index. Optimal cut-off points can be computed based on the ROC surface by finding the coordinate on the surface that minimizes the Euclidean distance from the point representing perfect classification for all categories. Alternatively, use of the Youden index selects the pair of cut-off points that maximizes the overall classification accuracy by maximizing the equation (x + y + z −1) / 2, where x, y, and z are the respective probabilities of correct classification for each of the different categories. The Youden index takes values between 0 and 1, with 1 representing a test with perfect diagnostic accuracy. The optimal cut-off points suggested by these two methods do not tend to be the same unless x, y, and z are equal. VUSs were compared using the method of omnibus ANOVA with Wald test (Xiong et al., Reference Xiong, van Belle, Miller and Morris2006). The classification accuracy reflects the percentage of subjects correctly classified (number of correct classification/sample size) (Choi and Park, Reference Choi and Park2016). In this study, the classification accuracy showed the percentage of correct classification for CN, MCI, and dementia.
Analyses were carried out using SPSS for Windows, version 20.0 (IBM Corporation, Armonk, NY, USA) and the R 3.2.4 software (The R Foundation for Statistical Computing). Statistical tests were two-tailed, and α was set at <0.05.
Results
Participant characteristics
Participant characteristics are summarized by each stage of cognitive dysfunction in Table 1. The mean age of the participants was 66.6 ± 9.1 years with 8.0 ± 4.2 years of education, and the sample was 40.8% female. The mean MMSE score of the participants was 23.2 ± 6.1 with 3.53 ± 0.53 on the IQCODE and 0.60 ± 0.47 on the CDR. As expected, MMSE, CDR, and IQCODE scores differed across each stage.
Demographic and clinical characteristics
CN, cognitively normal; MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; CDR, clinical dementia rating; IQCODE, informant questionnaire on cognitive decline in the elderly.
Correlations between IQCODE and other neuropsychological tests
Correlations with other neuropsychological tests were significant. The IQCODE scores were found to be negatively correlated with the MMSE scores (r = −0.655, p < 0.001) and positively correlated with the CDR scores (r = 0.663, p < 0.001).
Discriminating dichotomous stages of cognitive dysfunction
ROC curves were drawn for dementia versus non-dementia, as well as MCI versus CN, to determine the dichotomous discriminatory ability of IQCODE (Figure 1). AUC for dementia and MCI by the IQCODE were 0.91 (95% confidence interval [CI], 0.86–0.95) and 0.71 (95% CI, 0.62–0.70), respectively. These results show that the IQCODE differentiated these groups well, the corresponding values for the MMSE and CDR were presented in Figure 1. When discriminating dementia and MCI, the AUCs of IQCODE were not statistically different from these of MMSE and CDR (all p > 0.05). Table 2 presented the optimal cut-off points and various parameters from dichotomous screening.
Receiver operating characteristic (ROC) curve for the dichotomous screening for dementia from non-dementia (A) and MCI from CN.
MCI, mild cognitive impairment; CN, cognitively normal; AUC, area under the curve; IQCODE, Informant Questionnaire on Cognitive Decline in the Elderly; MMSE, Mini-Mental State Examination; CDR, Clinical Dementia Rating.
Dichotomous screening ability for the IQCODE
MCI, mild cognitive impairment; CN, cognitively normal; AUC, area under the curve; PPV, positive predictive value; NPV, negative predictive value; LR, likelihood ratio.
Discriminating three stages of cognitive dysfunction
The ROC surface for the IQCODE as a measure for discriminating CN, MCI, and dementia was analyzed. The VUS for the IQCODE was estimated as 0.60 (95% CI, 0.56–0.64). The plots in Figure 2 suggest that there was a separation of the distribution of scores between CN, MCI, and dementia, and the derived optimal cut-off points were 3.15 and 3.73. The Youden index for the IQCODE was estimated as 0.51 (95% CI, 0.47–0.54) and the derived optimal cut-off points were 3.33 and 3.70.
Optimal cut-off points of IQCODE for control, MCI, and dementia.
CN, cognitively normal; MCI, mild cognitive impairment; VUS, volume under the surface.
VUSs were constructed to measure the properties of the MMSE, CDR, and IQCODE in classifying three stages of cognitive dysfunction (Figure 3). The VUS of the IQCODE was 0.61 (95% CI, 0.53–0.69) and those of the MMSE and the CDR were 0.64 (95% CI, 0.56–0.71) and 0.64 (95% CI, 0.56–0.72), respectively. The VUS of the IQCODE did not differ from those of the MMSE and the CDR (all, p > 0.05). This result showed that the discriminatory ability of the IQCODE was not lower than those of the MMSE and the CDR in this study.
Receiver operating characteristic (ROC) surface plot of MMSE, CDR, and IQCODE.
IQCODE, Informant Questionnaire on Cognitive Decline in the Elderly; MMSE, Mini-Mental State Examination; CDR, Clinical Dementia Rating; VUS, volume under the surface.
Table 3 presents the overall classification accuracy, expressed as the proportion of correctly classified subjects. The overall classification accuracy by VUS was 58.3% and that by the Youden index was 61.8%.
Multi-category classification accuracy for the IQCODE
CN, cognitively normal; MCI, mild cognitive impairment; VUS, volume under the surface.
Discussion
This study investigated whether the IQCODE was able to classify not only dichotomous but also three stages of cognitive dysfunction. This study showed that the IQCODE was able to discriminate dichotomously between dementia and non-dementia, as well as MCI and CN. Moreover, the IQCODE was able to simultaneously discriminate three stages of cognitive dysfunction as CN, MCI, and dementia.
The IQCODE could prove useful in most clinical practice because of its ease of use. In this study, the correlations observed between the IQCODE and other neuropsychological tests proved statistically significant concurrent validity. The IQCODE is informant-based questionnaire, while the MMSE is a test performed directly with the patient. Previous studies have reported the correlation between the IQCODE and MMSE (Jorm, Reference Jorm2004; Lee et al., Reference Lee2005; Hancock and Larner, Reference Hancock and Larner2009; Razavi et al., Reference Razavi2014). The CDR is a structured interview with the patient and a reliable informant and it is a valid tool for assessing the severity of cognitive dysfunction (Lowe et al., Reference Lowe, Balsis, Miller, Benge and Doody2012). The discriminatory ability of the IQCODE was not different from those of the MMSE and the CDR by comparing VUSs. This study adds to the literature, supporting the IQCODE as a useful screening tool for cognitive dysfunction.
Most studies support the validity of the IQCODE in dementia screening, and some studies have evaluated its validity in MCI screening (Jorm and Korten, Reference Jorm and Korten1988; Jorm et al., Reference Jorm, Scott, Cullen and MacKinnon1991; Morales et al., Reference Morales, Bermejo, Romero and Del-Ser1997; Jorm, Reference Jorm2004; Isella et al., Reference Isella, Villa, Russo, Regazzoni, Ferrarese and Appollonio2006; Tokuhara et al., Reference Tokuhara, Valcour, Masaki and Blanchette2006; Ehrensperger et al., Reference Ehrensperger, Berres, Taylor and Monsch2010; Sikkes et al., Reference Sikkes2010; Cruz-Orduna et al., Reference Cruz-Orduna2012; Quinn et al., Reference Quinn, Fearon, Noel-Storr, Young, McShane and Stott2014). The cut-off points suggested in the literature ranges from 3.30 to 4.00 for detecting dementia (Jorm, Reference Jorm2004; Lin et al., Reference Lin, O'Connor, Rossom, Perdue and Eckstrom2013) and from 3.19 to 3.30 for discriminating MCI from controls (Isella et al., Reference Isella, Villa, Russo, Regazzoni, Ferrarese and Appollonio2006; Tokuhara et al., Reference Tokuhara, Valcour, Masaki and Blanchette2006; Lin et al., Reference Lin, O'Connor, Rossom, Perdue and Eckstrom2013). This study showed that the cut-off point between MCI and CN was 3.15–3.33 and that between dementia and non-dementia was 3.42–3.73. These patterns of cut-off points are consistent with those reported in previous studies on detecting cognitive dysfunction (Jorm, Reference Jorm2004; Isella et al., Reference Isella, Villa, Russo, Regazzoni, Ferrarese and Appollonio2006; Tokuhara et al., Reference Tokuhara, Valcour, Masaki and Blanchette2006; Lin et al., Reference Lin, O'Connor, Rossom, Perdue and Eckstrom2013). However, previous studies have evaluated only the ability to discriminate dichotomously. This study additionally evaluated the ability to simultaneously discriminate the three stages of cognitive dysfunction as CN, MCI, and dementia. This study suggested that these cut-off points could be used as screening and classification tools for not only dichotomous but also multiple stages of cognitive dysfunction in clinical practice.
This study showed that the classification parameters were relatively low or variable in MCI compared with those in dementia. Previous studies reported that the IQCODE can discriminate dementia successfully (Jorm, Reference Jorm2004). However, there exists controversy about the ability of the IQCODE to discriminate subjects with MCI from CN. Some studies have reported that the IQCODE effectively differentiates MCI from CN (Isella et al., Reference Isella, Villa, Russo, Regazzoni, Ferrarese and Appollonio2006; Ehrensperger et al., Reference Ehrensperger, Berres, Taylor and Monsch2010), but others have reported that it is less successful in identifying participants with MCI (Sikkes et al., Reference Sikkes2010; Razavi et al., Reference Razavi2014). The IQCODE was designed to assess changes in daily activities related to memory and intelligence, which were closely related to each other in cognitive dysfunction suffered by dementia (Jorm and Korten, Reference Jorm and Korten1988; Jorm, Reference Jorm2004). Patients with MCI may report difficulties concerning complex activities (Winblad et al., Reference Winblad2004). Such complex activities are not covered in the IQCODE questionnaire. The low complexity of the IQCODE items might explain why the IQCODE had the different properties of classification parameters on MCI compared to dementia.
Limitations of this study include that the sample was not representative of the entire population, covering only those who visit clinics. The relatively small sample size and matched samples design are another weakness of this study. The subtypes of MCI and dementia were not identified in this study. It might be possible that the discriminatory ability of the IQCODE differs across subtypes of MCI or dementia. Furthermore, it remains to be determined whether the short version of the IQCODE can classify multiple stages of cognitive dysfunction. The combination of a cognitive test has been known as a useful approach to screening for cognitive dysfunction. However, the combination of test is outside the scope of this study. Further study may expand on this study by including a combination of test to improve screening ability for cognitive dysfunction, including MCI.
In conclusion, this is the first study to investigate the ability of the IQCODE to simultaneously discriminate between CN, MCI, and dementia, a frequently encountered situation in clinical practice. The IQCODE is an easy, self-administered informant questionnaire, which is a useful to simultaneously distinguish not only dichotomous but also multiple stages of cognitive dysfunction. Because of its ease of use for classifying of cognitive dysfunction, the IQCODE could prove useful in clinical practice.
Conflict of interest
None.
Acknowledgments
We would like to thank all who made the study possible, especially the patients participating in this study. This study was supported by a grant from Korea University (K1609681).
