Introduction
Delirium represents a common and severe problem among hospitalized elderly people, affecting as much as 50% of these patients (Inouye et al., Reference Inouye, Westendorp and Saczynski2013). This syndrome is characterized by the rapid onset of symptoms which fluctuate, a disturbance of consciousness, with reduced ability to focus, sustain or shift attention, accompanied by global disturbance of cognition or perceptual abnormalities and evidence of a physical cause, substance intoxication/withdrawal or multiple etiologies (American Psychiatric Association, 2000).
The development of delirium in elderly people is usually multifactorial, resulting from a complex inter-relationship between vulnerable patients with several predisposing factors (e.g. dementia, multi-morbidity) and exposure to precipitating risk factors (e.g. poly-pharmacy, physical restraints) (Inouye and Charpentier, Reference Inouye and Charpentier1996).
It has also been recognized that delirium is associated with numerous negative outcomes, including increased length of hospital stay, risk of death, institutionalization and dementia (Leentjen et al., Reference Leentjens2012). Because of its clinical effects, this syndrome has important implications for healthcare utilization and costs, amounting to over 182 billion dollars per year in 18 European countries (including Portugal) (Inouye et al., Reference Inouye, Westendorp and Saczynski2013).
Despite this, delirium is often unrecognized, with non-detection rates as high as 69%. Failure to diagnose delirium potentially compromises patient's safety and may have downstream implications for clinical care and patient health, delaying identification and treatment of underlying medical illness (Inouye et al., Reference Inouye, Foreman, Mion, Katz and Cooney2001; McLafferty and Farley, Reference McLafferty and Farley2007).
Some studies have identified factors associated with unrecognized delirium. One of them (Inouye et al., Reference Inouye, Foreman, Mion, Katz and Cooney2001), identified four independent risk factors: hypoactive delirium, advanced age, vision impairment and dementia. Patients with three or four risk factors had a 20-fold risk of under-recognition. Some other factors have also been identified: the fluctuation course of delirium symptoms, the lack of knowledge about delirium and its identification, as well as the failure to consistently use standardized screening instruments in daily clinical practice (Inouye, Reference Inouye2006). Most health professionals recognize that delirium is a seriously under-diagnosed problem, but only a minority routinely screens for delirium and few use a specific tool for assessment (Ely et al., Reference Ely2004).
The use of these instruments in clinical practice can help not only in the detection of delirium, but also in rating clinical improvement and evaluating the effectiveness of various interventions (Grover and Kate, Reference Grover and Kate2012), contributing to the enhancement of patient outcomes and a decrease in the associated burden (Hughes et al., Reference Hughes, Brummel, Vasilevskis, Girard and Pandharipande2012).
Therefore, a variety of screening tools have been developed. Among them, the confusion assessment method (Inouye et al., Reference Inouye, van Dyck, Alessi, Balkin, Siegal and Horwitz1990) has been considered to be the most useful diagnostic or screening instrument for several reasons, including its brevity, and ease of use by trained non-psychiatric clinicians in both clinical and research settings (Wong et al., Reference Wong, Holroyd-Leduc, Simel and Straus2010).
It was originally developed based on the Diagnostic and Statistical Manual of Mental Disorders-III-R (DSM-III-R) criteria (American Psychiatric Association, 1987), assessing the presence, severity and fluctuation of nine clinical features: acute onset and fluctuating course*, inattention*, disorganized thinking*, altered level of consciousness*, disorientation, memory impairment, perceptual disturbances, psychomotor agitation or retardation, and altered sleep–wake cycle. This instrument also includes a diagnostic algorithm (short version), based on the four cardinal features of delirium (previously marked with an asterisk). Delirium diagnosis requires the presence of features 1 and 2, and either 3 or 4. This instrument also had a severity score based on the shortened version, with higher scores indicating increased severity (Inouye, Reference Inouye2003). In the original study (Inouye et al., Reference Inouye, van Dyck, Alessi, Balkin, Siegal and Horwitz1990), the CAM was validated against psychiatric diagnosis, showing high levels of sensitivity (94–100%), specificity (90–95%), and positive (91–94%) and negative predictive values (90–100%). It was also found to have good convergent validity with other cognitive measures (e.g. MMSE and DST), and high inter-rater reliability (k = 0.81–1.00).
The CAM has become the most widely-used instrument (Wei et al., Reference Wei, Fearing, Sternberg and Inouye2008), recommended by the most recent guidelines (National Institute for Health and Clinical Excellence, 2010). It has been used in more than 4,000 published studies so far and translated into at least 12 languages (Inouye et al., Reference Inouye, Westendorp and Saczynski2013). The aim of this study is to present the reliability and validity of the European Portuguese version of CAM (long version).
Methods
Translation process and pilot-study
The CAM was translated in accordance with standard translation guidelines of International Society for Pharmacoeconomics and Outcomes Research (ISPOR) (Wild et al., Reference Wild2005), which also included trained researchers, according to the original training manual (Inouye, Reference Inouye2003). Following this, the European Portuguese version of CAM was tested in a pilot-study, with 50 elderly patients, recruited from the same clinical settings where the present study was carried out. It was found to have a good level of comprehensibility, inter-rater reliability (k >0.81), as well as moderate sensitivity (73%) and excellent specificity (95%). These preliminary results suggested that this version showed good ecological, face and content validity. Supporting data are described in greater detail in a previous publication (Martins et al., Reference Martins, Simões and Fernandes2012).
Sample
The sample included elderly patients (≥65 years), who had been admitted for at least 48 h, into two IMCU of intensive medicine and surgical services in the university hospital - São João Hospital Centre (CHSJ), in Porto. Two days per week were selected at random. The exclusion criteria were: a total score ≤11 on the Glasgow Coma Scale (Teasdale and Jennett, Reference Teasdale and Jennett1974), blindness/deafness, inability to communicate and to speak Portuguese.
Procedures
All patients included were assessed by a trained researcher (SM), who completed CAM based on observations made during a clinical interview and a formal cognitive assessment, with the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975) (cognitive global measure) and DST (Wechsler, Reference Wechsler1997) (attention and work memory measure). The CAM was completed immediately after the interview to ensure accurate information. The patient's family or caregivers and nurses, were also interviewed about the patient's prior cognitive performance and any recent cognitive change.
Delirium severity was determined using the scoring system of the CAM algorithm. It was solely used for the purpose of calculating the correlations between CAM and other cognitive instruments.
The CAM interviewer had limited information concerning the patient and was blinded to the patient's medical records.
The reference standard for delirium was also assessed in all patients by one of the authors (LF), an experienced geriatric psychiatrist, according to the Diagnostic and Statistical Manual of Mental Disorders-IV-TR (DSM-IV-TR) criteria (American Psychiatric Association, 2000). This assessment comprised a patient clinical interview, mental status examination, family and nurse interviews, as well as a review of medical records.
These two assessments were independent and blinded, and were performed in most of the cases an average of 4 h apart from each other (between 1 and 6 h).
The inter-rater reliability analysis was carried out in a convenience sample of 40 patients, recruited only from the IMCU of the Intensive Medicine Service. Patients were interviewed by a trained psychologist (SM) and nurse (CL). The order of the interview was alternated. Both raters completed the CAM independently on the basis of their observations during the interview, blinded to each other. Test-retest was not performed due to the fluctuating course of this syndrome.
Complete demographic and clinical characteristics of patients and hospital-related data were also obtained through chart review. The sample was also described with regard to the presence or absence of cognitive impairment/dementia, based on doctors’ information registered in the medical records.
The Ethics Committee of the Hospital approved the study. Informed consent was obtained from the patient or their closest relative, if the patient was unable to decide for him/herself.
Statistical analysis
The statistical analyses were performed using the Statistical Package for the Social Sciences Version 21.0 for Windows software (SPSS).
Patient characteristics are presented as raw frequencies and percentages for categorical variables and as median, minimum and maximum for continuous variables, as normality could not be assumed.
For analysis of differences between the two groups with and without delirium (according DSM-IV-TR criteria), the Mann-Whitney test for continuous variables, Chi-square test for paired categorical variables and Fisher's exact test for dichotomous variables were used at a significance level of 0.05.
Concurrent validity was assessed by sensitivity, specificity, positive and negative predictive values, as well as likelihood ratio for the European Portuguese CAM against the reference standard (DSM-IV-TR criteria), calculated by standard formula, using 95% confidence intervals (CI).
Convergent validity was explored by calculating Spearman's p rank correlation coefficients between CAM severity score and total scores for MMSE and DST (forward and backward), at a significance level of 0.01.
Inter-rater reliability was calculated using Cohen's kappa coefficient, for each feature in the instrument. The strength of agreement of the kappa statistics was based on the guidelines from Landis and Koch (Reference Landis and Koch1977), which defined k >0.61 as substantial and k >0.81 as almost perfect.
Results
Between June 2012 and June 2013, 268 patients were enrolled in this study, of whom 20 refused to participate, 16 had already been included, 11 were excluded due to an incomplete interview (mainly due to care management or clinical procedures) and 10 who were in terminal condition, with 3 dying during the study period.
The final sample included 208 elderly patients, most of them (75.5%) recruited from the IMCU of the Intensive Medicine Service. The sociodemographic characteristics of the sample are presented in Table 1, and did not differ significantly between the groups with and without delirium, classified by DSM-IV-TR criteria. In relation to the clinical characteristics (Table 2), significant differences were noted for admission type (p = 0.008) and length of stay in the IMCU (p <0.001), with the delirium group presenting the highest duration of hospitalization. As expected, significant differences regarding total scores of MMSE (p <0.001), DST forward (p <0.001) and backward (p <0.001) were found, with the lowest values in the delirium group. There was also a verified difference for the total number of co-morbidities, revealing a fairly higher value (p = 0.049) in the group without delirium. The most common reasons for admission were cardiorespiratory and gastrointestinal problems, with no significant differences found between groups or for the number of daily medications.
Demographic characteristics of sample
Note: (1)Mann-Whitney test;(2)Chi-Square Independent test;(3)Chi-square's exact test;(4)Fisher's exact test; min.-minimum; max.-maximum.
Clinical characteristics of sample
Note: (1)Mann-Whitney test; (2)Chi-Square Independent test; (3)Chi-square's exact test; max.-maximum; IMCU – Intermediate Care Unit; MMSE – Mini Mental State Examination; DST – Digit Span Test.
Concurrent validity
The occurrence rate of delirium was 25% (n = 53), according to the reference standard DSM-IV-TR rating. In these patients, only 42 were also positive according to the CAM. The CAM rated 153 of the 155 patients negatively whom the psychiatrist also rated as not being delirious.
Using DSM-IV-TR as a reference standard, the CAM had a sensitivity of 79% and a specificity of 99%. The positive predictive value of the test in the sample was 95%, indicating a strong ability to confirm delirium with a positive test result, and the negative predictive value was lower (93%) (Table 3). A kappa statistic of 0.83 (CI95% 0.74–0.92) was found between the CAM and DSM-IV-TR.
Comparison between CAM and DSM-IV-TR
Note: 95% CI = 95% confidence interval; CAM – Confusion Assessment Method; DSM-IV-TR – The Diagnostic and Statistical Manual of Mental Disorders-IV-TR.
Convergent validity
Negative correlations between CAM severity score and the total of the MMSE (rs = −0.676; p ≤0.01), DST forward (rs = −0.605; p ≤0.01) and backward (rs = −0.487; p ≤0.01) were found.
Inter-rater reliability
In the inter-rater reliability, the total CAM algorithm and five clinical features presented k = 1.00. The k values for the other clinical features are presented in Table 4. In this study, the median time for the CAM interviews (with cognitive measures) was 20 minutes and for completion of CAM ratings, 5 minutes.
Inter-rater reliability
Note: k = Cohen´s k coefficient; 95% CI = 95% confidence interval.
Footnote: For inter-rater reliability analysis, forty elderly patients were included. The majority were female (53%), married (53%), with a low educational level (90% 0–4 years of education) and with a mean age of 78.5 (sd 6.9). The main admission reasons were cardiorespiratory (50%) followed by urology/nephrology problems (23%).
Discussion
This study clearly indicates that the European Portuguese version of CAM has good reliability and validity, with robust positive data on concurrent and convergent validity and inter-rater reliability.
Moderate sensitivity and excellent specificity were found, when compared with the original study (Inouye et al., Reference Inouye, van Dyck, Alessi, Balkin, Siegal and Horwitz1990), as well as with other previous validation studies (Fabbri et al., Reference Fabbri, Moreira, Garrido and Almeida2001; Monette et al., Reference Monette2001; Laurila et al., Reference Laurila, Pitkala, Strandberg and Tilvis2002; Gonzalez et al., Reference Gonzalez2004; Hestermann et al., Reference Hestermann2009; Ryan et al., Reference Ryan, Leonard, Guerin, Donnelly, Conroy and Meagher2009; Wongpakaran et al., Reference Wongpakaran2011). However, there is some heterogeneity in the methodological issues in these studies, regarding the setting, sample, cognitive assessment tools used and formal trainer researchers. In addition, some of them did not consider the inclusion of patients with dementia, depression or other psychiatric disorders (Rockwood et al., Reference Rockwood1994; Pompei et al., Reference Pompei, Foreman, Cassel, Alessi and Cox1995; Rolfson et al., Reference Rolfson, McElhaney, Jhangri and Rockwood1999; Gonzalez et al., Reference Gonzalez2004), who are easily confused with delirious patients. This can explain some higher values in sensitivity. In fact, any test can distinguish the severely diseased from the healthy, but more important is the test's ability to distinguish confounding cases (Jaeschke et al., Reference Jaeschke, Guyatt and Sackett1994). So, in this setting with a high prevalence of confounding factors diagnoses, a sensitivity of 79% can still be appreciated as being very good.
Two false-positives and 11 false-negatives were found. With dementia present, health professionals often overcalled delirium in dementia patients (Inouye et al., Reference Inouye, Leo-Summers, Zhang, Bogardus, Leslie and Agostini2005), due to the overlap of several features, which makes differential diagnosis more complex (Inouye, Reference Inouye2006). This can explain the two false-positive cases verified where delirium was incorrectly classified in the patients with moderate dementia.
On the other hand, health professionals also frequently misattributed delirium symptoms to underlying dementia (Inouye et al., Reference Inouye, Leo-Summers, Zhang, Bogardus, Leslie and Agostini2005). This appears as the main reason for four of the 11 false-negative ratings found. Determining whether a patient has delirium, delirium superimposed on preexisting neurocognitive disorder or a major neurocognitive disorder separate from delirium (such as dementia) is a major challenge for clinicians. This is complicated as these disorders frequently coexist. One study found that the frequency of delirium in patients hospitalized with dementia was as high as 89% (Fick et al., Reference Fick, Agostini and Inouye2002). Moreover, dementia is a leading risk factor for delirium (Inouye et al., Reference Inouye, Westendorp and Saczynski2013) and, conversely, delirium seems to create an increased vulnerability to the development of dementia (Witlox et al., Reference Witlox, Eurelings, de Jonghe, Kalisvaart, Eikelenboom and van Gool2010) and also contributes to the acceleration of cognitive decline in patients with dementia (Fong et al., Reference Fong2009).
In the present study, delirium symptoms were also misattributed to depression in one patient, and not detected in another patient with visual and hearing impairment, which is in agreement with the results of previous studies (Farrell and Ganzini, Reference Farrell and Ganzini1995; Inouye et al., Reference Inouye, Foreman, Mion, Katz and Cooney2001).
As delirium can rapidly fluctuate, the extensive interval (6 h) in five cases may have caused some discordant observations between research and psychiatric assessment. This discordant result may not be a reflection of the test characteristics but rather an implication of the fluctuating nature of delirium itself. Fluctuation is an intrinsic problem of studying delirium (McNicoll et al., Reference McNicoll, Pisani, Ely, Gifford and Inouye2005).
In the present study, a good agreement between CAM and DSM-IV-TR was found, according to the guidelines from Landis and Koch (Reference Landis and Koch1977).
Regarding the convergent validity, a significant inverse relationship was observed between the CAM severity score and other cognitive measures, particularly with MMSE and DST forward, which supports the detrimental effect of delirium on cognition (Voyer et al., Reference Voyer, McCusker, Cole, St-Jacques and Khomenko2007).
In this study, inter-rater reliability was excellent, with an agreement of 100% (k = 1.00) for the diagnosis of delirium and for the majority of the individual items (at least five). However, the disorganized thinking item presented the lowest value (k = 0.65), which was described as k = 1.0 in the original study (Inouye et al., Reference Inouye, van Dyck, Alessi, Balkin, Siegal and Horwitz1990), without further discussion. This result might suggest that the scrutiny of cognitive measures performance was not sufficient. The addition of some standardized questions or specific tasks, such as interpretation of proverbs or description of similarities and differences between words, which requires organized, rational, conceptual thinking, could probably enhance the assessment of this clinical feature. Another additional explanation could be the different backgrounds of the two raters (psychology/nursing), which may also have contributed to some disagreements.
The present study has some strengths. First, appropriate training of the interviewers and a formal cognitive assessment (in particular with an attention measure) were performed before the CAM rating. Second, the comparisons between raters were blind and were made for all patients independently of whether or not they had delirium according the CAM. Third, the study population represents patients for whom testing by the European Portuguese version of CAM in a clinical “real-life” setting would be useful, once they presented high risk (e.g. advanced age, dementia) for development of this syndrome. Moreover, despite the increased number of ICMU in hospitals, there are few data in the literature regarding delirium in this context, compared to the information on critically ill patients.
A limitation in the present study was the selection of a convenience sample, recruited from ICMU in the referred hospital, without previous screening all of the patients admitted in these two units.
In addition, some patients were not interviewed within a shorter time interval, due to technical reasons, clinical circumstances and the care management needed in the units.
On the other hand, as this is a two-site study, validation of the performance of this instrument will be required in other settings to assure generalizability. Further research should focus on the use of this instrument in various settings by different levels of health professionals.
Using the MMSE and DST in order to rate the CAM could be difficult for routine ward nurses who are already very busy with clinical duties. In these cases, when time is scarce, assessment with other brief instruments (e.g. Abbreviated Mental Test) and attention tasks (e.g. DST or listing days of the week backwards) can be alternatively used (Inouye et al., Reference Inouye, Westendorp and Saczynski2013).
In Portugal, standardized assessment of delirium is still scarce and no national strategy for routine monitoring, prevention or treatment of this syndrome exists. The implementation of the European Portuguese version of CAM in daily practice is feasible and could lead to raised awareness in health care professionals. Besides that, CAM is an internationally recognized and used scale, so another important benefit of this validated version is that it can be used as a tool for multinational clinical research and comparison with other countries.
The European Portuguese version of the CAM has been shown to be a valid and reliable instrument for delirium detection in elderly patients hospitalized in ICMU by non-psychiatric trained health professionals and could be a valuable tool for future studies, namely on delirium incidence, risk factors and outcomes.
Conflict of interest
None.
Description of authors’ roles
Lia Fernandes and Mário R. Simões participated in the definition and design of the study. Sónia Martins, Carla Lourenço and Lia Fernandes participated in the data collection and writing the paper. Sónia Martins, Mário R. Simões and Lia Fernandes contributed to the analyses and interpretation of the results, as well as to the critical revision of the paper with João Pinto-de-Sousa, Filipe Conceição and José Artur Paiva. All authors have approved the final manuscript.
Acknowledgments
The authors wish to thank clinical staff, patients and their families for the collaboration. The present work was supported by the Foundation for Science and Technology (SFRH/BD/63154/2009)
