Introduction
Our population is aging at an unprecedented rate; we can therefore expect increasing numbers of people admitted to acute hospitals to have dementia, with or without delirium. However, both disease entities are under-recognized. In an acute hospital in the UK, 42.4% of patients aged over 70 years had dementia, but only half of them had been diagnosed prior to admission (Sampson et al., Reference Sampson, Blanchard, Jones, Tookman and King2009). Concerning delirium, its reported rate in hospitalized patients ranged from 11% to 42% (Siddiqi et al., Reference Siddiqi, House and Holmes2006), but it is estimated that the condition may go undetected in up to 70% of patients (Cole, Reference Cole2004). A distinction between these disorders is important because delirium often represents a medical emergency and its prognosis can be poor without prompt treatment (Kakuma et al., Reference Kakuma2003; Lindesay et al., Reference Lindesay, Ames, Chiu, Lindesay and Shulman2010). Even though dementia is often thought to be clinically irrelevant in acute settings, its under-recognition has often been associated with suboptimal care, resulting in increased risk of adverse events, iatrogenic harm and greater functional decline after an acute hospital admission (Creditor, Reference Creditor1993; Sampson et al., Reference Sampson, Blanchard, Jones, Tookman and King2009).
Delirium is characterized by its acute onset and fluctuating severity of inattention and disturbance in consciousness (American Psychiatric Association, 1994; Meagher et al., Reference Meagher2007), whereas dementia runs a deteriorating course with disproportionate memory impairment and relatively preserved attention. Attentional deficit has been consistently revealed as a distinguishing cognitive feature of delirium (Bhat and Rockwood, Reference Bhat and Rockwood2007; Lowery et al., Reference Lowery, Wesnes and Ballard2007; Meagher et al., Reference Meagher, Leonard, Donnelly, Conroy, Saunders and Trzepacz2010), thus, various tests of attention have been studied for their usefulness in the assessment of delirium (O'Keeffe and Gosney, Reference O'Keeffe and Gosney1997; Meagher et al., Reference Meagher, Leonard, Donnelly, Conroy, Saunders and Trzepacz2010). In a recent study by Meagher et al. (Reference Meagher, Leonard, Donnelly, Conroy, Saunders and Trzepacz2010), the spatial span forwards (SSF), which is a more specific test of simple visual attention, has been suggested to be a useful screening tool for delirium. On the other hand, the spatial span backwards (SSB) test, which requires greater processing of information and working memory, was significantly lowered in all patients with cognitive impairment when compared with controls, but it did not help in differentiating delirium from dementia. In our study, the digit span tests (DST, which includes the digit span forwards and digit span backwards), the verbal equivalent of the SSF and SSB, were selected because they could be readily used at the bedside, and have been widely adopted as reliable tests of attention. O'Keeffe and Gosney (Reference O'Keeffe and Gosney1997) have identified a cutoff point on the digit span backwards to help with the screening of delirium; however, its potential rater bias (because the same investigator would have performed the DST and made the diagnosis of delirium) may limit its validity.
In the absence of local data on the prevalences of dementia and delirium, and the unavailability of validated screening instruments in detecting these two neuropsychiatric conditions in acute hospitals in Hong Kong, this study had two principal aims: (i) to validate the DST in the identification of and differentiation between dementia and delirium in older people who were admitted to an acute medical unit; and (ii) to ascertain the prevalences of these conditions within the same setting.
Methods
Subjects and setting
This was a cross-sectional validation study, which was approved by the Hong Kong Hospital Authority Ethical Committee and conducted during a two-week period in January/February 2010 at the United Christian Hospital in Hong Kong. The United Christian Hospital is one of the general hospitals in the region with 300 acute beds in its Department of Medicine and Geriatrics. Potential subjects were recruited from the acute admission unit of this department, and they included patients who were 75 years or older with an unplanned admission. Informed verbal consent was obtained from these subjects. For those who lacked the capacity to consent, assent was given by caregivers, with the proviso that the subject did not show unwillingness to participate. Patients who could not communicate verbally, or were unable to participate in clinical interviews, were excluded.
Sample size calculation
Based on the estimated prevalences of delirium and dementia at 10% and 20% respectively, it was determined that a sample size of n = 17 would be required in the positive population (i.e. those who were cognitively impaired), while n = 51 (3:1 control-to-case ratio) would be required in the negative population (i.e. those who were cognitively intact). These sample sizes would give a power of 0.8 and a Type I error of 0.05.
Testing procedure
Three general nurses who were going to perform the digit span tests were given a 15-minute briefing session on the use of these tests by a psychiatrist (JL) prior to the study. Patients who were admitted to the study unit and deemed eligible were recruited consecutively. Soon after each newly admitted subject had received the standard care procedures and any urgent management for their acute medical condition(s), one of the nurses from this study would perform the DST first. The two research psychiatrists (JL or GL) would then perform a standard psychiatric assessment on the same patient. The nurses and psychiatrists were blind to each other's findings, and both would complete their respective assessments within four hours of each admission. The psychiatrists’ evaluation included a formal interview with the patient, a mental state examination, a Cantonese Mini-Mental State Examination (CMMSE; Chiu et al., Reference Chiu, Lee, Chung and Kwong1994), followed by a medical record review and an interview with any informants, if they were available. Psychiatric diagnoses were made according to the DSM-IV criteria. Demographic and clinical data (age, gender, astheno-emotional disorder (AED) diagnosis, history of delirium and dementia) for all subjects were obtained by the psychiatrists from medical records.
Measurements
Gold standard
The DSM-IV depicts the explicit phenomenology of delirium, dementias and other organic mental disorders, enhancing diagnostic accuracy and serving as the “gold diagnostic standard” in this study.
Digit span tests
The subject was asked to listen carefully to a series of random numbers, which were presented at a rate of one per second. In the digit span forwards (DSF), the subject was asked to repeat the numbers in a forward order, while in the digit span backwards (DSB), the subject was asked to recall the numbers in reverse order. Starting with a two-number sequence, each correctly repeated series was followed by a sequence with one additional digit. For each sequence, the subject was given a second chance with another set of random numbers if he/she had failed in the first attempt. If the subject also failed in the second attempt, the test was interrupted and scored according to the longest series that was achieved. For instance, if a subject had correctly recalled a sequence of five digits in the DSF in either the first or second attempt but then failed in the sequence of six digits, he/she would get a score of 5 in the DSF. The DSF and DSB were scored separately. A maximum of eight digits were presented in the DSF, and a maximum of seven digits were presented in the DSB. Subjects were not penalized if they corrected themselves, and no time limitations were imposed.
Statistical analysis
All statistical analyses were performed using the Statistical Package for Social Sciences version 11.5, and a p-value of <0.05 was considered statistically significant. For comparisons between groups, the t test was adopted for continuous variables. Receiving operating characteristics curve (ROC) was used in conjunction with sensitivity, specificity and likelihood ratios to assess the performance of the tests. The agreement between the DST ratings and CMMSE scores was calculated using the Pearson correlation coefficient.
Results
Study population
A total of 172 people aged 75 years or above were admitted to the medical admission ward through the emergency department during the two-week study period. Of these, 28 were excluded for being unable to undergo the digit span tests (eight were on non-invasive ventilation, six were aphasic, five had severe hearing impairment, five had diminished arousal/vigilance, three were in severe distress and one had communication difficulties as he spoke in a dialect). None of the eligible patients or their relatives had refused to participate in the study and thus 144 (84%) were formally assessed and included in this analysis.
The patients who were excluded from this analysis were significantly older than those who were included (83.3 years v. 79.9 years, p = 0.029), and women were predominant in the excluded but not the included sample (61% v. 46%).
Clinical characteristics
Of the 144 patients who underwent cognitive assessment, 19 (13.2%) had a documented history of dementia and four (2.8%) had a documented history of delirium. Thirty-eight patients could not be assessed using the CMMSE for the following reasons: drowsiness (n = 13), inability to follow instructions (n = 12), language problems (n = 6), hearing impairment (n = 2), visual impairment (n = 2), severe shortness of breath (n = 2), and severe pain (n = 1). The ten most common diagnoses made in the emergency department are presented in Table 1.
Top ten diagnoses made in the emergency department (n = 144)
Prevalence and clinical characteristics
Of the 144 recruited subjects, 83 (57.6%) were cognitively intact. Table 2 shows the prevalence of: (1) cognitive-intact, (2) dementia alone, (3) delirium alone, (4) delirium superimposed on dementia and (5) cognitive impairment not otherwise specified (NOS), as well as the characteristics of the patients within these five diagnostic entities. Subjects who were cognitively intact scored significantly higher in CMMSE than patients with dementia alone (p = 0.00), delirium alone (p < 0.01) or delirium superimposed on dementia (p < 0.01). The cognitive-intact group also showed significantly better performance in DSB than the “dementia alone” (p = 0.00), “delirium alone” (p = 0.00) and “delirium superimposed on dementia” (p = 0.00) groups. With regard to DSF, the only observed difference was between the “cognitive-intact” and the “delirium superimposed on dementia” groups.
Prevalence of cognitive impairment and clinical characteristics of 144 participants
CMMSE = Cantonese Mini-Mental State Examination; DSF = digit span forwards; DSB = digit span backwards; NOS = not otherwise specified.
a p = 0.00; b p < 0.01; c p < 0.05.
Four patients were labeled as having “cognitive impairment NOS” because they demonstrated overt cognitive impairment in the psychiatric assessment but longitudinal observation was warranted for definitive diagnosis. Of these, all four patients received a working diagnosis of acute cerebrovascular accident in the emergency department. Owing to the small sample size (n = 4), comparison analysis was not performed on this group.
Psychometric properties of the Digit Span Tests
The Digit Span Tests were completed by nurses in under three minutes. In the first threshold analysis, the cognitive-intact group served as the normal controls (NC, n = 83) for comparison with those who demonstrated major cognitive disorders (MCD, n = 61), which included dementia alone, delirium with or without dementia, and cognitive impairment NOS. Second threshold analysis was carried out to study the usefulness of DST in the differentiation of delirium from non-delirium: patients with delirium alone or delirium superimposed on dementia were categorized in the “delirium” group (n = 26) so as to be compared with all other diagnostic entities (n = 118). Table 3 shows the sensitivity, specificity, likelihood ratios and the areas under the ROC curve (AUC) at the optimal cut-off scores of DST in both threshold analyses. The optimal cut-off was determined as the score where Youden's index (sensitivity+specificity-1) was maximized.
Psychometric properties of DSF and DSB in identification of major cognitive disorders (NC vs MCD) and differentiation of delirium from non-delirium
NC = normal controls; MCD = major cognitive disorders (includes delirium, dementia and cognitive impairment not otherwise specified); AUC = area under curve; LR = likelihood ratio; CI = confidence interval.
* Delirium includes “delirium alone” and “delirium superimposed on dementia”.
With regard to DSF, the areas under the ROC curve (AUC) were 0.62 and 0.65 for “NC versus MCD” and “delirium versus non-delirium” respectively. At the cut-off score of <8, DSF had unsatisfactory sensitivities in both analyses (0.46 for NC versus MCD and 0.58 for delirium versus non-dementia).
Concerning DSB, threshold analyses revealed that at the optimal cut-off score of <3, for NC versus MCD, the AUC was 0.83, sensitivity was 0.77, specificity was 0.78 and the positive likelihood ratio was 3.55 (CI: 2.31–5.45). In comparison between delirium and non-delirium, the AUC was 0.78, sensitivity was 0.81, specificity was 0.63 and the positive likelihood ratio was 2.17 (CI: 1.61–2.92).
In order to examine the usefulness of DST in the differentiation between delirium and dementia, subgroup analysis of “delirium with or without dementia” (n = 26) and “dementia alone” (n = 31) was performed. The AUCs for both DSF (0.62, CI: 0.47–0.76) and DSB (0.60, CI: 0.45–0.75) were not significant, and no optimal cut-off score could be identified for both tests.
Subjects who could not be assessed by CMMSE were excluded in the correlation analysis between DST and CMMSE. Scores on CMMSE were significantly correlated with DSF (r = 0.22, p < 0.05) and DSB (r = 0.52, p = 0.00).
Discussion
This study is the first to demonstrate the prevalence rates of dementia alone (21.5%), delirium alone (9%) and co-morbid dementia-delirium (9%) in an acute geriatric inpatient population in Hong Kong. The low documentation rates of dementia (13.2%) and delirium (2.8%) in medical records affirm the problem of under-recognition and under-documentation of cognitive impairment in the study population.
In this study, severe language or hearing deficits accounted for 20 of the 28 patients who could not undergo DST and who were therefore excluded (eight were on non-invasive ventilation, five had a severe hearing impairment, six were aphasic and one had communication difficulties through speaking in dialect). Because of this we argue that tests for visual attention – for instance, the spatial span tests (SPT) – can be good alternatives for attention assessment in patients with sensory (other than visual) or language deficits (Brown et al., Reference Brown, McGrory, McLaren, Starr, Deary and MacLullich2009). Despite the limitations of the DST, it was better tolerated than MMSE by frail geriatric patients (26% of patients who could be assessed by DST were unable to undergo MMSE) and its interpretation is spared from certain constraints such as upper limb impairment or illiteracy. Furthermore, DST can be easily learnt and deployed by general nurses in a busy acute medical setting.
This study confirms the hypothesis that inattention, as demonstrated by the compromised performance in DSF (mean of 7.4 in the dementia alone group, 6.4 in the delirium superimposed on dementia group, p < 0.05), is a notable characteristic of delirium, but not of dementia. However, the clinical usefulness of DSF as a screening instrument for delirium is not supported by the threshold analysis, which involved comparison between delirium (+/- superimposed dementia) and non-delirium (AUC: 0.65; sensitivity: 0.58, specificity: 0.72). This unexpected finding is at odds with the result in the study by Meagher et al. (Reference Meagher, Leonard, Donnelly, Conroy, Saunders and Trzepacz2010), which suggests the usefulness of SSF as a “differentiating screening test”. The apparent contradictory findings may be explained by the difference in modality of the attentional tests: DSF is a test of verbal attention, while SSF examines visual attention. Other than inattention, patients with delirium have shown to bear specific visual perceptual deficits that cannot be accounted for by their general cognitive impairment (Brown et al., Reference Brown, McGrory, McLaren, Starr, Deary and MacLullich2009). This may explain the superiority of SSF over DSF as a test for delirium. In addition, the study by Meagher et al. (Reference Meagher, Leonard, Donnelly, Conroy, Saunders and Trzepacz2010) was set in a palliative care service where dementia is not as prevalent as in elderly medicine settings, where our study was performed. This suggests that generalizability is one of the important considerations in the translation of research findings to various clinical settings, and we recommend that the cognitive tests are validated in specific target populations before being applied.
In general, DSB has a better overall performance than DSF in the detection of major cognitive disorders (AUC: 0.83 for DSB; 0.62 for DSF) and the distinction of delirium (AUC: 0.78 for DSB; 0.65 for DSF). However, the clinical usefulness of DSB in differentiating delirium was limited by its low specificity (0.63) and unsatisfactory likelihood ratios (+ve LR: 2.17, -ve LR: 0.31). We do not find this surprising. In addition to inattention, another cardinal feature of delirium – its acute onset and fluctuating course – can only be detected with knowledge of a patient's baseline cognitive functions, obtained either by careful history-taking from reliable informants or from longitudinal observation. This supports the use of a nursing observational scale (Gaudreau et al., Reference Gaudreau, Gagnon, Harel, Tremblay and Roy2005; Leung et al., Reference Leung, Leung, Leung and Pan2008) for screening of delirium.
With regard to the screening for major cognitive disorders – i.e. dementia, delirium and cognitive impairment NOS – DSB demonstrates reasonable clinical utility (sensitivity: 0.77; specificity: 0.78; +ve LR: 3.55). The compromised sensitivity is likely the trade-off of the ease-of-use of the test. Therefore, we recommend the use of DSB as a very quick screening for major cognitive disorders in busy acute inpatient settings. For the screened-positive patients or patients who present with cognitive or behavioral changes during hospitalization, history-taking from informants guided by a structured assessment (Harwood et al., Reference Harwood, Hope and Jacoby1997; Jorm, Reference Jorm2004) or the use of the nursing observational scale is instrumental in improving screening accuracy.
A design limitation of this study was that all the subjects were cognitively examined only once, soon after their acute admission, and no longitudinal observation was carried out. We were thus left with four cases of diagnostic uncertainty. Since all four of these patients were diagnosed with acute cerebrovascular accident in the emergency department, we acknowledge the need for a more comprehensive cognitive assessment, with a longer observation period for this group of patients.
In conclusion, cognitive impairment is highly prevalent but significantly under-reported in older people in acute medical settings. The administration of DSB is effective in identifying patients with major cognitive impairment in the study population. The clinical usefulness of the combined use of DSB and a brief informant-based assessment will be explored in future research.
Conflict of interest
None.
Description of authors’ roles
Dr. Leung was responsible for designing the study, data collection, data analysis and writing the paper. Dr. Lee and Mr. Lam participated in study design and data collection. Dr. Chan and Dr. Wu helped with data analysis and writing the paper.
Acknowledgments
The authors thank the nurses of the Acute Medical Unit of United Christian Hospital for their very kind support; and Dr. Meranda Leung and Dr. Frank Leung for proof-reading.
