Introduction
Delirium is a common neuropsychiatric syndrome associated with adverse outcomes including long-term functional deterioration, persistent cognitive impairment and mortality (Witlox et al., Reference Witlox, Eurelings, de Jonghe, Kalisvaart, Eikelenboom and Van Gool2010). It is a complex syndrome that has a highly heterogeneous and fluctuating phenomenological profile whereby approximately 50% of patients with delirium are either missed or diagnosed late across clinical settings (O’Hanlon et al., 2014). Subsyndromal delirium (SSD) is a state characterized by the presence of delirium symptoms but without full syndromal delirium (FSD) criteria. It is associated with outcomes intermediate between FSD and no delirium (Cole, 2013). Delirium can be further characterized by clinically defined subtypes, with best evidence for the designation of subtypes according to motor activity profile. Lipowski (Reference Lipowski1983) suggested “hyperactive” and “hypoactive” patterns, characterized by increased and decreased motor activity respectively, before adding a third “mixed” category in recognition that many patients experience elements of both within short time frames. It has been reported that clinical motor subtypes of delirium differ according to a number of factors including, pathophysiology, detection rates, treatment experience, duration of delirium episode, and clinical outcome (Meagher, Reference Meagher2009). In particular, hypoactivity has been associated with organ failure and metabolic causes, while hyperactivity is more connected to substance-related delirium e.g. delirium tremens. Unfortunately, such studies have been found to have inconsistent findings due to inconsistent methodology regarding motor subtype profiling. In particular, it has been demonstrated that between four commonly utilized subtype profiling methods that there was concordance of approximately 34% (Meagher et al., 2008b). However, despite these methodological limitations, hypoactive subtypes have been found to have a significantly poorer prognosis (Yang et al., 2009). In particular, hypoactive motor profiles have been found to have highest associated mortality independent of factors such as co-morbidity, age, delirium, and dementia severity (Kiely et al., Reference Kiely, Jones, Bergmann and Marcantonio2007). However, the association between hypoactive delirium and elevated mortality may be reflective of delayed detection of delirium, and hence more prolonged episodes (Gonzalez et al., Reference González2009). Conversely, hyperactive and mixed subtype delirium is associated with more frequent use of antipsychotics, higher detection rates, and better outcomes (Meagher et al., Reference Meagher, Leonard, Donnelly, Conroy, Adamis and Trzepacz2011; Reference Meagher, Leonard, Donnelly, Conroy, Adamis and Trzepacz2012). Subsequently, a variety of different definitions of motor subtypes have emerged that include a wide range of motor and non-motor features (Lipowski, Reference Lipowski1989; Liptzin and Levkoff, Reference Liptzin and Levkoff1992; O’Keeffe and Lavan, Reference O’Keeffe and Lavan1999), though only the DMSS has been validated using non-delirious controls or independent measurements of motor behavior. Studies have demonstrated concurrent and predictive validity for the original 13-item DMSS including comparison to objectively measured motor activity levels using bioelectronic methods (Godfrey et al., Reference Godfrey, Conway, Leonard, Meagher and O’Laighin2008; Reference Godfrey, Leonard, Donnelly, Conroy, ÓLaighin and Meagher2010). In addition, the DMSS has been applied to the longitudinal study of motor subtype profile in delirium (Meagher et al., Reference Meagher, Leonard, Donnelly, Conroy, Adamis and Trzepacz2011; Slor et al., Reference Slor2013). It has also been translated into Dutch and Japanese versions (Slor et al., Reference Slor2014; Uchida et al., Reference Uchida2015). No other rating scale for motor subtypes of delirium has been developed and validated with such rigor.
More recently, an abbreviated version of the scale has been developed that can allow for simple and rapid identification of delirium subtypes. Preliminary work with the DMSS-4 has indicated that it has high concordance with the original 13-item DMSS in cross-sectional assessment of at-risk populations (e.g. palliative care, old age psychiatry, and general adult psychiatry referrals) (Meagher et al., Reference Meagher2014). However, because delirium is a highly fluctuating condition that requires serial assessment in order to capture its full clinical presentation, the reliability of assessments conducted over time is an important feature of delirium tools. Moreover, the DMSS and DMSS-4 are observational tools that are designed for use by a variety of healthcare professionals who engage with delirious patients and as such the concordance between assessment conducted by different healthcare professionals and the inter-rater reliability of the tool are of interest.
The aims of this study were (i) to explore the attribution of clinical subtype in elderly medical inpatients at risk of delirium and to specifically examine the level of agreement between the DMSS and DMSS-4, (ii) to examine concordance between the DMSS and DMSS-4 over time, (iii) to examine the inter-rater reliability of the DMSS and DMSS-4 between doctor, medical student, and nursing staff.
Methods
Participants and study design
We conducted a prospective study of consecutive elderly general medical admissions in an acute hospital setting. Over a one-year period between August 2012 and August 2013, 277 elderly general medical patients, attending the emergency department of Cork University Hospital in the Republic of Ireland were screened for inclusion in the study. During the recruitment, all elderly (70 yrs+) general medical patients admitted within the past 36 hours were invited to participate in the study. Criteria for patient exclusion from this study were (1) requiring specialist intervention (e.g. hematology, oncology, orthopedic surgery, and psychiatry), (2) patients who were deemed too unwell to participate, (3) early discharge from hospital, (4) insufficient data collected, and (5) refusal to participate. Of the 277 assessed, 145 (52%) were included and 132 (48%) were excluded from the study. All subjects were assessed for phenomenology, demographic, and treatment data in a consistent manner by raters (N. O’R [doctor] and J.F. [medical student]) who were trained by an expert (D.M.) in the use of both the DRS-R98 (Trzepacz et al., Reference Trzepacz, Mittal, Torres, Kanary, Norton and Jimerson2001) and DMSS. Patients’ phenomenology was assessed at baseline and also daily (by J.F. and N. O’R) for up to a week using validated methods.
Informed consent
The procedures and rationale for the study were explained to all patients but because many patients had cognitive impairment at entry to the study, it was presumed that most were not capable of giving informed written consent. Due to the non-invasive nature of the study, ethics committee approval was given to augment patient assent with proxy consent from next of kin (where possible) or a responsible caregiver in accordance with the Helsinki Guidelines for Medical Research involving human subjects (World Medical Association, 2004).
Medication data
All medications prescribed for each patient were documented at each visit including regular and as required doses. The use of psychoactive agents was a specific focus, especially the use of antipsychotic, opioids, benzodiazepines, psychostimulants, and corticosteroids. Such agents have been reported to be associated with delirium as a risk factor or treatment modality (Clegg and Young, Reference Clegg and Young2011). Dose equivalents over the 24 hours prior to assessment were calculated for each of these drug classes according to accepted conversion rates, i.e. antipsychotics in chlorpromazine equivalents; opioids in morphine equivalents; benzodiazepines in diazepam equivalents; steroids in prednisolone equivalents (Taylor et al., Reference Taylor, Paton and Kapur2015).
Delirium assessment
The DRS-R98 is a widely used instrument to measure symptom profile in delirium and can be used both as a diagnostic and severity assessment tool (Trzepacz et al., Reference Trzepacz, Mittal, Torres, Kanary, Norton and Jimerson2001). It is designed to rate symptoms over the previous 24 hours. The DRS-R98 has high inter-rater reliability, sensitivity, and specificity for distinguishing delirium from mixed neuropsychiatric populations including dementia, depression, and schizophrenia (Trzepacz et al., Reference Trzepacz, Mittal, Torres, Kanary, Norton and Jimerson2001). It is a 16-item clinician-rated scale with 13 severity items and 3 diagnostic items. The DRS-R98 total scale score ranges from 0 to a maximum of 46. A cut-off score of ≥18 is typically applied to delirium diagnosis, especially where high specificity is desirable, but other work suggests that such cut-offs exclude many cases of DSM-IV delirium and lower cut-off scores are advocated where diagnostic sensitivity is the primary goal (Franco et al., Reference Franco, Mejía, Ochoa, Ramírez, Bulbena and Trzepacz2006; Kato et al., Reference Kato, Kishi, Okuyama, Trzepacz and Hosaka2010). For this general hospital population, in consultation with the DRS-R98 developer (PTT), we applied relaxed cut-off scores to enhance diagnostic sensitivity (especially to milder delirium) to equate total scores of 0–6 with no delirium, 7–11 with SSD, and ≥12 with FSD.
The DMSS (Meagher et al., Reference Meagher2008a) is a scale comprising 13 (5 hyperactive and 8 hypoactive) symptoms selected according to their reflection of motor phenomenology, relative specificity for delirium relative to controls, and demonstrated correlation with independent and objective measures of motor behavior (Meagher et al., Reference Meagher2008a; Godfrey et al., Reference Godfrey, Leonard, Donnelly, Conroy, ÓLaighin and Meagher2010). The DMSS can be rated by any healthcare professional familiar with the clinical presentation of delirium. Scoring requires at least two features to be present from either the hyperactive or hypoactive list to meet subtype criteria. Patients meeting both hyperactive and hypoactive criteria are deemed mixed subtype, while patients meeting neither criterion are labeled “no subtype”.
The DMSS-4 is an abbreviated version of the DMSS which includes two hyperactive (Increased amount of activity and loss of control of activity) and two hypoactive (decreased speed of actions and decreased speech) features derived through discriminant analysis of the original longer scale. Preliminary work has suggested has high concordance with the DMSS (Meagher et al., Reference Meagher2014).
Dementia assessment
For all patients, the medical case notes were reviewed for a diagnosis of preceding cognitive impairment or dementia made by a suitably trained physician. This was confirmed by the use of the Informant Questionnaire on Cognitive Decline in the Elderly Short-Form (IQCODE-SF) to assess pre-admission cognitive impairment and decline. This 16-item scale was scored by a caregiver or close relative and a mean item cut-off score of ≥3.5 was used to diagnose dementia (Jorm, Reference Jorm2004). Consensus discussion was used to apply diagnosis in borderline cases. In the absence of an assessment from ICQODE-SF/collateral history, patients were considered not to have dementia if they scored greater than or equal to 27/30 on MMSE.
Statistical analyses
Continuous variables were presented as means and SD, and categorical variables as proportions and percentages. Cohen's κ coefficient was used to calculate the agreement between the full (13 item) DMSS and the DMSS-4. Similarly, to estimate the inter-rater reliability among the three raters doctor, nurse, and student for each of the scales DMSS (13 item) and DMSS-4, ICCs were calculated. The PAWS (SPSS) v18 software was used for statistical analyses. ICCs and Cohen's κ coefficient >0.75, between 0.40 and 0.75, and <0.40 were interpreted respectively high, moderate, and low inter-observer reliability (Landis and Koch, Reference Landis and Koch1977). The PAWS (SPSS) v18 software was used for statistical analyses.
Results
Demographic and medical history
Of the 277 patients assessed, 145 (52%) were included and 132 (48%) were excluded from the study. The reasons for exclusion were as follows: (1) refusal to participate (n = 65), (2) patients who were deemed too unwell to participate (n = 32), (3) early discharge from hospital (n = 28), and (4) patients requiring non-general medical treatment (n = 7). The mean age was 80.2 (SD: 6.3) (range 70–94 years old). Sixty nine (47.6%) were females. From the 145 participants, 31 (25.4%) had a history of dementia (as indicated by clinical history and case note records, MMSE scores, and IQCODE scores). Only six participants were in receipt of one or more classes of psychotropic/sedative medication. Three were in receipt of antipsychotic medications throughout the hospitalization, two antidepressants, four benzodiazepines, four z-hypnotics, and six other sedative medications (e.g. opioids). Up to a third of patients were assessed for seven days.
Delirium status
The prevalence and incidence of FSD was 33.8% (n = 48) and 9% (n = 13) respectively. The percentage of patients who reached FSD at any point during the period of study was 42.2% (n = 61). Similarly, the percentage who reached SSD at any point was 37.2% (n = 53). Twenty seven patients who experienced SSD transitioned to FSD during the study period (18.6% of the total study population and 52% of those who exhibited SSD).
Motor subtypes
The frequency of motor subtypes according to the DMSS at baseline assessment (n = 143) was 47.6% hypoactive (n = 69), 4.1% mixed (n = 6), 2.8% hyperactive (n = 4), and 44.1% no subtype (n = 64). The frequency of motor subtypes according to the DMSS-4 across all assessments during the study was 44.8% hypoactive (n = 72), 2.1% mixed (n = 3), 2.1% hyperactive (n = 3), and 44.8% no subtype (n = 65). The frequencies of motor subtypes according to the DMSS and DMSS-4 are shown in Table 1.
Motor subtypes according to DMSS and DMSS-4 (n = 143)
Agreement between DMSS and DMSS-4
The overall agreement (K) between the DMSS and DMSS-4 was 0.75 across all four subtypes. Further analysis of the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each subtype comparing DMSS to DMSS-4 are shown in Table 2.
Agreement (κ), sensitivity, specificity, and predictive values between DMSS-4 and DMSS across each motor subtype
Inter-rater reliability
Table 3 shows agreement (K) between the DMSS and the DMSS-4 for the three raters separately across the seven days of assessments. Table 4 shows pairwise ICC between the DMSS and DMSS-4 for the three raters. There was very high concordance between any of the pairs of raters for the DMSS-4 and moderate concordance between any of the pairs for the DMSS. The ICC for ratings conducted by all three raters was 0.70 for the DMSS and 0.59 for the DMSS-4, indicating moderate inter-rater reliability. Analysis of the agreement between raters for individual DMSS items found higher concordance in respect of hypoactive features compared to hyperactive. Items 2 and 3 (Increased speed of actions and Loss of control of movement) were found to have lowest agreement (K) between nursing and medical staff (medical student vs. nurse = 0.49, 0.44; doctor vs. nurse = 0.56, 0.49), while the remaining items retained high agreement (K ≥ 0.75).
Pairwise agreement (k) between DMSS and DMSS-4 for the raters (Doctor, Medical student, and Nurse) over seven days of assessments
Inter-rater reliability (IRR) between pairs of raters using the DMSS and DMSS-4
Discussion
In this study of elderly medical admissions, we conducted a longitudinal assessment of motor profiling activity using two validated instruments (DMSS and DMSS-4). The overall concordance between the original DMSS and the abbreviated DMSS-4 was high and this was evident over longitudinal assessments. These findings highlight how the relatively brief four-item DMSS-4 scale can usefully substitute for more detailed and time-consuming methods of defining motor subtypes and thus allow for rapid assessment. These findings are thus relevant to both research that is primarily focused upon clinical subtypes, as well as secondary analyses of the impact of clinical subtype upon treatment response and prognosis – two areas that are particularly lacking in respect of data-based evidence to guide real-world practices (Meagher et al., Reference Meagher, McLoughlin, Leonard, Hannon, Dunne and O’Regan2013).
There was very high concordance between any of the pairs of raters for the DMSS-4 and moderate concordance between any of the pairs for the DMSS. Analysis of the agreement between raters regarding the individual items of the DMSS-4 was found to be higher for hypoactive compared to hyperactive items. This may reflect the greater challenge in assessing particular aspects of motor activity, for example, the lowest inter-rater agreement was for items 2 and 3 (Increased speed of actions and involuntary movements), both for medical student versus nurse and doctor versus nurse. The reasons for the lower agreement between nursing and medical staff for some individual items could be attributed to how these raters perceive and conceptualize this behavior. In particular, these features can be challenging to reliably assess in elderly medical patients (especially if they are sedentary) in the acute hospital setting (Blazer and van Nieuwenhuizenbut, Reference Blazer and van Nieuwenhuizenbut2012). Improved detection of these items could be facilitated by a more detailed explanation of their characteristics supported by specific training that focuses upon their identification in clinically relevant examples and how this information can be reliably recognized by both nursing and medical staff who typically differ in the consistency of contact with patients over the 24-hour cycle during which these behaviors can fluctuate considerably.
The final aim of this study was to investigate reliability between medical and nursing staff using the DMSS and DMSS-4. The ICC was found to be moderate for the DMSS and high for the DMSS-4. As such, these findings support the use of both the DMSS and DMSS-4 by assessors with nursing and medical backgrounds which is in keeping with the need to engage the multidisciplinary perspectives in order to optimize accurate delirium detection and ongoing assessment (O’Hanlon et al., Reference O’Hanlon2014).
Study limitations
The findings from this elderly general medical inpatient cohort do not necessarily generalize to all clinical settings (e.g. ICU and palliative care) where delirium is common and as such further studies specific to these settings are required. Another limitation of this study is the exclusion of 132 (48%) patients; this was due to our stringent exclusion and inclusion criteria applied to this hospital setting. Exclusion of this many patients may have reduced the number of subsyndromal cases and hence may have impacted upon the detected prevalence of the different motor subtypes. Moreover, up to a third of patients were assessed for seven days which may also have impacted upon the longitudinal assessment of motor subtypes detected. This was largely due to the discharge rate of this cohort by medical teams, as well as in patient changes to eligibility for inclusion in this study e.g. referred for specialist services. The group studied herein was predominantly hypoactive, which is congruent with other studies of older adults and contained relatively few patients with mixed and hyperactive motor subtypes (Meagher, Reference Meagher2008b; Blazer and van Nieuwenhuizenbut, Reference Blazer and van Nieuwenhuizenbut2012). We found relatively few cases of mixed motor subtype, despite the existing evidence suggesting that this type is a distinct subtype rather than a transition state (Meagher et al., Reference Meagher, Leonard, Donnelly, Conroy, Adamis and Trzepacz2012). Therefore, the concordance in respect of cases detected as mixed presentations is less convincing. Specificity across the four subtypes was high. However, although sensitivity for hypoactive and no subtype was high, it was lower for mixed and hyperactive subtypes. This may reflect the lower numbers of mixed and hyperactive motor behavior within this cohort.
Future work and conclusions
This study builds upon the development of the DMSS-4 as a well validated, conceptually robust, and simple method of assessing clinical subtypes of delirium. This study supports its usefulness both longitudinally and for use by a variety of healthcare professionals, including both medical and nursing staff by demonstrating high agreement and good reliability. Further work may examine motor subtypes in both SSD and FSD states in terms of bioelectronic analysis (for example in terms of modes of physical activity or energy utilization) and explore their relationship to issues such as delirium onset/resolution and specific symptom domains such as altered circadian integrity, (Godfrey et al., Reference Godfrey, Leonard, Donnelly, Conroy, ÓLaighin and Meagher2010; Fitzgerald et al., Reference Fitzgerald2013). Further work may also investigate predictors for persistent delirium over the seven-days period.
Conflict of interest
The authors have no conflicts of interest to report apart from that this work was supported by a grant from the Health Research Board (Ireland).
Description of author's roles
James Fitzgerald was involved in the design, data collection, data analysis, and writing of the paper. Niamh O’Regan was involved in the design, data collection, data analysis, and writing of the paper. Dimitrios Adamis was involved in the data analysis and writing of the paper. Suzanne Timmons was involved in the design, data collection, data analysis, and writing of the paper. Colum Dunne was involved in the design and writing of the paper. Paula Trzepacz was involved in the data analysis and writing of the paper. David Meagher was involved in the design, data collection, data analysis, and writing of the paper.
