Introduction
There is an extensive body of research recommending that palliative care (PC) should be incorporated into intensive care unit (ICU) practice (Aslakson et al., Reference Aslakson, Cheng and Vollenweider2014; Byock, Reference Byock2006; Nelson et al., Reference Nelson, Bassett and Boss2010; Truog et al., Reference Truog, Campbell and Curtis2008). For a long time, PC interventions were only associated with end-of-life care and patients were seen belatedly by PC teams during hospitalization. This practice leads to risk of unnecessary and futile life-prolonging interventions for patients presenting high risk of morbidity and mortality (Schneiderman et al., Reference Schneiderman, Gilmer and Teetzel2003; Teno et al., Reference Teno, Gozalo and Bynum2013). Recent studies have shown that PC consultations for this group of patients is associated with more frequent and earlier family meetings, better symptom management, and shorter ICU and hospital lengths of stay (Braus et al., Reference Braus, Campbell and Kwekkeboom2016; Norton et al., Reference Norton, Hogan and Holloway2007). Moreover, PC interventions promote support for patients and families, addressing complex decision-making, goal clarification, and coping with distressing symptoms (Campbell & Guzman, Reference Campbell and Guzman2003; O'Mahony et al., Reference O'Mahony, McHenry and Blank2010).
A current challenge of incorporating PC teams into acute care settings is how to correctly identify patients for whom a PC intervention is considered appropriate. For this purpose, several screening criteria have been suggested for triggering PC consultations in traditional ICUs (Nelson et al., Reference Nelson, Curtis and Mulkerin2013; Zalenski et al., Reference Zalenski, Courage and Edelen2014). Despite the growing number of studies in this field, screening tools for PC consultations have not yet been developed and validated for use in emergency department ICUs (ED-ICU).
ED-ICUs encompass units with critical care beds located within an ED. Usually, these units are set up in the ED to continue intensive care initiated in the emergency room when critically ill patients cannot be admitted to a traditional ICU (Tseng et al., Reference Tseng, Li and Chen2015; Weingart et al., Reference Weingart, Sherwin and Emlet2013). This situation is often a result of the combination between scarcity of ICU beds and excessive demand from the ED (ACEP, 2011; Chalfin et al., Reference Chalfin, Trzeciak and Likourezos2007; Mullins et al., Reference Mullins, Goyal and Pines2013). Frequently, patients are accepted or refused into ICUs based on specific criteria, aiming to improve the allocation of available resources (Sprung et al., Reference Sprung, Baras and Iapichino2012). As a result, ED-ICUs tend to become an alternative “refuse heap” for palliative care services, admitting a greater number of “reduced benefit” patients (Goldstein, Reference Goldstein2005; Weingart et al., Reference Weingart, Sherwin and Emlet2013). In comparison to traditional ICUs, ED-ICUs tend to admit patients with more severe conditions, less chance of survival, and who might benefit from advanced palliative care skills. Early identification of ED-ICU patients with PC needs, especially in institutions with scarce resources, may not only improve care for PC patients, but also increase the access of non-PC patients who need an ICU bed and are waiting for prolonged periods in the ED (Aslaner et al., Reference Aslaner, Akkaş and Eroğlu2015). The aim of the present study was to assess the criterion validity and inter-rater reliability of a PC screening tool for patients admitted to an ED-ICU.
Methods
Study design, setting, and population
This was a single-center, observational retrospective study, carried out between November 15, 2014, and November 14, 2015, at the Instituto Central do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (IC-HCFMUSP) in Brazil. This study was approved by the Ethics and Research Committee of the IC-HCFMUSP and informed consent was waived.
The IC-HCFMUSP is a tertiary teaching hospital with 1,100 beds, of which 110 beds are distributed among 10 traditional ICUs (medical and surgical). Within the ED, there is an ED-ICU with 17 critical care beds, admitting patients directly from the emergency room after initial clinical stabilization. The ED receives approximately 55,000 patients every year, resulting in 14,000 hospital admissions, with 4,500 in ICUs. Approximately 33% of ED patients who might benefit from critical care management receive treatment in the ED-ICU and do not have access to traditional ICUs. In our institution, when an ICU-bed becomes vacant, a critical care physician determines which patient will be admitted to the traditional ICU. ED patients who had an ICU bed requested but were not selected by the critical care physician are admitted to the ED-ICU. Currently, there are no objective and protocolized criteria for this selection process.
In the present study, we included all nontrauma patients older than age 18 who were admitted to the ED-ICU during the one-year study period. The exclusion criteria were: patients who did not have their PC screening criteria assessed; death within less than 24 hours of ED-ICU admission; transfers to other hospitals; and ED-ICU readmissions during the same hospitalization (only the first admission was considered).
Palliative care screening criteria
The Palliative Care Service of the IC-HCFMUSP developed a screening tool based on the Gold Standards Framework Prognostic Indicator Guidance (GSF-PIG), adapted to local patient's characteristics, aiming to provide non-PC physicians with a decision aid tool to identify patients in need of a PC consultation. The GSF-PIG is a validated screening tool widely used for patients likely to have a short life expectancy and for whom a PC intervention is indicated (O'Callaghan et al., Reference O'Callaghan, Laking and Frey2014). In the present study, different from GSF-PIG, screening criteria were based on the presence of specific clinical indicators of advanced disease and the use of two “surprise questions” to identify patients with a potentially life-limiting or life-threatening condition (Figure 1). Besides the traditional “surprise question” (“Would you be surprised if this patient were to die in the next 12 months?”), we added a modified “surprise question” (“Would you be surprised if this patient were to die during this hospitalization?”) to capture the context of the ED and ICU settings, where patients with PC needs have poorer short-term outcomes compared with same patients in ambulatory or general ward settings. Previous studies have reported the utility and predictive validity of a modified surprise question to capture short-term prognosis of ED patients (Hamano et al., Reference Hamano, Morita and Inoue2015; Haydar et al., Reference Haydar, Almeder and Michalakes2017). Although the relationship between poor prognosis predicted by the “surprise questions” and patient PC needs is not completely established, early identification of patients with advanced diseases and high-risk of death can trigger consultations of PC specialists, who can better manage patient and families, and avoid unnecessary invasive procedures and prolonged ICU and hospital lengths of stay (Braus et al., Reference Braus, Campbell and Kwekkeboom2016; Downar et al., Reference Downar, Goldman and Pinto2017; Norton et al., Reference Norton, Hogan and Holloway2007).
Algorithm framework for palliative care screening.
According to the proposed PC screening tool (Figure 1), ED-ICU patients who did not present at least one of the advanced diagnoses (question 1) were classified as category 1. For patients with one or more advanced diagnosis, the traditional “surprise question” (question 2) was applied, and a “yes” answer classified the patient as category 2. A “no” answer for question 2 led to the modified “surprise question” (question 3). A “yes” for question 3 classified the patient as category 3, and a “no” as category 4.
Screening procedures
An intensive care physician (SCCR) assessed the screening criteria of eligible patients within 72 hours of ED-ICU admission, based on the admission report registered in the electronic health record. Any contact between the patient and the ED-ICU physician (screener) occurred after the screening only. This physician worked at the ED-ICU from Monday to Friday on diurnal shifts (8:00 am–8:00 pm); the screening procedures were performed during this period.
Inter-rater reliability procedures
A sample of 100 patients, chosen randomly from the overall sample, was assessed by two raters using the proposed screening criteria. The interrater reliability of the screening tool was assessed by comparing the screening category rated by an intensive care physician (SCCR) with more than 7 years of ICU practice to the score of a second rater, an emergency physician (RDD) with more than 5 years of ED practice. Both raters, critical care and emergency physicians, had access to the same patient information (ED-ICU admission report). These reports were made available to the raters in a pdf format, and access to the entire electronic health record was given only after screening. In addition, the admission reports were never written by the ED-ICU or ED physicians involved in the screening.
Variables and outcome measurements
The following admission variables were registered: age; gender; time between ED arrival and ED-ICU admission (ED length of stay); diagnostic classification at admission according to APACHE disease classification system (Knaus et al., Reference Knaus, Zimmerman and Wagner1981); and PC screening category.
Outcomes were assessed by the investigators after patient discharge or death and based on the information provided in the electronic health record. The primary outcomes were those related to severity of illness and mortality: intrahospital and ED-ICU mortality; intrahospital and ED-ICU length of stay; transfer from ED-ICU to a traditional ICU; use of mechanical ventilation in the ED-ICU; Simplified Acute Physiology Score III (SAPS-3) at ED-ICU admission and standardized mortality ratio (SMR). The SMR was calculated between observed (intra-hospital) mortality and predicted mortality (by SAPS-3 admission score). The probability of death based on SAPS-3 was calculated using the logit customized to South America (Moreno et al., Reference Moreno, Metnitz and Almeida2005).
The secondary outcomes were those related to palliative care needs: order for a PC consultation during both ED-ICU and hospital stay; time between ED-ICU admission and PC consultation; and transfer to a specialized PC-bed during hospitalization. During the study period, the ED-ICU physicians did not have access to the screening tool results (categorization), and decided to order a PC consultation based on clinical judgment only.
Statistical analysis
Categorical variables were described as absolute numbers and proportions, and continuous variables were described as median and interquartile ranges (1°−3° interquartile range). Continuous data were tested for normality using the Kolmogorov-Smirnov test. Differences in continuous variables among category groups were assessed by the Kruskal-Wallis test. Differences among proportions were assessed by chi-square statistics. Post hoc analyses were performed to assess pairwise comparisons using the Dunn-Bonferroni approach for continuous data and the Bonferroni correction to proportions.
For sample size calculation, we referred to a similar study that found a hospital mortality of 6.6% and 37.3% in low and high screening categories, respectively (Zalenski et al., Reference Zalenski, Courage and Edelen2014). We estimated that, for a power of 90% and a two-sided 95% confidence level (CI 95%), a sample size of 93 patients would be necessary to find an estimated odds ratio for hospital mortality of 4.0, comparing screening category 1 with category 4 in our sample.
The CI 95% for the SMR was calculated using an online SMR analysis calculator, taking Fisher's exact and a CI 95% (Sullivan, Reference Sullivan2006).
Inter-rater reliability was measured by quadratic weighted kappa test and proportion of agreement. A sample size calculation for kappa analysis was performed based on a previous study (Walter et al., Reference Walter, Eliasziw and Donner1998). It was established that 86 observations performed by two raters achieve 80% power to detect a kappa value of 0.60 (considered a substantial agreement) with alpha error at 0.05 (two-sided). Kappa coefficient interpretation was based on the Landis and Koch study (Landis & Koch, Reference Landis and Koch1977).
Data were registered using a web-based software for data storage (RedCap) (Harris et al., Reference Harris, Taylor and Thielke2009). Statistical analyses were performed using the SPSS 22.0 software (SPSS Inc.). For all results, p < 0.05 was considered statistically significant.
Results
Demographic characteristics
During the study period, 1,503 patients were admitted to the ED-ICU, with 701 being trauma patients. Among the 802 eligible nontrauma patients, 238 did not have their screening criteria assessed and 54 were excluded (10 readmissions, 16 deaths within less than 24 hours of admission, and 28 transfers to other hospitals). A total of 510 patients (63.6% of all eligible patients) were included in the present analysis and their main demographic characteristics are described in Table 1.
Patient characteristics at ED-ICU admission
Continuous data are presented as median (1°−3° interquartile range); categorical data are presented as absolute number (percentage). ED-ICU, emergency department intensive care unit; PC, palliative care; SAPS-3, Simplified Acute Physiology Score III.
Of all included subjects, 173 (34.0%) had at least one of the nine advanced diseases assessed by question 1 in the PC screening tool. The most prevalent condition was cirrhosis (25.4%), followed by cancer (14.5%), chronic obstructive pulmonary disease (13.9%), chronic kidney disease (12.1%), frailty syndrome in elderly (11.0%), heart failure (9.3%), advanced dementia (8.1%), AIDS (3.5%), and severe neurological disability (2.3%). According to the PC screening tool, no patient was classified as category 2 (presenting one of the advanced diseases and expected to survive after 12 months from admission). Table 2 displays a comparison among the different screening category groups.
Comparison among screening category groups
Continuous data are presented as median (interquartile range, 1°–3°) with differences analyzed with the Kruskal-Wallis test; categorical data are presented as absolute number (percentage), with differences analyzed with the chi-square test. *95% confidence interval. ED-ICU, emergency department intensive care unit; PC, palliative care; SAPS-3, Simplified Acute Physiology Score III.
Severity of illness and mortality outcomes
A post hoc pairwise analysis (Table 3) demonstrated that patients in category 4 presented higher mortality (ED-ICU and intrahospital) and higher use of mechanical ventilation compared to category 1 (p < 0.05) and category 3 (p < 0.05) patients. However, there was no statistically significant difference between categories 1 and 3 regarding these outcomes. Similarly, SAPS-3 admission score was higher in category 4 compared with categories 1 and 3 (adjusted p < 0.001), but no statistically significant difference was found between categories 1 and 3 (adjusted p = 0.999).
Post hoc pairwise comparison among groups
The pairwise comparison for categorical variables used Bonferroni corrections and p value is expressed as <0.05 or ≥0.05 (NS, nonsignificant). *The Dunn-Bonferroni approach was used for continuous variables and the actual p value is reported. ED-ICU, emergency department intensive care unit; PC, palliative care; SAPS-3, Simplified Acute Physiology Score III.
Palliative care needs measures
According to the post hoc pairwise analysis (Table 3), patients in categories 3 and 4 had a PC consultation ordered more frequently than patients in category 1, considering either the ED-ICU stay (p < 0.05) or the entire hospitalization (p < 0.05). Category 3 patients did not differ significantly from category 4 patients regarding these metrics.
The time between ED-ICU admission and PC consultation was significantly shorter for patients in category 4 compared to category 1 (median of 4 vs. 15 days; adjusted p < 0.001), but no significant difference was found in other group comparisons. Similarly, category 4 patients presented a higher transfer rate to a specialized PC-bed compared with category 1 (16.4% vs. 12.2%, p < 0.05), but no difference was found in other group comparisons.
Inter-rater reliability analysis
Inter-rater reliability analysis of the PC screening tool shown that both critical care and emergency physician screeners agreed in 80% of the cases (patient classification in the same screening category). Kappa coefficient was 0.75 (CI 95% = 0.62, 0.88), representing a substantial agreement between both raters.
Discussion
In the present study, we assessed the criterion validity and inter-rater reliability of a PC screening tool for patients admitted to an ED-ICU. This PC screening tool was developed based on the GSF-PIG with the addition of a modified “surprise question” capturing the short-term prognosis of acute care patients. The aim of developing and validating this instrument is to provide non-PC clinicians working in ED-ICUs with a decision aid tool for early identification of patients in need of PC consultation.
Not all patients admitted to a hospital benefit from a PC consultation. As such, a wide variety of criteria has been developed to assess patient groups for whom a PC intervention may be appropriate (George et al., Reference George, Phillips and Zaurova2016; Nelson et al., Reference Nelson, Curtis and Mulkerin2013; Weissman & Meier, Reference Weissman and Meier2011). The majority of these tools use specific criteria, such as prolonged hospital and ICU lengths of stay (Braus et al., Reference Braus, Campbell and Kwekkeboom2016; Norton et al., Reference Norton, Hogan and Holloway2007), which are not applicable to patients admitted in an ED-ICU. In fact, the median time between ED arrival and ED-ICU admission in our study was less than 16 hours and the median ED-ICU stay was less than 4 days, making these criteria inapplicable in the ED-ICU setting.
The screening tool used in our study is based on the GSF-PIG and present two “surprise questions” to evaluate short-term (intrahospital) and long-term (12 months) survival prediction. We found that almost one-fifth of patients admitted to the ED-ICU were category 4 (presence of one of the advanced diseases and negative responses to both “surprise questions”). However, no patient was screened as category 2 (presence of one of the advanced diseases and positive response to the traditional “surprise question”). This might have occurred because the presence of one of the advanced diseases in a critically ill patient is, per se, an indicator of poor 12-month prognosis. Therefore, the traditional “surprise question” may not be helpful in a screening tool for ED-ICU patients. Accordingly, previous studies have demonstrated the utility of the traditional “surprise question,” but only for non-critically ill patients (Moroni et al., Reference Moroni, Zocchi and Bolognesi2014; O'Callaghan et al., Reference O'Callaghan, Laking and Frey2014; Small et al., Reference Small, Gardiner and Barnes2010), and recent studies have proposed a modified “surprise question” for ED patients (Hamano et al., Reference Hamano, Morita and Inoue2015; Haydar et al., Reference Haydar, Almeder and Michalakes2017).
Corroborating to the criterion validity of the proposed screening tool, we found severity of illness (SAPS-3 score and mechanical ventilation), mortality (ED-ICU and intrahospital), and PC-related measures (order for a PC consultation, time between admission and PC consultation, and transfer to a PC bed) were significantly different across groups, more evidently between categories 4 and 1. Category 3 patients presented similar outcomes to patients in category 1 for severity of illness and mortality. However, category 3 patients had a PC consultation ordered more frequently than category 1 patients, suggesting that category 3 patients present more PC needs than category 1 patients, despite similar severity and prognosis.
Interestingly, there was no difference regarding transfer from ED-ICU to traditional ICUs and length of stay (ED-ICU and hospital) among category groups. In part, this may reflect the lack of specific criteria for ICU admissions in our institution as well as the absence of PC screening tools to identify patients with poor prognosis and reduced benefit from critical care interventions. Many category 3 and category 4 patients may be receiving excessive and futile advanced care, with prolonged lengths of ICU and hospital stay. Previous studies have suggested that the lack of awareness regarding prognostication and prioritization criteria may lead to the ineffective use of ICU beds and the prolonging of futile and nonbeneficial interventions, such as mechanical ventilation (Ramos et al., Reference Ramos, Perondi and Daglius Dias2016). In our study, for instance, 51.8% of category 4 patients used mechanical ventilation, despite poor prognosis and advanced preexisting diseases. Additionally, only 14.3% of category 3 and 26.4% of category 4 patients in our sample had a PC consultation ordered during their ED-ICU stay. Future research can study the adoption of the proposed screening tool to trigger PC consultations for all patients in categories 3 and 4, investigating the impact of this intervention on unnecessary transfers to ICUs, prolonged length of stay, and quality of care for these groups of patients.
An important limitation of our study was the retrospective design and the nonsystematic inclusion of patients. This limitation occurred because only one researcher applied the screening tool and was not available full time. Despite this, we assessed the screening criteria of 70.3% of all admissions during the study, which we believe is a representative sample of our population. Future studies should proceed with a prospective inclusion of all consecutive patients to avoid possible selection bias. Another limitation was the inter-rater reliability analyses that involved only two expert physicians. Future research should study the inter-rater reliability among novice physicians because they may also use this screening tool. In addition, we used order for a PC consultation, time between admission and PC consultation, and transfer to a PC bed as proxy for PC needs. Because these decisions were based on clinical judgment, future studies can use standardized criteria to assess PC needs and the impact of PC interventions. Other PC needs metrics, such as symptoms management, family distress, communication quality, and goal clarification can also be used.
In summary, our research investigated the criterion validity of a PC screening tool for patients admitted to an ED-ICU, discriminating three groups of patients within a spectrum of increasing severity of illness, risk of death, and PC needs, and presenting substantial inter-rater reliability. Because the intention of this tool is to support decision-making in a clinical setting and our data were based only on retrospective electronic health record review, this study is the first step towards the utilization of a PC screening tool in the ED-ICU. Future research should investigate the implementation of these screening criteria into routine practice of an ED-ICU.
Acknowledgments
SCCR, RTC, JAR, and RDD contributed to the conception, design, and interpretation of data. Data acquisition was performed by SCCR and statistical analysis was performed by RDD. All authors contributed to drafting and revising the manuscript.
Conflicts of interest
The authors declare that they have no conflict of interest.
