Since early human history, the number of anthropogenic disasters (eg, Brussels bombings, 2016) and natural disasters (eg, West African Ebola outbreak, 2014) has continued. This led to growing concern about the ability of hospitals to absorb and treat large numbers of casualties during mass casualty incidents (MCI).
The ability of a health care facility to manage a sudden and unexpected increase in patient volume that would otherwise severely challenge or exceed its present capacity, generally, is referred to as surge capacity.Reference Hick, Barbera and Kelen 1 There are 3 kinds of surge capacities: public-health, facility-based, and community-based surge capacity.Reference Hick, Hanfling and Burstein 2
Public-health surge capacity is the ability of the public health system to increase its capacity not only for patient care but also for epidemiologic investigation, mass prophylaxis or vaccination, and other activities. Facility-based surge capacity refers to capacity-extending actions taken by a health care facility (eg, executing its emergency operating plans). When these resources are exceeded, community-based surge capacity actions (eg, establishment of off-site hospital facilities) may be implemented.Reference Hick, Hanfling and Burstein 2 In the pilot study reported in this article, our findings on a novel way to semi-automate decisions to deal with facility-based surge capacity will be presented. We will discuss, in particular, the impact of applying the concept of reverse triage through an information technology (IT) software application.
Reverse triage is a way to rapidly create inpatient surge capacity by prioritizing hospitalized patients who need the least amount of medical assistance and only have a small risk for serious complications resulting from early discharge. Such patients can immediately be discharged in MCIs to make room for the disaster victims, who are in greater need of hospital care and resources.Reference Pollaris and Sabbe 3 When looking at the 3 parts of overall surge capacity, reverse triage (also referred to as “surge discharge”) is only a small component of the first subset, conventional capacity.Reference Hick, Barbera and Kelen 1 Conventional capacity is the first phase of a health care facility’s reaction to an MCI. In this context, space, staff, and supplies used are consistent with daily practices, but the facility’s emergency operations plan has been activated. Nevertheless it is clearly understood that reverse triage contributes to the facility’s surge capacity by providing new available beds within 2-4 hours. Cancellation of elective procedures starts at the same time but, by comparison, has its first impact only 12-24 hours later.Reference Hick, Barbera and Kelen 1 A study conducted in the United States evaluated the potential surge capacities of an academic hospital, an affiliate hospital, and a community hospital by implementing an early-discharge protocol.Reference Kelen, McCarthy and Kraus 4 Reverse triage created the greatest additional capacity in each facility, yielding 50%, 55%, and 59% of new available beds, respectively.Reference Kelen, McCarthy and Kraus 4 Therefore the present study will focus only on the concept of reverse triage.
The aim of the study was to create an evidence-based IT application that uses the health electronic record (HER) of inpatients to speed up and guide the clinical decision-making in the reverse triage selection process. The application must be capable of detecting hospitalized patients who unequivocally cannot be discharged early when a disaster strikes. Doing this, we hope to reduce the number of inpatients who need to be clinically evaluated for potential early discharge in MCI situations.
Methods
Development
In 2006, Kelen, Kraus, and McCarthy took the first steps toward the development of inpatient disposition criteria for reverse triage.Reference Kelen, Kraus and McCarthy 5 They developed a patient classification system based on a risk assessment of consequential medical events (CME) resulting from early hospital discharge. The selection of criteria was guided by evidence-based materials and 39 expert panelists who discussed candidate criteria and rated them. CME was defined as an unexpected death, irreversible impairment, or reduction in function within 72 hours of hospital discharge for which an in-hospital critical intervention (CI) would have been necessary to stabilize or ameliorate the patient’s condition.Reference Kelen, Kraus and McCarthy 5 So the failure to initiate or continue a CI would potentially result in a CME. They identified 28 CIs that, if withheld, would significantly increase the risk of a CME after discharge. These interventions were then ranked on a 10-point Likert scale for the likelihood a CME would develop if a particular intervention was withheld or withdrawn (Table 1).Reference Kelen, Kraus and McCarthy 5 , Reference Allen and Seaman 6 As they succeeded in validating the 28 CIs, we started to create and evaluate the Reverse Triage Tool Leuven (RTTL) based on their original 28 CIs, with the same cut-off. Table 2 lists the criteria the RTTL uses to determine whether a patient is too medically compromised to be discharged during an MCI.
Critical Interventions (CI) of Kelen, Kraus, and McCarthy and the Likelihood of a Consequential Medical Event (CME) If the CI Is Withdrawn Or Withheld
Abbreviations: CPR, cardiopulmonary resuscitation; PPV, positive pressure ventilation; ADL, activities of daily life.
a Each intervention was ranked on a 10-point Likert scale for the likelihood a CME would develop if that particular intervention was withheld or withdrawn. Larger numbers mean a greater likelihood.
Criteria Used by the Reverse Triage Tool Leuven (RTTL) To Exclude Patients From Being Discharged Early
Abbreviations: MCI, mass casualty incidents; ADL, activities of daily life.
To achieve our goal, we did setup the RTTL to work by exclusion, such that the software would exclude patients based on their need for a CI. Patients who were excluded by the program were ones who were not healthy enough to be downgraded in care and who therefore had to remain in the health care facility, even under MCI situations. Patients who were not excluded were ones who could probably be discharged early during MCI situations because they lacked the need for any CI. The program displayed a list of these potentially dischargeable patients at the end of the filtering process. These patients, however, still required clinical re-evaluation by a multidisciplinary team (physician, nurse, social assistant) before they could be actually discharged home or downgraded in care.
The development of the RTTL occurred in three phases. In the first phase, a panel consisting of the authors was given the task of evaluating each CI to determine which would be retained or need to be adjusted, with the goal of identifying those that would be used as filters in the RTTL.Reference Kelen, Kraus and McCarthy 5 At the end of the panelists’ assessment, 20 CIs were retained, 3 CIs were omitted and 5 CIs were adjusted (see online data supplement). In the second phase, the selected CIs were translated into a description that is compatible with the HER of the University Hospitals Leuven, a large tertiary hospital in Leuven, Belgium. In the third and last phase, some additions to the criteria were made in order to refine the list of potentially dischargeable patients produced by the RTTL. Patients who do not need a CI but should not be discharged early or downgraded in care because of their condition (eg, being treated in a hematological isolation unit), as well as patients who are registered but are not actually occupying beds (eg, consultations), should be excluded from the pool of candidate dischargeable patients also (see online data supplement).
The study was approved by the commission of Medical Ethics of the University Hospitals Leuven and also registered at the Clinical Trial Centre of the University Hospitals Leuven (S-number S57389).
Pilot study
During a 3-week period in March 2015, a pilot study comprising 2 study periods was conducted at the Gasthuisberg campus of the University Hospitals Leuven, Belgium. The goal of this pilot study was to assess the RTTL’s ability to identify patients who could be safely discharged early in disaster circumstances.
During the first study period, we created a list of random patients (random group). The IT department delivered to us an alphabetical list of all patients registered at the University hospitals at 4:00 am on a daily basis; this list was a digital file in the form of a Microsoft Office Excel (Microsoft, Seattle, WA, USA) spread sheet. First, we reduced this list to those registered within the Gasthuisberg campus, and then randomly selected from it using the Excel® formula: =aselect(). This method is comparable to that used to accomplish a simple random sampling.Reference Polit and Beck 7 Then, we attempted to visit as many of that day’s patients as possible, following the randomized order. After the patients provided informed consent, they were evaluated and a registration form was completed, which included a checklist to score the RTTL and a checklist based on the original 28 CIs.Reference Kelen, Kraus and McCarthy 5 The random group enabled us to compare the outcome of possible dischargeable patients produced by the RTTL with the outcome of possible dischargeable patients produced using only the original 28 CIs of Kelen et al.Reference Kelen, Kraus and McCarthy 5
During the second study period, the programmed RTTL ran daily within the HER and composed an alphabetical list of patients who could be discharged early (ie, the filtered group). We followed the same randomization, visitation, and registration procedures as described above. The filtered group enabled us to determine the degree to which potentially dischargeable patients were appropriately identified by the RTTL. This, in turn, enabled us to identify any potential programming and registration error.
To evaluate whether both groups were comparable in terms of age and gender, we used an unpaired t-test and a χ 2 test, respectively. To compare the outcome of the RTTL with the already available and validated instrument of Kelen, Kraus, and McCarthy, we performed a χ 2 test and a 2-tailed Fisher’s exact test.Reference Kelen, McCarthy and Kraus 4 All statistical analyses were conducted using the Statistical Package for the Social Sciences (SPSS®) (IBM Corp. Released 2013. IBM SPSS Statistics for Windows, Version 22.0, IBM Corp., Armonk, NY, USA).
Results
Although the variables age and gender showed a t=0.239, df=488, P=0.811 and a χ 2 =0.009, df=1, P=0.928, respectively, we can conclude that the random and the filtered group were comparable in these terms.
Random group
During the first study period we were able to visit 417 from a total of 8828 patients on the alphabetical lists we received from the IT department. As shown in Figure 1, some patients were absent from their rooms at the time of visits, some had just died, or declined, or could not give informed consent. These were excluded from further analysis. We also excluded patients if their HER had missing data.
Selection of the Study Subjects.
A total of 230 patients were included for further analysis (Figure 1). Of these, 116 were male individuals and 114 were female individuals(Figure 2). Their age distribution is presented in Figure 3. Of these 230 patients, 67 (29%) met none and 163 (71%) met one or more of the RTTL exclusion criteria. Thus, using the RTTL criteria, 67 patients (29%) of the random group were identified to be evaluated for possible early discharge. By contrast, when the original 28 CIs were used, only 21 patients (9%) did not require any of the 28 CIs (ie, eligible for potential early discharge) and 209 patients (91%) needed 1 or more CIs (Figure 4).Reference Kelen, McCarthy and Kraus 4
Age and Gender Distribution for the Random and Filtered Group.
Age Distribution for the Random and Filtered Group.
Numbers of Potential Early Discharges by Group and Method.
Next, the above-mentioned data were organized into a 2×2 table (Table 3) and evaluated statistically. The tests produced a χ 2 =12.024 (df=1) and a P=0.002. When the validated list of the 28 CIs from Kelen, Kraus, and McCarthy was used as the gold standard, the RTTL had a sensitivity of 0.62, a specificity of 0.74, a positive predictive value (PPV) of 0.19, and a negative predictive value (NPV) of 0.95.Reference Kelen, McCarthy and Kraus 4
2×2 Table: RTTL Versus Kelen et al Criteria
Filtered group
An alphabetical list of 3186 potentially dischargeable patients was generated by the RTTL software during the second study period. Of these, we visited 417 patients. As with the random group, some patients were absent, some had just died, or some could not give informed consent (Figure 1). After excluding those patients, 260 remained for study inclusion (Figure 1). Of these, 130 were male individuals and 130 were female individuals (Figure 2). Their age distribution is presented in Figure 3.
Due to programming or registration errors in 3 criteria, the RTTL inappropriately selected 130 patients (50%) for early discharge. Most of these errors related to how the Katz scaleReference Katz, Downs and Cash 8 was implemented (128 patients). One patient who was inappropriately selected had a major surgical procedure, and 1 had a blood transfusion. When the original 28 CIs were applied to the filtered group, 43 patients (16.5%) did not require any CI, and 217 patients (83.5%) needed 1 or more CIs (Figure 4).Reference Kelen, McCarthy and Kraus 4
It is important to note that a reduction of 63.9% of the initial study population (random group average: 1261 patients/day) could be programmed, resulting in an average daily alphabetical list of 455 patients/day in the filtered group.
Discussion
In the present study, we developed and evaluated an evidence-based IT application to help guide clinical decision-making during the reverse triage selection process. We were particularly motivated to create the software for disaster situations in which MCIs occur, leading to surge capacity needs in hospitals. The RTTL assesses all hospitalized patients using objective medical criteria, and filters out those who do not qualify for early discharge under disaster circumstances, leaving a list of only those who possibly could be discharged. This culled list of patients still requires clinical re-evaluation by a multidisciplinary team before they can be actually discharged home or downgraded in care.
We compared outcomes between the RTTL and the already validated instrument of Kelen, Kraus, and McCarthy.Reference Kelen, McCarthy and Kraus 4 Our null hypothesis was that both instruments were equal in judging which patients should remain in the hospital when an MCI occurs. χ 2 and Fisher exact tests were performed, which produced a χ 2 of 12.024 (df=1) and a two-tailed Fisher exact P-value of 0.002. This means that the RTTL was not equal to the validated instrument in judging whether a patient should stay in the hospital in case of an MCI.Reference Polit and Beck 7 In the random group, the RTTL pre-selected 67 patients, whereas the validated instrument selected only 21 patients to be discharged early (Figure 4). Moreover, the instrument of Kelen, Kraus, and McCarthy detected almost twice as many patients to be discharged early in MCI situations in the filtered group than it did in the random group. Thus, despite the considerable number of errors found in programming and registration of some criteria, the RTTL still resulted in a better selection than the random method. When interpreting these results, it is important to stress several points. We adjusted some parameters in the RTTL, because they were not well defined in the original publication.Reference Kelen, Kraus and McCarthy 5 This partly contributed to the differences observed between the instruments. However, not all of the original 28 CIs could be translated into an IT program compatible with the HER. This again accounts for some of the differences.
It is important to recall that the RTTL was developed based on exclusion criteria. With a sensitivity of 0.62 and a PPV of 0.19, the first iteration of the RTTL would appear to be an inadequate instrument. However, the RTTL never had the goal of predicting whether patients would be able to be discharged early in disaster situations. It was developed to do the exact opposite: to predict which patients certainly could not be discharged, based upon specified exclusion criteria. Therefore, specificity and NPV are of much greater value in determining the predictive validity in this context. Thus, with a specificity of 0.74 and an NPV of 0.95, the predictive validity is reasonably good for the RTTL. Besides, the existence of registration and programming errors is also partly to blame for the lower sensitivity and PPV.
Drafting exclusion criteria instead of inclusion criteria resulted in both advantages and disadvantages. It is safer for an IT system to exclude patients who cannot be discharged, than predicting which patient should be discharged without multidisciplinary re-evaluation. When creating inclusion criteria, one always has to consider every possible context. This makes it much harder to be exhaustive (eg, some patients with IV therapy can be switched to oral treatment and therefore be discharged early in MCI situations but others cannot be). Furthermore, exclusion criteria are easier to translate into technical terms used by programmers. It is more preferable to have too many false positives to take to a multidisciplinary re-evaluation than false negatives. The necessity of a multidisciplinary re-evaluation is a major disadvantage.
Perhaps we have been too cautious with the creation of the exclusion criterion concerning the ADL-dependency, scored by the Katz scale.Reference Katz, Downs and Cash 8 If we excluded only patients scoring a 4 on at least 1 of the 6 items, we would have only excluded patients fully dependent on that item (or multiple items). Obviously, they cannot be discharged early in disaster situations. With the current criterion, any patient who was partly dependent on any item (score of 3) was also excluded.
As demonstrated in the supplemental information, we could not translate the 13th CI (transfusion) of Kelen, Kraus, and McCarthy into the HER. It would be too burdensome for the system to call in the hemoglobin value for every patient within the study population.7 Fortunately, no patient in the random group had a hemoglobin lower than 7 g/dl who then would have been recommended for a transfusion. Thus, the omitted item for the present study had no influence on the results presented here.
Limitations
The main limitation of the present study is that we did not assess the excluded population in the filtered group using the original 28 CIs, or the items of the RTTL. In addition, the “excluded patients” did not undergo a multidisciplinary reassessment to determine their actual preparedness for early release. Both are necessary to completely validate the RTTL as an independent reverse triage instrument. Another limitation was that many patients were excluded because their patient records had missing data. Incomplete or inadequate registration of the Katz scaleReference Katz, Downs and Cash 8 values by the nurse staff accounted for most of this missing data. This problem could have been avoided if the management would have underscored the importance of systematic and correct registration of the ADL-independent scale. If they had done so, the number of patients included would have increased by 40 in the random group and 25 in the filtered group. Another concern was the presence of multiple programming and registration errors, which resulted in half of the visited patients to be wrongly selected for early discharge. Finally, the external validity of this study was rather low. The RTTL is only applicable to the HER and patient population of UZ Leuven. However, as the HER system is used in several hospitals in Belgium, our findings may be generalizable to a limited extent locally.
Conclusion
In the present study, we developed the RTTL, an IT application for reverse triage. The RTTL was based on the original 28 CIs of Kelen, Kraus, and McCarthy, which was modified into a feasible translation for the HER system of the academic hospitals of UZ Leuven. The results demonstrated that the filter still needs significant optimization. We detected several registration and programming errors associated with the RTTL. Therefore, it needs to be refined by altering, omitting, or adjusting some of its filter items. Nevertheless, the probability of identifying dischargeable patients according to the original 28 CIs was nearly twice as high after the patients were pre-selected by the RTTL, compared with random selection of patients. The RTTL, therefore, saves time in 2 ways: (1) It reduces the pool of patients to be evaluated for early discharge to approximately one-third and (2) it doubles the probability of actual discharge in disaster circumstances. After the above-mentioned refinements to the RTTL are made, the RTTL would need to be validated as an independent tool that excludes non-dischargeable patients in disaster circumstances. Once validated, the RTTL could become a valuable clinical-decision aide for reverse triage. Furthermore, after extensive fine-tuning, its potential usage during daily surge situations in hospitals should be investigated, as emergency departments experience crowding situations on nearly a daily basis.
Funding
This research was supported by University Hospitals of Leuven.
Conflicts of Interest
None.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2017.115
