By definition, disasters and mass casualty incidents overwhelm health care resources. Although these events vary in scope, duration, and number of patients affected,Reference Mace, Jones and Bern 1 a common problem in disasters is the need to use limited resources to benefit a large number of patients. During disasters, patients are triaged on the basis of the severity of their illness or injury and the likelihood that they will survive.Reference Lerner, Cone and Weinstein 2 , Reference Baxt, Jones and Fortlage 3 It is often emergency medical services (EMS) providers, specifically, paramedics and emergency medical technicians (EMTs), who perform disaster triage at the scene of the event.
When pediatric patients are among the disaster victims, the problems of resource management and triage are compounded. EMS providers care for children infrequently, with children making up fewer than 10% of daily EMS patients.Reference Seidel, Henderson and Ward 4 Children with special health care needs are even more vulnerable in disasters and infrequent in EMS practice.Reference Sutton, Stanley and Babl 5 In general, pediatric EMS patients tend to be less seriously ill or injured than their adult counterparts.Reference Richard, Osmond and Nesbitt 6 Opportunities to triage children in actual disaster situations are fortunately rare, but in order to assist providers with this role, multiple pediatric disaster triage (PDT) strategies are used. The evidence behind these PDT strategies is limited, including studiesReference Romig 7 - 9 and systematic direct comparisons of their accuracy.Reference Lerner, Schwartz and Coule 10 Few head-to-head comparisons of the efficacy of such strategies have been performed, and those that do exist are often statistical analyses prone to the shortcomings of database research.Reference Emerman, Shade and Kubincanek 11 - Reference Cross and Cicero 14 Therefore, it is unclear which PDT strategy best determines severity of illness or injury and likelihood of survival.
At the disaster site, EMS providers perform primary triage. Common features of PDT systems include rapid assessment of patients and little if any treatment provided to the patients until all victims have been assessed. However, there are differences among states, and even neighboring EMS systems, regarding what PDT strategy is used. For example, at the inception of this study, the adjacent states of Connecticut, Rhode Island, and Massachusetts each used a different PDT strategy. Standardization of the PDT strategy could allow for better communication among EMS providers and between EMS and receiving hospitals.
Most PDT strategies are algorithmic, assessing a patient’s ambulatory status, airway, breathing, circulation, and neurologic status.Reference Romig 7 , Reference Cone, Serra and Kurland 8 , Reference Wallis and Carley 12 , Reference Lerner, Schwartz and Coule 15 , Reference Sacco, Navin and Waddell 16 Some strategies calculate a numeric scoreReference Sacco, Navin and Waddell 16 , Reference Navin, Sacco and McGill 17 or allow EMS providers to use their clinical judgment about triage decisions without the use of an algorithm.Reference Qazi, Kempf and Christopher 18 , Reference Fries, McCalla and Levitt 19
Using a sample of EMS providers from a prospective cohort study, we conducted a cross-sectional analysis to compare the triage accuracy of 2 commonly used algorithm-based PDT strategies (JumpSTART and Smart) and clinical decision-making (CDM) with no algorithm. The primary outcome of the study was to determine if 1 of the 3 triage strategies yielded more instances of accurate triage than the others.
Methods
Participants
Participants were EMS providers in Rhode Island, Massachusetts, and Connecticut. Participants were recruited via e-mail and in-person invitations. The participants were enrolled in a PDT curriculum, and prior to participation in the curriculum, the participants completed a survey using a 5-point Likert scale to record their self-efficacy in performing PDT and treating pediatric victims. Comparative self-efficacy for treating pediatric asthma and head injuries, 2 common reasons for EMS transport, were also assessed. The Likert scale was anchored at novice (rating 1) and expert (rating 5). Finally, the pre-participation survey also included items about (1) previous disaster experience, (2) previous disaster coursework, (3) which of 3 curricular disaster triage simulations the participant completed for this study, and (4) the number of years working as an EMT or paramedic. The participant EMS providers included EMTs, paramedic students (most of whom were already EMTs), and practicing paramedics. The institutional review boards of the 3 academic institutions that conducted this project reviewed and approved the study.
Curriculum
We used a modified Delphi method to create 3 high-fidelity simulations and attendant evaluation tools for the participants who completed the simulations. In the standard Delphi method, participants are anonymous; however, in this study, subject matter experts were known to each other. A total of 8 experts were involved in the process. The development of the simulations has been previously described.Reference Cicero, Brown and Overly 20 The 3 simulations each included 10 patients with evaluation tools. The simulations were a multi-family house fire, a school shooting, and a school bus crash. The selected scenarios were chosen because such events are common when compared to other types of multiple patient incidents. Furthermore, the likelihood of these unnatural events should not be affected by geography or weather patterns.
Within each simulation, each of the 10 patients had a different illness or injury. Across the simulation scenarios, the same presentations and injuries were represented. Furthermore, we represented the same sounds (eg, sirens), smells (eg, smoke), and injuries at each site. Examples include a patient who was unconscious with a head injury and another with tachypnea after a chest injury. Table 1 shows the triage domains assessed for each patient and the simulation modality used to represent the patient.Reference Cicero, Brown and Overly 20
Simulated Disaster Victims With Learner Domains for Triage Assessment and ManagementFootnote a
a Abbreviation: CSHCN, child with special health care needs. X denotes the domain was to be assessed for the patient. MegaCode Kid, SimMan, and SimBaby were from Laerdal (Wappingers Falls, NY).
Using a modified Delphi method,Reference Qazi, Kempf and Christopher 18 subject matter experts from across North America assigned expected triage levels for each of the patients. Delphi experts determined 1 of 4 triage levels: RED (Emergent), YELLOW (Delayed), GREEN (Ambulatory), and BLACK (Deceased). For all 3 PDT strategies, RED is defined as patients with life-threatening illness or injury who might survive if treated immediately, YELLOW is a nonambulatory patient without life-threatening injuries, GREEN is an ambulatory patient, and BLACK is a patient who is dead or expected to die given the disaster situation and current resources.
Each site used one PDT strategy, according to local policy. Smart TriageReference Cone, Serra and Kurland 8 was used in Connecticut, CDMReference Qazi, Kempf and Christopher 18 in Rhode Island, and JumpSTART/STARTReference Romig 7 Triage in Massachusetts. In Connecticut, the Smart algorithm and length-based triage tape were available to participants. In Massachusetts, the JumpSTART algorithm was available to participants. In all 3 states, participants were instructed to use the local PDT strategy, by name. The curriculum was 5 hours in total, including prebriefings, the simulations, individual debriefings, and an interactive e-learning module.
Evaluation of Triage Performance
Performances on the first simulation, before any educational interventions, were rated by using a checklist-based tool.Reference Cicero, Brown and Overly 20 The timing of the simulation in this study is shown in the project flow diagram (Figure 1). We chose to compare triage outcomes before any educational interventions to approximate real-world performance of the 3 PDT strategies. The participants’ performances in their first disaster simulation are most likely to correlate with performance during actual disasters, without the benefit of recent triage training.
Design of the Prospective Cohort Study. The cross-sectional study presented in this article occurred in the study onset phase.
Participants were video-recorded by use of a single handheld video camera and a wearable microphone by a single videographer as they completed the disaster triage simulations. Video recordings were de-identified and stored in a password-protected online file repository. The same subject matter experts who participated in curriculum design served as evaluators and reviewed the learner video recordings. At least 2 evaluators reviewed each video, with a third reviewer evaluating the video, breaking the tie if there was a disagreement about triage accuracy. The evaluators used iteratively designed, checklist-based evaluation instruments and a global assessment of performance to assess each learner. The evaluation instruments were designed alongside the simulations during the modified Delphi process. Accuracy was defined as participant triage agreement with the pre-determined Delphi gold standard and was agnostic to any single triage strategy.
Statistical Analysis
Unadjusted comparisons of triage accuracy among the 3 types of triage strategies were performed by using chi-square or Fisher’s exact tests, grouped by the following variables:
-
∙ Each triage level, meaning the collective accuracy of triage for all of the patients expected to be triaged into each of the 4 triage categories (RED, YELLOW, GREEN, BLACK).
-
∙ The triage of individual patients, considering the nature and gravity of each patient’s illness or injury.
Adjusted analysis of the effect of triage strategy on triage accuracy included participant factors such as level of training (EMT student, EMT, paramedic student, or paramedic), prior disaster training, prior disaster experience, self-efficacy assessment, and which of the 3 disaster scenarios was completed (house fire, school shooting, or school bus rollover). To perform these analyses, generalized estimating equations modeling of the binary outcome (accurate triage, yes/no) was used. This approach allows estimating the effects of explanatory variables on triage accuracy, taking into account the dependency in the data from the repeated observations of 10 types of patients within each study participant. Data were summarized by using odds ratios (ORs) and 95% confidence intervals (95% CIs). Because the primary comparison of interest was between algorithm-based PDT strategies (JumpSTART, Smart) and CDM, the alpha level was adjusted by using the Bonferroni approach for multiple comparisons to be 0.05/2=0.025.
Results
Participant Characteristics
For the study period of 2011 to 2013, a total of 273 participants completed the simulation and were evaluated. Of these 273 participants, 175 completed the house fire simulation, 89 completed the school bus rollover simulation, and 9 completed the school-shooting scenario. Table 2 shows the participant characteristics, including level of training, previous disaster coursework and experience, and self-efficacy in disaster and nondisaster clinical domains. To reiterate, the Smart triage group (n=122) represented Connecticut, the JumpSTART group (n=71) comprised the Massachusetts participants, and Rhode Island (n=80) used CDM with no formal PDT strategy. The participants differed across sites, notably regarding number of years experience in EMS, level of training, and previous disaster experience. The JumpSTART group was skewed toward more junior participants. Despite their more junior status, the JumpSTART group rated themselves more highly in self-efficacy with PDT than did the Smart group, and the JumpSTART and Smart groups had similar self-efficacy with asthma management, whereas this varied in the CDM group.
Preparticipation Characteristics of Pediatric Disaster Triage Study ParticipantsFootnote a
a Abbreviation: EMT, emergency medical technician. P values of 0.05 or less were considered significant.
Odds of Triaging a Patient Accurately By Triage Strategy
There was no significant difference in performance between Smart triage and CDM with no algorithm (P=0.43; OR: 0.87; 95% CI: 0.61, 1.24). However, when JumpSTART triage was used, the odds of selecting the correct triage level were higher than when either Smart (P=0.002; OR: 2.03; 95% CI: 1.30, 3.17) or CDM (P=0.02; OR: 1.76; 95% CI: 1.10, 2.82) was used.
Accuracy of Each Triage Strategy to Determine Each Color-Coded Triage Level
There was a significant effect of predetermined color-coded triage level (RED, YELLOW, GREEN, or BLACK) on triage accuracy. Patients color-coded as BLACK or GREEN had higher odds of being triaged accurately than did patients triaged to the other colors (P<0.0001). Patients with expected triage RED were more likely to be accurately triaged than were YELLOW patients (P<0.001).
In addition to its main effect on the accuracy of triage, the triage levels significantly affected the associations of the type of triage strategy on the outcome (P=0.03), as well as the type of scenario on the outcome (P=0.02). Table 3 summarizes the effects of explanatory variables on the triage accuracy stratified by the triage levels of patients considered in the simulations. Of note, the analysis was not powered to detect significant interactions, and after stratification, a number of P values were not significant.
Comparison of Triage Accuracy For RED, YELLOW, GREEN, and BLACK Patients by Pediatric Disaster Triage Strategy, Simulation Scenario, and Previous Disaster CourseFootnote a
a Abbreviations: CI, confidence interval; OR, odds ratio. JumpSTART outperformed Smart and CDM when there was a significant difference. P values of 0.05 or less were considered significant.
b Insufficient data to estimate parameters.
For RED triage, there were differences in triage accuracy, with JumpSTART outperforming Smart (P=0.05), whereas no difference was observed between Smart and CDM or JumpSTART and CDM. The same directions of associations, but with larger magnitudes, were detected for patients who should be triaged as BLACK. For simulated patients who were designed to be triaged YELLOW, JumpSTART significantly outperformed both Smart (P<0.001) and CDM (P<0.001). For ambulatory, or GREEN patients, there were no significant differences among the PDT strategies.
Effect of Disaster Scenario on Triage Accuracy
The performance in the first of 3 disaster simulations each learner completed was evaluated in this study. The school-shooting scenario was not presented as the first scenario for most of our learners as a result of a decision related to the timing of the training session and events that occurred at Sandy Hook Elementary School in Newtown, Connecticut. A total of 175 learners were evaluated who completed the house fire simulation, 89 who started with the school bus rollover, and only 9 who started with the school shooting (prior to December 2012).
The comparison of overall accuracy based on the 3 different simulations showed a difference between the house fire scenario and the school bus rollover (P=0.01; OR: 1.34; 95% CI: 1.07, 1.68), with greater accuracy in the house fire scenario. However, there was no significant difference in overall accuracy between the school shooting and the school bus rollover (P=0.33; OR: 1.20; 95% CI: 0.83, 1.74) or between the school shooting and the house fire (P=0.57; OR: 1.11; 95% CI: 0.77, 1.61). The patients’ triage levels modified these associations, with no significant differences by scenario observed among deceased patients (coded as BLACK) (Table 3).
Effect of Learner Characteristics on Triage Accuracy
Learners who had taken a previous disaster course had greater triage accuracy than did those with no prior disaster training (P=0.02; OR: 1.4; 95% CI: 1.1, 1.8). When intended color-coded triage level was considered, learners with previous disaster coursework triaged RED patients (P=0.02; OR: 1.53; 95% CI: 1.09, 2.13) more accurately than did learners without prior disaster coursework. However, learners’ previous disaster coursework was associated with no significant difference in triage accuracy for the other color codes of patients (Table 3).
Other learner characteristics were not associated with a difference in triage accuracy. These characteristics included prior disaster experience, learner’s self-efficacy with disaster triage, and number of years of experience as a paramedic or as an EMT.
Finally, there was no significant difference in triage accuracy based on training level. Comparing paramedic students to paramedics (P=0.62; OR: 1.09; 95% CI: 0.82, 1.39), paramedic students to EMTs (P=0.77; OR: 0.59; 95% CI: 0.71, 1.22), or paramedics to EMTs (P=0.35; OR: 0.87; 95% CI; 0.64, 1.17) showed no significant difference.
Accuracy of Each Triage Strategy to Determine Triage Level For Individual Patients
Regarding the individual patients, irrespective of triage level, there were few differences in triage accuracy, as shown in Table 4. Comparisons were made for patients in the house fire scenario and the school bus rollover scenario. Patients for whom there was a significant difference in triage accuracy when different PDT strategies were used included a lifeless infant in the school bus rollover scenario. This infant was more likely to be accurately triaged BLACK by paramedics and EMTs who used the JumpSTART algorithm than by those using Smart (P=0.01) or CDM (P=0.02).
Number of Correct Triage Outcomes by Pediatric Disaster Triage Strategy for Individual PatientsFootnote a
a Abbreviation: CDM, clinical decision-making. School shooting scenario data were omitted because no participants used JumpSTART or CDM without an algorithm to triage school shooting patients.
Discussion
This simulation-based study of PDT strategies showed that, overall, JumpSTART outperformed Smart triage and CDM with no algorithm. At the color-coded triage level, JumpSTART outperformed Smart for RED patients, outperformed CDM for BLACK patients, and outperformed both Smart and CDM for YELLOW patients.
There were noteworthy differences in PDT strategy performance in the different triage scenarios. Specifically, RED patients were more likely to be triaged accurately in the school-shooting scenario than in the school bus rollover, and GREEN patients were more likely to be accurately triaged in the house fire scenario than in the school bus rollover. This finding was unexpected: the simulations were designed to be equally difficult, with similar numbers of patients in each color-coded triage level. Possible explanations include concerns about children with special health care needs in the school bus rollover, or participant’s concerns about penetrating trauma in the school-shooting scenario versus blunt trauma in the school bus rollover. Another possible explanation is the degree of explicitness of the moulage used to depict the victims’ injuries. For example, the gunshot wound victims had more obvious, bloodier injuries than did the school bus rollover victims.
Considering the accuracy of the 3 PDT strategies for individual patients, the results mirrored the color-coded triage outcomes in aggregate. The better performance of JumpSTART for patients with unstable airways is of unclear significance. However, because children are more prone to airway and respiratory emergencies than are adults, disaster planners and EMS educators may wish to consider this finding when choosing a PDT strategy.
Some participant factors were associated with increased likelihood of triage accuracy. Notably, participants with prior disaster education, and not those with prior disaster experience, demonstrated greater accuracy than did participants without prior disaster education. This finding may be heartening to disaster educators, but our findings are limited in that we did not assess the scope of prior disaster training nor the timespan from the prior training until participating in our intervention. Furthermore, participants’ self-efficacy in disaster triage and treatment, their years of experience in EMS, and their highest training level (EMT vs paramedic), had no demonstrable association with triage accuracy outcomes. Some possible explanations for the last finding are that triage requires no paramedic-specific skills, and that triage strategies are often designed as algorithms, which do not require the additional training paramedics undertake.
Limitations
Our study had several limitations. First, there was heterogeneity in the learner populations at our 3 intervention sites. The JumpSTART group comprised paramedic students, with some paramedics and no EMTs, whereas the Smart and CDM groups had greater proportions of both paramedics and EMTs. Whether the JumpSTART group performed more accurate triage than did the other 2 groups because of a different blend of participants is worth considering. However, another factor, previous disaster coursework, was similarly distributed across the 3 groups. Previous disaster coursework correlated with better triage performance.
The school shooting, multiple-family house fire, and school bus rollover are, unfortunately, relatively common pediatric multiple casualty incidents.Reference Fries, McCalla and Levitt 19 , Reference Kuisma, Hiltunen and Määttä 21 , Reference Schenk, Wijetunge and Mann 22 Furthermore, the incidence of such events is unlikely to be influenced by geographic region of the United States. Another limitation was the relatively small scale of our disaster scenarios, which included 10 patients. While this meets the common definitions of disaster for EMS systems, it is unclear whether differences in performance of PDT algorithms may be observed in larger-scale disasters such as natural disasters or biological or chemical disasters that would require consideration of decontamination.
Although simulation is a common training modality, there will always be limitations related to the realism of the simulation and minor variations. Significant effort went into standardizing the simulations with scripts, identical moulage, programmed high-fidelity manikins, identical background noise, and other aspects to increase the fidelity. Ultimately, differences will always remain between reality and simulation. The effects of this limitation could affect the results, although one could argue that the limitations of simulation were shared across all 3 sites.
A final limitation of the study was that other PDT strategies, such as CareFlight,Reference Cross and Cicero 14 the Sacco Triage Method,Reference Sacco, Navin and Waddell 16 or Sort-Assess-Life-Saving Treatment (SALT) 9 were not assessed. We limited our study to a comparison of JumpSTART, Smart, and CDM because these were the prevalent PDT strategies in Connecticut, Massachusetts, and Rhode Island at the onset of our study. Had we included additional prevalent and emerging PDT strategies in the comparison, the generalizability of our findings, and value of the study for disaster educators and planners, would have been even greater.
Our findings regarding triage accuracy among 3 different PDT strategies may serve as the basis for several additional investigations. First, a similar study incorporating additional sites and comparing additional PDT strategies, such as SALT or the Sacco Triage Method, would yield greater understanding of modern disaster triage efficacy. Next, a study in which laypeople and non-EMS health care workers conduct triage might lend generalizability of our conclusions and aid in disaster preparedness efforts. Another domain for study is how PDT education affects disaster response and patient outcomes in the prehospital setting, within hospitals, and in military disaster response.
Though CDM did not perform as well as Smart and JumpSTART in some domains, this approach may be a viable alternative to algorithm-based strategies. The effect of educational interventions on CDM-based PDT accuracy is worthy of additional study.
As a final note, physically attending multiple simulation and education sessions during the curriculum proved arduous for some of our learners. Implementing screen-based education, including video games on EMS performance of PDT, may affect disaster response.
Conclusion
Our data suggest that when JumpSTART is used, the highest priority RED category patients were triaged more accurately than when Smart was used. JumpSTART outperformed CDM when BLACK patients were triaged. Furthermore, JumpSTART outperformed Smart and CDM when YELLOW patients were triaged. Future investigations can focus on the accuracy of other triage methods and the use of simulation in disaster triage education and assessment.
Funding
This work was supported by an Emergency Medical Services for Children Targeted Issues Grant, HRSA grant #H34MC19349.
