Introduction
Mass-casualty incidents (MCIs) involving children are rare, and adequate preparation often requires disaster simulation. While potentially helpful to teach and reinforce uncommon procedures, such as triage for children involved in MCIs, drills require coordination, space, equipment, and participation. One potential way to decrease the personnel required for a disaster drill is to eliminate live moulaged actors as the “patients.” High-fidelity simulators are well-accepted by participants in disaster drills,Reference Gillet, Peckler and Sinert 1 with equivalent triage times,Reference Schultz, Skrzypczak and Raith 2 but also with longer times to perform critical actionsReference Wallace, Gillet and Wright 3 and with conflicting results regarding correct performance of critical actions.Reference Gillet, Peckler and Sinert 1 , Reference Schultz, Skrzypczak and Raith 2 A virtual-reality setting has also been tested and seems to provide a reasonable surrogate to standardized patients.Reference Andreatta, Maslowski and Petty 4 - Reference Vincent, Sherstyuk and Byrgess 7 However, both of these options are not available globally in all drill settings. Table-top exercises, while simpler to create, may lack the realistic feel and discourage a step-wise approach to gathering and interpreting physical findings on patients.
Interactive screen-based simulation is not a new concept in medical education. It ideally is suited for disaster preparation by allowing flexibility of environment and access to providers who may not be able to attend a scheduled drill. Computerized scenarios can also be used to simulate large numbers of patients in an MCI without significant equipment or personnel costs. Computerized scenarios have been used to improve response to a pandemic influenza surge and for mass-casualty planning on a larger scale.Reference Leaming, Adoff and Terndrup 8 , Reference Scheulen, Thanner and Hsu 9 A handful of groups have designed online MCI virtual scenarios for provider education, demonstrating equivalent learning to simulation mannequinsReference Youngblood, Harter and Srivasta 10 and overall receptiveness of participants.Reference Heinrichs, Youngblood and Harter 11 A few papers have described interactive computerized games for MCI teaching.Reference Roy, Sticha and Kraus 12 , Reference Yu and Ganz 13 To the authors’ knowledge, no prior studies directly have compared the fidelity and outcomes of a computerized scenario for pediatric triage in an MCI with a live actors scenario.
The objective of this study was to compare the triage accuracy, time, and fidelity using the child version of the Simple Triage and Rapid Treatment (JumpSTART) triage algorithmReference Romig 14 between computerized, pediatric, MCI patients to live, moulaged, adult actors portraying children in an MCI simulation.
Methods
Study Population and Setting
A disaster drill was conducted at a single university medical school with pre-clinical medical students who were recruited through an Emergency Medicine student interest group. Students were given an initial 15-minute lecture discussing pediatrics MCIs and the JumpSTART triage method. University of Southern California (Los Angeles, California USA) Institutional Review Board approval was secured for this study.
Simulated Patient Development
Each simulated patient scenario was developed based on actual trauma patient records. Eighteen pediatric trauma charts from a single Level 1 trauma center were reviewed for this study. Authors of the study assigned JumpSTART triage categories based on available prehospital data. Using a modified Delphi method over two iterations, a total of 12 cases, representing all four triage levels, were chosen for the drill. Eight cases were allocated for the computerized cases, and four cases were allocated for the human portrayal. The appropriate outcomes at each JumpSTART step and at the overall scenario were converted into an actor script by an investigator. Scenarios are described in Table 1.
Scenarios (All Results of an Earthquake)
Four moulaged medical students, who were peers to the participants, portrayed the four human scenarios. The moulage was specific to the injury; the adults were not made to look more child-like, but they were asked to act as an appropriately aged child. Each student was provided a script describing the full history and physical exam for their patient scenario with appropriate prompts for the JumpSTART triage algorithm and training from one of the physician investigators.
Computerized patients were authored using Articulate Storyline (New York, New York USA) using a two-screen, progressive, disclosure case report method using minimal text. Existing patient photos were moulaged using Adobe Photoshop 6.0. (San Jose, California USA). Realistic backgrounds, such as fires and rubble, and a soundtrack of amalgamated distress sounds, including crying infants and adults, sirens, and explosions, were added. For each case, the participant could gather physical exam information by clicking on the relevant patient body part which would provide a closer image. The participants could also select a set of simple command buttons to perform JumpSTART actions (eg, chin lift and check pulse) which then provided the appropriate response either in text form or in visual form. For pulses and respiration rates, a real-time “beating” or “inflating” symbol to convey the appropriate physiological rate was used. No numbers were provided. All cases provided information relevant to the JumpSTART algorithm when prompted by the participant, and each case required a triage level assignment prior to advancement (Figure 1 and Figure 2).
Screenshot of Computerized Scenario.
Screenshot of Computerized Scenario.
Drill Performance
The students were assigned to evaluate and triage four human moulaged actors portraying patients and seven computerized patients. Using a block randomization scheme, the order in which each subject encountered the human scenarios and the computer scenarios was pre-determined to maximize participant flow and to minimize the effect of ordering on their performance. No formative feedback was given during the session. Following the drill, a survey was administered assessing the students’ impressions of the fidelity of the drill scenario. Students received a gift card as compensation for participation.
A single board-certified or board-eligible pediatric Emergency Medicine physician was assigned to observe each live patient encounter directly and record time, accuracy, and performance of critical actions. Timing was determined using a stopwatch until verbal declaration of the triage level and placement of the appropriate triage tag onto the desk or patient, signifying expectant, immediate, delayed, or minor categories, was completed. Each case represented one of the four triage levels. No soundtrack or background setting was altered for the human scenarios. Each scenario was conducted in a private area.
For all computerized scenarios, student participants had a private laptop with headphones and went through a brief tutorial on how the program worked. The file was hosted on local learning management system software (Moodle Systems; Perth, Australia) which enabled the program to track the student's assigned triage level, the actions within the JumpSTART algorithm chosen, and the duration of each scenario. Technical problems were encountered that eliminated one computer case from analysis, and four of seven cases did not have accurate scenario durations. The technical problems only affected data tracking, not the user experience of the simulation itself.
Outcome Variables
Triage category accuracy using the JumpSTART algorithm and speed (in seconds) of designating a triage category were measured. Final survey responses measured perceived cumulative “stress” of the two modalities on a 10-cm visual analog scale (VAS) followed by a simple Boolean yes-no question on which scenario participants felt was more realistic. Inappropriate actions, defined as an unnecessary or missed component of the JumpSTART algorithm, were also counted as a secondary outcome per case. Under-actions, defined as actions that should have been completed for a given scenario but were not, and over-actions, defined as actions that were not required in a certain scenario but were done in error by the participant, were compared. JumpSTART has six potential actions that were assessed (Figure 3).
Rendering of the JumpSTART Algorithm Abbreviations: AVPU, alert, voice, pain, unresponsive scale; RR, respiratory rate; START, Simple Triage and Rapid Treatment.
Data Analysis
Data were characterized using descriptive statistics. Wilcoxon rank sum was used to compare median times to triage, and a 95% confidence interval of the difference was estimated using the Hodges-Lehman estimate. When comparing durations per scenario, the Wilcoxon rank sum was relegated to three scenarios from the computerized scenarios that did not experience technical problems and three live actor scenarios corresponding to the identical triage levels (expectant, immediate, and minor). A Mantel-Haenszel chi square was used to compare triage accuracy between modalities and to report relative risk. The Kruskal-Wallis test was used to compare duration across triage classes. Over-actions, under-actions, and stress levels were analyzed using a Wilcoxon Rank sum test with Hodges-Lehman estimate. Finally, a regression analysis using generalized estimating equations to look for the effect of modality (computerized vs live actor) was performed; the model adjusted for each participant, cast, triage acuity, and number of over- and under-actions. All data were analyzed using SPSS v19 (Armonk, New York USA) and SAS (Cary, North Carolina USA).
Results
Thirty-four students were enrolled in this study. However, since one student did not complete all scenarios and was eliminated from data analysis, 33 students were analyzed. All were first or second year medical students. Eleven scenarios were included, for a total of 363 patient encounters (231 computerized and 132 with live actors).
For the four live actor scenarios, the median time to triage was 57 seconds versus 80 seconds for the three computerized patients (median difference=23 seconds; 95% CI, 10-36 seconds). Table 2 summarizes median durations between live actor and computerized scenarios, stratified by triage level.
Time to TriageFootnote a
a Computerized delayed case experienced technical difficulty, resulting in exclusion.
The overall accuracy of triage category assignment was 85.7%. For the live scenarios, 122/132 (92.4%) patient encounters were given an accurate triage designation versus 189/231 (81.8%) patient encounters for the computerized scenario. This represented a relative risk of 1.3 (95% CI, 1.1-1.6) for live actors over computerized scenarios for correct triage assessment (P=.005).
In the computerized scenarios, participants performed a significantly higher number of tasks that were not necessary to complete the triage (using the JumpSTART algorithm) and missed a significantly higher number of actions that would have been considered necessary by the same algorithm (Table 3).
Comparison of Missed and Unnecessary Actions between Live Simulated Scenarios vs Computer Simulated Scenarios
The moulaged actors were felt to offer a more realistic encounter by 88% of the participants (SD=33%). Ninety-seven percent of participants felt that moulage contributed to the realism of the live actors. On a VAS-created de novo for this exercise, the participants ranked the median stress level during the live actor simulations at 47 (IQR=23-68.5). This was significantly more stressful than median stress levels during computerized scenarios (22; IQR=5.5-30). The median difference was 25.5 (95% CI, 16.5-35).
Finally, after adjusting for correlations within each student, as well as the case, triage acuity, and the number missing or unnecessary actions using a generalized estimating equations model, computerized scenarios were associated with an average increase in triage duration of 74 seconds (95% CI, 23.4-124.6; P=.004).
Discussion
This study focused on delineating differences between simulation modalities: computerized scenarios or live adult actors portraying children. Although a number of different triage algorithms exist for children, for the purposes of this drill, the team focused on JumpSTART, as JumpSTART is commonly used by US first responders. Jones et al found 66% accuracy when medics applied the JumpSTART triage system, with an over-triage rate of 23% and an under-triage rate of 11.2%, and a triage time of 23 seconds, when tested with live child actors and high-fidelity simulators.Reference Jones, White and Tofil 15 Cicero et al found that when residents learned JumpSTART using live adult actors and simulation mannequins, they demonstrated an improvement from 69% to 80% triage accuracy that was maintained over time.Reference Cicero, Auerbach and Zigmont 16 Sanddal et al identified 6.22 of 11 moulaged children correctly triaged by nursing and medic trainees using JumpSTART, which improved to 8.25 with appropriate training.Reference Sanddal, Loyacono and Sanddal 17 Wallis and Carley applied JumpSTART to emergency department patients and found a low sensitivity.Reference Wallis and Carley 18 Though the participants showed greater triage accuracy compared to previously published data, they took a longer time to perform triage in either modality.
For accuracy of triage designation, time required to assign a triage designation, and performance of indicated actions, use of live moulaged patients proved superior. Several potential reasons for this could be postulated. Performance of actions on the computerized scenarios required only the quick click of a button compared with the necessity of executing or verbalizing the action in the live drill. Also, actions which may have been uncomfortable to simulate on a peer (eg, rescue breaths) tended to be unnecessary over-actions on the computerized patients. Both of these factors may have contributed to the performance of over-actions on the computerized patients and the total time to assignment of a triage category. The higher perceived fidelity of the live patient scenarios may have contributed to greater accuracy of triage category.
This study was the first to examine the concept of learner-perceived fidelity between two disparate modalities for portraying pediatric patients. Each modality – computerized and adult actors – posed different limitations in fidelity. The data suggest that the interactivity and responses from the humans lent themselves to a higher level of psychological fidelity as measured by stress levels on a VAS. It appears that functional fidelity, the ability of a simulation to portray realistic inputs and outputs by the participant, was most important for the students. This is consistent with simulation literature from Norman et al and others describing the importance of functional fidelity over most other forms, including physical fidelity.Reference Norman, Dore and Grierson 19 That the realistic soundtrack and visual portrayal of a disaster arena in the computerized format appeared to have minimal effect on the sense of realism is also consistent with other work that posit functional fidelity as one of the more important concepts for fidelity and realism within simulation.Reference Norman, Dore and Grierson 19 - Reference Rudolph, Simon and Raemer 22
Limitations
This was a small group of participants, all of whom were medical students with an interest in Emergency Medicine with an immediately antecedent lecture on the triage algorithm. Comments are unable to be made on learning decay over time. The results may not be extrapolatable to first responders with triage training not immediately prior to a given MCI. Many other potential options for drill subjects exist, including moulaged children, high-fidelity simulators, and table-top scenarios, which were not tested in this particular scenario. In this situation, the young age of the children precluded the use of children to portray the victims consistently over multiple encounters. It was felt that the ability to replicate the patient scenario was more important for a research study than use of a child as “victim,” yet it was acknowledged that this would detract slightly from the realism. Further, though it seems unlikely that learner triage performance with delayed patients would be different than with the three triage levels reported here, due to a technical error, conclusions were unable to be drawn about computer-based simulation versus live actor simulation for such non-ambulatory, physiologically stable patients. In addition, the computerized scenarios relatively were static and did not use full virtual-reality capabilities, which can be used to improve functional fidelity.
Conclusion
Among a limited group of pre-clinical medical students, the use of live adult actors portraying children in an MCI allowed for higher JumpSTART triage accuracy and shorter time to triage when compared to computerized scenarios.
