Introduction
External disasters causing mass-casualty incidents (MCIs) are unpredictable and can create an influx of patients to hospitals, often overwhelming available human and material resources. Preparation for such events involves risk assessments, creation of standard operating procedures, procurement of equipment, space, rostering of on-call staff, and training at regular intervals. 1
Triage is a crucial step in patient care during an MCI. When correctly and efficiently executed, triage categorizes patients based on medical acuity such as to prioritize medical management and optimize the allocation of limited resources. The extraordinary patient influx to triage during an MCI can easily overwhelm health care providers, thus multiple MCI triage tools have been developed to enhance and simplify the response in both the hospital and prehospital setting. A commonly used tool is the Simple Triage and Rapid Treatment (START), which divides patients into four categories: Green (minor); Yellow (delayed); Red (immediate); and Black (dead or expectant).Reference Jenkins, McCarthy and Sauer 2 Kahn, et al have shown that although START has a tendency to over-triage, the tool is 100% sensitive in identification of the most critically ill patients.Reference Kahn, Schultz, Miller and Anderson 3 While START is commonly used in the prehospital setting, it can be adapted to the hospital setting if needed. In locations without a prehospital system or in situations whereby many patients self-present to hospitals, a hospital-based MCI triage approach has been proven effective.Reference Shah, Rehman and Sayyed 4 Additionally, Lee and Franc demonstrated that applying hospital-based START triage followed by Canadian Triage and Acuity Scale (CTAS) during an MCI had similar accuracy and efficiency as applying START alone.Reference Lee and Franc 5
In the rapidly evolving era of new technologies, computerized solutions are also emerging to enable rapid triage processing and to streamline patient coordination during an MCI. Schell, et al demonstrated the feasibility of using electronic, hand-held devices to perform triage during an MCI in a blinded, crossover study.Reference Schell, Wohl, Rathe and Schell 6 Also, START can be incorporated into existing emergency department (ED) information systems, eliminating the need for paper triage cards, and integrating care during an MCI with usual ED operations.
This study compares both accuracy and speed (triage time) of computer-based (electronic) to traditional paper-based (manual) START triage during an MCI in a hospital setting. It was hypothesized that electronic triage would have a faster triage time and have similar accuracy when compared to manual triage. In addition, the user preference of each method was assessed. High end-user adoption and preference for electronic triage by both providers and patients were initially hypothesized.
Methods
Study Design & Population
A randomized, crossover study was conducted on May 26, 2016 at the Montreal General Hospital (MGH; Montreal, Quebec, Canada) during a hospital-based MCI disaster response simulation of a train derailment and chemical spill. The MGH, a tertiary care teaching hospital within the McGill University Health Centers (MUHC; Montreal, Quebec, Canada), is an urban, Canadian, Level 1 trauma center. This study was approved by the MUHC Research Ethics Board.
Accuracy of assigned category and time of triage were assessed for three health care provider types: two paramedics (PMs); two physicians/medical doctors (MDs); and two registered nurses (RNs). The providers were volunteers and were blinded to study objectives and outcomes. The manual method used traditional paper charts and cards with triage category color to be place around a patient’s neck. The electronic method used computer software and mobile computer units. A total of 30 simulated patients were triaged, both manually and electronically, by each health care provider type; simulated patients therefore acted as their own controls in the design. The simulated patients were volunteer students from undergraduate health science programs. There were no exclusion criteria for participants.
Three separate simulation exercises, one for each health care provider type, were conducted. Within each exercise, crossover of triage mode (electronic or manual) used by each provider occurred after 15 patients.
Simulation Exercises
On the day of the simulation, health care providers received a 30-minute training session on the use of the START disaster triage interface on the MUHC ED’s information system.
Simulated patients were randomly assigned individual scenarios and prepped with moulage on the morning of the simulated exercise. A total of 30 patients participated in the exercise, with 14, six, nine, and one patients simulated to have injuries consistent with Green, Yellow, Red, and Black triage categories, respectively.
Three different exercises were conducted throughout the course of this study. Exercises 1 and 2 were triaging exercises separately performed in a room outside of the ED by PMs and MDs, respectively. These were completed prior to Exercise 3, which was conducted in the ED during the live MCI. In Exercise 3, the live MCI, RNs were the triaging providers. In each exercise, patients sequentially underwent manual then electronic triage. Within each exercise, the providers switched triage modality (manual or electronic) after the first 15 patients had completed both stations. All simulated patients were triaged by each provider type twice, once with each modality, within each exercise. Figure 1 depicts the flow of activities for the three simulated exercises.
Flow of Activities.
Randomization Method
On arrival, simulated patients were randomly assigned numbers corresponding to scenarios matched to the pre-determined triage categories. The software Research Randomizer (Social Psychology Network; Middletown, Connecticut USA) was used to generate a pre-determined number sequence to randomize the order in which simulated patients presented to health care providers in Exercises 1 and 2. Simulated patient presentation order was not randomized nor pre-determined in Exercise 3, as this exercise was integrated into the hospital-based MCI simulation. However, patients in Exercise 3 still presented first to the manual station for triage, followed by the computer station.
For each of the three exercises, providers were randomly assigned to begin with either the manual or electronic triage method. To decrease inter-provider variability, each health care provider switched triage method after 15 patients; each individual provider triaged all 30 patients once, 15 by each method.
Outcome Measures & Data Collection
This study’s primary outcomes were accuracy and triage time. Accuracy was defined as the proportion of provider-assigned START triage levels correctly matched to the pre-determined assigned categories. Triage time was defined as time in milliseconds measured from the point of first contact with the patient by the health care provider until the allocation of a triage color to the patient, indicated by a manual checkmark or mouse click in the manual and electronic methods, respectively.
Primary outcome data were collected by two research assistants using pre-designed data collection forms and smartphone-enabled stopwatches. Assistants were randomly assigned to either the manual or electronic triage stations, where they remained through all three exercises. Both research assistants were blinded to study objectives.
Secondary outcomes were measured using survey data. All participants were asked to identify their preferred modality (ie, manual, electronic, or no preference) and to provide open-ended comments to explain their selections. These data were collected using a standard feedback form following the exercises. Health care providers and simulated patients were asked to complete the forms without communication with each other to minimize bias.
Post-hoc Analysis
The triage time definition, consisting of START category selection as an endpoint within each modality, was designed to ensure that each method had comparable and consistent measures. However, the study’s triage time outcomes were inconsistent with the secondary outcomes suggesting end-user perception that the electronic method was faster. Post-hoc, it was hypothesized that the original timed endpoints used for triage time under-estimated the actual complete time required for manual triage. It failed to account for the time required by an assistant to code the START paper card and place the triage card around the patient’s neck following START category selection. Both these actions were not required in the electronic version (Figure 2).
Process.
For exploratory purposes and to motivate future studies, a post-hoc analysis was conducted assessing the time required to complete these additional actions. To accomplish this, the extra steps were replicated after the original exercise in the same location by one provider (MD) and with one simulated patient for each triage category. No moulage was used since determination of triage category was not a factor in this post-hoc simulation. Only the additional steps, starting from the initial manual endpoint (checkmark) and ending with the card placed around the patient’s neck, were timed. These steps were timed twice for each of the triage categories and averaged in the final reporting. These post-hoc time data collected were added to the original data collected for the manual timed-triage section in a separate data analysis; this adjusted time is referred to as complete manual triage time in study results and discussion.
Statistical Analysis
Triage times are reported as means and standard deviations, with triage accuracy and preferences reported as percentages. While the results are mostly intended for descriptive purposes, the analysis also implemented random effects mixed models for inference, with method, color, and health care provider type and their interaction as fixed effects, and individual patients and health care provider being entered as random effects. Differences in triage times between electronic and manual methods were estimated using the least squares means from these mixed models. Differences in rates were tested using the chi-square test. A P value of<.05 was used to determine significance and was corrected for multiple comparisons using Bonferroni procedures. SAS version 9.2 (SAS Institute Inc.; Cary, North Carolina USA) was used for all analyses.
Results
Primary Outcomes
The accuracy of the electronic and manual methods was similar overall, 83% and 80% (P=1.0), respectively, and did not differ significantly across any of the individual health care provider types (Table 1). In addition, accuracy of triage for electronic and manual methods were similar across all triage categories (Table 1).
Accuracy of Triage by Health Care Provider Type and START Category
Note: The P values for a difference in accuracy between electronic and manual methods are Bonferroni corrected for multiple comparisons.
Abbreviations: MD, medical doctor; PM, paramedic; RN, registered nurse; START, Simple Triage and Rapid Treatment.
On average, the least square means triage time using the manual method was estimated to be 8.4 seconds faster (P<.001) for PMs (Table 2). While small differences in triage times were observed for MDs and RNs, they were not significant after accounting for other factors according to the mixed models (Table 3).
Mean Time (seconds) for Measured Time of Triage by Health Care Provider Type and START Category, for Electronic and Manual Methods
Abbreviations: MD, medical doctor; PM, paramedic; RN, registered nurse; START, Simple Triage and Rapid Treatment.
Estimated Difference (seconds) between the Mean Electronic Method Minus the Manual Method, According to the Least Squares Means Obtained by the Mixed Model
Note: The P values for difference in times between electronic and manual methods are Bonferroni corrected for multiple comparisons and bolded for P<.05; negative differences indicated the electronic method was faster.
Abbreviations: MD, medical doctor; PM, paramedic; RN, registered nurse; START, Simple Triage and Rapid Treatment.
a The Complete Manual estimate refers to the estimated difference after adjusting for additional steps needed from the manual method.
In contrast to the primary outcome data, secondary outcome data pointed to a perception that the electronic method was faster both by health care providers and simulated patients. As this was not congruent with the primary outcome data, research methodology was reviewed to identify a source for the discrepancy between the objective and subjective findings. As described in the post-hoc analysis of the Methods section, two additional START card manipulation steps are required in the manual method, which were not accounted for using the original triage time endpoints (Figure 2).
When including the additional steps as part of manual triage time, an average of 10.7 seconds was required for triage for Green-, Yellow-, and Red-labeled patients, while Black-labeled patients required an additional 16.1 seconds as they also required manual lifting of their heads to place the triage card around their necks. According to the complete manual triage time, the electronic method was 9.2 seconds and 15.7 seconds faster for MDs and RNs (P<.001), respectively, while RNs were the fastest providers overall (Table 3).
Secondary Outcomes
All six health care providers completed the standard feedback form. The health care providers preferred the electronic method with only one provider (a PM) who indicated a preference for manual triage and one provider (an RN) who indicated no preference (Table 4).
Preferred START Triage Modality
Abbreviation: START, Simple Triage and Rapid Treatment.
Of those providers who preferred the electronic method, all four reported that their preference was linked to the ease of use. One health care provider elaborated by stating that initially the electronic method was slightly more difficult because of the novelty, but as the exercise progressed, the electronic method became easier to use. Both MDs stated they believed the electronic method was faster and more efficient, emphasizing that it did not require help from another staff member. Health care providers also believed that the electronic method would result in fewer errors as triaged patients were automatically added into the patient-management system and easily tracked throughout their entire medical encounter by any provider. The provider who preferred the manual method stated speed as the reason for his preference but specified that faster speed was specific to the triage decision and did not include the time required for the assistant to place the card around the patient’s neck.
Twenty-nine of 30 simulated patients completed the standard feedback forms. Aggregated together, the results did not point to a strong preference for either of the two modalities of triage. The plurality of votes went to “no preference.”
Simulated patients who preferred the electronic triage method provided reasons such as speed, perception that the providers were better organized, and the feeling that their information would be better coordinated and passed on between providers amidst the chaos. The simulated patients who preferred the manual method felt that they were more likely to be triaged accurately as the interactions with providers were more personal. This personal attention also helped calm them down amidst the chaos of the MCI. Those who stated “no preference” fell into one of two general categories: those who were indifferent and simply trusted providers to use the best possible modality available, and those who were indecisive when weighing the aforementioned pros and cons between the two methods.
Discussion
During MCIs, triage is one of the most important steps to ensure effective patient management and proper allocation of limited resources. Many medical centers are adopting electronic charting for patient encounters, but the feasibility and impact of electronic MCI triaging has not been well studied. This study provides objective and subjective data to assist in a health care system’s choice of MCI triage modality.
In this study, triage modality (electronic versus manual) had no significant effect on triage accuracy overall, across providers or across triage categories. This is an important finding, yet not unexpected given that the cognitive process of working through the START algorithm and assigning a triage group is constant, regardless of the triage modality.
Both modalities having equivalent triage times overall was unexpected and incongruent with secondary data. When the triage process was mapped out for analysis, a potential reason for the discrepancy was identified as the failure to account for the time required to code and place the START triage card around the patient’s neck in the manual method. Simulated patients and health care providers experienced these additional steps as part of the manual triage process, potentially giving the perception that the electronic triage method was faster, thus explaining the discrepancy between primary and secondary data.
To explore this theory, the post-hoc analysis was conducted and the estimated least square-means triage time using the electronic method was 9.7 seconds faster than the manual method. When analyzed by health care provider type, RNs had the largest reduction in triage time using the electronic method (15.7 seconds). Applied to all 30 simulated patients, this represents a potential time saving of eight minutes.
Faster overall and electronic triage times by RNs could be attributed to two factors. Firstly, they triaged during the high-stress environment of the live mass-casualty simulation, whereas MDs and PMs triaged in the controlled-environment outside the context of a live-disaster simulation. Secondly, RNs triaged patients using the CTAS via computerized entry on a daily basis. Thus, they have more familiarity with not just triaging, but also using a computerized workflow environment as opposed to the other provider types. Similarly, the only significant difference in triage times in the original analysis was for PMs who had shorter triage times with the manual method, a system with which they have more familiarity. These findings suggest that perhaps familiarity with a given modality has more effect on triaging speed than the modality itself.
The majority of the health care providers and patients preferred electronic triage to manual triage, while patients overall showed no preference. Some patients noted that the electronic triage process felt less personalized. Despite this impression, the equivalent accuracy between triage methods demonstrated that patient safety was not compromised. This highlights the importance of not only accurate triage, but also maintaining patient contact in this stressful environment.
This study will guide the development of future research projects assessing the use of electronic triage methods in MCIs. An additional consideration for their use is their capacity to be integrated into the hospital information system, and thus, their ability to track patients beyond triage. Given that electronic MCI triage methods use a bar code for patient tracking, the potential to eventually integrate tracking of patients from the first encounter with Emergency Medical Services in the field up until hospital discharge is another potential benefit of this modality. In addition, this method also allows for visualization of all patients (those present in the ED before the MCI and those arriving from the MCI scene) without changing normal ED operations.
Limitations
This study was conducted at a single center during a disaster simulation. Results may differ from those obtained during an actual MCI. Furthermore, Exercises 1 and 2 took place in a separate area prior to the live disaster simulation. Although unlikely to have impacted results in a meaningful way, it is important to acknowledge this difference with Exercise 3, the live simulation in the ED.
In all three exercises, including the live simulation, the electronic triage commenced after simulated patients had already been manually triaged. The impact that the order of triage modalities had on timing differences is unknown. One known limitation to this methodology was that simulation patients arrived at the initial manual station in an unpredictable manner, while the flow to the second electronic station was more orderly, due to the bottleneck created at the manual station.
The conclusions drawn from the post-hoc analysis are also limited by the different setting in which these additional data were collected, as the additional measurement was done with one provider and one patient only.
Conclusion
The analysis demonstrated that hospital-based electronic (computer-based) START triage had the same accuracy as hospital-based manual (paper-based) START triage, regardless of triage provider type or acuity of patient presentations. Time of triage, using triage category selection as an end point, is not significantly different between the two methods overall, but may be influenced by the provider’s familiarity with the modality.
Additionally, both health care providers and simulated patients perceived that electronic triaging was faster, more organized, and created a safer continuity of ongoing patient care, making it a feasible option for triage. However, the findings also highlight the need for triaging health care providers to maintain good contact as simulated patients perceived the electronic triage to be less personal.
