Introduction
Since 2011, over 600 major disasters were reported in the United States alone. These include earthquakes, severe storms, terrorist attacks, and other emergencies. 1 Several of these events made hospital evacuations necessary. Health care facilities are required to actively prepare for emergencies. However, there is little to no evidence to guide hospital evacuations. This is caused by the difficulty in conducting prospective research for these rare, but potentially catastrophic, events. Therefore, all current guidelines are based on recorded or published experiences and expert opinion rather than data. It has been reported that 22.2% of hospitalized patients would have special needs in a possible evacuation, including intensive care, isolation, active labor, and dependency on intravenous (IV) drugs, respiratory assist, or controlled mechanical ventilation.Reference Rimstad and Holtan2 In these patients, an evacuation would be particularly challenging. The American College of Chest Physicians (CHEST; Glenview, Illinois USA) released a consensus statement on evacuation from the intensive care unit (ICU) based mainly on expert opinion.Reference King, Niven and Beninati3 This statement emphasized the need for substantial improvements in provider education on the vertical evacuation process.Reference King, Niven and Beninati3
It is to be expected that patients in the operating room (OR) would be particularly affected by a disaster evacuation.Reference Norcross, Elliott, Adams and Crawford4 In this situation, it is critical to perform a timely evacuation while maintaining respiratory and circulatory support, avoiding damage to the patient from the exposed surgical site, and taking a supply of necessary equipment to bridge the time period until help arrives.
Periodic revision and training of emergency procedures are essential, especially when evaluating responses to rare events.Reference Peck, Scullard and Hedberg5 Modalities of clinical training include lectures, virtual training, and simulation.Reference Miller, Rambeck and Snyder6, Reference Olson, Scheller and Wey7 Especially, in-situ simulation provides the additional opportunity to test procedures and improve the evacuation process.
In this in-situ simulation study on the evacuation of an anesthetized patient from the OR, it was hypothesized that evacuation checklists would: (1) improve completion of pre-defined actions that are critical for patient safety, (2) reduce evacuation time, and (3) reduce complications. The aim of this study was to compare a simulated evacuation with and without the use of an evacuation checklist.
Methods
This was a prospective, randomized simulation study. This study was approved by the Institutional Review Board at University of California, San Francisco (UCSF; San Francisco, California USA; approval number: 14-13691). Informed consent was obtained from all participants prior to the study. Data were collected via high-fidelity simulation scenarios from June 2015 through April 2016. Randomization was performed according to a randomized number table. The checklists used were established by researchers in a preliminary study on high-fidelity, in-situ simulation of evacuation procedures.Reference Rich, Hirsch and Fidler8 Checklists were based on problems identified in the simulation sessions of this preliminary study, or on suggestions from participants in this study. The checklist is shown in Figure 1 and the pre-defined critical actions are the items within the checklist, including bringing additional drug and equipment supplies, securing the airway and IV lines, and bringing an oxygen tank. Securing the airway was defined as any attempt by the anesthesia provider to ensure the endotracheal tube would not move during transport. Dropped items were recorded as those representing potential complications of the evacuation. Participants in the study were anesthesia residents in their second and third year of residency that were rotating through the hosting institution, and nurse anesthetists working at the hosting institution. Participating OR nurses were volunteers that were willing to participate. Exclusion criteria were previous participation in the study or previous experience as a team member in simulation at the hosting institution. Participants did not receive any training on the checklist prior to the simulation scenario, and the anesthesia providers did not have any prior training on the evacuation system. The primary outcome measures were the completion of pre-defined critical actions, and total evacuation time. Secondary outcome measures were the assessment of the individual opinions about the checklists and in-situ evacuation simulation.
Evacuation Checklist for the Anesthesia Provider Used in the Simulation Study.
Scenario Setup
Scenarios were set up in a real OR on the third floor of the medical center (three levels above the ground). The setup included an intubated and mechanically ventilated, high-fidelity manikin (SimMan3G; Laerdal Inc.; Norway) undergoing a simulated open appendectomy. Simulation team members played the roles of the initial primary anesthesiologist, surgeon, and assistant surgeon. The circulating nurse was staffed by a simulation team member from the simulation team if no nurse was available for simulation. Besides oxygen tank, essential components included a high-fidelity manikin weighing 85 pounds and an evacuation system (Evacusled; Evacusled Inc.; Toronto, Ontario) as used at the hosting institution. 9 The evacuation system was placed on a gurney and stored out of sight close to the OR during the scenario setup phase.
Interventions
During the scenario setup phase, participants waited outside of the OR. One of the simulation team members gave a small briefing to participants about the high-fidelity manikin and the study protocol. All participants were told that they were going to participate in a simulated evacuation scenario. Physicians or nurse anesthetists were assigned as the team leader. Once the participants arrived at the OR, they were asked to take over from the simulation team after a hand-off. At the beginning of the scenario, the manikin displayed normal vital signs compatible with the procedure. Then, a hypotensive episode was simulated to focus the participants’ attention on the scenario. The anesthesia provider was required to manage this situation. Once the appropriate interventions were made, the vital signs normalized. Subsequently, the room lights were turned off, a fire alarm siren was played, and the team was asked via loudspeaker to immediately evacuate the patient out of the hospital. A timer was started at this point. According to a previously created table, participants were randomized into two groups. The checklist group (CG) was handed a checklist at the beginning of the evacuation procedure and the non-checklist group (NCG) did not receive a checklist. For both groups, the simulation team members provided the evacuation system and the gurney, obeyed the team leader’s orders, and assisted with manual tasks such as transferring and carrying manikin and equipment. The simulation team members indicated as well the evacuation path and endpoint without providing additional help.
Demographics of the participants (age, medical experience in years, and profession) were noted. Timelines and critical actions were recorded according to standardized forms. Critical actions assessed during the scenario were carrying the oxygen tank safely, bringing drug and equipment supplies, and securing the airway and vascular access. If the team supported the airway and vascular access with additional security measure to prevent pulling out, it was assessed positive. Additionally, any dropped items during the evacuation were recorded. If the team picked up a code bag, they received credit for bringing drug and equipment supply during the evacuation.
Simulation was performed along the pre-determined evacuation path. This included transporting the manikin along a 150 feet hallway on a gurney and continued using the evacuation system, down a stairway to the outside of the building. Upon reaching this point, the scenario was ended and the time was stopped. During all evacuation steps, vital signs were given in normal ranges, the monitor part was carried, and team leader was given opportunity to observe the monitor.
After the simulation, a debriefing session was performed. Subsequently, the participants were asked to complete a survey. Questions in this survey included if the participants had any previous training on evacuation and if they had ever used the evacuation system. Then they were asked to rank five statements on a Likert scale (one = extremely agree and five = extremely disagree).
Power Analysis
Initially, the authors determined that an effect size of 0.3 would be a meaningful result. In preliminary experiments, it was found that the scenario time had a standard deviation of 15%-20% of the mean time. To calculate the sample size that would be necessary to detect this effect size, the calculator tools of the UCSF clinical and translational science institute were used.Reference Hulley, Cummings, Browner, Grady and Newman 10 A two-tailed test, a significance level of 0.05, a Type II error rate of 0.2 (power of 0.8), and a standard deviation of 0.2 were specified. It was determined that a total of 14 scenarios (seven in each group) would be necessary. Since the researchers were advised that due to the use of the t-distribution, the calculation of the necessary sample sizes for small samples may be slightly under-estimated, it was decided to run at least nine scenarios per group.
Statistics
Descriptive statistics were presented as a frequency (percentage) for categorical variables. The Kolmogorov-Smirnov test was used to test the distribution of the data. Data were presented as mean (standard deviation [SD]) for normally distributed data. Student’s t-test was used to compare the groups with normally distributed data. The Chi-Square test was used to compare categorical variables. All statistical tests were performed with the Predictive Analytics Software (PASW, version 18; SPSS Inc; Chicago, Illinois USA). A P value of less than .05 was considered statistically significant.
Results
A total of 19 scenarios, nine with use of the checklist (CG) and 10 without use of the checklist (NCG), were evaluated. All groups completed the evacuation scenarios from OR to ground floor and exited the building. None of the scenarios were ended prematurely. The CG had 13 participants and NCG had 15 participants. Mean age was 36.0 (SD = 11.1) years for all participants; there was no significant difference between groups (P = .87). The CG and NCG showed homogeneous distribution for the profession of the participants (P = .935). A total of four participants had previous evacuation training experience without statistically significant difference between the groups (P = .630; Table 1).
Demographics and Comparison of the Study Groups
a Student’s t-test.
b Chi-Square test.
Mean scenario time (from mock fire alarm start to exit from building) was 593.8 (SD = 77.0) seconds without statistically significant difference between groups (P = .369). All time data are shown in Table 2. Further analyses of partial times were conducted to investigate different challenges in the scenario. The time from the start of the scenario to leaving the OR was a representation of challenges in decision making, collecting equipment, and transferring the patient on to the evacuation system and on to the gurney. This time was not statistically significantly different between the groups (P = .462). The wait time at the stairwell prior to navigating the stairs represented another challenge in evacuation continuum, as the manikin must be carried with only the evacuation system from that point forward. This time was also not statistically different between the groups (P = .929).
Comparison of the Study Groups for Timeline and Critical Actions
Abbreviations: IV, intravenous; OR, operating room.
a Student’s t-test.
b Chi-Square test.
In the analysis of completion of critical actions, the CG group showed significantly better performance in securing the airway, taking additional drug supplies, and taking additional equipment supplies (P = .047, .001, and .001, respectively). There was no statistically significant difference between the groups in bringing an oxygen tank during evacuation, but it is noteworthy that all simulation teams in the CG group remembered bringing the oxygen tank while two simulations in the NCG did not bring an oxygen tank. Groups did not show any statistically significant differences in securing IV lines (only one participant from either group took the time to secure the IV line prior to evacuation) or dropping materials (P = .279 and .405, respectively). All critical action assessment data are also shown in Table 2.
According to post-evacuation surveys, 26 out of 28 of the participants answered that the use of standardized checklists would be useful in a real evacuation, as shown in Table 3. Twenty-seven out of 28 agreed that using standardized checklists would improve the evacuation process (Table 3). There was no statistically significant difference between CG and NCG in these responses (P = .948 and .615, respectively).
Comparison of Survey Answers
Note: 1 = Extremely Agree, 2 = Agree, 3 = Neutral, 4 = Disagree, 5 = Extremely Disagree.
a Chi-Square test.
Discussion
The most important result of this study is that the use of standardized checklists significantly increased the completion rate of critical safety actions during the vertical evacuation of a patient under general anesthesia from OR. This significantly higher performance in the completion of safety-related procedures may translate into a reduced risk of adverse effects during the evacuation of these critically ill patients.
There was not any statistically significant difference in evacuation times between the groups. This is probably partially due to a longer preparation time in the OR (time to leave the OR) in the CG, although this difference was not statistically significant. It is noteworthy that, in spite of this initial delay, the participants completed the evacuation in approximately the same time as the control group without a checklist.
The possible increase in patient safety supports the recommendation of the CHEST consensus report to develop detailed vertical evacuation plans.Reference King, Niven and Beninati 3 This consensus statement also suggested the use of simulated disasters to test readiness, in particular for training in the use of special evacuation equipment such as the evacuation system. The need of such training is emphasized by recent experiences in real disasters. For example, Hurricane Sandy (2012) revealed that ICU providers were not trained adequately; King, et al reported that only 21% of the staff had an evacuation drill in the prior two years.Reference King, Dorfman, Einav, Niven, Kissoon and Grissom 11 These authors also found that transport sleds were the second most useful equipment after flashlights.Reference King, Dorfman, Einav, Niven, Kissoon and Grissom 11
In a previous simulated evacuation study, Dhondt, et al reported that three critical care departments, including dialysis unit, surgery clinic, and burn unit, faced the same challenges in the evacuation drill.Reference Dhondt, Lauwaert and Hendrick 12 While the current study tested the vertical evacuation of OR patients, the findings can probably be generalized to the ICU and other departments having disabled and/or critically ill patients. Gildea, et al tested the evacuation of ICU patients from the fourth floor of the hospital and found that the mean evacuation time was 14.7 minutes, 3.75 minutes per floor.Reference Gildea and Etengoff 13 This group reported no adverse events for the simulated patients nor the personnel from the evacuation.Reference Gildea and Etengoff 13 It is noteworthy that while the current study utilized a high-fidelity manikin to simulate an evacuation of a ventilated patient from the OR located at the same floor level, the mean evacuation time was similar to Gildea’s finding, an average of 9.88 minutes or 3.29 minutes per floor, also without complications for patient or participants. In both studies, the weight of the patient or manikin is probably under-estimated, since Gildea used slender volunteers and this study’s full-size manikin weighs only 85 pounds.
Iserson reported that most hospital evacuation plans are based on fire-fighter evacuations and are trained using stretchers.Reference Iserson 14 However, the availability of a large number of stretchers may not be realistic in a real disaster. Moreover, authors determined in earlier evacuation trainings that stretchers do not fit in the stairways in the hosting hospital, a key point since elevators cannot be used during a disaster.
The mattress-sheet method has been proposed as a simple and fast method for vertical evacuations.Reference Iserson 14 However, while mattresses and sheets are cheap and most likely universally available, this method has several shortcomings. These include the lack of handles with risk to patients and health care professionals from opening of knots in the sheets, the risk of tearing of sheets, the time that is necessary to put mattresses on the stairs, and a potential lack of mattresses. The commercially available evacuation equipment used in this study is conceptually based on the mattress-sheet method, but is designed to avoid these shortcomings. Therefore, evacuation system and similar systems appear more suitable for emergency evacuations. However, both methods require skill and continued training for safe operation.
In the hosting hospital, the evacuation system is stored adjacent to each OR. However, very few participants had ever used this system, and many participants had never noticed the bright yellow, clearly labeled evacuation system next to each OR door. In the present study, it was found that only four of 28 participants had undergone previous evacuation training. The trained individuals were all nurses who had received this training when they started to work at the institution. It is obvious that for actual disaster preparedness, health care providers must be trained in regular intervals to use equipment properly and to ensure completion of critical actions.
While computer-based disaster preparedness assessment tools are a cost-effective alternative to some aspects of this training, other parts, such as team interaction and equipment operations, need to be trained in-person.Reference Debacker, Van Utterbeeck, Ullrich, Dhondt and Hubloue 15 In-situ simulation can not only be used for this part of training, but has additional uses in testing quality improvement interventions in a safe environment.Reference Yager, Collins and Blais 16 Moreover, it is a way to evaluate and compare the strengths and weaknesses of different equipment options and modalities prior to committing to one plan.Reference Debacker, Van Utterbeeck, Ullrich, Dhondt and Hubloue 15 In-situ simulation scenarios for disaster preparedness with high-fidelity manikin and real equipment can give opportunity to detect the risk for having problems, such as strained muscles or any material which limits passing the hallways; however, any data on this topic were not collected.
Based on recent disaster experiences, evacuation by staff with available resources, also dubbed “drop everything and go,” seems to be a more suitable concept for hospital evacuation training than assuming a large number of additional helpers (firefighters) or additional equipment (stretchers and blankets) will be present. Helpers and equipment are likely in short supply in many disaster situations. This emphasizes the importance of evacuation training by staff members. In addition, staff members are less-experienced in an actual evacuation procedure and its associated risk than, for example, firefighters. Therefore, successful evacuation needs to emphasize precautions to prevent problems associated with airway, IV lines, drains, and sterile procedures. In this study, the CG performed significantly better in securing the airway and in bringing drug and equipment supplies along during the evacuation than the group without a checklist.
Moreover, a cornerstone of the “available resources” concept is that for their own protection, staff cannot go back into the building to get additional drugs or equipment. This further emphasizes the need for checklists to take all necessary equipment.
Limitations
This is a single center, simulation-based study. Therefore, it may be limited in actual application and the study may lack external validity. This study had a limited number of scenarios that did not allow parametric statistical analysis. Moreover, this study primarily tested the actions in the team leader’s task list because of limitations in availability of the other team members for simulation. Additionally, specific evacuation equipment may vary from hospital to hospital.
Conclusions
Standardized checklists increase the completion rate of pre-defined critical actions in evacuations out of the OR. The completion of these actions likely improves patient safety. The use of the checklist did not have any effect on the total evacuation time.
Conflicts of interest/funding
Dr. Acar was compensated by The Scientific and Technological Research Council of Turkey (Ankara, Turkey) for this research with the project number of 1059B191500459. All other funding was from departmental sources only. There are no conflicts of interest for any of the other authors to disclose.
