Active shooter incidents in hospitals are events that pose extreme and immediate risk to staff, patients and visitors. Although overall infrequent, there as many as 154 incidents in the media and peer-reviewed medical literature of armed assailants in hospitals in the decade leading up to 2011. Reference Kelen, Catlett and Kubit1-Reference Nir4 In American hospitals, rates of hospital-based shootings increased from 9 per year (between 2000 and 2005) to 17 per year (from 2006 to 2011). Reference Kelen, Catlett and Kubit1 These incidents occur quickly, without warning, and leave hospital personnel and security with minimal time to respond to multiple casualities. 5
Potentially catastrophic outcomes involving an armed assailant within a hospital highlight the need for proactive training of hospital staff and the creation of robust policies to prepare for these events. Multiple agencies across North America have called for the establishment of health-care facility active shooter policies, often under the hospital’s “code” system, along with training programs to prepare staff and community services. 5-7 In our provincial jurisdiction, this has been termed “Code Silver.” 6 Likewise, multiple case reports outlining experiences with simulated training exercises have been published in recent years. However, none propose a clear methodology to analyze training simulations and inform modification to active shooter protocols and assist in policy implementation. Reference Goralnick and Walls2,Reference Adashi, Gao and Cohen3,Reference Phelps, Russell and Doering8-Reference Mannenbach, Fahje and Sunga11
While there is no clear consensus for the evaluation of protocols for rare, high-risk events in health-care facilities, interprofessional in situ simulation coupled with framework analysis represents a promising approach. In situ simulation is defined as simulation conducted within the actual clinical environment, Reference Petrosoniak, Auerbach and Wong12 and it is particularly effective for the identification of latent safety threats (LSTs), defined as “system-based threats to patient safety that can materialize at any time and are previously unrecognized by health-care providers.” Reference Fan, Petrosoniak and Pinkney13-Reference Wetzel, Lang and Pendergrass15 In situ simulation allows simulation educators to re-create previously reported or anticipated critical events within the clinical environment, providing a unique opportunity to identify context and situational specific issues and challenges.
The inherent latent nature of LSTs requires a robust, reliable, and accurate identification process given the complex and variable circumstances under which they may manifest. Video-based framework analysis, a qualitative research methodology that classifies data according to themes that are then organized in an output matrix, is a novel approach to in situ simulation-based LST identification. Reference Petrosoniak, Auerbach and Wong12,Reference Petrosoniak, Fan and Trbovich16 The framework analysis matrix end-product concisely summarizes LSTs into traceable and transparent themes that facilitates an understanding of their frequency and context in which they occur. Reference Hamman, Beaudin-Seiler and Beaubien14,Reference Tallentire, Smith and Skinner17 In addition, using a framework analysis to identify LSTs offers a methodical modality to highlight independent human factors that may be overlooked during traditional postsimulation debriefs.
Here, we describe the design and implementation of an Active Shooter Simulation In-Situ Training (ASSIST) exercise to evaluate the hospital’s draft Code Silver active shooter policy and protocol. This high-fidelity multiple causality simulation study sought to evaluate a hospital-wide response to an active shooter incident using a modified framework analysis. Findings informed subsequent modifications to the institutional policy and protocol.
METHODS
Active Shooter Protocol Development
A hospital-wide Code Silver policy and protocol outlining the response to an active shooter incident on hospital property was developed for a Canadian academic, inner-city, level-1 trauma center. The protocol, developed by the hospital emergency preparedness committee (EPC), received input from hospital administration, communications, security, and medical personnel in response to regulatory requirements introduced in Ontario mandating an Emergency Preparedness and Response Plan for an individual in possession of a weapon. 6 Before final hospital approval, the draft Code Silver policy and protocol were piloted and evaluated using in situ simulation.
Simulation Objectives and Planning
We prospectively designed and conducted a hospital-wide in situ simulation to test our hospital’s draft Code Silver policy and protocol and identify LSTs related to its implementation. An overview of this process can be found in Figures 1 and 2. A team of simulation experts, hospital administrators, and emergency department (ED) clinicians developed goals and objectives for a multiple casualty scenario. The team met biweekly over 2 mo to finalize the simulation cases and plan the logistics necessary to conduct an ASSIST exercise within the hospital. We identified several objectives to address during this exercise:
-
1. Test the proposed transport protocol of injured patients within the hospital during a Code Silver event;
-
2. Identify LSTs related to the medical care of injured patients, administrative response during the event, and safety and security of hospital staff, patients, and visitors; and
-
3. Identify educational opportunities to increase awareness about the Code Silver policy and protocol among hospital staff.
Process Overview.
Overview of the Code Silver Policy and Protocol Drafting and In Situ Testing Process.
This study received institutional research ethics board approval (REB #15-046).
Outline of Simulation Scenario
One week before the ASSIST simulation, we notified hospital leadership, ED and trauma staff by means of email outlining the objectives and proposed date and time for the exercise. A reminder email was circulated 1 day prior. As an additional safety measure, the hospital-wide overhead call used to start the exercise specified “mock” Code Silver. Based on our experience with announced and unannounced simulations as well as similar simulation exercises in the literature, we announced the time and date of the ASSIST exercise in advance. Reference Wexler and Flamm10,Reference Petrosoniak, Auerbach and Wong12,Reference Fan, Petrosoniak and Pinkney13 We decided that it was preferable to announce this exercise to further encourage clinician buy-in and ensure patient and visitor safety during the exercise.
The ASSIST exercise consisted of a shooting during a meeting in a boardroom located on the hospital’s ground floor. The exercise was undertaken late in the morning on a weekday, a time where typically the trauma volume in our hospital is low, limiting potential conflict with a real trauma activation. The scenario began with an armed assailant, portrayed by an actor, “shooting” several individuals inside the boardroom, who then exits the hospital. A noninjured collaborator called the real in-hospital emergency line to report the shooting and simulated injured patients, activating the simulated Code Silver protocol. The ASSIST exercise involved 3 simulated casualties: patient 1 (PT1), a manikin with life-threatening head and abdominal gunshot wounds; patient 2 (PT2), a standardized patient (SP) with hypotension from an abdominal gunshot; and patient 3 (PT3), an SP with minor injuries and significant psychological distress. Each casualty case was developed and peer-reviewed by simulation experts. The cases were pilot tested for clarity, flow, and logistical considerations by emergency medicine residents not part of the study and modified accordingly before the exercise.
Upon notification of the “mock” Code Silver, the triage nurse activated the on-call trauma team, as per the draft Code Silver policy and protocol, through the hospital paging system to manage potential injured patients. The law enforcement team was notified and was involved in securing premises after the shooting while communicating with our medical evacuation team. Patients, visitors, other uninvolved staff, and nonmedical personnel were informed of the exercise through several channels, including posters at the hospital entrance, internal email communication, overhead announcement, as well as directed by support staff wearing clearly labeled high-visibility vests.
We piloted a novel ED-based medical evacuation team to transport internal patients to the real trauma bay. This is an ad hoc team comprised of available on-duty personnel, including unarmed security and nursing staff, with the principle mandate to safely transport injured patients within the hospital to the trauma bay while limiting treatment efforts to rapid hemostasis by means of temporizing interventions, such as tourniquets and bandages. The EPC formulated this specific medical evacuation team consisting of available on-duty personnel such that this team would be able to form at all times in the event of a Code Silver.
Debriefing
The postsimulation debriefing occurred in 2 phases, with the goal of identifying (1) challenges related to protocol application and (2) elements that functioned well within the draft Code Silver policy and protocol. We sought perspectives of simulation participants and stakeholders who observed the exercise. Two members of the ASSIST team, each with greater than 5 years of simulation and clinical debriefing experience conducted the semi-structured debriefing in both phases.
Phase 1 (“hot” debrief – Figure 3C) occurred immediately after the simulation in the trauma bay for all clinical personnel and participants. The hot debrief focused on the practical application of the draft Code Silver policy and protocol within the clinical space, including the transport of injured patients from the scene, the notification process and communication within and outside of the ED. The debriefing leaders addressed clinical trauma care, but it was not a primary focus.
Overview of Video Ethnographic Observation of Shooting Victims.
Phase 2 (“cold debrief”) took place 1 h after the simulation exercise was completed and included hospital leadership, administrative staff, and law enforcement. In contrast to Phase 1, this session addressed the practical application of the draft Code Silver policy and protocol within the incident command center and across the institution, beyond the ED and trauma bay. The debrief focused on communication challenges (and opportunities for improvement) among all clinical personnel, law enforcement and paramedics, corporate command center and with patients, family, and the media.
Data Collection
We conducted video-based ethnographic observation of clinical participants (2 GoPro Hero4 2015, 2 GoPro Hero). Audio was captured using an overhead microphone (Aputure V Mic D1 Directional Condenser Shotgun) hanging from the trauma bay ceiling and from audio provided by the Go Pro Cameras. We selected 4 locations for camera placement to capture multiple salient elements of the ASSIST exercise: (1) on the stretcher to capture transport of the manikin (PT1) from the shooting scene to the trauma bay; (2) overhead within the trauma bay to capture all patients in the trauma bay; (3) the chest of the standardized patient (PT2) who suffered abdominal injuries to capture the patient perspective; and (4) the chest of the other standardized patient (PT3) who was managed in the acute area of the ED, adjacent to the trauma bay. Video recording began immediately following the simulated shooting and continued throughout the transport process and during the medical care of the injured patients. Figure 3A,B provides an overview of the video analysis and simulation exercise.
A Modified Video Framework Analysis to Identify LSTs
We used a modified framework analysis to examine synchronized video and audio recording in detail. Reference Mannenbach, Fahje and Sunga11 Two authors (N.A., W.L.) independently performed multiple reviews of the recordings to familiarize themselves with the ASSIST exercise. Any perceived medical or safety significance were independently transcribed into a preformatted spreadsheet with standardized time-stamps on the video (eg, at video time 2 min, “RN escorts PT3 down the hallway and commented no security is present”) and were compiled into a cohesive consensus list of significant events.
The 2 reviewers (N.A., W.L.) then independently assigned LST code(s) to each transcribed event (eg, lack of security presence during transport). These codes were developed inductively and are a concise description of the LST observed. A single LST code may be used to describe multiple events with similar features, and conversely, multiple LST codes may be used to describe a single event. LST codes were then further grouped with other relevant codes into LST themes. Themes were selected based on preliminary results from a trauma-based in-situ simulation trial using similar methodology performed at our hospital and modified by consensus between both raters. Reference Fan, Petrosoniak and Pinkney13 For example, the LST code, “equipment placement impeding clinical care” was grouped with “monitors not visible to team members,” under the theme of “Physical Environment and Location.”
The 2 authors (N.A., W.L.) reviewed their collective codes to arrive at consensus of all LST events, creating a final analytical framework based on themes. Interrater reliability was calculated using Cohen’s Kappa test based on independently coded themes before consensus being achieved.
Once consensus was achieved, 2 authors (A.P., N.A.) with clinical trauma resuscitation experience independently applied a hazard score for each code based on the Healthcare Failure Mode and Effect Analysis (HFMEA) for each of the coded themes. Reference Petrosoniak, Auerbach and Wong12,Reference DeRosier, Stalhandske and Bagian18 A hazard score, the product of the frequency and severity of each LSTs, allows for the quantification and subsequent prioritization of each LST. The frequency is rated on a 4-point scale: remote, uncommon, occasional, and frequent, in increasing value from 1 to 4. The 4-point severity scale spans 1 for minor LSTs (no significance impact on patient or provider), to 4 for catastrophic LSTs (life or limb threat to patient or provider). Reference Tallentire, Smith and Skinner17 Themes with a hazard score of 8 or higher are designated as a priority if there is no preexisting process or backup in place to capture, mitigate, or prevent the threat. Reference DeRosier, Stalhandske and Bagian18 The HFMEA decision matrix is included in Figure 4.
Example of a Healthcare Failure Mode and Effect Analysis (HFMEA) Matrix.
We conducted a video-based analysis of the ASSIST exercise, rather than rely on participant recall for LST identification. Reference Patterson, Geis and Falcone19,Reference Wheeler, Geis and Mack20 Notes taken by study investigators during the 2 phase debriefings were reviewed and cross-referenced with the framework analysis data for additional LSTs, although none specific to the draft Code Silver policy and protocol were encountered. We have published the rationale and methodology for this approach previously. Reference Fan, Petrosoniak and Pinkney13
RESULTS
Simulation Implementation
Seventy-six multidisciplinary and interprofessional clinical and administrative staff and trainees participated in the 2-h ASSIST exercise. The scenario was conducted without interruption from clinical activities. We assigned observers as a safeguard to monitor clinical care during the simulation, and no disruptions in care were noted. Additionally, support staff wearing highly visible vests were available to guide patients, visitors, and the uninvolved around the exercise.
Latent Safety Threats Identification
The framework analysis revealed 110 LSTs including 15 LSTs during transport, 85 LSTs in the trauma bay, and 10 LSTs during the care of PT3 in the ED. All events were coded by 2 separate coders independently and organized into 23 LST codes that were independently categorized into 7 themes (Table 1). Interrater reliability for theme coding was K = 0.812.
LST Themes and Examples Identified Using a Video-Based Framework Analysis
Abbreviations: BVM, bag valve mask; FAST, focused abdominal sonography of trauma.
A hazard score ≥8 was assigned to 15 LSTs (Table 2). Following the HFMEA decision tree, 3 of these 15 LSTs warranted further attention as they either lacked an effective control measure or were not readily discoverable before they occurred. These 3 LSTs are described in detail below:
-
1. A lack of security presence during transport and in the ED during patient care. A single police officer was present during the transport of 1 of 3 injured patients, while the other 2 patients were transported by clinical staff alone. Law enforcement officers effectively managed the immediate threat but did not communicate with ED staff or hospital security to signal when an area was safe for evacuation, treatment, and transport. This was not noted during the debriefing.
-
2. A lack of situational awareness of events outside the trauma bay by the clinical team. This occurred when the trauma team leader requested an update of the situation, the current status of the shooter, and whether an ongoing safety threat existed to the team. No one present could provide an update nor was there any communication with the command center. The team had no sense of whether they remained under threat, which was also expressed by the team during the debriefing.
-
3. Critical tasks were not consistently coordinated between team members. A salient example occurred when the porter assigned to transport blood products between the blood bank and trauma bay, was unsure whether they could safely travel to obtain blood products for the patients given the rest of the hospital was under lockdown. The porter did not know who to speak with to clarify this, resulting in a delay to blood product administration. Many of these situations were not highlighted in the debriefing sessions.
Findings from the ASSIST exercise were submitted as a report and presented to the hospital’s emergency preparedness committee for further review and consideration. Specifically, the remaining 12 of 15 LSTs that scored ≥8 were highlighted in our report to the hospital acknowledging that, while there were control measures in place, system-based changes could be beneficial to mitigate these threats. The draft Code Silver policy and protocol was then amended to reflect the new information gleaned from the ASSIST exercise, and changes are highlighted in Appendix 1, which is available online.
Framework Analysis Hazard Scores
DISCUSSION
Active shooter events are increasingly frequent situations that challenge hospitals to establish an emergency preparedness strategy. Reference Hauk21 Prespecified protocols and longitudinal training are essential to efficiently and effectively support the institutional and clinical response. The rarity of this type of event precludes reliance on a real situation to test the policy and protocol. Implementing an active shooter protocol without testing it is akin to certifying a car as road worthy without ever crash testing it. A full-scale, in situ exercise allows for a realistic re-creation of a hospital-based active shooter event, and as a result, an opportunity to “crash test” the process. In situ simulation, represents a resource-efficient and effective technique to detect LSTs, and it may be superior to simulation within a laboratory setting. Reference Mannenbach, Fahje and Sunga11,Reference Petrosoniak, Auerbach and Wong12,Reference Patterson, Geis and Falcone19
In this study, we identified 110 LSTs during a 2-h active shooter scenario. Using a hazard matrix, we categorized 15 LSTs that represent high-risk threats. Further application of the HFMEA decision tree allowed us to focus on 3 specific threats that were both not readily apparent when they occurred and lacked effective control measures (ie, a barrier that significantly reduces the likelihood of the event occurring).
Each of these 3 threats are closely tied to the safety of clinicians and team members. An active shooter event is a unique circumstance, where clinical care team members may experience direct threats to their health and safety. As a result, a Code Silver policy and protocol must include elements that serve to protect those caring for the injured. We identified the need for enhanced security and law enforcement personnel during the transport of patients and staff to the trauma bay. During the ASSIST exercise, participants expressed uncertainty about whether it was safe to enter premises where the shooting occurred. This highlights the need to engage local law enforcement in the design, implementation, and training for such events. At our hospital, we amended the draft Code Silver policy and protocol to state explicitly that the ED medical evacuation team will only begin care once the premises are secured by law enforcement and to clearly state the stage of events occurring. Subsequent transport of patients now also mandates an accompaniment by security staff.
During the care for the patients, the limited communication between the hospital command center, law enforcement, and the clinical team became clear. The situational awareness of the clinical team regarding their own safety and the status of the shooter was poor. We have highlighted this issue and measures are being designed to ensure regular updates occur to our ED and trauma personnel from the command center.
Lastly, movement in and out of the ED is considerably compromised and potentially unsafe during an active shooter event. While the trauma bay and ED are primarily self-sustaining, we do not keep blood products in our trauma bay or ED. Administration of blood requires a porter to travel from the blood bank to the trauma bay with a blood cooler, which is part of our institutional process when blood products are requested during code trauma activations. We observed uncertainty among the clinical team whether it was safe to travel throughout the hospital, an occurrence closely tied to the above-mentioned poor situational awareness of the team. This highlights an important educational opportunity to train not just clinical staff, but also support staff regarding the situational awareness during an active shooter event. Our hospital’s draft Code Silver policy and protocol is undergoing additional refinements to ensure safe travel of essential personnel to and from the trauma bay and ED during such an event, with security escorts, and using the route deemed safest by law enforcement. Additional modifications to the announcements are being made to increase clarity of overhead announcements in regard to when it is safe to travel through the hospital.
This study significantly adds to the body of literature in active shooter simulation and preparation by demonstrating the unique use of video framework analysis as a systematic tool for the identification and categorization of threats based on frequency and severity. Reference Wexler and Flamm10,Reference Mannenbach, Fahje and Sunga11,Reference Sanchez, Young and Baker22 This analysis facilitated amendments to our draft Code Silver policy and protocol directed at the highest risk LSTs. This is in contrast to previous active shooter simulation exercises that relied on direct participant debriefing, postevent survey, observer evaluation, and or nonstructured video review to identify critical events and themes. Reference Wexler and Flamm10,Reference Mannenbach, Fahje and Sunga11,Reference Sanchez, Young and Baker22 There are numerous pitfalls to exclusively relying on these nonstructured modalities for LST identification. Postsimulation survey in a similar exercise only achieved a response rate of 50.5%. Reference Mannenbach, Fahje and Sunga11 Simulation evaluators may not observe all salient safety threats and lacks the bird’s-eye perspective gleaned from systematic video framework review. In addition, simulation participants may not identify LSTs due to memory lapses, recency bias, and recall bias given the possibility of task overload compromising recall. For example, members of the ED medical evacuation team may not appreciate the potential risk of traveling unaccompanied within the hospital when they are focused on patient care.
This study has several limitations. First, this is a single-center study at a Canadian, academic, inner-city, Level-1 trauma center, and the findings may not be generalizable to other centers, especially among institutions with different levels of security presence. However, our application of an objective and systematic method to LST identification through in situ simulation can be applied broadly and may represent an important alternative to methodologies that rely on participant recall. Additionally, our findings highlight concerns related to general topics of clinical team safety during patient transport and within the ED. Any institution designing a Code Silver active shooter policy and protocol will be well served to at least consider our identified LSTs to ensure they have mitigation strategies for such issues in place. These LSTs were reviewed by a multi-disciplinary study team, who all work within the trauma bay and ED, and these LSTs align with our experiences as well as with previously published work. Reference Fan, Petrosoniak and Pinkney13
Second, we only conducted this exercise once. However, even a single in situ simulation was sufficient to identify over >100 LSTs and 3 that we determined to be high risk. For local quality improvement initiatives, limited sample sizes may be sufficient to prompt a more detailed analysis of protocol and process issues. Reference Etchells and Woodcock23 Third, we may not have identified all LSTs. We relied on video review and as a result, elements not amenable to video capture are not analyzed. This highlights the importance of multi-modal LST analysis. A larger scale comparative study is required to understand the types of LSTs identified during video review and how they may differ from debriefing. Finally, much of the active shooter literature addresses the RUN, HIDE, FIGHT paradigm that teaches participants preferentially to RUN from the area, HIDE if they are unable to run, and as a last resort, FIGHT the shooter. 7 This simulation did not evaluate that response as we focused on the medical and logistical issues once the shooter had left the scene. Specific testing of such a response is needed but is beyond the scope of this simulation.
CONCLUSIONS
This study describes the application of an in situ simulation coupled with a video-based framework analysis in the evaluation of a draft Code Silver policy and protocol. Our findings identified latent safety threats related to team safety, clinical logistics, and medical care that helped inform and support modifications to our institution’s draft Code Silver policy and protocol, as well as areas where further staff education may be needed, particularly related to situational awareness and staff safety. We propose that in situ simulation be routinely used to identify and mitigate LSTs for newly developed protocols for rare but high-risk events, such as an active shooter scenario.
Acknowledgments
The authors thank the staff at the Allan Waters Family Simulation Centre (Nazanin Khodadoust, Susan Zelko, Hentley Small, and Ashley Rosen) for their contributions to the simulation design and logistical planning support. The authors also thank the actors, staff, residents, and leadership who participated in the exercise and for their support of this study.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2019.161
