Introduction
Mass casualty incidents (MCIs), including those occurring within hospital settings, present a significant challenge to health care systems, requiring rapid coordination of clinical care, patient flow, and interdepartmental communication under conditions of extreme stress.Reference Berzon, West and Jacobs1 The ability of hospitals to implement an effective mass-casualty response plan rapidly and effectively is critical.Reference Waxman, Chan and Pillemer2–Reference Pek, Quah and Valente4 Despite the widespread development of institutional MCI protocols, prior studies have demonstrated variability in hospital preparedness and a persistent discordance between written plans and real-world performance.Reference Skryabina, Betts and Reedy5–Reference Naru, Churruca and Long8
Simulation has emerged as a high-fidelity method for evaluating hospital preparedness and testing MCI protocols under realistic conditions.Reference Pek, Quah and Valente4, Reference Tallach, Schyma and Robinson9, Reference Hitti, Cheaito and Kazzi10 In particular, simulation-based clinical systems testing (SbCST) provides a structured approach for assessing health care systems under operational stress and identifying latent safety threats (LSTs), defined as previously unrecognized system-based conditions that may contribute to patient harm if unaddressed.Reference Lounsbury, Tomlinson and Wakeling11, Reference Mohtady Ali, Ranse and Roiko12 Furthermore, integrating SbCST with structured risk analysis methods such as Failure Modes and Effects Analysis (FMEA) enables prioritization of identified threats and supports targeted mitigation strategies.Reference Holmes, Chipman and Barbour13–Reference Holmes, Chipman and Gray15
The use of in situ simulation to identify LSTs has been well described across diverse clinical settings including procedural suites, neonatal resuscitation programs, and newly commissioned facilities.Reference Sarwal, Morris and Crumpler16–Reference Rubio-Gurung, Putet and Touzet18 However, its systematic application to internal MCI preparedness remains substantially underexplored. Existing disaster simulation studies have largely employed tabletop exercises, external MCI scenarios, or regional full-scale drills, with relatively few employing in situ SbCST to evaluate system performance during simulated internal events.Reference Tallach, Schyma and Robinson9, Reference Mahdi, Jafri and Allana19 Furthermore, whether the system-level vulnerabilities identified through this methodology are consistent across different institutional contexts, an important consideration for developing generalizable preparedness frameworks, remains poorly characterized.
The objective of this study was to use a multicenter SbCST approach to identify and characterize LSTs during simulated internal MCIs at two hospitals, and to prioritize these threats using an FMEA-based framework, with the aim of characterizing cross-institutional patterns of system vulnerability in internal MCI preparedness.
Methods
Study Design and Setting
This study describes a multicenter SbCST initiative conducted at 2 hospitals to evaluate preparedness for internal MCIs. Both exercises were performed in situ and simulated a missile strike within an inpatient hospital ward. The simulations were designed to assess system performance under conditions approximating real-life operational stress. Each exercise incorporated multiple operational zones, including inpatient wards, evacuation pathways, casualty collection points, emergency department (ED) intake areas, and designated holding areas.
This study was conducted as a quality improvement initiative. Institutional Review Board (IRB) exemption was obtained for one participating site, and the second site was included as part of operational quality assurance activities. No patient data were used.
Project Team and Preparation
The project was led by SimulTech, the simulation center of Clalit Health Services. An interprofessional, multidisciplinary project team was established, including simulation experts, institutional clinical leadership, emergency preparedness leads, and emergency medical services (EMS) personnel.
A structured needs assessment was conducted at each participating institution, including a detailed review of local internal MCI protocols and operational workflows. Based on this assessment, each simulation scenario was tailored to reflect the specific organizational structure, patient flow pathways, and emergency response processes of the respective hospital.
The simulation team developed site-specific structured observation tools for each exercise. These tools were aligned with institutional emergency protocols and informed by standardized observation templates from the Home Front Command, enabling systematic evaluation of system performance across key domains.
Simulation Scenario and Participants
The simulation scenarios were designed to represent internal MCIs within a hospital ward. Each scenario began at the point of impact and included the full continuum of care following site-specific protocols for patient movement to the ED or evacuation to a tertiary center.
To ensure maximal realism, the exercises utilized a “hands-on” approach where all clinical and logistical actions were performed in real time. Standardized patients (actors) were used to simulate conscious casualties, while high-fidelity manikins were utilized for patients requiring invasive procedural interventions.
The exercises incorporated systematic patient search and evacuation, followed by patient flow through predefined hospital pathways. Depending on institutional protocols and operational models, patients were either evacuated to the ED for further evaluation and treatment or transferred to a designated holding area for staging and subsequent evacuation.
Approximately 35 participants were involved at each site, representing a comprehensive cross-section of the hospital response team, including:
-
• Attending staff and residents
-
• Nursing staff
-
• Security personnel
-
• Emergency response teams
-
• Emergency medical services
-
• Patient transport teams
Identification and Processing of Latent Safety Threats
A multi-source data collection approach was used, triangulating observations from 3 complementary streams:
-
1. Structured field observations—Designated trained observers were embedded across distinct operational zones during each exercise: the ward of origin, evacuation pathways, the casualty collection point, and ED intake. Each observer remained assigned to a single zone for the duration of the scenario. Observers used site-specific structured observation tools, developed a priori and aligned with institutional emergency protocols, to document system behaviors in real time.
-
2. Facilitator observations—Lead simulation facilitators monitored overall system performance across multiple zones and recorded system-level observations independently from the zone-based observers.
-
3. Participant self-report—During structured debriefing, participants were explicitly prompted to identify moments in which the system (rather than individual performance) created difficulty, delay, or risk. Verbal observations were documented in real time by a designated note-taker and reviewed for accuracy by the facilitation team.
Following data collection, all observations from the three streams were compiled and reviewed by the core simulation team. Duplicate observations, defined as descriptions of the same underlying system condition captured by more than one stream, were consolidated into single entries, requiring agreement between at least 2 team members. Disagreements were resolved through group discussion. The resulting list was then abstracted into underlying system-level threats.Reference Holmes, Chipman and Barbour13, Reference Congenie, Bartjen and Gutierrez20, Reference DeRosier, Stalhandske and Bagian21
LSTs were assigned to system-level domains using the HFMEA framework by a multidisciplinary review group comprising simulation specialists, clinical leads, and emergency preparedness coordinators. A preliminary domain assignment was proposed by the lead analyst for each LST, then reviewed by the full group. Discrepancies were discussed until consensus was reached in all cases. The process was conducted independently at each site, and assignments were subsequently compared across institutions.
FMEA Scoring and Prioritization
Each LST was evaluated using an FMEA approach to determine its relative priority for mitigation.Reference Holmes, Chipman and Barbour13, Reference DeRosier, Stalhandske and Bagian21, Reference Colman, Stone and Arnold22
Each LST was assessed across three domains: severity, probability of occurrence, and detectability. Scoring was performed by a panel of five domain experts including clinical simulation specialists, an emergency physician, a senior nurse educator, and an institutional emergency preparedness lead. To reduce anchoring bias, each panelist scored all LSTs independently before group discussion. Following independent scoring, scores were collated and compared. LSTs on which all panelists’ scores fell within one point on each subdomain were assigned the panel mean directly. LSTs with greater variation (>1 point discrepancy on any subdomain) were reviewed in a structured consensus meeting, in which each panelist articulated their reasoning before the group reached a final agreed score. This process was conducted separately and independently at each site. Each domain was scored on a scale from 1 (low) to 4 (high). Detectability was defined as the likelihood that the failure would not be recognized before causing harm, with a score of 4 indicating the lowest probability of detection.
A composite priority score (range 1-64) was calculated as the product of the 3 domain scores. Based on previously described methods, LSTs were categorized into 4 priority levels: low (1-16), medium (17-31), high (32-47), and very high (48-64).
Potential mitigation strategies were identified during debriefing sessions at both sites. These included both immediate operational changes and system-level interventions, such as modifications to documentation processes, implementation of alternative communication tools, and augmentation of emergency equipment availability.
Responsibility for implementation of mitigation strategies was assigned to hospital leadership and emergency preparedness teams at each institution.
Results
LSTs
Across both SbCST exercises, approximately 75-80 raw observations of LSTs were identified during the simulations and subsequent debriefing sessions. Following consolidation of duplicate observations and grouping into system-level categories, a total of 53 unique LSTs were identified across the 2 institutions. The number of LSTs identified per exercise was comparable between sites.
The identified LSTs spanned multiple domains, including clinical care processes, patient flow, communication, infrastructure, equipment, and organizational systems. The distribution of LSTs across system-level domains by site is presented in Figure 1, demonstrating a similar pattern across institutions.
Distribution of latent safety threats (LSTs) by system-level domain across the 2 institutions. A similar pattern of LST distribution was observed across sites, particularly in domains related to patient flow, communication, and clinical processes.
Figure 1. Long description
The Y-axis is labeled Number of L S Ts with a scale from 0.0 to 4.5 in increments of 0.5. The X-axis lists eight domains from left to right. Each domain has a blue bar for Site 1 and an orange bar for Site 2.
* Leadership: Site 1 is 3.0, Site 2 is 4.0.
* Clinical Care: Both sites are 3.0.
* Patient Flow: Both sites are 4.0.
* Triage: Both sites are 3.0.
* Communication: Both sites are 4.0.
* Documentation: Site 1 is 2.0, Site 2 is 3.0.
* Interfaces: Site 1 is 2.0, Site 2 is 3.0.
* Infrastructure: Both sites are 2.0.
Horizontal dashed grid lines align with the Y-axis increments. A legend in the top right corner identifies the blue bars as Site 1 and orange bars as Site 2.
Importantly, many of these threats were not confined to isolated clinical tasks but occurred at transition points between stages of care and across operational units.
To further characterize system-level vulnerabilities, key LSTs were grouped into major domains, as summarized in Table 1. These domains reflect recurrent patterns observed across both institutions, particularly in leadership, patient flow, communication, and system interfaces. These domains were derived through grouping of LSTs according to the HFMEA framework, allowing for structured system-level analysis.
Key system-level latent safety threats identified across both institutions
Table 1. Long description
The table consists of four columns: Domain, Description of threat, Representative example, and Potential impact.
* Leadership and Command: Lack of defined roles leads to fragmented decision-making, causing delayed prioritization and inefficient resource allocation.
* Clinical Care: Inconsistent execution of life-saving interventions, such as failure to perform A B C D E assessments, increases the risk of preventable morbidity and mortality.
* Patient Flow and Evacuation: Inefficient processes and bottlenecks, like congestion in transport routes, prolong time to definitive care.
* Triage and Prioritization: Variability in accuracy and delayed identification of high-acuity patients leads to suboptimal resource allocation.
* Communication and Coordination: Breakdowns between teams, such as delayed recognition of an internal M C I, reduce situational awareness and system activation.
* Documentation and Patient Tracking: Lack of standardized documentation and incomplete record transfers result in loss of critical clinical information.
* System Interfaces: Poor coordination with services like imaging or the blood bank disrupts workflow and delays treatment.
* Infrastructure and Equipment: Physical limitations and restricted evacuation routes reduce operational efficiency and compromise safety.
FMEA Scores
FMEA is commonly paired with SbCST to prioritize identified LSTs based on their potential impact on patient safety.
Application of FMEA scoring demonstrated a broad distribution of LST priority levels across both sites, with a substantial proportion categorized as high or very high priority, as presented in Figure 2.
The distribution of FMEA priority levels by site, demonstrating a similar pattern across institutions, with a predominance of medium-to-high priority LSTs.
Figure 2. Long description
A stacked bar chart titled Distribution of F M E A Priority Levels by Site. The y-axis is labeled Number of L S Ts with a scale from 0 to 30 in increments of 5. The x-axis identifies two categories: Site 1 and Site 2. A legend in the top-left corner defines four grayscale shades: light gray for Low, medium-light gray for Medium, medium-dark gray for High, and dark gray for Very High.
* Site 1: The total bar height reaches 23. The Low segment spans from 0 to 3. The Medium segment spans from 3 to 12. The High segment spans from 12 to 18. The Very High segment spans from 18 to 23.
* Site 2: The total bar height reaches 30. The Low segment spans from 0 to 5. The Medium segment spans from 5 to 16. The High segment spans from 16 to 24. The Very High segment spans from 24 to 30.
Both sites show a similar proportional distribution where Medium and High categories constitute the largest portions of the total count.
Across both institutions, the highest-scoring LSTs were primarily associated with system-level processes, particularly patient flow, communication, and interdepartmental coordination. Median FMEA scores were comparable between sites (Site 1: 30 [IQR 24-36]; Site 2: 32 [IQR 24-48]) (Table 2).
Number of LSTs, proposed mitigations, and FMEA scores by site
Table 2. Long description
The table consists of six columns and four rows including the header.
Column headers from left to right are: Site, L S T s identified, L S T s scored, Median L S T s per exercise, Proposed mitigations (percent of L S T s), and Median F M E A (I Q R).
Row 1: Meir Medical Center. L S T s identified: 45 to 50. L S T s scored: 23. Median L S T s per exercise: 23. Proposed mitigations: approximately 15 (approximately 65 percent). Median F M E A (I Q R): approximately 30 (24 to 36).
Row 2: Magen Ha’ari Hospital. L S T s identified: approximately 30. L S T s scored: approximately 30. Median L S T s per exercise: 30. Proposed mitigations: approximately 18 (approximately 60 percent). Median F M E A (I Q R): approximately 32 (24 to 48).
Row 3: Total. L S T s identified: approximately 75 to 80. L S T s scored: approximately 53. Median L S T s per exercise: 26.5. Proposed mitigations: approximately 33 (approximately 62 percent). Median F M E A (I Q R): approximately 31 (24 to 42).
Mitigations
Mitigation strategies were identified for the majority of LSTs during structured debriefing sessions at both sites, with approximately 60-65% of LSTs associated with proposed interventions.
These strategies included both immediate operational changes and broader system-level modifications. Examples included implementation of duplicate documentation processes to ensure continuity of patient information, use of alternative communication tools such as megaphones in environments with limited communication infrastructure, and augmentation of emergency equipment availability through coordination with EMS upon arrival.
In addition, several recurring system-level recommendations were identified, including standardization of patient search and evacuation processes, reinforcement of triage and prioritization protocols, clarification of team roles and leadership responsibilities, and strengthening of communication pathways between clinical units and supporting services.
Responsibility for implementation of mitigation strategies was assigned to hospital leadership and emergency preparedness teams at both institutions.
Discussion
This multicenter SbCST study identified numerous LSTs during simulated internal MCIs across 2 institutions. The use of in situ simulation enabled evaluation of real-world system performance under conditions approximating operational stress, providing insight into vulnerabilities that may not be apparent in routine clinical practice.
Key Findings and Significance
The primary finding of this study is the consistency in both the type and severity of LSTs identified across 2 distinct hospital settings. A substantial proportion of LSTs were categorized as high or very high priority, with similar median FMEA scores across sites. The most critical threats were related to patient flow, communication, triage, and coordination across system interfaces. Importantly, many of these vulnerabilities occurred at transition points within the system rather than within isolated clinical tasks. These findings are significant as they highlight that hospital preparedness for internal MCIs depends largely on system-level processes, including coordination, communication, and workflow design, rather than solely on individual clinical performance.
Comparison With Existing Literature
Our findings are consistent with prior studies demonstrating that in situ simulation can effectively identify LSTs before they result in patient harm.Reference Holmes, Chipman and Barbour13, Reference Kotagal, Falcone and Daugherty14, Reference Congenie, Bartjen and Gutierrez20, Reference Grace and O’Malley23 In line with previous SbCST studies, including Kerrey et al.,Reference Kerrey, Boyd, Shoemaker and Zackoff24 the majority of LSTs identified in our study were related to system processes and interdepartmental coordination rather than individual technical performance. Similarly, studies in simulation-based disaster preparedness have consistently identified communication breakdowns, patient flow challenges, and deficiencies in situational awareness as key contributors to system-level failure.Reference McElroy, Steinberg and Keller25–Reference Naru, Long and Churruca28 Previous work has also demonstrated that integrating SbCST with FMEA provides a structured approach for prioritizing safety threats and guiding mitigation efforts.Reference Colman, Stone and Arnold22, Reference Nagpal, Vats and Ahmed29, Reference Weinstein, Linkin and Sausman30 In contrast to much of the in situ literature, which focuses on routine clinical environments, the present study extends the SbCST framework to a high-stakes, low-frequency scenario type with distinct operational demands, and demonstrates that the resulting vulnerability profiles are reproducible in character across institutions.
The recurrence of these themes across different institutions and settings supports the generalizability of these findings and reinforces the role of SbCST as an effective tool for uncovering system vulnerabilities.
Strengths
This study has several strengths. The multicenter design enabled comparison across 2 distinct institutional environments, enhancing the generalizability of the findings. The use of a structured methodology combining SbCST with FMEA facilitated both systematic identification and prioritization of LSTs.
A key strength of this work was the use of fully immersive, hands-on in situ simulations rather than tabletop or discussion-based exercises. This approach allowed participants to engage in real-time clinical and operational actions, increasing the realism of the scenarios and more accurately replicating system stress. As a result, it enabled the identification of LSTs that may not have been apparent in more theoretical or method-based simulations.
In addition, the inclusive data collection strategy, incorporating input from facilitators, trained observers, and participants, increased the likelihood of capturing a comprehensive range of system vulnerabilities. Finally, the integration of mitigation strategies into the debriefing process generated actionable mitigation targets for hospital leadership and emergency preparedness teams.
Limitations
This study has several limitations. First, it included only 2 institutions, which may limit generalizability. Second, FMEA scoring was based on expert consensus and is therefore subject to inherent subjectivity. Third, although in situ simulation approximates real-world conditions, it cannot fully replicate the complexity of actual MCI events.
The most significant limitation of this study is the absence of follow-up data on the implementation and impact of the identified mitigation strategies. While mitigation strategies were developed for the majority of LSTs during debriefing and responsibility for implementation was formally assigned to hospital leadership at each institution, the present study did not systematically track which strategies were implemented, at what fidelity, or on what timeline, nor did it assess whether implementation produced measurable improvements in system performance or preparedness readiness.
As demonstrated by Lago et al. in their FMEA-based quality improvement study of pediatric medication safety, re-scoring priority threats after implementing corrective actions combined with structured auditing at defined intervals provides substantially stronger evidence of impact and represents the natural closure of a QI cycle.Reference Lago, Bizzarri and Scalzotto31 The present study was designed as a structured initial systems assessment; future work should incorporate this critical follow-up phase, including re-simulation after targeted mitigation and systematic reassessment of priority LSTs at defined time points.
Conclusions
In situ SbCST identified multiple high-priority LSTs during simulated internal MCIs, with consistent vulnerability patterns observed across 2 institutions. The most critical threats involved system-level processes rather than individual clinical performance. The cross-institutional consistency of these findings suggests they may reflect generalizable vulnerabilities in internal MCI preparedness rather than site-specific gaps, and supports the use of SbCST combined with FMEA as a structured approach for identifying and prioritizing system-level risks. Prospective tracking of mitigation implementation and re-simulation following targeted interventions represent the essential next step in translating these findings into measurable improvements in hospital emergency preparedness.
Acknowledgments
We thank the emergency preparedness teams at the participating institutions for their contribution to the execution of the simulation exercises.
Author contribution
HKD: Conceptualization, project leadership, study design, data interpretation, manuscript drafting, and critical revision of the manuscript; RS: Data collection, simulation implementation, data review, and revision of the manuscript; ST: Simulation implementation, data collection, data review, and revision of the manuscript; HM: Emergency preparedness expertise, simulation implementation, data collection, and revision of the manuscript; AS: Simulation implementation, data collection, data review, and revision of the manuscript; AP: Institutional oversight, emergency preparedness, and health-system expertise, interpretation of findings, and revision of the manuscript; KB: Simulation coordination, nursing and educational expertise, data collection, data review, and revision of the manuscript.
All authors approved the final version of the manuscript and agree to be accountable for all aspects of the work.
Funding statement
No external funding was received for this study.
Competing interests
None.
Ethics approval
This study was conducted as a quality improvement initiative. Institutional Review Board (IRB) exemption was obtained at Meir Medical Center. The second site was included as part of institutional quality assurance activities.
