Learning Points and Transferability

We identified numerous reported learning points from MCE simulations that were applicable and transferrable to any hospital. Identification of the themes provides an opportunity to recommend areas where hospitals could ensure they have optimized their preparedness in responding to a major incident. This corresponds to the WHO Hospital Emergency Response Checklist. ^{Reference Gortolin, Sorensen and Zane26} This supports the assertion that simulation training helps to strengthen real response when the learning points from simulation and from real response can be superimposed.

The impact of human factors is pervasive throughout these 11 themes. How we respond to an MCE relies on the system in place, the people within it, and how they interact with the system and each other. These learning points highlight system error that could be minimized if identified before the volatile environment of an MCE, as well as institutional safety that contributes to effective response. ^{Reference Smith and Plunkett27}

Implementation of Actions From Identified Learning Points

Only 59% of the papers included reported action points or stated that action points were made to act upon. One of the main purposes of major incident simulation training is to improve preparedness by identifying areas that require attention. Although there might be an element of reporting bias, the target for reporting lessons learnt and action points implemented should be 100%. The lack of reporting of learning points gained from the simulation suggests a greater focus on the exercise as an endpoint, rather than the exercise being a tool for further improvement.

Central Register of Scenarios, Performance Matrix, and Lessons Learned

Designing a robust scenario for an MCE simulation may be one barrier to considering training the hospital team in this manner. It may also be why the simulation itself is regarded as the end point because so much preparation goes into making it a robust learning opportunity for the staff involved. If a central database of patient profiles, facilitator roles, scenarios, and imaging existed, this would be a positive step toward making this form of training manageable, as most of the preparation work could simply be lifted from it. The department, hospital or agency could, therefore, focus on the human factors involved in gaining engagement from all parties before the exercise, and on implementation of the learning points gained following the exercise for their specific risks.

Types of Exercise

Live exercises are time and resource rich. For this reason, we anticipated live exercises to be less commonly reported; however, this was not the case. Among the exercises that reported learning points, there was an abundance of live exercises compared with tabletop and virtual reality simulations. In contrast, a recent survey on major trauma preparedness in MCEs involving 4 developed countries reported tabletop exercises were far more common than live exercises. ^{Reference Gabbe, Veitch and Curtis28} Due to the significant amount of time and effort going into planning and executing a live exercise, there may be a publication bias toward live exercises as they may be considered more desirable for publication.

Departments Involved

Exercises can be departmental (1 department), multi-departmental (1+), or multiagency (eg, across emergency services). Our findings show highest rates as single departments in the hospital (45%) or multiagency (41%). The role of agencies such as emergency servicesFootnote ^a , the military, public health bodies, and charities are fundamental to MCE response and the findings are not surprising. The multiagency rates are comparable to a survey report where 52% of MCE exercises in New Zealand, Australia, England, and Canada were multi-agency exercises. ^{Reference Gabbe, Veitch and Curtis28} However, the same report states 48% of in hospital exercises were multi-departmental in contrast to our findings of 14%. This could be explained by not all exercises carried out being published in literature.

If the perception is that simulation is not worthwhile unless it tests the whole response, then hospitals could miss important learning opportunities. In high-performance sports, athletes are coached to break down their activity into its component parts and examine each action in detail. ^{Reference Haugen, Seiler and Sandbakk29} This approach can also be applied to the component parts of a hospital’s mass casualty response: call-in, triage, restocking of supplies, phone communication, role allocation, flow through radiology, expanding intensive care, etc.

Using the Clinical Space

Training courses in mass casualty response provide excellent experience and education for the participants who attend them. However, in this review, we were interested in the learning or benefit gained for the organization, and this is greatest when the exercise is conducted within the health-care setting that would be called upon in a real event. ^{Reference Sørensen, Østergaard and LeBlanc30} A PubMed search for “in-situ simulation” and screening for team training, revealed no papers published before 2000, three papers published between 2000 and 2010, and 29 published between 2011 and 2020. This reflects how simulation training has moved from a classroom environment to being embraced in the workplace. Mass casualty preparedness exercises appear to also reflect this shift. Most papers we identified were single department exercises, although three papers examined interdepartmental teamwork. Only one mentioned “whole hospital response”. It is our belief that this is reflective of the actual proportion of exercises. The logistics of an entire hospital engaging with a mass casualty exercise while contemporaneously maintaining the standard of care to current patients precludes this scale of exercise being done frequently.

Time of Publication

One would expect a rise in the number of major incident simulations taking place following real major incidents. We identified the highest numbers of publications in 2012 and 2014 followed by 2003 and then 2019. We could not correspond these dates of publication with specific mass casualty events. However, it is evident that the number of global deaths from terrorism started rising in 2011 and more than quadrupled by 2014. ³¹

Prehospital Versus in Hospital Exercises

Prehospital simulations were expected to outnumber in-hospital exercises. There is almost unlimited space to train teams in MCE response outside, but difficult to run training exercises alongside ongoing service provision inside health-care facilities. Nonetheless, more papers described in hospital than prehospital simulation exercises. We do not believe this is a true representation of the number of exercises carried out pre and in hospital. Every year, prehospital multi-agency exercises are carried out and reported in the media, however, not always in literature. ^32–34 It appears that simulation training has become standard practice in the prehospital environment; thus, the yearly exercises are not always reported in literature. It is also possible that our searched databases did not necessarily include all reports of such exercises. It is perhaps an indication that simulation exercises to enhance mass casualty preparedness are not the standard practice within hospitals and, therefore, results in more papers written and published. This is something that we believe needs to change. If “no plan survives first contact with the enemy,” that first encounter should not be facing the real enemy.

Does Simulation Training Improve Response in Real Mass Casualty Incidents?

The ultimate aim of simulation is transferable advantage in real situations. This is almost impossible to quantify due to the unpredictable nature of mass casualty incidents. Confidence in responding to the Manchester Arena attack in May 2017 was attributed to having partaken in a recent simulated exercise, but difference in performance and contribution between the staff who did and did not attend the training remains elusive. ^{Reference Skryabina, Betts and Reedy35} However, the main themes found from our systematic review are reproduced in learning points from many real mass casualty events. ^36,37 The number of debriefs from mass casualty incidents that credit recent simulations for the cohesion of their response also gives credence to this method of training. ³⁶

Limitations

Bias in evaluation of exercises was a theme in the selected studies. Their findings were mostly self-reported by the participants. Some studies also reported response bias as only a subset of individuals responded to post-exercise questionnaires. Furthermore, there was no standardized way of evaluating performance. It is plausible that the collective learning points based on these feedbacks may have been influenced by such limitations.

It is plausible to assume variance in knowledge and in pre-exercise experiences between studies. These variances included regular training courses such as Adult Trauma Life Support® in preparation for the exercise, 1-hour pre-exercise explanation before a tabletop exercise or a simulation done twice but lessons reported together. It should be appreciated that these could have had an impact on the lessons learnt from an exercise.

The greatest limitation of our study was publication bias. ^{Reference Dickersin38} We are aware of major incident simulations that take place annually, yet they are not published in literature. We also note high number of CBRNE scenarios reported in literature, however, they are not as commonly encountered events. It is possible that only very large and effortful exercises are considered “worthy” of publication, creating a gap in important learning points that smaller scale exercises can identify.