%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
Delivering quality medical care during a disaster requires adequately prepared teams of health-care providers working together to save patient lives. Few providers receive competency-based training to respond to low-probability, high-risk events such as natural disasters, terrorist attacks, or other mass-casualty scenarios. The 2014 American College of Emergency Physicians report on the state of emergency care in the United States gave the country a “C-” in disaster preparedness. 1 Overall, 14 states received an “F” grade, partly because of inadequate education and funding for emergency preparedness training (EPT). 1
Despite recommendations from the Association of American Medical Colleges and the Institute of Medicine (IOM), US medical schools have been slow in developing stand-alone EPT curricula. 2 , 3 Although Joint Commission regulations require all “emergency services” facilities to perform yearly disaster drills, hospitals have been reluctant to develop comprehensive EPT programs on their own. 4 Consensus reviews suggest that health-care worker training programs in the United States lack clarity, objectivity, competency-driven goals, scientific rigor, prospective validation, and consistency across medical specialties.Reference Subbarao, Lyznicki and Hsu 5 - Reference Blanchard and Dosa 8
In a mass-casualty event, there are 2 fronts of personnel. Forward-deployed first responders include emergency medical technicians, police, firefighters, paramedics, and others at the scene of an emergency. The “Second Front,” or first receivers, includes clinicians, hospital workers, mental health providers, public safety officials, community volunteers, trainees, and administrative or other lay personnel. Experience has shown that in the immediate aftermath of a mass-casualty event, more than half of the victims will often bypass the first responders and report directly to hospitals. For example, during the 1995 Tokyo sarin-gas attack, more than 80% of patients self-transported to hospitals where staff and patients were exposed to sarin gas because of deficits in decontamination, lack of personal protective equipment (PPE) and awareness, and inadequate training.Reference Tokuda, Kikuchi, Takahashi and Stein 9
The hospital-coordinated and bystander responses to 2 recent large-scale mass-casualty incidents illustrate the importance of coordinated health-provider responses to save lives. In the recent Paris mass-casualty incident in November, 2015, 130 patients died before reaching the hospital. Of the 302 victims who presented to 16 different hospitals, only 4 (1%) died at the hospital. In addition to a swift and coordinated mobilization of resources, credit for this successful response has been given to the presence of health-care professionals able to perform triage as well as prehospital damage control.Reference Hirsch, Carli and Nizard 10 In the Boston Marathon bombings of April, 2013, 281 individuals were injured and 3 patients died before reaching a hospital. Of the 127 victims who presented to a Level 1 Trauma center, no patients died thanks, in part, to prehospital interventions and organized risk-stratification of victims.Reference Gates, Arabian and Biddinger 11 In both Paris and Boston, Second Front providers were able to rapidly identify and risk-stratify patients during a moderate patient surge.
Educational research has demonstrated that collaborative, multimodal and problem-based learning environments increase learner satisfaction and may lead to better performance and long-term trainee retention.Reference Prince 12 - Reference Vernon and Blake 15 With this in mind, and in response to a great need for EPT among health-care professionals, a competency-based curriculum was developed, which included a mixture of didactics, small group discussion, problem-based learning, and collaborative live scenarios. The goal was to evaluate the effect of a short, performance-based EPT course to improve the ability of a diverse group of disaster-care providers to respond effectively to mass-casualty scenarios.
METHODS
Curriculum Development
The Center for Health Professional Training and Emergency Response (CHPTER) was founded in 2009 with the mission to develop state-of-the-art, performance-based EPT curricula utilizing community resources from clinical, public health, private, governmental, and non-governmental coalitions (www.musc.edu/chpter). In 2010, a CHPTER curriculum task force established 9 learning objectives, 18 competencies, and 24 performance objectives for a 5-hour EPT course. Course elements were designed for a diverse group of learners with various degrees of expertize as well as different learning styles.
Detailed descriptions of the course goals, development of the curriculum task force and curriculum content, evaluation measures, and preliminary data have been published previously. These publications include preliminary cognitive and self-assessment data as well as limited small group and mass-casualty-incident (MCI) skill assessment data.Reference Scott, Smith, Jones, Manaker, Seymore and Fulkerson 16 - Reference Jones, Staub, Seymore and Scott 18
Participants
Trainees were enrolled on a first come, first serve basis between November 2011 and August 2014. Advertising for the course included the use of university e-mail, social media, paper fliers, and word-of-mouth. The target audience included any provider who might render patient care during a disaster, including physicians, nurses, medical students, and other interested medical-university learners and employees. The course was voluntary and offered at no charge. The study was approved by a university-based Institutional Review Board.
Course
Training was initiated with brief multimedia presentations, followed by small group exercises to provide trainees with the knowledge and skills they would need to mitigate multipatient scenarios at the conclusion of the course. Didactic presentations included background on emergency preparedness, PPE, teamwork, communication, and disaster triage. During the first small group exercise (SGE1), teams of 4-6 participants worked on large jigsaw puzzles to experience and practice disaster-scene teamwork and communication. The performance objectives for SGE1 included having a prehuddle, establishing a team leader, assigning roles, understanding the roles, discussing the threat, completing the puzzle, and describing the scene. The second small group exercise (SGE2) challenged teams to triage 60 patients simulated by small toys with printed description and vital signs. The performance objectives for SGE2 included establishing a chain of command, verbal communication, relaying key information and communication accuracy, conciseness, and clarity. Performance objectives for the small group exercises were either “met” or “not met,” according to facilitator assessment.
Following small group exercises, teams were assessed on their ability to mitigate 2 different multipatient MCI training scenarios. In the first scenario (“Storm Surge”), teams triaged live patient actors in a dark, chaotic environment. During “Storm Surge”, teams were assessed by trained facilitators using 6 performance objectives based on a 10-point Likert scale in which 1=Did Not Meet Expectations and 10=Exceeded Expectations (Table 2).
In the second scenario (“Influenza-Like Illness”), teams were confronted with patient surge and a possible terrorist threat from outside the emergency department (ED).
During “Influenza-Like Illness,” teams were assessed by facilitators using 12 performance objectives based on a 10-point Likert scale in which 1=Did Not Meet Expectations and 10=Exceeded Expectations (Tables 2 and 3). For example, if providers secured the ED from the outside threat, a performance objective was met. If they failed to secure the ED, they would face a chaotic patient environment and a potentially uncontrolled bioterrorism threat.
Trained facilitators helped organize both small group and MCI scenarios. When available in sufficient numbers, facilitators – all of whom were volunteers – were asked to submit written assessments of team performance. Although facilitator evaluations were helpful in assessing trainee performance, facilitators were commonly unavailable. Performance data are limited to training with facilitators working in sufficient numbers to evaluate trainees.
Evaluation Measures
Trainees answered a 24-question online test both before and immediately following the EPT course. Of the 24 questions, 18 were identical and form the basis of the pre- and postcomparison; whereas the pretest was required to start the training day, the posttesting was voluntary and not all students completed posttest assessments. In addition, whereas all trainees completed the didactic component of the course, only about half completed the small group and MCI portions of the course, largely due to time constraints of the trainees.
To increase enrollment, trainees were allowed to simply show up on training day. As a result, estimates of which students completed the full course are simply estimates based on the date of online pretest completion.
An early version of the course utilized high-fidelity human simulators in addition to the standard live-actor patients. In an effort to make the course easily reproducible in a variety of training contexts, high-fidelity mannequins were entirely replaced by patient actors after 2012.
Statistical Analyses
Descriptive statistics including means, standard deviations, frequencies, and percentages are reported. Pre- and posttesting were carried out using unpaired t-tests. When the course was first offered, data collected from trainees were not linked between pre- and posttest responses. As the course was improved for content, the data-collection procedures were also improved to allow for paired responses from the trainees. As the entire data set could not be paired, analyses are performed on unpaired records.
RESULTS
A total of 708 individuals enrolled and participated between November 2011 and August 2014. The participants included 353 (49.9%) physicians, 226 (31.9%) medical students, 51 (7.2%) nurses, and <1% each of emergency managers, mental health providers, EMS personnel, law-enforcement officials, and others.
Of the 708 participants who completed the 18-question pretest, 509 completed an identical posttest. The average number of correct answers increased from 39% to 60% (P<0.01). Average trainee self-assessment of overall disaster-preparedness skill improved from 36 to 73 points out of 100 (P<0.01) (“Overall comfort level performing to protect and save human life during a disaster”). Average training self-assessment of overall disaster-preparedness knowledge improved from 33 to 74 points out of 100 (P<0.01) (“Overall knowledge of disaster preparedness and disaster medicine”) (Table 1).
Pre- and Post-Cognitive Assessment and Trainee Self-Assessment
A total of 251 participants completed a course assessment. Most felt that the course was relevant to patient-care providers (96%), that the size of the training class was “adequate size to achieve course objectives” (92%), and that they had an “adequate length of time to achieve course objectives” (86%). Overall, 92% highly recommended the training.
Facilitators evaluated 38 teams who completed small group exercise 1 (SGE1) and 37 teams who completed small group exercise 2 (SGE2). During the jigsaw-puzzle communication exercise (SGE1), teams met, on average, 5.08±2.01 of the 7 performance objectives (average of 6 participants in each team). During the triage tabletop exercise (SGE2), teams met, on average, 5.76±0.55 of the 6 performance objectives (average of 6 participants in each team, n=37 total teams). In all, 89% (33/37) of SGE2 teams met all of the objectives and 100% (37/37) of SGE2 teams were able to establish verbal communication with incident command and convey an accurate description of the disaster scene.
Facilitators evaluated 12 teams during MCI scenario “Storm Surge” and 24 teams during “Influenza-Like Illness” (average of 6 participants per team). During “Storm Surge,” 6 performance objectives were assessed using a 10-point Likert scale (1=Did Not Meet Expectations and 10=Exceeded Expectations). Overall, teams scored >6 out of 10 on all performance objectives, with the highest average scores for teams’ recognition of a disaster (9.0±1.1), utilization of resources (7.8±1.3), and personal and staff safety (7.8±2.4) (Table 2).
“Storm Surge” and “Influenza-Like Illness” Mass-Casualty Training Scenarios, Facilitator Assessment of Team Performance by Performance Objectives Using 10-Point Likert Scale (1=Did Not Meet Expectations, 10=Exceeded Expectations)
During the MCI scenario “Influenza-Like Illness,” 12 performance objectives were assessed. Teams successfully used proper PPE (24/24), established team roles at the beginning of the scenario and followed team roles throughout (23/24), adhered to personal and team safety (22/24), established a communication link (21/24), established a chain of command (16/24), communicated needs with incident command (15/24), and reassessed patients and the scene (13/24). Teams were least successful at securing the facility and preventing an influx of contaminated patients (Table 3).
“Influenza-Like Illness” Mass-Casualty Training Scenario; Number of Teams that met Each Performance Objective as Assessed by Facilitator (Binary Did or Did Not Meet), Out of a Total of 24 Teams
Abbreviations: PPE, personal protective equipment; ED, emergency department.
DISCUSSION
This study demonstrates that a half-day, easily reproducible EPT course can improve knowledge and foster the performance skills necessary for a diverse group of trainees to respond to a clinical disaster training scenario. Trainees increased their knowledge of emergency preparedness, improved their comfort and confidence with handling clinical disasters, and highly recommended the course. Trainees completing the full course also successfully completed a series of challenging small group and multipatient scenarios.
The course was completed in a medical-university setting without compromising existing course schedules, demonstrating that disaster training can occur in addition to existing university course requirements. An unexpected finding of the curriculum-development process was that the majority of training was completed at little or no cost. Unique, low-cost tools such as puzzles and toys significantly reduced the financial burden of the course. In addition, using trainees as patients during 2 (simultaneous) scenarios eliminated the need for paid actors or costly simulators. Further iterations of the curriculum could benefit from cost per provider, cost per training hour, and in-kind cost assessments associated with utilizing volunteers.
The low average score on the knowledge-based pre-assessment underscores the need for introductory-level EPT among health-professional and medical-student populations. This study contributes to a growing understanding that poorly trained staff members can place patients and providers at significant risk during even a small-scale disaster.
As seen earlier in previous publications, the percentage of correct answers to knowledge questions increased from pre- to posttest (P<0.01) following the course.Reference Jones, Staub, Seymore and Scott 18 In terms of the curriculum, trainees were more likely to succeed in mechanical objectives such as triage, donning PPE, and providing emergency communication. Other skills, such as teamwork, general communication skills, ability to comfort patients in distress, and ability to problem solve under pressure were not as easily mastered during a half-day course.
It could be argued that trainee performance during MCI simulation was poor when compared with typical 100% scales. For example, teams on average met 68.5% of objectives when asked to manage a disaster scene with multiple patients in a dark, chaotic environment. However, the MCI scenarios were intentionally designed to be loud and difficult to manage. In the “Influenza-Like Illness” scenario, for example, the vast majority of groups failed to prevent the threat from entering the ED. Though facility safety is addressed in the didactic component of the course, the chaos and timing of this scenario proves to be a significant challenge to trainee groups. As the threat breaches their facility, participants experience a simulated version of what could be a very costly mistake in the event of a real disaster or bioterrorism event. The room becomes noisy and crowded, and a potentially lethal substance is introduced. Experiencing the consequences of the mistake, along with a debriefing following the simulation, allows for a more profound learning experience by allowing for failure. Further long-term assessment of this curriculum will hopefully show that students learn more, perform better, and retain knowledge and skills longer when they confront failure via a challenging testing environment.
Limitations
This study has several additional limitations. First, the study included a pre- and posttest for knowledge and personal assessment of skills, but performance was not measured before the course. Because of this, any success trainees may have demonstrated could have been related to previous experience. It would be ideal to understand how profession, subspecialty, and degree of previous training impact performance during the course, although given the low self-assessment and pretest cognitive scores, the impact of previous disaster experience in this study sample is arguably low.
The study was also limited to 1 site and mostly to medical students and physician participants, therefore generalization of the curriculum to other trainees and lay people may be limited. Because of the challenges of enrolling voluntary trainees and facilitators (during their free time), not all students completed the full course and post-assessments. Therefore, conclusions about which components of the course contribute to student success are limited.
Although reliance on volunteer facilitators reduced the burden of hiring trained researchers, it also greatly limited our sample size to measure performance. The small sample for team assessments limits the generalizability of the curriculum and its expected impact on clinical disaster performance. Finally, further iterations of this study could provide a more robust assessment of cost including cost per provider, cost per training hour and in-kind costs associated with utilizing volunteers.
CONCLUSIONS
This course does not represent comprehensive training for disaster response; however, as shown in this study, a relatively short, intensive EPT course can improve knowledge and comfort level. It can also foster the performance skills necessary for a diverse group of trainees to respond to disaster-medicine training scenarios. Components of the course may be adapted for use in “just-in-time” training for medical students, nurses, hospital teams, or other personnel to foster knowledge, comfort level, and skill.
The AAMC, IOM, and other organizations have been clear in their call for action to improve emergency preparedness at all levels. Although further research is needed, this course is an innovative, effective, potentially affordable, and easily reproducible contribution to the clinical EPT ecosystem.
Acknowledgments
The authors would like to thank the hundreds of trainees who have participated in this course. Because of the interactive nature of the training day, conducting this course would not have been possible without everyone’s participation and enthusiasm. Also, the authors would like to thank the volunteers for serving as facilitators and for their participation in data collection, trainee evaluation, and feedback that helps us to improve the course with each iteration.
Funding
The project described was supported by Award Number UL1TR001450 from National Institutes of Health (NIH) Clinical and Translational Science Award (CTSA) program. The CTSA program is led by the NIH’s National Center for Advancing Translational Sciences (NCATS). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Ethical Approval
Approved by the Institutional Review Board.
Previous Presentations
A preliminary study was published in Prehospital Disaster Medicine: Jones J, Staub J, Seymore A, Scott LA. Securing the second front: achieving first receiver safety and security through competency-based tools. Prehospital Disaster Medicine.
