Introduction
Health ministries and the community-at-large expect hospitals to be prepared to cope with all types of emergencies, including the mass arrival of casualties following a disaster of any type. It is well recognized that children are a population segment especially vulnerable to disastersReference Bank and Khalil 1 because of their unique anatomical, physiological, psychological, and developmental particularities.Reference Waisman, Aharonson-Daniel and Mor 2 , Reference Cichon, Fuchs and Lyons 3 It is therefore important that pediatric hospital disaster responders be trained specifically and be prepared to receive and manage children in a multiple/mass-casualty incident.
Hospitals worldwide use various systems of emergency coding in order to alert staff quickly and to provoke an efficient response. One such system is color coding, in which Code Orange denotes an external disaster that may provoke the arrival of multiple or mass casualties. Training hospital personnel to respond to such rare, high-impact events is a difficult task. This particularly is true of events involving children, as this expertise is limited. In addition, disaster response and management rests on a fundamental paradigm shift in the way that medical practitioners provide care. It is based on the idea that resources are not unlimited and may be overwhelmed by the demand of an increased number of patients requiring care. Although simple in principle, it is a shift that is difficult to apply in practice as most health care workers have not had to work under such conditions.
Simulation as a modality has been a core means for training and evaluating performance of health care workers involved in a disaster response.Reference Weiner 4 - Reference Scott, Maddux and Schnellmann 7 Some evidence has demonstrated that simulation offers a superior and longer-lasting learning experience than traditional didactic methods alone when training medical staff to manage and treat pediatric disaster victims.Reference Cohen, Sevdalis and Taylor 8 The experiential learning activity focusing on disaster management must construct a realistic incident in which providers care for multiple injured patients while simultaneously coping with numerous stressors designed to tax an institution’s resources. In disaster simulation, the use of high-fidelity simulators and live-actor patients have equivalent results in prompting critical actions in mass-casualty drills and have increased the perceived reality of such exercisesReference Behar, Upperman and Ramirez 9 as they provide life-like medical scenarios with real-time stressors. In addition, studies have shown a superior knowledge acquisition and retention following simulation-based learning in medical students and residents,Reference Gillett, Peckler and Sinert 10 , Reference Anderson, Murphy and Boyle 11 but this has not been demonstrated in advanced learners such as the practicing pediatric emergency physician (PEP).
It was hypothesized that an experiential learning experience, focused on the principles central to the paradigm shift that is fundamental to a disaster response, would provide a realistic learning experience, and that such an exposure would improve practicing Pediatric Emergency Medicine (PEM) physicians’ and senior residents’ perceived abilities to manage pediatric patients in a mass-casualty event and increase their confidence in such a situation. An evaluation was proposed of the effects of such a learning activity on the advanced learners’ perceived knowledge and confidence with regards to medical decision making and management during a disaster response. Furthermore, the observed behaviors were hoped to help determine the learning needs of this group of responders, thereby contributing to the development of future disaster preparedness curriculum.
Methods
Program
In order to provide a disaster experience to physicians who ultimately would be first receivers in a mass-casualty event, a needs-assessment-based, 4-hour simulation workshop was created focusing on the paradigm shift that is fundamental to a disaster response.
Study Population/Inclusion Criteria
All PEM core physicians, senior PEM physicians, and pediatric critical care and pediatric surgery residents were invited to participate in the disaster workshop in December 2011. Twenty-seven participants took part in the workshop. Seventeen of the participants were PEM attending physicians, two were senior pediatric surgery trainees, two were senior pediatric intensive care trainees, three were senior PEM trainees, two were PEM clerical staff, and one was a PEM nurse. This exercise targeted the advanced learner who may be implicated directly in a disaster response and who likely would be expected to take on a leadership role in such circumstances. Due to limited disaster training resources in the given context, the group of physicians and trainees that was targeted had only been exposed to didactic sessions and table-top exercises. This particular group was not thought to have been exposed to any disaster simulation-based exercises involving direct management of standardized patients or mannequins in the past, therefore presenting a unique opportunity to describe the effects of an experiential learning experience on the advanced learner.
Exclusion Criteria
Non-physicians and any resident below their fourth post-graduate year of training were excluded from this study.
Needs Assessment
The workshop was created with the needs of the participants in mind. Due to recent experience in the Montreal (Quebec, Canada) area with a local college shooting, participants voiced a concern in being prepared for a disaster and wanted to experience managing a multiple-casualty incident in a safe learning environment. Round table discussions with the PEM group, as well as previous concerns in table-top exercises, identified participants’ needs.
Educational Intervention
The workshop was composed of a 30-minute, mini-plenary session which reviewed principles of disaster management and details of the hospital disaster plan. Participants subsequently rotated through three clinical stations, each involving two or more patient encounters: Triage (12 minutes for 30 cases and 18 minutes for debrief); Yellow (12 minutes for three cases per group and 18 minutes for debrief); and Red (20 minutes for two cases per group and 45 minutes for debrief). These care stations were based on the color triage system where Red represents the most urgent cases, Yellow those who can wait a short time before receiving treatment, Green are the walking wounded, and Black is the expectant treatment category. Upon completion of all cases, a brief wrap-up session took place to further discuss disaster-management strategies in a pediatric environment and allowed learners the opportunity to ask their remaining questions.
The simulations themselves used a combination of high, medium, and low-fidelity simulators, as well as standardized patients, to achieve the following learning objectives for Crisis Resource Management (CRM), Code Orange (Table 1), and medical objectives. Evaluators, who were selected for their experience in disaster preparedness and/or simulation and debriefing, observed all patient interactions. They provided feedback to participants in the debriefing sessions that followed each portion of the scenario.
Learning Objectives Specific to the Simulated Patient Care Areas in a Code Orange
Abbreviations: ER, emergency room; START, Simple Triage and Rapid Treatment.
The added challenge when dealing with an advanced learner being taught by peers is that there may be a concern that their performance is being assessed and their competence may be judged based on that performance. The “safe learning environment” aspect of a simulation was emphasized and enforced during both the simulation and the subsequent debriefings. In addition, during the opening plenary, participants were reminded of the simulation center code of conduct that includes this principle explicitly. Evaluators had received their assigned cases and evaluation grids approximately one month prior to the workshop. Evaluator training consisted of two, 1-hour, open evaluator training sessions to address any questions specific to each case assigned, and a 30-minute briefing immediately prior to the simulation, reviewing the evaluation process and reiterating the need for a respectful and safe learning environment.
The workshop was based on a scenario describing an earthquake affecting the surroundings of a major metropolitan city causing particular damage at the site of an amusement park. The initial confirmed number of patients to be transported to the emergency room (ER) in question was 13, with another 10 to a secondary pediatric center. The total number of casualties remained unknown.
Assessment Instruments
Perceived Ability to Manage Medical and CRM Components of the Care of a Pediatric Patient in a Disaster Situation
In order to assess the participants’ perception of their ability to manage the cases in the different disaster care areas, they were asked to complete a questionnaire at the end of the workshop. The survey used a retrospective pre-post formatReference Tsai, Harasym and Nijssen-Jordan 12 , Reference Mancini, Soar and Bhanji 13 to assess change in perceived abilities to manage disaster victims, including triage, re-triage, medical management, and CRM management in a disaster context before and after the workshop (Survey Appendix 1; available online only). In addition, six months after the simulation session, the participants were asked to fill out the same questionnaire again to assess their perceived retention and confidence.
Objective Assessment of Medical and Disaster Management
Evaluation grids for each case and each treatment area were created based on pre-determined medical and disaster-related learning objectives categorized by CanMeds roles (Queen’s University; Kingston, Ontario, Canada). Evaluators completed the evaluation grids as participants progressed through each case or treatment area (Appendix 2; available online only).
Data Analysis
Pre- and retrospective post-data from the questionnaire were analyzed using the Wilcoxon rank sum test for nonparametric data using SPSS version 21 (SPSS; IBM Corp.; Armonk, New York USA). A result of P<.05 was considered a statistically significant change from pre to post.
Objective data from each case were derived from the evaluation grids. They were collated and then categorized into areas done well/completed (“yes” column in the grid) and areas needing improvement (“borderline” and “no” columns in the grid). Results helped to inform further learning needs in disaster management for this group of learners.
Ethics
The Research Ethics Board of the Faculty of Medicine, McGill University (Montreal, Quebec, Canada) granted ethical approval for the study. Workshop participants provided written informed consent for their participation in the study.
Results
Perceived Ability to Manage Medical and CRM Components of the Care of a Pediatric Patient in a Disaster Situation
Improvement in perceived ability to manage the medical and CRM components of the care of a pediatric patient in a disaster situation demonstrated statistical significance for all retrospective pre-post survey questions completed at the time of the workshop, as well as at six-months post-simulation (Table 2). In addition, participants felt that the pediatric disaster simulation day was valuable to their learning (5.6 on 6 point Likert scale). Furthermore, confidence was retained after six months.
Retrospective Pre-Post Survey (initial and 6 months after completion of the Disaster/Code Orange workshop)
Abbreviation: START, Simple Triage and Rapid Treatment.
Objective Assessment of Medical and Disaster Management
All evaluator grids were compiled and collated in order to identify strengths and areas needing improvement in each disaster care area: Red, Yellow, and Triage (Table 3). Within each case, many strengths were found; yet certain areas were identified as needing improvement, such as: prioritizing resources within the disaster context; downgrading or upgrading patients triage category once the initial triage had been completed; and difficulty in stopping resuscitative measures in an unsalvageable patient in order to care for new incoming patients.
Opportunities for Improvement Identified by the “Borderline” or “No” Checkboxes within the Medical Evaluation Grids
Abbreviations: CT, computerized tomography; ICU, intensive care unit.
Discussion
Hospital disaster preparedness rests firmly on a solid education and maintenance of competence program. A combination of didactic teaching and various types of simulation has been the traditional way to train medical personnel for disaster response. The impact of a half-day, simulation-based, disaster workshop was created and studied for the advanced learner. Simulations are resource intensive. Their effectiveness has been shown in residents,Reference Gillett, Peckler and Sinert 10 , Reference Anderson, Murphy and Boyle 11 but few studies exist of their effectiveness in the advanced learner, more specifically, the PEP already in practice who is likely to assume a leadership role in the event of a disaster.
Analysis of subjective data demonstrated that there were significant perceived improvements in the participants’ abilities in all areas of disaster management. Specifically, Questions 3-7 of the survey (Table 2) relating to disaster triage showed a significant change in perceived ability at the end of the simulation, and this perceived improvement was retained at six months. Likewise, Questions 10-12 addressing medical management particular to a disaster situation showed a significant change in perceived ability at the end of the simulation, and this perceived improvement also was retained at six months. Most importantly, the participants felt more confident about their ability to respond to a disaster after the session, reflected in Question 15, and this confidence was retained six months later. In addition, participants felt that the pediatric disaster simulation day was valuable to their learning (5.63 on 6 point Likert scale). This suggests that disaster simulation, when framed appropriately, can be an effective means of training and maintaining the competence of the advanced learner.
Analysis of the objective data allowed the determination of areas of medical management and decision making that were strong and those that need attention in future teaching and training sessions. All data collected from the evaluation grids filled in by the evaluators were collated and simply classified as “Strengths” and “Areas Needing Improvement.” Overall, these participants were able to apply basic principles of pediatric trauma management despite the increased stress of having multiple victims to care for with limited resource availability. It was shown, however, that at times, the secondary survey was incomplete and that certain infrequently encountered traumatic pathologies (such as fat embolus) were not recognized consistently. Participants generally were able to apply the concept of paradigm shift central to disaster response, such as a change in triage, but had difficulty with prioritization of patient management and resources when multiple casualties arrived concurrently. This was likely due to participants’ low frequency of previous exposure to multiple-casualty events. Debriefing of the cases addressed these challenges, and discussion amongst participants on how to mentally reframe the scenario given the disaster context was stressed. The change of triage categories once initial triage was complete also posed a problem for some participants. This may be due to the infrequent need to formally re-triage patients on a regular basis in the emergency department. The need and importance of re-triage in a disaster situation was re-emphasized during the debriefings. It also was noted that there was some difficulty with adjusting actions based on scenario evolution, specifically implementing the “unsalvageable” triage category despite a worsening scenario and increased demands on the already limited resources. Further discussion of the unsalvageable category occurred in the debriefing session. The principle of “stabilize and dispose” generally was felt to be understood, except for a few clinical situations (Table 3). The stabilization of the most critically injured patients, although a strength, could have been accelerated in order to transfer care to the intensive care unit or treating surgeon. This would have freed the ER team to take care of the next casualty. On the whole, the participants were able to recognize and apply many pediatric-specific disaster principles.
Many coordination aspects of the disaster response, such as patient tracking, inter-area communication, and inter-departmental communication, were not addressed in this simulation. It was noted, however, that patient identification and chart assignment were inconsistent. Thus, the data gathered allowed identification of opportunities for improvement and future training needs for this cohort of responders, thereby contributing to the disaster preparedness curriculum development.
Limitations
The study was evaluated using self-assessment surveys and subjective medical evaluations. The study was based on an educational activity with limited space for participants, thus the number of participants completing the workshop and survey was limited. This was known previously and expected. Another potential limitation was the site in which the study and simulation took place. Although holding a simulation at an off-site center has certain advantages, such as less disruption of normal hospital activity and patient care, it presents some important limitations. In reality, the team of responders includes many professionals, such as nurses, patient care attendants, respiratory therapists, psychologists, spiritual care professionals, clerical staff, and several other ancillary service members. It was not possible to have a significant number of non-physician participants. Therefore, many aspects of the team interaction were not realistic and could not be simulated adequately. As well, some key aspects of disaster management concerning inter-departmental communication and coordination of care could not be assessed at the simulation center. These were therefore not included in the learning objectives for this simulation. Despite these limitations, the realism of the patient encounters allowed for the participants to feel a similar stress to what they would experience in the event of a true disaster response and provided an initial experience for participants and organizers regarding disaster preparedness education.
Conclusions
Advanced learners who may be implicated directly in a disaster response believed that the simulation workshop improved their ability to manage patients in a disaster situation and felt that it was valuable to their learning. This confidence was preserved six months later. Evaluator grid collated data identified the learners’ needs and contributed to subsequent disaster management curriculum development.
Acknowledgement
The authors would like to acknowledge the Steinberg Centre for Simulation and Interactive Learning at McGill University (Montreal, Quebec, Canada) for its support in the implementation of the simulation session associated with this research study.
Supplementary Material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1049023X16000704
