As a matter of course, in most countries, those who respond first to the outbreak of a disaster are specialized governmental organizations such as civil protection, national emergency services, fire brigade, police, coast guard, military forces, and so on. For health care workers in these fields, contrariwise to their civilian colleagues, disaster preparedness begins during their university studies, having periodical training as an integral part of their profession. Reference Mortelmans, Lievers, Dieltiens and Sabbe1 Nevertheless, since the potential for disaster is growing worldwide, future generations of civilian health workers are likely to respond to such events during duty. Taking cognizance of the prospective around disaster nowadays, both civilians and professionals are potentially involved in disaster response and the public have expressed the need for better preparedness. Reference Heinrichs, Youngblood, Harter, Kusumoto and Dev2-Reference Tavan, Menati, Azadi, Sayehmiri and Sahebi4
The provision of high-quality medical care in the challenging and highly dynamic environment of a disaster requires specific knowledge, technical skills and non-clinical competencies from health care personnel. Reference Nasrabadi, Naji, Mirzabeigi and Dadbakhs5,Reference Yang, Xiao, Cheng, Zhu and Arbon6 The main challenges of medical response in such situations are often, due to poor understanding of the existing medical disaster protocols, lack of experience and inadequate education and training of the involved personnel. Reference Baack and Alfred7,Reference Thorne, Oliver, Al-Ibrahim, Gucer and McDiarmid8 Training is widely accepted as the key action that can reduce the negative effects of disasters, by creating skilled health care professionals. Nonetheless, although numerous competencies for disaster health care personnel have been developed and endorsed by governmental and professional organizations, universal acceptance and application of these competencies is lacking, resulting in diversities in mass casualty incidents (MCI), disaster preparedness, as well as training curricula. Reference Daily, Padjen and Birnbaum9,Reference Ripoll Gallardo, Djalali and Foletti10 Additionally, significant limitations in the design of evaluation methods of disaster preparedness and training have been addressed. Reference Hsu, Jenckes and Catlett11
In this context, the medical approach to disaster preparedness has proven to be a problematic issue. Reference Gioffrè-Florio, Murabito, Crupi and Famà12 Earlier experiences in teaching emergencies and disaster preparedness have shown that blended learning approaches increase staff´s knowledge and their satisfaction. Reference Khorram-Manesh, Ashkenazi and Djalali13 Throughout the combination of theoretical and practical teaching, learners experience reality by simulation, make controlled errors, and develop an understanding of the doable and the undoable, Reference Khorram-Manesh, Ashkenazi and Djalali13,Reference Leaming, Adoff and Terndrup14 as demonstrated in fields beyond health care, where outcomes are also dependent on individual and team skills. Operation-based exercises (field exercises) represent a typical example of teaching activity simulation, considered to be an important and integral part of emergency preparedness activities, since they intend to enhance real experiences with guidance in a standardized manner. Reference Cannon-Diehl15,Reference Bond, Deitrick and Arnold16 Operation-based exercises typically involve responding to a scenario designed to simulate an emergency situation under more realistic conditions. Such exercises can range from smaller activities such as drills to practicing specific skills or procedures (such as triage, evacuation or communication) to field-based exercises designed to replicate as closely as possible a response to a real emergency. Because the primary purpose of most of such exercises is addressing organizational emergency preparedness (such as testing plans or procedures), little is known on the personal benefits from exercising on the involved staff. Only few quantitative studies included reports of improved knowledge, competencies, improved confidence, and understanding. Again, there is a lack of evidence about whether these positive changes persist over time. Reference Skryabina, Reedy, Amlôt, Jaye and Riley17
According to Bruner, repetition is the first principle of all learning. Repetition of training especially via experiential learning has positive results regarding the retention and persistence of knowledge. Reference Bruner18-Reference Ford and Schmidt20 This assumption has been confirmed in our case. The overarching purpose of the present study is to assess the role of field exercises in improving individual preparedness of health care professionals. We therefore present and discuss immediate and medium term results on the effectiveness and benefits of exercises to prepare health emergency professionals for responding to emergencies and disasters, based on their self-perception after participating in a set of 4 inter-professional field exercises organized and implemented in the framework of the MSc in Global Health-Disaster Medicine program of the School of Medicine, the National and Kapodistrian University of Athens, Greece, which was carried out yearly, from 2016 to 2019. The course on ‘Management of Disaster Victims’ is one of the flagship activities of the MSc in Global Health-Disaster Medicine that takes place every year in the third semester of the study. It is organized in collaboration with the local authorities and involves the participation of academics, experienced field workers and civil protection agencies. The scope of the course is to train students, as well as civilian health care professionals and representatives of humanitarian organizations who enroll in the course on how to deal with victims in MCI/disasters in practice.
METHODS
This study focused on the following research questions:
-
1) What is the educational value for those participating as trainees in field exercises, in terms of technical and non-technical skills, as a preparatory educational tool for disaster management performance, as well as acquisition of core competencies (technical and non-technical)?
-
2) What kind of impact do emergency exercises have on the immediate and medium term emergency preparedness of individuals?
The study presents quantitative and qualitative data analysis of the feedback (self-assessment) given by all trainees participating in the field exercises in 2016, 2017, 2018, and 2019 (a total of 228 participants). The questionnaires were distributed in 3 phases, before, 1 week and 10 months after their participation in the course (for those of 2016, 2017 and 2018). These questionnaires were identical for the 4 years. The response rate of the survey was 88%.
The questionnaire was self-administered following an explanatory session on its aim with the participants. The questions were finalized after a pilot phase to assess proper phrasing, time required to complete, and coding. The pre- and post-exercise evaluation questionnaire included 14 questions and 15 sub-questions, categorized in 3 groups each. The first group focused on demographic data. The second group assessed the level of subjective personal knowledge, experience and preparedness related to disaster response as at the time of the course. The third group assessed the personal perception of effectiveness on individual’s performance after MCI scenarios. The majority of questions were formulated so that participants could ‘agree’ or ‘disagree,’ while always having the option of ‘not sure.’ There were some questions in which the level of agreement was evaluated according to a 5-point Likert scale, from ‘strongly agree’ to ‘strongly disagree,’ and from ‘extremely’ to ‘not at all.’ In the end, there were some open questions. The self-reported evaluation compromises an integral part of assessing the impact of an intervention, as it measures the participants’ personal reflections and attitudes to specific elements. As such, the use of the Likert scale is widely used in facilitating the qualitative assessment of health interventions in order to provide valuable information on some of its characteristics in the way these are observed by the participants using an agreement scale. The validity of our qualitative results is observed through assessing the correlations of the different answers given.
The interdisciplinary exercises are launched annually in the third semester of the study program. The study aims to capture the short- and long-term effect(s) of participating in the MCI scenarios. Accordingly, questionnaires were distributed to capture the short and long term effects of the simulation exercise on the participants in the period between 2 consecutive exercises which is estimated as 11 months. The period of 1 week was chosen to capture the immediate effect of the training and the period of 10 months, as the longest period before the next simulation exercise of the program to capture the post-intervention effect. This choice was also required in order to avoid the introduction of information bias as the participation to similar exercises during this period was set as an exclusion criterion.
The educational syllabus of MCI scenarios was derived from disaster medicine literature, focusing on (1) Command-Control-Coordination Deployment, (2) Communication, (3) Triage, (4) Basic Trauma Care, (5) Safety-Security Precautions, and (6) Transport Management. The interdisciplinary exercise of MCI in Rhodos Island was executed in cooperation with the fire-brigade, law enforcement personnel, national ambulance service, coastguard, regional port authority, regional hospital, armed forces, regional civil protection service, civil aviation service and experienced field workers (e.g., Greek Red Cross).
Scenarios included mass casualty incidents after (1) a terrorist attack (active shooter- blast scenario) in a passenger ship at Rhodos Port, (2) emergency aircraft landing on regional airport, (3) fire in tourist location, and (4) a hospital preparedness scenario for mass casualty influx incident. Students were allocated in 9 different teams (Triage, Incident Command Centre, Green Team, Yellow Team, Red Team, Logistics, Evacuation Centre, Psychological First Aid, and Media). The duration of the exercise was approximately 1 to 1.5 hours, after which they were gathered together with the facilitators and the external evaluators, for the debriefing process. During this phase, an analysis of the events and discussion regarding the decisions made during the exercise was performed, based on the active reflection of participants.
RESULTS
Study Population
The median age of participants was 39 years old, represented by 56% women and 44% men. The sample consisted of 78% health care professionals (44% medical doctors [41% civilians, 3% military doctors], 22% registered nurses, and 12% other health care professionals), 16% social scientists and 6% other professionals. Although most participants had no previous exposure to a disaster management incident (65%), those who answered positively (25%), were involved in refugee crisis incidents (66%), earthquakes with multiple casualties (20%), and complex road accidents (14%). Many participants (69.6%) claimed that, if they were involved in a disaster, they could provide only limited support since they did not have previous training in disaster management. Only 30.4% of the participants had received training before the MSc.
Majority of the participants (83%) stated that they had participated in adult learning methodologies in Emergency Medicine prior to this course, 70% had taken medical simulation courses (BLS, ALS, etc.), real time MCI simulations (10%), problem-centered learning cases (8%), and other kind of exercises (12%). The above exercises had been undertaken in the context of personal initiative (46%), university studies (22%), continuing professional education (23%), and other non-specified activities (9%). With regard to the evaluation of this ‘prior exposure,’ the majority of the trainees (69%) assessed it as ‘moderately’ realistic (Table 1).
Demographic Data of Sample Population and Prior Experience to Disaster Management
Educational Outcome
The overwhelming majority (95%) agreed that MCI exercises are appropriate for disaster management training for comprehending the theory of disaster management, as well as for team training. Interdisciplinary training and preparation of all involved stakeholders in the response of a major incident was highly valued (86%) among participants. Regarding the educational method, all participants considered that blending theory and practice represents the most appropriate way of training. In order to achieve better preparedness in disasters, 83% of trainees consider the provision of introductory courses to be an absolute need. Interdisciplinary MCI Simulation led to a better understanding of the theoretical and practical knowledge acquired, according to 91% of the respondents ( Figure 1 ). Extensive briefing and debriefing after each field exercise was perceived as a valuable tool for better understanding of key learning points for the vast majority of the respondents (89% and 93% respectively). Based on the outcomes of the survey, participation in interdisciplinary MCI exercises requires special skills and knowledge for most respondents (> 50%) (Table 2). As necessary knowledge for a better mass casualty management, students mentioned the topic of emergency medicine, as well as disaster medicine. Considering the technical-skills domain, BLS, ALS skills, Triage, and Radio procedures were among the appropriate competencies. In the non-technical skills domain, students identified situational awareness, team working, decision making, leadership, and communication as important skills prior to a mass casualty incident (Figure 2). Challenges were faced primarily on non-technical skills; Communication (10%), Situational Awareness (21.25%), Team Working and Conflict resolution (23%), Psychological Issues and fatigue (14%), Coordination (8%), Simulation Issues (17%), and Low Preparation (7%) (Figure 3).
Knowledge Retention and Understanding of Disaster Management and Response.
Educational Effectiveness
Essential Competencies in Field Exercise.
Challenges During a Field Exercise.
Readiness
In case of a real MCI, 22% of participants declared themselves to be ready to respond prior to the MCI exercises. Upon completion of the course, the overall perception of readiness among participants was raised to 77%. Overall, participants reported feeling more prepared and confident for disaster response when compared to before the training (Table 3). The positive feeling and confidence seems to persist 10 months after the MCI exercises (65%) (Figure 4).
Perception of Readiness to Respond to Mass Casualty Incidents
Readiness to Respond a Real MCI.
Satisfaction
Overall, a vast majority of participants (80%) reported that the exercises met their expectations and evaluated it as excellent and above average, regarding the organization of the field exercise, and the knowledge and skills acquired. However, prior to the exercises, participants believed that advanced medical knowledge such as ALS and ATLS was necessary (70%). After participating in the course, first aid and triage were the only type of medical knowledge mentioned.
Eighty-one percent (81%) of trainees were willing to participate in future disaster management courses in order to maintain acquired knowledge and skills while 67% expressed their interest in becoming trainers (Table 4).
Individual Satisfaction and Willingness to Future Participation
Technical Skills
Trainees reported improvement of technical skills such as identification of critically injured patients (35%) and triage (37%).
Non-Technical Skills
Trainee feedback indicated enhancement of non-technical skills (87%) such as decision-making, communication, conflict resolution, teamwork, and coordination among stakeholders. Non-technical skills were assessed as more valuable, when compared with technical skills and special knowledge by the trainees (75%) (Figure 5). Although, before the exercises, trainees sustained that medical knowledge and skills are more important than administrative and non-technical skills, after participating in the course, a change in their mindset had been addressed, proving non-clinical competencies to be more important (66%). Furthermore, regarding the non-technical skills after the exercises, the participants indicated the ones mentioned before the course, and added leadership and collaboration, which had not mentioned previously. Finally, the percentage of the participants that didn’t know or didn’t express an opinion before the course, was reduced to 0%, indicating that participating in the course contributed to improved understanding (Table 5).
Perception of Skills Value for Optimal Field Exercise Performance.
Improvement of Technical and Nontechnical Skills
DISCUSSION
The most common challenges encountered and reasons for failure during disaster response are detected in missing or inaccurate information, time pressure, adverse circumstances, communication, teamwork, and coordination among professionals coming from different backgrounds and agencies. Reference Skryabina, Reedy, Amlôt, Jaye and Riley17 At the hand of the complex environment of a disaster, it must be ensured that professionals involved in disaster management are prepared to face the complexities of such events, and to make critical decisions in rapidly changing settings. In order to achieve this, they must acquire specific knowledge and technical skills as well as develop nonclinical competencies, which is difficult in daily practice. Reference Khorram-Manesh, Ashkenazi and Djalali13,Reference Ingrassia, Foletti and Djalali23 Nowadays, there is an increasing interest regarding the need for disaster preparedness among civilian health care professionals Reference Tan, Liao, Su, Li, Xiang and Dong3,Reference Khorram-Manesh, Ashkenazi and Djalali13,Reference Ford and Schmidt20,Reference Kolb21 although a review of post-event reports indicated inadequate training that results in repeated failing patterns. This calls for a new universally-accepted approach in disaster management training. Reference Khorram-Manesh, Lupesco and Friedl24
There are several teaching methods, from traditional learning, through lectures, to full-scale, real-time interdisciplinary exercises. A mixture of both theory and practice is nowadays suggested to be one of the best educational models in disaster management. Reference Khorram-Manesh, Ashkenazi and Djalali13 There have been several studies recognizing adult learning methodologies as a valuable tool for interactive training of health care professionals for disaster response. Reference Ingrassia, Foletti and Djalali23,Reference Alfred, Chilton and Connor25,Reference Liaw, Chen, Klainin, Brammer, O’Brien and Samarasekera26 Training among adult learners should incorporate andragogic principles according to literature. Adult learners are assumed to be self-directed with great accumulated experience, interested in integration of training with everyday life via problem centered approaches. Reference Kaufman27
Learning by doing, meta-schematizing real life experiences into an educational process facilitates inter-sectoral and inter-disciplinary collaboration, and enables a problem analysis and task prioritization while leading to decision making and action undertaking. Kolb’s Experiential Learning Theory (1984) defines experiential learning as “the process whereby knowledge is created through the transformation of experience. Knowledge results from the combination of grasping and transforming experience.” Reference Kolb21 There is evidence that experiential learning including simulations and role-playing games along with field exercises and drills can be highly engaging for students and can lead to better long-term memory. Experiential learning has the ability to lead to deeper understanding and develops the individual’s soft skills such as problem-solving, critical thinking, improved communication skill, and knowledge management. In particular, it enables learners to better manage highly complex situations that cross disciplinary boundaries and subject domains where the boundaries of knowledge are difficult to manage.
MCI exercises can be used as a research tool to study human decision-making behavior under crisis, representing a teaching instrument to familiarize the participants with disaster management issues such as teamwork, communication and coordination among all the stakeholders, and prepare for disaster management by revealing possible omissions and weaknesses in the existing operational plans. Reference Gardner, DeMoya and Tinkoff28-Reference Walker, Giddings and Armstrong31 Its advantages include offering learners an opportunity to develop know-how, not only know-what; developing skills and not just knowledge; helping in developing team spirit and communication skills under conditions closely matched to a real emergency event. Its disadvantages include requirements for careful and lengthy development and testing, a great deal of preparation and resources which can limit the number of learners and full engagement on behalf of tutors. Reference Khorram-Manesh, Ashkenazi and Djalali13 Overall, reported results are in line with existing literature, as multiple personal benefits from exercise participation have been reported with evidence of positive post-exercise change in participant knowledge, perceptions, and attitudes. Reference Skryabina, Reedy, Amlôt, Jaye and Riley17,Reference Coles, Dumas, Fielding and Watson32 Our self-reported data showcased increased confidence, an improved perception of preparedness and an improved understanding, not only from the respective trainee roles, but also from all stakeholders involved in pre-hospital care. There were reports of significant improvements in participant knowledge of emergency activities, policies and procedures, post-exercise. Significant improvements in competence and perceptions of network effectiveness, training, equipment adequacy, and teamwork were also reported. Our study goes even further as it provides initial evidence that in a medium term of 10 months to 1 year after the exercises, the positive impact of these benefits persists.
We consider therefore that blending theory and practice, practicing technical skills, and empowering nonclinical competencies through cross-sectoral and multidisciplinary training can be a remedy to the apparent disconnection between theory and methodology used in disaster preparedness training. The fact that the training took place in a multidisciplinary environment, in which trainees were deeply engaged in skills training by attempting to solve problems in an environment that was similar to the ones to be encountered in real life was of high educational value. Additionally, elements such as detailed briefing and debriefing, repetitive practices, and escalation of difficulty levels for the exercises, allow multiple learning strategies, provides a range of clinical scenarios, ensures a safe and educationally supportive learning environment, and allots both team and individualized learning, which have been defined as our outcomes and are scientifically validated. Reference Skryabina, Reedy, Amlôt, Jaye and Riley17
Evidence of the impact of the exercises on individuals is limited in the short term and continues to decrease with time. Reference Skryabina, Reedy and Amlôt19 Immediate post-exercise measures are flawed indicators of learning as they may only indicate temporary change rather than lasting learning. Reference Ford and Schmidt20 Taking all the above mentioned into account, we consider that our results are of significant value, improving mainly non-technical skills, persisting over time and potentially able to be translated into an improved emergency response.
CONCLUSION
This study points out immediate and medium term results showcasing positive impact regarding technical and mainly non-technical skills, and multiple personal benefits from trainee participation in the described set of interprofessional field exercises organized and implemented in the framework of the MSc Global Health-Disaster Medicine yearly, from 2016 to 2019. As such, the proposed set aforementioned can be considered for replication in other geographical areas, and for other types of disasters.
LIMITATIONS
Certain limitations should be taken under consideration in the interpretation of this study. Sample population was small and consisted mainly of health care providers, partially because it was conducted in the context of an MSc program of the Faculty of Medicine. Nevertheless, it makes generalization to other professionals cumbersome. It is crucial to expand disaster training to other actors involved in disaster management. The second limitation is that this study was performed over 4 years. Although analysis of the participants characteristics did not demonstrate significant differences, lack of synchronous, blinded, randomized controlled trials present the possibility of potential bias. Questionnaires reveal participant perceptions regarding the effectiveness of these methodologies. Although mass casualty drills provide a realistic real time evaluation among participants, lack of objective standardized assessment tools do not allow further generalization. The evaluation process was developed by the same group that developed the training.
Disclaimer
The opinions or assertions contained herein are those of the author/speaker, and are not to be construed as official or reflecting the views of the Department of Defense, the Uniformed Services University of the Health Sciences, or any other agency of the US Government.
