Background
Natural and man-made disasters have significant detrimental effects on human life and the environment. They result in fatalities, injuries, and disruptions to economic and social systems within the affected regions.Reference Mohammadi, Jafari and Etemadi 1 Historically, from 1900 to 2024, there have been 25,836 recorded disasters globally, with 69.41% attributed to natural disasters.Reference Cvetković, Renner, Aleksova and Lukić 2 One of the most important phases of disaster risk management is preparedness, and this phase entails the planning and implementation of actions prior to an emergency to ensure an effective response.Reference Oloruntoba, Sridharan and Davison 3
Health systems, encompassing both prehospital and hospital care play a crucial role in reducing mortality and morbidity during disasters.Reference Ginex, Dickman and Elia 4 , Reference Baack and Alfred 5 The health care system in any country can effectively manage during disasters if it successfully controls the situation from the outset and at the incident site. This necessitates efficient management of the incident during the prehospital phase.Reference Beyrami Jam, Aminizadeh and Akbari-Shahrestanaki 6 The more proficient and well-trained the prehospital system is, the more effective it will be in minimizing casualties and fatalities in various disaster scenarios. There is a global consensus regarding the importance of prehospital system coordination, efficiency, and preparedness in disaster response. Therefore, the preparedness of prehospital system is crucial in mitigating the impact of disasters.Reference Beyramijam, Farrokhi and Ebadi 7
One way to reduce damage caused by disasters is to improve the preparedness of health centers for a correct and timely response. On the other hand, the most important way to create, maintain, and increase preparedness is to design, perform, and evaluate disaster exercises.Reference Zhuge, Ruzieva and Chang 8 Running exercise courses in different sectors of health system is one of the important steps to prepare and deploy disaster risk management programs, especially in the response phase.Reference Reddin, Bang and Miles 9 , Reference Alshowair, Bail and AlSuwailem 10 Exercises simulate the realistic conditions so that people improve their mental and physical skills in situations similar to real conditions and provide an appropriate response based on existing programs to emergencies and disasters.Reference Elendu, Amaechi and Okatta 11 Disaster exercises can be used for testing and validating policies, programs, procedures, teaching personnel, their roles and responsibilities, as well as improving individual performance and improving interorganizational communication and coordination.Reference Andreassen, Elvegård, Villanger and Johnsen 12 , Reference Abualenain, Alhajaji and Alsulimani 13
To effectively execute an exercise, a series of methodical steps must be undertaken. All exercises should begin with the identification and analysis of the exercise need to determine the aims and objectives. This could be to test, evaluate, assess, practice, or demonstrate aspects of policy, plans, procedures, systems, or training. Given the aim and objectives, the scope, style, and participants are determined.Reference Hazard 14 A range of pre-exercise activities will need to occur before an exercise begins. These activities are required to prepare agencies and individuals for their participation in the exercise. These activities may take the form of a final review of exercise documentation, final technology and communication requirements and checks, establishing facilities, equipment familiarization, discussion exercise(s), seminar(s), workshops, and development or review of plans and procedures.Reference Hazard 14 , Reference van Westen and Damen 15 Exercise evaluation is a crucial element in the design and execution of exercises, as it systematically assesses the extent to which the exercise has achieved its intended objectives.Reference Abualenain, Alhajaji and Alsulimani 13 Furthermore, it identifies the strengths and weaknesses of the disaster exercise program. Following the exercise, evaluators should convene with all key participants and representatives from the involved units to discuss the exercise in detail.Reference Sheikhbardsiri, Nekoei-Moghadam and Yarmohammadian 16 The purpose of this session is to gather insights regarding individual performance and the application of information, which will aid in the revision and enhancement of the preparedness program and response processes.Reference Legemaate, Burkle and Bierens 17
Studies conducted on assessing the functional preparedness of hospitals and prehospital services during disasters through exercises suggest that the selection of key indicators for disaster exercises should be considered during design phase of the exercise. Additionally, the selection of indicators should be based on the expected effectiveness and efficiency of each different section separately. Furthermore, in the development, conduct, and evaluation of disaster exercises, the satisfaction indicators for each expected performance should be determined.Reference Khankeh, Lotfolahbeygi and Dalvandi 18 -Reference Heidaranlu, Ebadi and Khankeh 20 Careful planning and management are crucial for exercises; even minor exercises necessitate a coordinated strategy encompassing design, execution, and evaluation. To ensure success, it is essential to establish a planning team and define quality indicators for the functions requiring management.Reference Grunnan and Fridheim 21 , Reference Russo and Rindone 22
Quality indicators such as those endorsed by professional health provider organizations, institutions, accrediting bodies, and government agencies like the Agency for Healthcare Research and Quality (AHRQ) represent a key type of quality measure.Reference Bobrovitz, Parrilla and Santana 23 , Reference Stelfox, Bobranska-Artiuch and Nathens 24 Quality indicators are a key component of health care quality measurement, designed to assess the extent to which health services enhance the probability of favorable health outcomes and align with contemporary professional standards. These indicators offer benefits beyond documenting care quality; they facilitate quality and performance benchmarking, inform priorities for improvement initiatives, and promote accountability and transparency in health care.Reference Howard, Cameron and Wallis 25 , Reference Mainz 26
To identify the factors affecting the development, conduct, and evaluation of prehospital disaster exercises. Our analysis was designed to gain further information on existing quality indicators for creating standardized, evidence-based measures of prehospital disaster exercises. The findings of this study are expected to aid planners of disaster exercises in prehospital system for design and execution of standardized exercises, thereby improving the competencies of prehospital response teams. The indicators highlighted in this study are anticipated to enhance the effectiveness of prehospital exercises, improve team coordination, reduce performance errors, and elevate the quality of prehospital emergency care during emergencies and disasters.
Method
This study was conducted by a systematic review with the Prospero code CRD42024548450. This study was performed based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, and thematic content analysis was used for data analysis.Reference Moher, Shamseer and Clarke 27
Search Strategy
This study was conducted in May 2024 to review all published English articles in the field of disaster exercises in prehospital system for identifying and conceptualizing quality indicators influencing the structure, conduct, and outcome of disaster exercises. For this purpose, it has been studied databases including ISI Web of Science, PubMed, Scopus, and Google Scholar from January 1, 2000 to May 2024. The search keywords are “Exercise,” “Simulation,” “Drill,” “Functional Exercise,” “Full-Scale Exercise,” “Discussion Exercise,” “Table-Top Exercise,” “Operational Exercise,” “Workshop,” “Seminar,” “Hot Wash,” “Cold Wash,” “Disaster,” “Emergency,” “Crisis,” “Prehospital,” “Emergency Medical System,” “EMS,” “Prehospital System,” “Evaluation.” We used the “OR” and “AND” operators to develop the search strategy. Finally, we created a specific research strategy for each primary database.
Article Selection Process
Independent reviewers (HF and AA) systematically screened abstracts and titles for eligibility. If the reviewers determined that an abstract or title warranted further consideration, full-text copies of the corresponding articles were obtained and assessed for eligibility by both reviewers. In instances where discrepancies arose between the reviewers, the reasons for these differences were identified, and a conclusive decision was reached with the input of a third reviewer (MN).
Inclusion and Exclusion Criteria
The inclusion criteria consisted of published papers from the year 2000 to May 2024, written in English, and published in indexed, peer-reviewed research journals, specifically focused on prehospital system disaster exercises. We excluded review articles, case reports, case series, theses, editorials, opinions, and correspondence. Additionally, studies that did not provide essential information related to the design, implementation, and evaluation of prehospital system disaster exercises were also excluded.
Database Search
The initial electronic database search of the literature resulted in a total of 3083 documents. At the next step, duplicates, books, dissertations, and presentations were filtered, and the number of documents decreased to 2813 articles. In the first stage of the systematic screening process, titles and abstracts were reviewed to identify studies providing relevant information on the design, implementation, and evaluation of prehospital disaster exercises. This process yielded 23 eligible articles. In the second stage, the full texts of these 23 articles were assessed, and ultimately, 10 studies that reported quality-related indicators for prehospital disaster exercises were included in the final analysis. Figure 1 shows the search strategy and the selected articles in accordance with the PRISMA guidelines.
Flow diagram showing selection of articles reviewed, in accordance with the PRISMA guidelines.
Study Quality Assessment
The quality of the final extracted articles was evaluated using the CASP tool. 28 The scores of quantitative studies ranged from 5 to 11. Cross-sectional studies achieved a satisfactory quality score. Only one study did not specify the sample size. None of the interventional studies were randomly assigned, and the participants weren’t “blinded” to assess/analyze outcomes. Failure to adhere to proper randomization and blinding may introduce bias into the results and reduce their generalizability. The quality of the final extracted articles is shown in Table 1.
The quality of the final extracted articles using Critical Appraisal Skills Program (CASP) 28
Data Analysis
This research employed thematic analysis to analyze the data. This qualitative research approach combines and interprets various findings, focusing on the study subject. Two researchers examined all influence quality indicators extracted from the articles. Duplicate quality indicators were eliminated, and the remaining principal quality indicators were thoroughly studied and evaluated multiple times. Similar quality indicators were grouped into subcategories based on thematic similarity. In the next stage, after several rounds of study and revision, these subcategories were further grouped to establish the main categories.
Ethics Approval and Consent to Participate
The Ethics Committee of Kerman University of Medical Sciences approved this study. The Ethics approval Code is IR.KMU.REC.1403.522. All methods were performed following the relevant guidelines and regulations; this article does not contain any studies with animals. All methods were performed in accordance with the relevant guidelines and regulations; this article does not contain any studies with animals performed by any of the authors.
Results
Demographic Finding
After finalizing the articles and submitting them to research databases, the extracted data was compiled into a Microsoft Word file. This file included the journal name, article title, year of publication, author name, country, research method, and study results for analysis.
This review includes articles from Thailand,Reference Lee, Lee and Kim 29 Sweden,Reference Hagiwara, Kängström and Jonsson 30 , Reference Lennquist Montán, Hreckovski and Dobson 31 the United States,Reference Burns, Robinson and Lowe 32 -Reference Lee, McLeod and Van Aarsen 34 Taiwan,Reference Lin, Tay and Tsai 35 Germany,Reference Pietsch, Knapp and Ney 36 the Netherlands,Reference Bijl, Vianen and Van Lieshout 37 and Italy.Reference Carenzo, Gamberini and Crimaldi 38 Characteristics and finding focus of articles included in the systematic review are presented in Table 2.
Characteristics and finding focus of articles included in the systematic review
Thematic Analysis
Based on the studies included in this systematic review, the quality indicators of prehospital system exercises in emergencies and disasters were divided into 3 themes, 8 categories, and 21 subcategories. The primary themes and related main categories included: Exercise structure QIs (knowledge promotion and cognitive skills, supply of exercise hardware and software requirements and resources desirable management), Exercise conduct QIs (practical proficiency in essential skills and decision-making capacity), and Exercise outcome QIs (evaluation and reporting of exercise, promotion of managerial capabilities and competencies, and development of psychological capabilities). Summarizes the results of the content analysis shown in Figure 2.
Influence QIs for prehospital system disaster exercises (structure, conduct, and outcome), extracted from thematic analysis.
Theme 1: Exercise structure QIs
The first theme emerging from thematic analysis regarded studies published on prehospital disaster exercises. Three main categories are included in this theme.
Main category 1: Knowledge and cognitive skills
Enhancing knowledge and cognitive skills necessitates a sustained focus on delivering continuous discussion and operation-based exercise programs over time. Operational skills encompass the technical, procedural, and implementation capabilities crucial for executing specific tasks. Cognitive skills, on the other hand, pertain to mental abilities such as problem-solving, critical thinking, decision-making, analysis, and information processing. The objective of improving knowledge and cognitive skills is to elevate individual and group competencies through education, practice, and innovative technologies, thereby maximizing performance during incidents and disasters. Three subcategories are included in this main category: Choosing suitable style of exercise; short-term and long-term training of exercise concepts; and creative teaching of exercise evaluation methods.
Subcategory 1.1: Choosing suitable style of exercise. Studies showed that aims and objectives of the exercise will guide the selection of the most suitable style or styles. The exercise management team is not limited to a single style; the development of progressive exercise programs or the incorporation of multiple styles is often beneficial.Reference Lee, McLeod and Van Aarsen 34 , Reference Lin, Tay and Tsai 35 Functional exercises can be specifically designed to test higher decision-making levels without deploying tactical-level units; therefore this style of exercise can be cost-effective.Reference Burns, Robinson and Lowe 32 Cost-effectiveness, resource limitations, and the heterogeneity of medical services available in the prehospital emergency setting are factors known to influence exercise style preferences.Reference Yu and Coffey 33 , Reference Lin, Tay and Tsai 35
Subcategory 1.2: Short-term and long-term training of exercise concepts. Studies indicate that effective training requires adequate time, and it is essential to allow sufficient time for familiarization with the simulation system.Reference Lennquist Montán, Hreckovski and Dobson 31 While short-term training courses are beneficial for enhancing initial preparedness in critical situations, their impact is transient without ongoing practice. Short-term training demonstrably improves functional skills and reduces functional errors.Reference Hagiwara, Kängström and Jonsson 30 , Reference Lee, McLeod and Van Aarsen 34 Participants in long-term training courses scored higher on quantitative assessments of speed and performance. They performed better in prehospital emergencies.Reference Bijl, Vianen and Van Lieshout 37 Training has been shown to enhance self-assessment and improve safety and resource management skills.Reference Hagiwara, Kängström and Jonsson 30 Continuous training and repeated simulations are critical to sustaining and advancing skills needed in emergencies, especially for Emergency Medical System (EMS) nurses who may encounter these situations infrequently.Reference Bijl, Vianen and Van Lieshout 37
Subcategory 1.3: Creative teaching of exercise evaluation methods. Using hypothetical casualty scenarios of varying severity helps uncover critical limitations.Reference Lennquist Montán, Hreckovski and Dobson 31 Modern training methods, such as interactive scenarios, effectively enhance personnel skills, improve coordination, and strengthen decision-making in disaster response. Integrating advanced technologies, which provide enhanced data, significantly contributes to better decision-making processes. Developing cutting-edge tools is vital for the efficient management of large-scale incidents. Leveraging modern technologies for precise training in clinical diagnosis and crafting simulation scenarios to address complex, multidimensional conditions enhance rescuers’ capabilities.Reference Lennquist Montán, Hreckovski and Dobson 31 Advanced tools bridge the gap between additional data and its interpretation by responders. Training with these tools ensures optimal resource management, reduces over-triage, boosts the efficiency of emergency systems, and minimizes human errors.Reference Hagiwara, Kängström and Jonsson 30 , Reference Lennquist Montán, Hreckovski and Dobson 31
Main category 2: Supply of exercise hardware and software requirements
As an effective educational method for preparing medical teams, simulation requires technical infrastructure and advanced equipment. Advanced hardware equipment facilitates the collection of accurate data on learners’ performance, aiding in the better assessment of clinical skills. The appropriate combination of hardware and software technologies allows for the simulation of actual conditions, enabling precise performance evaluation, improvement of operational skills, and reduction of medical errors. Two subcategories are included in this main category: simulation models and technologies and utilizing advanced medical education tools and equipment’s.
Subcategory 2.1: Simulation models and technologies. Analysis of existing data shows that using Dynamic Clinical Cards (DCCs), smart sensors, and virtual reality (VR) models significantly improves the preparedness of rescue and medical teams.Reference Pietsch, Knapp and Ney 36 , Reference Carenzo, Gamberini and Crimaldi 38 Designing a standard simulation environment requires the correct combination of hardware and software technologies. Based on the obtained data, simulation technologies and new simulation models, simulating incident conditions, and using modern medical training tools for evaluating exercises are noteworthy aspects of training.Reference Lennquist Montán, Hreckovski and Dobson 31 Creating accurate assessment tools and advanced training models can enhance readiness for real-life scenarios.Reference Burns, Robinson and Lowe 32 , Reference Lin, Tay and Tsai 35 Incorporating simulated emergency conditions alongside modern educational techniques significantly boosts operational skills while minimizing medical errors.Reference Lin, Tay and Tsai 35 Utilizing technologies such as virtual reality and biometric sensors, online training platforms can streamline the learning process and elevate the quality of emergency responses.Reference Lin, Tay and Tsai 35 , Reference Carenzo, Gamberini and Crimaldi 38 Simulation equipment and methods, simulated training environments, mobile hospital use, and training scenarios increase accuracy and reduce diagnostic errors.Reference Lee, Lee and Kim 29 , Reference Pietsch, Knapp and Ney 36 The need to improve communication hardware and software facilities for better response is one of the issues that should be considered in training.Reference Burns, Robinson and Lowe 32
Subcategory 2.2: Utilizing advanced medical education tools and equipment. Successful simulation exercises require advanced physical equipment such as DCCs, biosensor devices, advanced medical models and interactive displays, helicopter mock-ups, and realistic scenarios that recreate incident conditions and accurately assess personnel performance.Reference Hagiwara, Kängström and Jonsson 30 , Reference Pietsch, Knapp and Ney 36 , Reference Carenzo, Gamberini and Crimaldi 38 Using digital tools for performance assessment has resulted in reduced performance errors compared to traditional teams.Reference Bijl, Vianen and Van Lieshout 37 Simulations also allow more effective learning and increase operational and non-technical skills.Reference Pietsch, Knapp and Ney 36
Main category 3: Resources desirable management
The provision of medical resources and equipment, logistics, and the proper selection of human resources, followed by the prioritization and appropriate allocation of resources during exercises, are important indicators that must be considered to achieve educational and operational goals during the exercise. Furthermore, the capacities of Collaborating organizations should not be overlooked when enhancing the quality of the drills. Two subcategories are included in this main category: prioritization and optimization of exercise resources and selection of personnel based on exercise type and objectives.
Subcategory 3.1: Prioritization and optimization of exercise resources. Research indicates the importance of prehospital systems allocating a specific budget line within their annual budget for disaster preparedness exercises.Reference Lennquist Montán, Hreckovski and Dobson 31 Resource deficiencies, communication equipment deficiencies, incomplete guidelines, and weaknesses in infrastructure management during exercises are significant barriers to preparedness in real-world disaster scenarios.Reference Lee, Lee and Kim 29 Prioritizing and securing resources for exercise performance enhances the exercise’s quality and the team’s efficiency during the drill. Providing educational and medical resources and equipment and aligning resource provision with medical needs commensurate with the severity of the incident ensures resource compatibility with disaster conditions. It improves resource management in emergencies.Reference Lin, Tay and Tsai 35 Utilizing the capacities of other collaborating organizations can optimize resource allocation.Reference Lee, McLeod and Van Aarsen 34
Subcategory 3.2: Selection of personnel based on exercise type and objectives. Selecting personnel based on the specific type and objectives of rescue and simulation exercises greatly influences the quality of execution and the learning outcomes for participants. To enhance the accuracy and efficiency of team performance, it is crucial to involve experienced and specialized individuals in facilitating the exercise. Ensuring participants’ skill and knowledge levels align with the intended exercise level is vital for effective exercise design.Reference Carenzo, Gamberini and Crimaldi 38 Selecting personnel familiar with specialized protocols and emergency scenarios is essential for enhancing exercise effectiveness.Reference Bijl, Vianen and Van Lieshout 37 Personnel who assume the role of simulated casualties must undergo standardized exercise to ensure training scenarios are as close to reality as possible.Reference Carenzo, Gamberini and Crimaldi 38
Theme 2: Exercise conduct QIs
The second theme emerging from thematic analysis regarded studies published on prehospital disaster exercises. Two main categories are included in this theme: practical proficiency in essential skills and decision-making capacity.
Main category 4: Practical proficiency in essential skills
Active engagement in field exercises and simulations enhances problem-solving capabilities, promotes effective team collaboration, and facilitates learning through experiential scenarios. Addressing unforeseen limitations and challenges, alongside technical operational considerations, contributes to improved readiness, enhanced decision-making speed and resilience under pressure, and optimized communication strategies. Critically evaluating exercises to identify weaknesses, refine protocols, and anticipate future requirements is essential for bolstering safety, optimizing performance, and elevating the caliber of response in real-world scenarios. Three subcategories are included in this main category: learning through practice, teamwork, and problem solving.
Subcategory 4.1: Learning through practice. Simulation serves as a valuable training modality for disaster response, provided it is meticulously designed to encompass all aspects of the response, including interdepartmental coordination and resource limitations.Reference Lennquist Montán, Hreckovski and Dobson 31 Participants reported that operational exercises significantly enhanced their understanding of the real-world challenges encountered during crises and improved decision-making in critical scenarios. Replicating authentic situations through operational training fosters increased preparedness for responding to critical emergencies.Reference Hagiwara, Kängström and Jonsson 30 EMS nurses who participated in operational exercises demonstrated superior skills and faster emergency response readiness than their counterparts.Reference Bijl, Vianen and Van Lieshout 37 Experiential learning, through exposure to real-life situations and unexpected contingencies, surpasses the effectiveness of purely theoretical instruction.Reference Hagiwara, Kängström and Jonsson 30
Subcategory 4.2: Teamwork. Coordination and teamwork lead to improved technical and non-technical skills.Reference Lennquist Montán, Hreckovski and Dobson 31 , Reference Pietsch, Knapp and Ney 36 By optimizing communication problems, it is possible to eliminate the poor coordination between teams and improve team activities.Reference Burns, Robinson and Lowe 32 Inadequate preparedness in relief teams leads to delays in appropriate response.Reference Lee, Lee and Kim 29 Findings show that the inadequate preparedness of relief and rescue teams in response to natural disasters leads to weakness in understanding the incident command system and coordinating teams for effective response.Reference Lee, Lee and Kim 29
Subcategory 4.3: Problem solving. Exercise is a way to identify and address deficiencies.Reference Burns, Robinson and Lowe 32 Potential errors are identified and corrected by simulating hazardous conditions, directly increasing patient safety in disasters.Reference Hagiwara, Kängström and Jonsson 30 By identifying deficiencies and Performance problems in incident response, teams can be improved. During exercises, gaps can be identified, and protocols can be modified to improve response in critical real-life situations. Exercises are valuable not only for identifying current problems but also for anticipating and managing potential future activities. Exercises are an important tool for increasing preparedness and improving emergency performance.Reference Burns, Robinson and Lowe 32
Main category 5: Decision-making capacity
Effective decision-making at various levels of disaster response is a fundamental requirement of crisis management that requires experience, expertise, and communication skills. Holding exercises improve teamwork, decision-making, and problem-solving. As a result, individuals gain mutual understanding and deeper insight into roles and responsibilities, increasing the speed and accuracy of their decision-making in emergencies and improving overall system performance in responding to emergencies. Three subcategories are included in this main category: better understanding, being cognitively able to make a choice, and improvement of abilities implied in decision.
Subcategory 5.1: Better understanding. Understanding refers to the capacity to understand what exercise planners are being told and what is being proposed to them to exercise players, therefore including understanding information related to the disaster exercise.Reference Lee, Lee and Kim 29 , Reference Burns, Robinson and Lowe 32 , Reference Lin, Tay and Tsai 35 In the context of training programs, understanding encompasses individuals’ comprehension of task descriptions, the disaster command hierarchy, and the overarching training goals. Studies indicate that training initiatives can enhance participants’ understanding of their operational responsibilities, which, in turn, leads to improved team performance in practical scenarios.Reference Lee, Lee and Kim 29 , Reference Burns, Robinson and Lowe 32 , Reference Lin, Tay and Tsai 35 .
Subcategory 5.2: Being cognitively able to make a choice. Simulation allows one to reflect on learning and improve decision-making.Reference Hagiwara, Kängström and Jonsson 30 Exercises that require rapid decision-making under uncertain conditions improve participants’ decision-making skills.Reference Hagiwara, Kängström and Jonsson 30 , Reference Lennquist Montán, Hreckovski and Dobson 31 , Reference Pietsch, Knapp and Ney 36 Individuals with more experience in rapid decision-making are less likely to delay responding to a crisis.Reference Lee, McLeod and Van Aarsen 34 , Reference Carenzo, Gamberini and Crimaldi 38 Exercises lead to improved awareness of the critical situation, favorable patient assessment, and accurate and rapid decision-making.Reference Hagiwara, Kängström and Jonsson 30 One of the main challenges during the crisis is the lack of effective communication among patients and the various teams responding to it. This issue is especially pronounced in international settings where language barriers exist. Poor communication can result in delays or errors in decision-making.Reference Lee, Lee and Kim 29 On the other hand, effective communication among relief teams also reduces decision-making time and increases efficiency.Reference Lennquist Montán, Hreckovski and Dobson 31 Exercises can increase managers’ understanding and insight into their tasks and roles.Reference Lee, Lee and Kim 29
Subcategory 5.3: Abilities implied in decision. Attending drills can lead to improved cognitive processing and social coordination, resulting in increased accuracy and speed of decision-making in critical situations. Simulated exercises improve implicit decision-making abilities, such as rapid decision-making during triage.Reference Yu and Coffey 33 Attendance and participation in drills will help prehospital staff identify their potential abilities and apply them in real-world events.Reference Lin, Tay and Tsai 35
Theme 3: Exercise outcome QIs
The third theme emerging from thematic analysis regarded studies published on prehospital disaster exercises. Three main are included in this theme: evaluation and reporting of exercise, promotion of managerial capabilities and competencies, and development of psychological capabilities.
Main category 6: Evaluation and reporting of exercise
The purpose of exercise evaluation is to compare and examine the behaviors and performances that took place in the exercise, its consequences, and the proper performance of the teams. All activities and how the exercise was performed are recorded in the exercise evaluation. While comparing them with the desired performance, the strengths and weaknesses of the exercise program are identified. Three subcategories are included in this main category: Post-action reporting process analysis, Immediate and delayed reporting or debriefing, and Accuracy analysis in immediate feedback implementation.
Subcategory 6.1: Post-action reporting process analysis. Evaluation and standardization of assessment tools, including checklists, increase the accuracy of evaluations and enable team performance improvement.Reference Burns, Robinson and Lowe 32 The evaluation of exercise outcomes and results must be based on defined criteria.Reference Lennquist Montán, Hreckovski and Dobson 31 Providing post-exercise feedback using exercise data is essential for evaluating the exercise itself.Reference Carenzo, Gamberini and Crimaldi 38 Assessing accuracy, identifying error patterns, and measuring task duration with standardized templates are vital.Reference Lee, McLeod and Van Aarsen 34 Analysis and reflection play a key role in all simulation exercises, providing opportunities for feedback and enhancing performance.
Subcategory 6.2: Immediate and delayed reporting or debriefing. Performance improvement and the exact identification of problems are possible through post-exercise debriefing.Reference Lee, Lee and Kim 29 Analyzing the exercise after its completion helps to correct its weaknesses. Debriefing and post-simulation feedback bridge the gap between theory and practice, solidifying knowledge. In addition to consolidating learning, post-simulation debriefing sessions improve participant performance in future exercises.Reference Hagiwara, Kängström and Jonsson 30
Subcategory 6.3: Accuracy analysis in immediate feedback implementation. Statistical analysis to evaluate the accuracy and time taken for specialized functions reduces errors during disaster response operations. The presence of experienced personnel enhances precision in disaster simulations.Reference Carenzo, Gamberini and Crimaldi 38 By analyzing and examining performance differences between seasoned and novice responders and providing targeted feedback, the operational skills of these individuals can be improved.Reference Carenzo, Gamberini and Crimaldi 38 Discrepancies in accuracy and identified errors are attributed to the inexperience of these individuals.Reference Lee, McLeod and Van Aarsen 34 Potential errors are identified by evaluating and analyzing accuracy and errors during disaster preparedness exercises, and the opportunity to correct them is provided.Reference Hagiwara, Kängström and Jonsson 30
Main category 7: Promotion of managerial capabilities and competencies
Developing and managing professional skills effectively improves teams’ readiness to participate in the exercise. Simulation exercises provide an opportunity to improve communication skills and professional abilities. Therefore, adopting appropriate policies and strategies to maintain and sustain the effect of the exercise is essential to ensure readiness in responding to crises. Three subcategories are included in this main category: professional behavior of participants, improving clinical competency and core skills, and developing new policies for role improvement.
Subcategory 7.1: Professional behavior of participants. Personnel flexibility and adaptability to unexpected situations enhance their capabilities in the response phase.Reference Burns, Robinson and Lowe 32 Individuals who better understand their roles and responsibilities during exercise perform better in emergencies.Reference Lee, Lee and Kim 29 The professional experience of training participants leads to their proper utilization of available resources.Reference Lin, Tay and Tsai 35
Subcategory 7.2: Improving clinical competency and core skills. Attention to resource management and resource optimization during exercises leads to improved specialized clinical functions. Exercises provide an opportunity to improve clinical diagnostic skills and enhance the professional capabilities of participants, which leads to increased efficiency of emergency systems.Reference Hagiwara, Kängström and Jonsson 30 , Reference Carenzo, Gamberini and Crimaldi 38 Identifying weaknesses during exercises, communication skills with patients and other collaborating organizations, and coordination between different teams can be improved.Reference Lennquist Montán, Hreckovski and Dobson 31 , Reference Burns, Robinson and Lowe 32
Subcategory 7.3: Developing new policies for role improvement. Training and preparing evaluators lead to increased quality of exercise evaluations.Reference Burns, Robinson and Lowe 32 Continuous training is an effective strategy that strengthens individuals’ working memory in emergencies.Reference Pietsch, Knapp and Ney 36 Developing functional exercises and exposure to real-life scenarios is also recommended as a strategy to enhance preparedness for disaster response.Reference Hagiwara, Kängström and Jonsson 30
Main category 8: Development of psychological capabilities
Simulation exercises play a significant role in strengthening the psychological skills of relief teams. Exposure to complex and stressful situations helps them gain more self-confidence and self-efficacy. Mental preparedness in relief teams allows them to react in real situations with more excellent psychological stability and optimal performance. Two subcategories are included in this main category: enhancing adaptability to realistic and challenging environments, strengthening self-perception of competence.
Subcategory 8.1: Enhancing adaptability to realistic and challenging environments. Structured exercises and simulation-based training significantly boost self-confidence and enhance the performance skills of EMS nurses.Reference Bijl, Vianen and Van Lieshout 37 Exposure to complex and stressful scenarios can sharpen rescuers’ readiness for real-world crises. Challenging drills mitigate stress and anxiety among staff, ultimately leading to improved performance.Reference Hagiwara, Kängström and Jonsson 30 , Reference Bijl, Vianen and Van Lieshout 37
Subcategory 8.2: Strengthening self-perception of competence. Engaging in accurate and realistic simulated scenarios significantly bolsters individuals’ confidence when facing real-world situations.Reference Hagiwara, Kängström and Jonsson 30 , Reference Yu and Coffey 33 Increased confidence within medical teams profoundly impacts the management and leadership of mass casualty incidents.Reference Yu and Coffey 33 Simulation enhances individuals’ self-efficacy and improves decision-making skills in complex situations (30).
Discussion
The main goal of this study was to identify and conceptualize QIs influencing prehospital disaster exercises across structure, conduct, and outcome. Based on the findings of this study, knowledge promotion and cognitive skills, supply of exercise hardware and software requirements, desirable management resources, practical proficiency in essential skills, decision-making capacity, evaluation and reporting of exercise, promotion of managerial capabilities and competencies, and development of psychological capabilities are the 8 main influences on QI for prehospital system disaster exercises.
Knowledge Promotion and Cognitive Skills
Adopting comprehensive and continuous training strategies to improve mental and functional skills of rescue and medical personnel is essential. Combining theoretical and operational training methods and using modern technologies will significantly improve the capabilities of this group of employees. Attention to the design and implementation of standard exercise programs is particularly important to ensure rescue teams’ readiness in emergencies. The study (Kou & Niu, 2024) showed that in harsh environments and unpredictable conditions, in addition to having theoretical knowledge and operational skills for independent learning nurses, decision-making, situation assessment, and resource management skills are particularly important. Relief teams face situations that require rapid decision-making, flexibility, and effective teamwork. Individual characteristics, commitment, and teamwork are key factors for success in emergencies. Strengthening training programs, simulating realistic scenarios, and promoting interdepartmental coordination are essential to increasing the efficiency of nursing teams.Reference Deng, Kou and Ma 39 VR technology significantly affects the speed and accuracy of decision-making in emergencies by increasing trainees’ confidence and skills during training, reducing operational errors, and improving the effectiveness of training programs.Reference Luo, Zhu and Pang 40 , Reference Pietschmann and Pickhardt 41 Prehospital emergency personnel are forced to make decisions under stressful conditions due to complex and unfamiliar situations and lack of access to medical support and advanced diagnostic tools at critical moments. Providing access to diagnostic technologies, developing training programs for infrequent but critical scenarios, and using telemedicine and remote support can help improve their cognitive skills.Reference Bai, Zhang and Xu 42
Continuous training programs, including simulations and operational exercises, and structured training for prehospital emergency teams increase their decision-making, communication, and incident management skills, improving their understanding and performance in emergency management.Reference Daniel, Gist and Grock 43 -Reference Caviglia, Da Silva-Moniz and Venturini 45 Executing exercises leads to improved knowledge and skills needed in emergencies, which are enhanced if the exercises are repeated and continued. Greater focus on emergency training and participation in simulated exercises improves personnel readiness to face crises.Reference Zhuge, Ruzieva, Chang, Wang, Qi, Wang, Wang, Kang, Liu and Wu 46
Supply of Exercise Hardware and Software Requirements
Using modern technologies in simulating critical conditions increases the accuracy in evaluating the performance of prehospital emergency teams. Advanced hardware equipment, such as smart sensors, virtual reality simulators, the development of internet-based simulation software, and their combination with biometric sensors, play a significant role in reconstructing real operational environments and can help improve clinical skills, reduce medical errors, and enhance training effectiveness.Reference Luo, Zhu and Pang 40 , Reference Davidson, Waxenegger, Mohamed, McConnell and Sanderson 47 , Reference Purohit, Dubrow and Bannan 48 Using artificial intelligence to simulate exercise emergency conditions can improve training and enhance prehospital emergency operational capabilities.Reference Toy, Warren and Wilhelm 49 , Reference Chee, Chee, Huang, Mazzochi, Taylor and Wang 50 Exercises based on realistic simulations are proposed to increase diagnostic capabilities and clinical decision-making and improve the efficiency of emergency systems.Reference Bozorgnejad, Najafi and Haghani Sh 51
Resources Desirable Management
Financial resources are a critical factor in conducting an emergency exercise in an organized and efficient manner. Inadequate equipment and facilities and a lack of adequate financial support prevent the exercise from being carried out as planned. The exercise budget covers the costs of materials, site rental, and personnel training required for the exercise process. Budget constraints, lack of coordination across departments, and inadequate leadership attention negatively impact response capacity and effectiveness.Reference Zhuge, Ruzieva, Chang, Wang, Qi, Wang, Wang, Kang, Liu and Wu 46 The availability of adequate equipment and facilities during exercise enhances participants’ performance throughout the exercise and improves functional and specialized skills in emergencies. Payment of subsidies to personnel involved in exercise can motivate these individuals.Reference Zhuge, Ruzieva and Chang 8 Given the exercise’s goal, various resources are needed for implementation, such as personal protective equipment, medical treatment facilities and equipment, appropriate medical waste disposal methods, vehicles, and rescue and rehabilitation sites. However, many countries lack the resources to implement exercises.Reference Zhuge, Ruzieva and Chang 8 The presence of training scenarios in exercise design, as one of the exercise resources, directly impacts improving the process and flow of the exercise. Therefore, exercise scenarios should be designed to be as realistic and diverse as possible to ensure the quality of simulating emergency conditions.Reference Yuan, Ding and Wang 52 Controllers, facilitators, actors, role-players, and exercise evaluators, as human resources for the exercise, must be identified before the exercise and receive the necessary training to participate. The presence of experienced individuals with training skills and the ability to manage multiple aspects of the exercise simultaneously and who are familiar with techniques such as role-swapping between doctors and paramedics can help to understand needs better and collaborate effectively during the exercise.Reference Tariq, Sood and Goodsman 53 , Reference Duhamela, Broheza, Delvosallea, Van Daele and Vandestrate 54 Countries must make the necessary provisions to improve the quantity and quality of emergency reserves and provide adequate facilities and equipment for emergency drills to enhance personnel performance.
Practical Proficiency in Essential Skills
Conducting prehospital exercises leads to the enhancement of essential and technical skills of operational teams. Improving rapid decision-making skills, communication, and teamwork is achieved through consistent exercises.Reference Koçkaya, Üçkuyu, Keysan, Gurcuoglu, Şener and Gülbaş 55 Exercises provide an opportunity to simulate prehospital emergency encounters with complex and high-pressure situations to improve the mental readiness of personnel, alongside the enhancement of technical skills. Since mental readiness and stress management are not sufficiently addressed in training, prehospital emergency teams lack the necessary skills when faced with unexpected situations.Reference Hugelius, Edelbring and Blomberg 56 Exercises improve active participation and problem-solving on the spot. Participating in simulated prehospital emergency exercises improves the practical understanding of events and increases decision-making ability in high-pressure and stressful situations.Reference Koçkaya, Üçkuyu, Keysan, Gurcuoglu, Şener and Gülbaş 55 , Reference Sung and Hsu 57 Furthermore, simulated exercises can improve coordination and inter-team interactions and reduce errors when dealing with real-world situations.Reference Ingrassia, Ragazzoni and Carenzo 58 Emergency medical personnel are weak in decision-making and critical thinking skills, and considering the importance of this issue in operations, these topics should be considered in training.Reference Heidari and Shahbazi 59 It seems that attention to teaching critical thinking and decision-making, and consistency in conducting exercises to strengthen individual and group performance, should be considered in the preparedness program. The use of blended training methods including technical training, realistic simulation, and training in stress management, problem solving, and team leadership is recommended.
Decision-Making Capacity
Situational awareness and decision-making skills are essential for effective performance in EMS that operate in high-risk and unpredictable environments. Upon arrival at the scene, EMS teams must quickly gather information, interpret information, and anticipate future situations to make rapid decisions.Reference Sedlár 60 Decision-making is a process that includes identifying the problem, generating alternatives, evaluating alternatives, choosing an alternative, implementing the decision, and evaluating decision effectiveness.Reference Lunenburg 61 There is a growing need to enhance non-technical cognitive skills in EMS training programs. Enhancing decision-making skills through targeted training can lead to better performance in emergency medical settings.Reference Sedlár 60 Management Information Systems (MIS) technologies, such as Decision Support Systems (DSS), Geographic Information Systems (GIS), and predictive analytics, play a pivotal role in accelerating decision-making and enhancing the accuracy of emergency responses. Incorporating information technology systems, including the International Classification of Diseases (ICD), within simulated exercise scenarios can significantly expedite decision-making during training exercises.Reference Rahman, Islam and Borna 62 , Reference Mitura, Snarska and Celiński 63 Managing emergencies and making quick decisions in this situation requires teamwork. Weak team coordination can lead to delayed decision-making and disruption of the crisis response.Reference Bearman, Grunwald and Brooks 64 Emergency management skills training should be considered to improve collaboration and coordination among emergency response teams. Trained nurses can be effective team members in responding to emergencies, working closely with physicians and other health professionals to provide integrated and coordinated care. Good teamwork can increase the efficiency and effectiveness of emergency response and reduce communication errors or breakdowns.Reference Yap, Daniel and Knott 65 Digital tools and decision support technologies can be important in improving coordination and information transfer between team members during exercise. Increasing interactions between team members and developing joint training programs can significantly improve the quality of decision-making in emergencies.
Evaluation and Reporting of Exercise
In the post-exercise reporting process, the use of checklists and standardized tools to evaluate the exercise increases the accuracy of results analysis and reduces bias. Attention to the importance of collecting accurate exercise data to identify strengths and weaknesses and improve team performance is essential.Reference Soghikian, Chipman, Holmes, Calhoun and Mallory 66 Conducting immediate and delayed debriefing is essential to ensuring the quality of prehospital training. Debriefings allow for the identification of strengths and weaknesses, review of information, and analysis of experiences.Reference Jaffrelot, Boet and Floch 67 Review sessions after each exercise improve the learning process and reduce the gap between theoretical knowledge and operational performance.Reference Jaffrelot, Boet and Floch 67 , Reference Tierney 68 Therefore, the exercise evaluation process, as one of the quality indicators of exercise, should be considered structured, based on real data, and include immediate and delayed feedback in exercises.
Promotion of Managerial Capabilities and Competencies
Training in functional skills for emergency response plays a crucial role in increasing nurses’ knowledge and skills, improving their reaction time in critical situations, and ultimately, enhancing the quality of medical care. Sufficient familiarity with emergency diagnostic and therapeutic measures, such as heart attacks, strokes, and trauma, enables individuals to identify problems more quickly and provide appropriate care to patients.Reference Nigatu, Debebe and Tuli 69 , Reference Deitche and Burda 70 Awareness of their roles and responsibilities in emergency response is a crucial element in the flexibility and adaptability of emergency personnel to unexpected situations, leading to improved emergency response performance.Reference Sukonthasarn, Wangsrikhun and Chanpransit 71 Also, increasing nurses’ knowledge and skills in emergency management enhances their focus, reduces their stress, and improves the quality of patient care.Reference Mowbray, DeLaroche and Parker 72 Continuous professional skills training and development are essential to maintaining competence and increasing the efficiency of prehospital emergency personnel.Reference Sukonthasarn, Wangsrikhun and Chanpransit 71 To evaluate the exercises, it seems necessary to consider an index to assess the level of personnel familiarity with roles and responsibilities and whether the exercise promotes essential personnel skills in emergencies.
Development of Psychological Capabilities
Providing specialized training in exercises increases the self-confidence and psychological readiness of prehospital emergency personnel.Reference Van Dillen, Tice and Patel 73 Confidence and self-belief are important elements in prehospital emergency personnel’s success and optimal performance, which lead to reduced stress and increased job satisfaction. These individuals will be more motivated to provide services to patients. Familiarity with treatment skills specific to emergencies increases nurses’ confidence in responding to emergencies.Reference Mowbray, DeLaroche and Parker 72 Self-efficacy is recognized as an alternative measure of training effectiveness.Reference Jonson, Pettersson and Rybing 74 Simulation-based training can increase confidence, motivation to learn, critical thinking, and self-efficacy, enhancing participants’ perceived abilities in communication, teamwork, leadership, and management of clinical scenarios.Reference Watters, Reedy and Ross 75 , Reference Stefanidis, Scerbo and Montero 76 The presence of emergency personnel in simulated operational exercises increases their self-confidence. These individuals have a more remarkable ability to manage actual situations. Short-term training based on computer simulations has increased the self-confidence of emergency nurses in managing major crises.Reference Jonson, Pettersson and Rybing 74 Prehospital exercises should be designed in a way that, in addition to promoting technical skills, strengthens the confidence and self-belief of personnel. Holding workshops, providing constructive feedback, encouraging and appreciating positive performance, and creating suitable opportunities for professional growth and development of personnel are suggested for this purpose.
Conclusion
The findings of this research present a knowledge framework that can help exercise planners in prehospital settings in designing scientifically sound and standardized exercises aimed at enhancing disaster response processes. Furthermore, the implementation and evaluation of both discussion-based and operation-based disaster exercises informed by these identified quality indicators can foster the development of knowledge and promote behavioral change among prehospital staff and facilitate a standardized response to emergencies and disasters. In designing exercise programs and improving functional and specialized skills, it is necessary to consider enhancing psychological abilities to increase perceived self-efficacy, stress management, and personnel confidence. The use of modern technologies and innovations enhances the quality of exercises. A shortage of logistical and support resources, as well as skilled human resources, is one of the obstacles to implementing functional exercises.
Prehospital systems must anticipate mechanisms for providing sustainable financial resources and evaluating capable personnel to implement exercises continuously. It is recommended that the long-term effectiveness of training programs implemented with modern technologies, including the adoption of digital tools for scenario simulation, interprofessional education to strengthen teamwork, and the standardization of post-exercise evaluations to identify performance gaps and enhance readiness and cost-effectiveness of these exercises, should be evaluated in future research. Additionally, the development of non-technical cognitive skills, such as stress reduction and improved decision-making, should be integrated and anticipated in the training programs of Prehospital emergency personnel. Identifying innovative training methods to maintain and enhance functional skills with minimal use of available resources is crucial.
Consent for publication
Not applicable.
Data availability statement
The data sets generated during the current study are available from the corresponding author.
Acknowledgments
The authors would like to appreciate all authors whose articles were used in this study.
Author contribution
AA and HF wrote the main manuscript text and prepared the figures and tables. The project was supervised by MN and AS. The manuscript was reviewed by MA, AS, and AT. The search strategy was reviewed by HF and AA and where agreement was not reached. MN further reviewed the papers and decided on inclusion. All authors read and approved the final manuscript.
Funding statement
This study was financially supported by the Kerman University of Medical Sciences.
Competing interests
None.
