Disasters are events that cause or may cause damage to people’s lives, bodies, property, and the nation, 1 resulting in severe dysfunction that exceeds the ability of the community to respond. Reference Rutherford2 They also have a significant mental health impact on individuals who are directly or indirectly affected. Reference Lee, Shim and Han3
In the case of a fire disaster, the victims may experience sleeplessness, nightmares, guilt, and fear of fire, leading to a compulsive behavioral instinct to escape, avoiding confined spaces, repeatedly checking for exits, obsessing about checking for gas, or not being able to get close to a fire. It has also been reported that hyperarousal to sounds reminiscent of fire alarms is a typical symptom of post-traumatic stress. Reference Mecke4 Depending on the severity of the fire, psychological problems such as anxiety, dissociation, and post-traumatic stress persist for a longer period, Reference Van Loey, van de Schoot and Faber5 which, if overlooked, can complicate the return to daily life and produce long-term disorders or even suicidal and life-threatening behavior. Reference Tucci, Moukaddam and Meadows6 Therefore, treating psychological trauma caused by fire should be considered just as important as physical recovery, and support activities for psychological stability and prevention of post-traumatic stress disorder (PTSD) should be prioritized. 7
In the immediate aftermath of a disaster, crisis intervention—activities performed to help disaster victims with their practical needs and reduce psychological problems caused by shock and distress—is called psychological first aid (PFA). Reference Uhernik and Husson8,Reference Vernberg, Steinberg and Jacobs9 PFA is an important initial response to a disaster, as it helps victims recover from the stress response and return to normal life more quickly. Reference Birkhead and Vermeulen10 Aid workers practicing PFA assist victims in disaster situations by providing psychological support and connecting them to necessary resources. Reference Choi11 With the Sewol ferry disaster in 2014 and the Pohang earthquake in 2017, many first responders had not received training in PFA and therefore lacked an understanding of the psychological problems of victims and their families under such circumstances. Reference Hyun12 As the frequency and scale of disasters gradually increase and society becomes more complex, there is a need to develop PFA training programs that are easy for disaster relief workers to learn and free from time and space constraints.
Web-based simulation education can be accessed through the Internet and participation can be made free. Reference Cant, Cooper, Sussex and Bogossian13 Further, unlike traditional simulation methods, it has the advantage of not being limited by time and space, Reference McGowan14 allowing for repeated participation and immediate feedback. Reference Cook, Levinson and Garside15 The effectiveness of these web-based simulation training programs has been studied in acute care nursing, Reference Kim and Cho16 respiratory care, Reference Kim, Kim and Lee17 depression screening, Reference Rota18 and physical education such as CPR training. Reference Boada, Rodriguez-Benitez and Garcia-Gonzalez19 Programs have been developed for various physical education topics, but the scope of PFA education is limited in Korea, as the only identified web-based PFA programs center on earthquake disasters Reference Choi11 and infectious diseases. Reference Ko20 As the curriculum should be implemented based on the characteristics of PFA performed in special situations such as disasters, content that is not realistic and tangible may decrease learners’ curiosity and participation. Reference Lee, Na and Shim21
Therefore, this study attempted to provide learners with a sense of realism and authenticity in studying disaster situations through a web-based training program. Further, it sought to develop and apply the training program to expand the psychological support capacity of disaster relief workers and to verify the program’s effectiveness, based on previous research showing that simulations using standardized patients rather than graphics are perceived as more realistic.
Methods
Research Design
The study adopted a nonequivalent control group pretest-posttest design to determine the contextualized PFA simulation training program’s effectiveness for disaster relief workers.
Participants
Participants were disaster relief workers who agreed to participate in the program and were not restricted by their department or specialty. Reference Park, Choi and Choi22 To recruit participants, announcements were posted on the online bulletin boards of academic societies and associations. After the purpose and method of the study were explained to those who registered and their consent obtained, participants were randomly assigned to the experimental and control groups.
The G-Power 3.1.9.2 program was used to determine the number of participants. According to the repeated-measures analysis-of-variance method with a significance level (α) of 0.05, power (1-β) of 0.80, and effect size (d) of 0.8, 26 participants were required for each group, but considering the dropout rate, the total number of participants required was calculated as 60 (30 in the experimental group and 30 in the control group).
This study was a single-blind trial, meaning that participants knew that they were participating in the PFA training program but did not know whether they were in the experimental or control group. This ensured that the results would not be influenced by placebo effects or researcher preference.
Measurements
Core Competencies in Disaster Response
The core competencies for disaster response were measured using the Perceived Competence Scale for Disaster Mentally Healthy Workforce (PCS-DMHW) developed by Yoon and Choi. Reference Yoon and Choi23 This tool assesses the competencies required for mental health workers’ effective response in disaster situations and has 2 subscales—individual competence and team competence—but only the individual competence scale was used in this study. The 24-item measure consists of 6 knowledge items (understanding disasters, tailoring support), 9 attitudinal items (calling, ethics, qualities), and 9 skills items (problem-solving, communication, information transfer). Items are rated on a 5-point Likert scale, with possible scores ranging from 0 to 96 and higher scores indicating higher perceived competence in the area. Reference Yoon and Choi23 At the time of its development, the scale’s Cronbach’s α was reported as 0.95.
Self-Efficacy
Self-efficacy is the belief that a particular behavior will produce the expected outcome and beliefs about one’s ability to achieve the expected outcomes. Reference Bandura24–Reference Bandura26 Self-efficacy was measured using an instrument developed by Ayres Reference Ayres27 and translated by Park and Kwon. Reference Park and Kweon28 The instrument consists of 10 items, with a reported Cronbach’s α of 0.94 at the time of development. Items are rated on a 7-point Likert scale ranging from 1 (“Not at all”) to 7 (“Very much so”) and higher scores indicating higher perceived competence in the area.
Problem-Solving Process
The problem-solving process was measured by the Process Behaviors on Problem Solving Performance instrument developed by Lee Reference Lee29 and modified by Park and Woo. Reference Park and Woo30 The measure consists of 25 items, including 5 items on problem discovery, 5 items on problem definition, 5 items on devising a solution to the problem, 5 items on implementing the solution, and 5 items on reviewing the solution. Items are rated on a 5-point Likert scale ranging from 1 (“Never”) to 5 (“Almost always”), with possible scores ranging from 25 to 125. The higher the score, the more effective the individual’s problem-solving process. Park and Woo Reference Park and Woo30 reported the scale’s Cronbach’s α to be 0.90.
Procedure
This simulation training program was designed based on the ADDIE model Reference Leshin, Pollock and Reigeluth31 to enhance fire disaster relief workers’ PFA capabilities. The study proceeded according to the following steps: analysis, design, development, implementation, and evaluation. A nonequivalent control group pretest-posttest study assessed the program’s effectiveness (Table 1).
Study design
The training program consisted of 4 stages: pre-learning, pre-briefing, simulation, and debriefing. In the pre-study, theoretical knowledge was disseminated so that participants could learn the content of the program, including fire disasters, disaster mental health, and PFA methods. The pre-briefing provided explanations and information on how the next stage of the simulation would be conducted. The simulation consisted of a video with a quiz presented in the middle. This provides immersive and experiential learning simultaneously, which can have the same effect as interaction Reference Buykx, Kinsman and Cooper32 and is conducive to practice and knowledge acquisition. Reference Verkuyl, Romaniuk and Mastrilli33
The pretest was an online questionnaire provided via Google Forms. The researcher sent a link to the pretest questionnaire to participants’ SMS for them to complete. The pretest consisted of items assessing general characteristics, core competencies for disaster response, problem-solving process, and self-efficacy, taking approximately 15-20 minutes to complete. All 60 participants completed the pre-questionnaire. There was no subsequent dropout in the study’s later stages.
The experimental group accessed and completed the training program through the URL sent by the researcher (Figure 1), whereas the control group was provided a written educational material regarding general disaster.
Web-based fire disaster victim PFA simulation training program screenshots (pre-learning, pre-briefing, video, and quiz).
Posttest 1 was designed to be completed immediately after the training. For the experimental group, the last item in the training program provided a link to Posttest 1. The control group accessed the survey through the URL sent by the researcher.
The experimental group and the control group accessed Posttest 2 through a link provided by the researcher 1 week after the program ended. Again, all participants completed items assessing current core competencies, problem-solving process, and self-efficacy of disaster response via Google Forms.
Data Analysis
Data were analyzed using SPSS 23.0 (IBM Corp, Armonk, NY). First, descriptive statistics were computed for participants’ general characteristics. Second, homogeneity between the experimental and control groups in terms of the characteristics and the research variables was analyzed using Fisher’s exact tests and t-tests. Finally, effects of the PFA simulation training program for disaster relief workers were analyzed using analysis of variance (ANOVA), independent t-tests, and paired t-tests.
Ethical Considerations
This study’s content and methods were approved by the ethics center at the authors’ institution (No.1041078-202107-HR-227-01), and the research protocol was registered with CRIS (KCT0008351). Participants were fully informed before the study and provided voluntary consent to participate. It was clearly stated and explained in the consent form that participants could withdraw from the study at any time if they wished and that they would not be penalized for doing so. Participants’ privacy and confidentiality were strictly protected, and all identifying information was eliminated.
Results
Homogeneity Tests for General Characteristics and Validation of Measures
The participants in this study were disaster relief workers, consisting of nurses, firefighters, police officers, social workers, and psychological counselors, and homogeneity between groups was confirmed (P = 0.202). The participants’ total work experience was 90.52 ± 58.07, with 93.80 ± 68.32 for the experimental group and 87.23 ± 46.58 for the control group, confirming homogeneity (P = 0.435) (Table 2). In terms of PFA training experience before this study, both the experimental and control groups were the same, with 20 (66.7%) of the experimental group having never received PFA training and 10 (33.3%) of the control group having received PFA training.
Homogeneity tests for general characteristics between the groups (N = 60)
Cont., control group; Exp., experimental group.
Verification of Training Program Effectiveness
Table 3 and Figure 2 show the results of the analysis of the training program’s effectiveness. First, the 2 groups’ core competencies for disaster response showed a difference between the groups in terms of change over time (F = 15.87, P = 0.000). When assessing differences in self-efficacy, a difference between the groups in terms of change over time was also significant (F = 4.13, P = 0.029). Lastly, when assessing differences in the problem-solving process, it was significant (F = 3.709, P = 0.038) as well.
Group differences in web-based PFA simulation education program effects focusing on fire disaster (N = 60)
Cont., control group; Exp., experimental group.
Mean differences between experimental and control groups by variables.
Discussion
The PFA simulation training program developed in this study was designed to train disaster relief workers to provide psychological support to victims of fires. Disaster response core competencies significantly increased in the experimental group immediately after training and 1 week later, and these increases were significantly greater than those of the control group.
This result is consistent with the demonstrated effectiveness of extant PFA simulation training programs for disaster situations developed by Choi Reference Choi11 and Ko. Reference Ko20 However, in these previous studies, only a pretest and 1 posttest were administered, whereas in this study there was a second posttest 1 week after the training, data from which demonstrated the continued effectiveness of the program. Furthermore, the improvement of the experimental group’s core disaster response competencies over time is likely due to the web-based learning that provides learners with a realistic and immersive disaster experience, transcending the limitations of traditional laboratory learning. Reference Currie, Kourouche and Gordon34 The experimental group’s ability to fully comprehend and apply the program’s components also may have been enhanced by learning in stages.
Disaster relief workers’ self-efficacy significantly increased in the experimental group immediately after the training and 1 week later, whereas there was no significant change in the control group. Self-efficacy is a key predictor of learners’ engagement in education and is positively related to academic achievement. Reference Al-Abyadh and Azeem35 This is consistent with the findings of Ko, Reference Ko20 who concluded that the experimental group in their program experienced the process of applying psychological support by identifying and solving problems on their own in disaster situations, which may have improved self-efficacy through active learning. Kim and Cho Reference Kim and Cho16 report that online learning in university education has a direct effect on self-efficacy, and although it is not the same educational program as that of this study, it is also web-based, suggesting that this program may have increased self-efficacy.
Moreover, the problem-solving process of disaster relief workers in the experimental group increased significantly over time, both immediately after the training and 1 week later. In the control group, there was a significant increase immediately after the training, but a slight decrease was observed 1 week after the training compared to immediately after the training. This is consistent with previous studies. Reference Choi11,Reference Kim, Kim and Lee17,Reference Ko20 This finding suggests that the prior learning facilitated by this training program improved the problem-solving process because learners participated in the simulation by identifying the characteristics of the problems to be solved based on cues and awareness of psychological support. In addition, the game-style quizzes in the program may have enhanced the learning process by providing immediate feedback and allowing for active participation in the problem-solving process.
Although empirical research on the effectiveness of PFA in disaster situations is lacking, mental health experts and global humanitarian guidelines continue to recommend PFA as appropriate support for individuals experiencing severe mental distress after a disaster. In Japan, PFA training is mandatory for firefighters, medical responders, local officials, schoolteachers, and others classified as disaster relief workers. As the importance of PFA training has increased in Korea, studies Reference Choi11,Reference Ko20,Reference Park, Choi and Choi22,Reference Kang and Choi36 have supported the effectiveness of PFA simulation training. Importantly, the web-based simulation training developed in this study resulted in understanding the content (specifically, fire disaster victim support) and enhanced self-efficacy and problem-solving process by facilitating active learning through realistic disaster scenarios.
Limitations
A limitation of this study is the purposive sampling of participants, so caution should be exercised in generalizing the findings to all disaster relief workers. Additionally, although we identified whether participants had attended PFA training and evenly distributed the number across groups, this may have affected the study results because different learning levels about PFA may vary depending on the number of PFA training attended. Furthermore, the posttests occurred immediately after the program and 1 week later, so longer follow-ups are recommended. Despite its limitations, the study is significant in that it developed an effective program allowing disaster relief workers of various professions to participate in realistic fire victim support training without restrictions on time and place.
Conclusion
Fire disaster victims need psychological support activities for psychological stability and PTSD prevention because they may experience various psychological problems as well as physical damage. This study was conducted to identify the effect of developing and providing a PFA simulation training program for fire disaster victims to disaster relief workers. According to this study, the web-based PFA simulation game showed significant effects in all areas of core competencies in disaster response, self-efficacy, and problem-solving process of the subjects. These results suggest that the web-based simulation game is a good education method that can effectively increase the learning effect in special situations of disaster situations.
