Introduction
When disasters occur, Emergency Physicians are on the front-lines of the response. They must respond to disaster events immediately and professionally, with no time available for refresher or remedial education in disaster medicine. The American College of Emergency Physicians (ACEP; Irving, Texas USA) has defined a comprehensive set of core competencies in disaster medicine spanning 19 domains that are relevant to all Emergency Physicians.Reference Schultz, Koenig, Whiteside and Murray 1 In addition, numerous federal and national guidance documents make disaster education a requirement for front-line hospital personnel such as Emergency Physicians. 2 , 3 However, in a 2014 study, more than one-half of residents in Emergency Medicine reported that they believed they did not receive enough training in disaster medicine during their residency to feel competent.Reference Sarin, Cattamanchi, Alqahtani, Aljohani, Ardalan and Ciottone 4 Moreover, other studies have demonstrated poor understanding of disaster medicine concepts among practicing Emergency Physicians who have completed residency.Reference Paganini, Borrelli and Cattani 5 Therefore, there is a clear need to improve and expand educational opportunities in disaster medicine for Emergency Physicians to enhance the readiness of the nation’s emergency care systems.
However, there are several challenges that make improved education in disaster medicine difficult in the current environment. First, there are already substantial practical limitations on expanding classroom instructional time for disaster medicine in residency programs, with many programs struggling to incorporate all of the required content of Emergency Medicine within a three- or four-year curriculum. Second, practicing Emergency Physicians can be hard to gather in a single educational setting, given work schedules and other commitments. Third, some Emergency Medicine residency programs, and most community hospitals, lack direct access to experts in disaster medicine who are capable of providing in-person education on a regular basis to large numbers of students. Emergency Medicine residents from 201 Emergency Medicine residency training programs need to be supported by the limited number of faculty who are capable of this instruction. 6 In fact, the entire membership of the ACEP Disaster Medicine section totals 616 members who are already engaged with multiple commitments. Consequently, there is a need to develop innovative methods to provide education to Emergency Physicians in disaster medicine competencies.
Faced with these challenges in educating both new and practicing Emergency Physicians and nurses in the ever-expanding body of knowledge of disaster medicine, video-based learning (VBL) has been suggested as a promising alternative to the use of traditional lecture (TL) methods.Reference Cardall, Krupat and Ulrich 7 If the effectiveness of VBL could be demonstrated, it could offer an acceptable substitute for, and complement to, TL for teaching the ACEP disaster medicine core competencies to Emergency Medicine residents. This study tested the hypothesis that there would be no significant difference in the effectiveness of TL and VBL in educating Emergency Medicine residents in the disaster medical topics of triage, patient decontamination, and personal protective equipment (PPE) use.
Methods
Study Design
A randomized, controlled trial was conducted to determine the effectiveness of VBL compared with TL for selected disaster medical topics presented to Emergency Medicine residents within a single residency program. The Institutional Review Board at Mount Sinai School of Medicine (New York, New York USA) determined that this study was exempt from the institutional review board process.
Participants
The study was conducted at an urban, academic, tertiary referral center that provides training to Emergency Medicine residents in a four-year program. All Emergency Medicine residents who attended the weekly educational conference were recruited for this study. As there were not any more residents available to participate at this single-site study, the number needed to study for significance was not determined.
Study Protocol
The educational content for both the TL and VBL training courses was developed under the supervision of an Emergency Medicine faculty member who is also the hospital’s Director of Emergency Management. Specific content for the training modules was adapted from training materials of the US Federal Emergency Management Agency’s (Washington, DC USA) Center for Domestic Preparedness Technical Emergency Response Training course, 8 and the Chemical Casualty Care Division of the United States Army’s Medical Research Institute (Frederick, Maryland USA) of Chemical Defense Hospital Management of Chemical, Biological, Radiological/Nuclear, and Explosive Incidents course. 9 Additional resources utilized in the course development included multiple US military publications, 10 - 15 educational materials from the Hazmat for Healthcare website, 16 the Duodote autoinjector (Meridian Medical Technologies, Inc.; Columbia, Maryland USA) instruction sheet, 17 a United Kingdom Strategic National Guidance resource, 18 and the US Occupational Safety and Health Administration’s (Washington, DC USA) Best Practices for First Receivers guidance. 3
A PowerPoint (Microsoft Corp., PowerPoint version 12; Redmond, Washington USA) presentation for the TL control group, and a training video for the VBL experimental group, were created. Both educational modalities contained identical educational content. The video for the VBL group was filmed using a camcorder (Sony USA, model HDR-HC7; New York, New York USA) with an amateur cast and crew at Mount Sinai Hospital (New York City, New York USA). Voice-over content was created using professional audio-visual scriptwriting software (Final Draft, Inc., Final Draft AV version 2.5.2; Calabasas, California USA). Recording was performed using a professional long-gun microphone head (Sennheiser Electronic Corporation, ME 67 Back-Electret Microphone Head; Old Lyme, Connecticut USA). Editing of the recording was accomplished using non-linear editing software (Apple Inc., Final Cut Pro version 6.0.6; Cupertino, California USA). An Emergency Medicine resident in the residency program’s disaster medicine specialty track wrote, directed, and edited the video. A screen capture of the video can be viewed in Figure 1. The educational slides for the TL group were created using a presentation program (Microsoft PowerPoint version 12). The TL modality was delivered by a faculty expert in disaster medicine. The VBL modality was setup and viewed without faculty interaction.
Video-Based Learning.
On the day of the study, and prior to the educational presentations, all participants completed a pre-test to establish their baseline knowledge in the disaster medicine topics to be presented. To preserve anonymity, study participants identified themselves on all study tests and surveys using employee identification numbers rather than their names. The study staff did not have any means to identify participants by their employee numbers and did not attempt to do so. A department research division consultant conducted a stratified randomization of residents by post-graduate year (PGY) class level (PGY 1 and 2 versus PGY 3 and 4) and assigned them to either the experimental (VBL) group or the control (TL) group. Participants in the VBL group viewed the video, while those assigned to the TL group attended the lecture. Upon completion of the educational presentations, all participants completed a written test to assess their knowledge of the subject material, as well as a survey to assess their comfort in applying this knowledge to practice. Staff from the department’s research division proctored the knowledge tests and comfort surveys.
Prior to the day of educational instruction, evaluators knowledgeable in the principles of triage, use of PPE, and patient decontamination completed a mandatory training session regarding the course content and educational objectives. On the day of the educational instruction, these individuals served as evaluators and scored the participants’ practical exercises using critical action checklists. All evaluators were blinded as to which study participants had participated in the TL modality and which participated in the VBL modality. The complete protocol of the study is outlined in Figure 2.
Study Protocol. Abbreviation: PPE, personal protective equipment.
Procedure
Outcomes were measured by participants’ scores, responses, and performance using three instruments: (1) a knowledge test; (2) a comfort survey; and (3) a practical exercise (Appendices A, B, and C; available online only). The knowledge test consisted of a 123-item questionnaire assessing content presented in the VBL and TL modalities. Each participant completed the knowledge test both immediately before and immediately after the educational intervention. Participants also completed the knowledge test again three months later. Each correct answer received one point for a total maximum score of 123.
The comfort survey utilized a five-point Likert scale to determine the participants’ self-reported comfort level in applying their knowledge of triage, use of PPE, and patient decontamination in practice. They survey also asked about residents’ prior experience with disaster training to determine if prior training had an effect on comfort. Each individual completed the survey immediately before and after the practical exercise, as well as again at three months post-intervention.
Recognizing that increases in knowledge and comfort scores do not necessarily translate into competent performance of skills in practice, a practical exercise component to the study was added. This allowed for the testing of the learners’ demonstration of practical skills and use of judgment in each of three skills stations. At Station One, participants were asked to don Level C PPE. Participants subsequently wore Level C PPE for the remainder of the skills station sessions. At Station Two, the participants were asked to demonstrate the proper technique for administration of the Duodote autoinjector to a simulated victim of nerve agent poisoning. Participants were then asked to use the Simple Triage and Rapid Treatment (START) triage system (a commonly used triage system in the US originally developed in 1983 by the Newport Beach, California fire department) 19 in a series of six different disaster scenarios that were described on cards attached to inflatable training manikins (MedicTech, Mass Casualty Incident Man; Sandy, Utah USA). At Station Three, participants were asked to decontaminate inflatable training manikins simulating contaminated victims of a hazardous materials incident. Following completion of the third station, the participants doffed their Level C PPE and were asked to complete the post-exercise comfort survey.
Analysis
Individual responses on the knowledge test and on the comfort assessment were tabulated and converted into percentages. The skill-based practical assessments often involved manual skills that require one or more partners, therefore a team-based approach was used to evaluate practical skills. Evaluators’ assessment of team performance in the skills stations was recorded. Percentage scores for knowledge and comfort items were calculated for individuals, and practical performance scores were calculated for teams. Data were analyzed by SAS statistical software Version 9.4 (SAS Institute Inc.; Cary, North Carolina USA). The chi-squared test and t test were for categorical and continuous variables, respectively, to examine the comparisons of interest and determine the level of significance between test groups. The dependent variable was pre- and post-intervention differences for the percentage scores. The independent variable was the educational presentation (video versus lecture).
Results
A total of 30 Emergency Medicine residents were recruited at the weekly educational conference on October 28, 2009. Due to scheduling conflicts, the research protocol was completed by 26 participants. Of the 15 residents randomized to the TL group, 13 participants completed the entire training and all associated tests and surveys. Of the 15 residents randomized to the VBL group, 13 completed the pre-instruction and post-instruction knowledge test, but one did not complete any of the comfort surveys. Residents that did not complete the training or associated assessments could not do so due to clinical duties that restricted their ability to participate. Furthermore, one participant from the VBL group was not able to participate in the practical exercise, also due to clinical duties causing this participant to leave early. Thus, a total of 13 residents randomized to the TL group and 13 residents randomized to the VBL group underwent the training on the instruction day.
The mean Emergency Medicine residency PGY for the VBL group was 2.15 versus 2.46 for the TL group (P=.41). Females comprised 46.0% of the VBL group and 54.0% of the TL group (P=.69). A baseline knowledge test and comfort survey were administered prior to instruction and the TL and VBL groups did not differ significantly in their scores (P=.77 and P=.36, respectively). The baseline characteristics of the participants are listed in Table 1.
Baseline Characteristics
a N=10 in this cell because 3 participants arrived late and did not complete this portion of the survey.
Both groups showed improvement in knowledge and comfort scores after the educational instruction. The TL and VBL groups did not differ in their post-instruction knowledge score (78.7% vs 80.1%; P=.65). Knowledge scores improved similarly in the TL group (14.1% improvement) and the VBL group (14.9% improvement; P=.74). Likewise, comfort scores improved among participants in the TL group (15.8% improvement) and the VBL group (18.3% improvement), but the improvement did not differ significantly between the two groups (P=.65). Post-instruction comfort scores were also similar between the TL and VBL groups (40.4% vs 43.4%; P=.64).
In the practical skills session, the VBL group significantly outperformed the TL group overall (70.4% vs 55.5%; P < .0001). When broken down by skills, the VBL group performed significantly better in the donning of PPE (78.1% vs 47.4%; P=.003) and decontamination (63.3% vs 46.9%; P=.004; Figure 3). The two groups did not differ in score for the triage portion of the practical skills session (P=.38). The post-instruction results are reported in Table 2.
Practical Exercise Mean Performance Score.
Post-Training Results
Note: Two sample t-tests were used in comparing means and generating P values.
a N=12 in this cell because 1 participant did not complete the survey.
b N=9 in this cell because 4 participants didn’t complete one of the surveys.
c N=12 in these cells because 1 participant did not complete the practical exam.
Although not part of the original study design, a three-month follow-up repeat of the knowledge test and comfort survey was obtained from six residents from the VBL group and six residents from the TL group. Although both groups demonstrated knowledge extinction from their post-intervention knowledge score, there were no significant differences between the two groups in their raw score or degree of knowledge extinction (Table 3). The VBL group did appear to have decreased comfort at three months compared to the TL group (19.0% vs 32.5%), although it was not statistically significant (P=.09).
Three Month Post-Hoc Survey Results
a N=5 in this cell because 1 participant did not answer this part of the survey.
Discussion
There are numerous potential advantages to augmenting, and selectively replacing, TL instruction with VBL instruction. The information provided in a video is standardized, does not vary with the skill and knowledge of the lecturer, and does not require the simultaneous physical presence of the participants and the lecturer. Video is capable of depicting actions, scenes, and processes that may be difficult or impossible to recreate or describe accurately in the classroom environment. With editing, video can compress time sequences, thereby facilitating description of the stepwise performance of tasks and procedures. Additionally, video instruction allows learners to see the actual equipment in use, which may be impractical in a classroom. Video instruction can also be viewed by groups or individuals and requires little preparation or supervision.Reference Gray 20 Of note, VBL is more than simply recording a lecture. In fact, when possible, it attempts to adhere to the filmmaking principle of “show, don’t tell.”
The effectiveness of VBL has been demonstrated with respect to teaching other practical skills. Cardiopulmonary resuscitation (CPR) research has demonstrated that when comparing VBL with the traditional American Heart Association (Dallas, Texas USA) Heartsaver training course, both methods resulted in similar cardiopulmonary skill performance retention at two months post-instruction.Reference Einspruch, Lynch, Aufderheide, Nichol and Becker 21 The research continued the work of a prior study that demonstrated similar CPR short-term skill performance when comparing VBL to a traditional Heartsaver training course.Reference Lynch, Einspruch, Nichol, Becker, Aufderheide and Idris 22 Furthermore, videos have already been advocated as effective means of educating Emergency Physicians in disaster medicine principles.Reference Curtis 23
Video-based learning has also previously been utilized successfully to educate registrars, advanced physician trainees, and others on the emergency department disaster response plan and on principles of hospital disaster management, and it has been associated with an 82.0% increase in the pass rate when participants are tested two weeks after instruction.Reference Bartley, Fisher and Stella 24 The US Centers for Disease Control and Prevention (CDC; Atlanta, Georgia USA) has also utilized VBL for disaster education, producing a 17-minute training video that covers key principles and procedures relevant to a radiation emergency event. 25 Research using pre- and post-tests published by the CDC about this modality revealed improvements in trainees’ knowledge after receiving video education.Reference McCurley, Miller and Tucker 26 In addition, VBL not only improved testing performance, it has been shown to increase student satisfaction. 27
Despite these previously reported successes, as compared with other specialty areas within Emergency Medicine (such as ultrasound education), VBL is still not commonly utilized in disaster medical education. Perhaps this is because, to date, there has been little research in disaster medicine that directly evaluates the effectiveness of VBL in direct comparison with other educational modalities. While one literature reviewReference Hsu, Jenckes and Catlett 28 characterized video education as inexpensive and convenient, the review recommended that more research be performed in order to adequately evaluate its effectiveness. Indeed, only one study identified by the authors has been published that documents the effectiveness of VBL compared with TL for disaster education.Reference Hsu, Jenckes and Catlett 28 That study tested instruction of staff about a hospital’s disaster plan utilizing a 40-minute VBL session and found that the mean post-instruction quiz scores of nurses and paramedical staff were 27.0% better than the comparison group who had only read the written disaster plan.Reference Gray 20
In this pilot study, it was demonstrated that video modules can be utilized successfully to train Emergency Medicine residents in two of the ACEP disaster medicine core competencies: patient triage and patient decontamination; VBL and TL have been shown to be similarly effective for imparting knowledge in these competencies immediately, and this effectiveness persists similarly at three months. Video-based learning may also be a better method of instruction for practical skills than TL in disaster medicine as the differences in exercise scores between the two groups were substantial and approached (though did not reach) statistical significance. This may have to do with the inherent advantages of VBL discussed previously (such as the opportunities to showcase processes and equipment in use during the didactic portion of training). It is possible that additional, larger studies may be able to confirm whether specific disaster medicine skills may be better taught through VBL, and to identify which skills specially benefit from VBL.
Limitations
This study had some limitations. The study site was a single PGY 1-4 residency program based at a university hospital in a large city with frequent activations of its emergency operations plans. Therefore, there is the possibility that these findings may not be fully generalizable to other environments. The study was also limited by its small sample size. Future studies could be performed with an increased sample size and participation of multiple residency sites, randomizing the residents to either a live, teleconferenced, PowerPoint-based lecture or broadcast VBL. In addition, during the practical exercise, because of practical considerations (especially during decontamination), participants completed the stations in teams. It is possible that some participants with greater experience guided those who otherwise would have performed poorly in the stations, confounding the study’s ability to discriminate group differences in the effect of the intervention. Future studies could design exercise stations in which the participants complete the stations individually.
Conclusion
This study suggests that VBL modules can be as effective as, or in some cases more effective than, TL in educating Emergency Medicine residents about disaster medicine as measured by knowledge scores, comfort scores, and performance in practical exercise stations. Because of limitations on classroom time in residency programs, as well as other limitations on providing disaster education to practicing emergency physicians, VBL may provide a promising opportunity to remedy previously described deficiencies in Emergency Physicians’ knowledge of disaster medicine topics. Future studies may further explore the effectiveness and limitations of increased use of VBL.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X1700718X
