Introduction
There has been an increased emphasis on hospital preparedness for chemical/biological/radiological/nuclear (CBRN) incidents in the preceding 20 years. Management of these incidents requires personal protective equipment (PPE) utilization to ensure the safety of victims, facilities, and providers; therefore, providing effective and accessible training in its use is crucial.Reference Olivieri, Ingrassia and Corte1 These concerns have largely been driven by real and perceived threats of terrorism, though accidental hazardous material (HAZMAT) incidents are far more common.Reference Williams and Sizemore2,Reference Levitin, Koenig and Schultz3 Concerns have been heightened by the 2014 and 2018-2019 outbreaks of Ebola virus disease in Africa.Reference Fischer, Weber and Wohl4 While this emphasis has been accompanied by increased government funding in the United States, such as that available through the Weapons of Mass Destruction Act of 1996, there is concern that hospitals and health care professionals are no more prepared now than previously.Reference Keim, Pesik and Twum-Danso5-Reference Wetter, Daniell and Treser8 In addition, it is difficult to measure preparedness. Considerable disagreement remains about the manner to best train and prepare health care providers.Reference Gowing, Walker, Elmer and Cummings6
Personal protective equipment has an important role to play in the evaluation and treatment of patients with suspected exposure to HAZMAT, though more effective methods of exposure control exist.Reference Fischer, Weber and Wohl4,9 Many of the challenges with PPE are thought to arise from the health care provider’s ability to consistently choose the right type and size, as well as use it correctly.Reference Levitin, Koenig and Schultz3,10,Reference Kang, Kim and Choi11 There have been attempts to standardize the types or levels of PPE by both national organizations (including the Centers for Disease Control and Prevention [CDC; Atlanta, Georgia USA], the Occupational Safety and Health Administration [OSHA; Washington, DC USA], and the American College of Emergency Physicians [ACEP; Irving, Texas USA]) and by hospital systems.12-16 While OSHA mandates that first receivers in hospitals receive a certain amount of PPE training, no gold standard method is dictated.13
Most hospitals perform a standard, annual in-person demonstration, or may occasionally use video or slide presentations.Reference Kang, O’Donnell and Colaianne17 However, multiple studies have also demonstrated the effectiveness of more interactive “hands-on” training methods, ranging from computer simulation to self-directed workbooks.Reference Hung, Choi and Chiang18-Reference Doll, Feldman and Hartigan22 Kang, et al state that “evidence shows that traditional learning methods (eg, watching educational videos, learning PPE guidelines) are inferior to immersive learning methods, including active learner involvement using simulations that include feedback on performance.”Reference Kang, O’Donnell and Colaianne17
Nevertheless, emergency situations demanding the immediate use of PPE may not allow for the use of these more immersive methods. If a CBRN incident occurs in the middle of the night, there may not be a qualified staff member available to demonstrate to health care providers the proper application of PPE. Other training methods may be necessary.
The goal of this pilot study was to compare the effectiveness of video training versus in-person training of 20 Emergency Medicine (EM) residents in the donning and doffing of PPE.
Methods
Study Design and Setting
This was a randomized, controlled pilot trial set in the emergency department of an urban acute care hospital in central Brooklyn, New York (USA) with 376 in-patient beds and >68,000 annual emergency department visits. The study design was submitted to the State University of New York (SUNY) Downstate (Brooklyn, New York USA) Institutional Review Board and was exempted from full committee review (Protocol 1248482-1).
Description
A convenience sample of subjects was chosen from EM residents in attendance at their weekly educational conference. Twenty residents were stratified by their post-graduate year (PGY) level (two through five) and then randomized into four distinct groups, with the same number of each PGY residents in each group to minimize the possible effect of previous training exposure. The EM residents not in attendance at the conference were excluded.
Group 1 and Group 2 acted as the experimental groups and were shown a nine-minute video presentation created by the Emergency Preparedness Team (EPT), which can be provided upon request to the corresponding author. The video demonstrated EPT personnel performing the correct donning and doffing of Level C PPE, which included Tychem CPF 3 chemical-resistant coveralls with attached butyl gloves (DuPont; Midland, Michigan USA); rubber boots; and a Powered Air Purifying Respirator (PAPR) device. Prior to viewing the video, one random member of each group was provided with supplies and instructions to create a donning/doffing checklist based on the video to be used during the drill itself.
Group 3 and Group 4 were taken to a separate location and served as the control groups. They received an approximately ten-minute in-person demonstration on the correct donning and doffing of the same PPE by the same member of the EPT featured in the video demonstration. They were also asked to create a checklist to use during the drill.
The four groups were then taken to separate locations and provided with the abovementioned PPE. Within each group of five residents, one was designated to actually perform the donning/doffing procedure, two were assigned to physically assist with the procedure, one was responsible for reading the checklist and making sure it was followed, and one was providing general assistance, as needed.
Each group then performed the donning procedure while being observed by evaluators. Once completed, the individual in the PPE passed through an active decontamination shower room as an additional assessment of the integrity of the applied PPE. The individual then returned to the previous area and began the doffing process with the rest of the group.
Study Endpoints
The primary study endpoint was the completion of the doffing process by all four groups.
Evaluation and Data Collection
Sixteen PGY 1 residents were designated as independent evaluators. They had received in-person PPE training separately the day prior and were trained on their roles in this drill at that time. Eight of the evaluators were assigned to the donning process and eight of the evaluators were assigned to the doffing process. The evaluators were then further sub-divided so that two evaluators were assigned to each of the four groups during each process. The evaluators were blinded to the type of training their particular group had received. Donning evaluators were given a donning evaluation tool while the doffing evaluators were responsible for evaluating the doffing process (Figure 1 and Figure 2). These evaluation tools were then collected for analysis by the EPT.
Donning Evaluation Tool.
Doffing Evaluation Tool.
Following the drill, a group debriefing (“hot wash”) session was performed with all the drill participants, evaluators, and the EPT. Written and verbal feedback was elicited at this time. The 20 drill participants were given a brief survey concerning the drill utilizing a five-item Likert scale (Figure 3). This survey was intended to assess the participants’ confidence in donning/doffing both prior to and after their respective trainings, as well as other subjective aspects of the trainings.
Self-Evaluation Tool.
Results
Donning
During the donning process, each of the four groups were evaluated on 14 critical protocol tasks (Figure 1). These tasks were designed to happen in sequence and were specifically included in both the video and the in-person trainings.
Evaluators of Group 1 and Group 2 (video trained) identified a total of nine protocol task violations out of 55 evaluated possibilities (Table 1). These violations included tasks that were either performed incorrectly or omitted entirely. This led to an error rate of 16.0% (95% CI, 11.2% to 32.0%). The most frequently failed tasks involved not checking the PAPR for flaws and functionality, and not performing a final safety check by the donning assistant (a buddy check) prior to entering the decontamination showers.
Donning Tasks Completed versus Violated
Evaluators of Group 3 and Group 4 (in-person trained) identified a total of 11 protocol task violations out of 56 evaluated possibilities (Table 1). This led to an error rate of 19.6% (95% CI, 8.6% to 28.5%). Their most frequently failed tasks also involved PAPR inspection and forgotten buddy check, but there were additional omissions such as failing to check vital signs or providing hydration during the process.
Using a Fisher’s exact test to compare the number of protocol task violations between the two sets of groups demonstrated a two-tailed P value of .81, demonstrating no statistically significant difference between donning errors in the two sets of groups.
Doffing
During the doffing process, each of the four groups were evaluated on 11 critical protocol tasks (Figure 2). These tasks were constructed to appear in sequence and were specifically included in both the video and the in-person trainings.
Evaluators of Group 1 and Group 2 (video trained) identified a total of 15 protocol task violations out of 44 evaluated possibilities (Table 2). This led to an error rate of 34.1% (95% CI, 21.8% to 48.9%). The task violations were largely evenly distributed.
Doffing Tasks Completed versus Violated
Evaluators of Group 3 and Group 4 (in-person trained) found a total of 12 failures out of 44 evaluated possibilities (Table 2). This led to an error rate of 27.3% (95% CI, 16.2% to 42.0%). Their most frequent task violations involved the removal of the PAPR hood and the rechecking of vital signs after decontamination was complete.
Using a Fisher’s exact test to compare the number of failed tasks between the two sets of groups demonstrated a two-tailed P value of .64, demonstrating no statistically significant difference in doffing errors between the two sets of groups.
Self-Evaluation
The drill participants each completed a self-evaluation Likert-scale survey (with a 100% response rate) assessing their confidence in donning/doffing before and after the different trainings, the ease of the training, and the captivity of the training (Figure 3). Unsurprisingly, both the video and the in-person trainings showed a statistically significant difference between the before and after confidence levels of the participants in the donning and doffing procedures (Table 3). In addition, a two-tailed T test revealed a significant difference between the two groups for the explanation of donning, but not for doffing (see Question 3 in Figure 3/Table 3). A comparison of the two sets of groups regarding the ease of the different trainings or how captivating they were detected no significant differences.
Self-Evaluation Data
Discussion
The analysis demonstrated no statistically significant differences in the number of protocol task violations in both donning and doffing between the two training modalities. This would seem to show the two methods are equally effective, at least based on the criteria the investigators chose for evaluation. Therefore, it is not possible to reject the null hypothesis at this time. However, it is conceivable that different criteria may have elicited a difference (eg, time to completion). The investigators focused on the quality of the donning/doffing rather than the time needed to accomplish the actions. It is possible that a group trained by a certain modality would complete the procedures more quickly, which would be advantageous if the error rates were similar between the groups.
While the two training modalities appear equally effective, they also appear equally ineffective. The error rates for donning ranged from 16% to 20% for each modality, with even higher doffing rates. This is especially concerning given that the risk of contamination has been shown to be highest in doffing.Reference Fischer, Weber and Wohl4,Reference Suen, Guo and Tong23 There are multiple possible explanations for these high rates, but examination of the evaluator data shows that the majority of the failed tasks involved PAPR inspection, use, and removal. This might indicate that more emphasis should be placed upon this portion of the training in both the video and in-person modalities.
The self-evaluation identified a statistically significant difference with respect to the donning training between the two modalities. Eighty-two percent of the participants in the in-person demonstration group “strongly agreed” that the training explained the steps of donning adequately, compared to only 22% of the video group participants. Several explanations can be posited for this difference, including the inability to ask questions of a presenter. However, this difference was not borne out in the doffing group and the remaining 78% of the video group participants “agreed” with the statement, so the difference may not represent a fundamental flaw with the video training.
There was some initial concern that the participants in the video training would be less likely to pay close attention. However, the self-evaluation survey data demonstrated high levels of interest that was not significantly different from the in-person training. While survey data are susceptible to biases, it is also possible that the use of dramatic music and visual effects in the video kept the participants interested.
Importantly, this study was designed as a pilot study, examining whether these methods of training and evaluating the participants were feasible and could then be applied to a larger sample size. Based on the quality of the data collected and participant feedback elicited from this study, a larger scale version appears feasible, with enough equipment, participants, and time. This study also garnered suggestions for additional investigation in a larger setting, such as measuring time to completion by each arm.
Limitations
The primary limitation of this study is the sample size. A power analysis shows it would require 1,771 subjects in each group to show differences in donning technique and 721 subjects in each group to achieve statistical significance in doffing. The sample was limited by the number of PAPRs available (four) and the number of EM residents in conference that week. While the experiment could have been expanded beyond the emergency department and repeated to obtain greater numbers, that might have introduced a selection bias, as these additional non-frontline individuals are less likely to be faced with a HAZMAT situation.
Conclusion
In this pilot study, there were no significant differences found in the error rates of the donning and doffing of Level C PPE by EM residents following video versus in-person training. Both types of trainings increased the confidence of the participants in the procedures and were perceived as equally interesting and easy, though the in-person training may have been more useful to teach donning. Furthermore, it was demonstrated that this method of collecting comparison data is feasible and can be replicated in a larger scale. Overall, this study indicates that video training on donning and doffing may provide a useful alternative to in-person demonstration training in certain situations, though further study with an appropriately powered sample is warranted to further explore this equivalence.
Acknowledgements
This publication is the result of a thesis submitted in partial fulfilment of the requirements for the degree of Master of Science in Disaster Medicine (European Master in Disaster Medicine). It was supported by a grant from the US Department of Health and Human Services (HHS) Hospital Preparedness Program (HPP) and Public Health Emergency Preparedness (PHEP) aligned Cooperative Agreements (the HPP Grant and the PHEP Grant, respectively) for the purpose of building and sustaining New York City’s public health and health care preparedness capabilities.
Conflicts of interest/funding
This manuscript was supported by a grant from the US Department of Health and Human Services (HHS) Hospital Preparedness Program (HPP) and Public Health Emergency Preparedness (PHEP) aligned Cooperative Agreements (the HPP Grant and the PHEP Grant, respectively) for the purpose of building and sustaining New York City’s public health and health care preparedness capabilities. The authors have no conflicts of interest.
