Introduction
The last 15 years have demonstrated an increase in the frequency and cost of natural disasters that have struck the United States. 1 These disasters, especially recent events like Hurricanes Sandy (2012), Katrina (2005), Irma (2017), and Harvey (2017), have led to the evacuations of hospitals and other health care facilities. 2 Infrastructural disasters such as the Northeastern power outage of 2003 have also led to forced hospital evacuations. Vertical evacuation of patients is especially complicated, requiring dedicated equipment and training to be performed safely.Reference Manion and Golden 3 , Reference Gildea and Etengoff 4 For hospital evacuations to proceed safely and securely, hospital staff must train and practice disaster management and response drills involving institution-specific evacuation protocols.
The Emergency Preparedness Team (EPT) at the New York Institute of All-Hazards Preparedness at the State University of New York (SUNY; Brooklyn, New York USA) Downstate determined that renal transplant patients are especially vulnerable during evacuations and designed a vertical evacuation drill of the Samuel Kountz Kidney Transplant Unit (KTU) in the University Hospital of Brooklyn (UHB; Brooklyn, New York USA).
The drill simulated a power outage after a scheduled upgrade of emergency power outlets affecting only the KTU. Thirteen simulated patients, 11 of which were non-ambulatory, were then vertically evacuated from the KTU eight flights down a stairwell to the ambulance entrance on the ground floor within 3.5 hours.
Study Objective
The goal of this study was to evaluate staff performance and self-assessment during a vertical evacuation drill.
Methods
Study Design and Setting
This was a descriptive survey study in an urban acute care hospital in central Brooklyn, New York with 376 beds and 68,000+ annual emergency department visits.
Development of the Evacuation Protocol
A formal protocol for evacuation was developed for SUNY Downstate UHB by reviewing hospital evacuation plans from across the United States and modifying existing plans and protocols from the institution’s previous evacuation drills. 5 A working group including experts in renal transplant surgery, infection control, intensive care, and emergency planning developed the evacuation protocol.
Drill Preparation
In the weeks prior to the drill, the EPT (which included trained facility staff, administrative staff, and physicians) conducted multiple trainings with the KTU staff. These trainings included a PowerPoint (Microsoft Corporation; Redmond, Washington USA) presentation on the Incident Command Structure (ICS), an eight-minute video describing safe use of the Med Sled (ARC Products LLC; Des Peres, Missouri USA) plastic portable evacuation sled, and hands-on training session with these sleds. 6
Drill Description
The full-scale exercise (Appendix 1; available online only) was conducted using 13 healthy volunteers (recruited from Emergency Medicine) to act as patients. Volunteer patients were each assigned a fabricated medical profile. These medical profiles contained a brief history, vital signs, and physical exam findings consistent with recent renal transplant operations. Hospital staff members who participated in the drill itself either were on-duty at the time of the drill or were recruited by hospital leadership to participate. The evacuation was operated almost exclusively by the KTU department physicians and registered nurses and clerks. Given the drill scenario, the timely recruitment of hospital personnel was felt to reflect “real-world” response.
Study Endpoints
The primary study endpoint was the successful evacuation of the KTU drill victims from the eighth-floor unit with medical records to the hospital ambulance bay awaiting transportation. A post hoc analysis compared the percentage of successfully evacuated victims. The study was submitted to the SUNY Downstate Institutional Review Board and was exempted from full committee review.
Evaluation and Data Collection
Specific drill objectives included the staff establishing specific roles during the evacuation process, demonstrating proper knowledge of the KTU’s evacuation plan, utilizing the portable evacuation sled correctly, and safely physically evacuating all of the mock patients from the unit in a timely fashion.
Nine EPT evaluators were selected to evaluate certain components of the drill. Three of the evaluators were previously trained in emergency preparedness and had evaluated several drills previously. Two were medical students who had worked on the planning and implementation of the specific drill. The remaining four were selected from the Emergency Medicine residents based on availability. They received the evaluation materials and forms one month prior to the event and underwent a one-day training session on the nature of the exercise protocol, public health recommendations, and the use of the forms.
Following the drill, a group debriefing (“hotwash”) session was performed with the KTU staff, the mock patients, the Emergency Command Center staff, the drill controllers, and the drill evaluators. Verbal and written feedback was elicited during the debriefing session. The 62 participants were given a brief evaluation concerning the drill (Figure 1). This post-drill evaluation contained 15 questions based on a Likert scale of one to five (Poor to Excellent) and was intended to evaluate the participants’ perception of the drill and their confidence and understanding of the ICS and the evacuation plan in general.
Post-Drill Evaluation.
Results
The post-drill evaluation demonstrated that the respondents considered the drill grossly successful. The majority of responses were positive to most questions. Responses were considered positive if they were scored a four (very good) or five (excellent) on the Likert scale. The full responses of the drill participants can be seen in Table 1, but some excerpts include:
∙ 46/58 (79%) responded positively about accomplishing their defined roles during the drill;
∙ 34/47 (72%) responded positively that the preceding training prepared them adequately for the drill;
∙ 27/51 (53%) responded positively about delays in the evacuation; and
∙ 28/55 (51%) responded positively about transporting the patients.
Evaluation Responses
These post-drill evaluation results came in stark contrast to the observations of the EPT. While the participants may have considered the drill a success, EPT members perceived a number of challenges to that perception, such as observed staff reticence, lack of training retention, and shortfalls of leadership. While the drill was not geared to objectively measure these challenges, the impact of these challenges on the drill was noticeable and is worthy of discussion.
Discussion
When the drill was announced, there was obvious participant reticence. There was a lack of perceived urgency and motivation by the KTU clinical staff to respond and participate in the drill. This was particularly true of the nursing and ancillary staff. Approximately 10 minutes passed before the staff actually began to “buy-in” and engage in the drill, which led to delays in implementing ICS and preparing for the patient evacuation.
This observed reticence may be attributed to several factors. While the actual patient census for the KTU was relatively low on the day of the drill, the staff did still have clinical responsibilities to attend to while participating in the drill.Reference Rubin 7 There was a general lack of enthusiasm in participating in any additional tasks beyond the ones mandated by the staff’s position. In addition, a trusted agent in a senior staff position was utilized to motivate the staff and emphasize the importance of the drill. However, this agent was new to the institution and it was unclear whether they had the full support of the unit staff at the time of the drill.
The EPT also observed a considerable lack of training retention during the drill. The KTU staff forgot how to use ICS, how to access the various Job Sheets for the ICS roles, how to safely use the portable evacuation sleds, and even how to use the handheld radio devices. Some of this was anticipated based on the staff’s observed attitudes toward the initial training, which were well-attended but had participants with a wide range of enthusiasm and interest in the subject material.
Lastly, shortfalls in leadership were observed during the drill. Senior clinical staff, particularly nurses and physicians, were reluctant to take initiative during the drill. This was partially expected, as the nurse supervisor for the KTU was on extended leave before the drill and not present. However, other staff members were expected to fill the leadership void, but this did not occur. Beyond creating an environment of uncertainty during the drill, senior staff were also hesitant to allow more motivated junior staff to step up and fill the leadership void as the event simulated required. Eventually the unit clerk, who is usually tasked with secretarial duties, took responsibility in organizing the evacuation process and tracking patients.
Beyond these three challenges, examination of the objective evaluation results and the EPT observations also demonstrated a marked difference in perception between the two. The evaluations completed by the KTU staff and participants were largely positive about their performance in the exercise, while the EPT identified several serious flaws. Likert scale surveys are at-risk for response and acquiescence biases and it is possible the participants’ responses were influenced by their desire for the drill to be perceived as successful.Reference Ray 8 , Reference Smith 9 Revisiting the evaluation after the debriefing session revealed some potentially complicated wording in the questions that may have led to some confusion as well.
Limitations
Limitations of the study, beyond the previously discussed potential biases in Likert scale surveys, included inconsistently answered questions in the post-drill evaluation, resulting in different denominators of replies. The post-drill evaluation was not designed to measure the EPT’s observations, and surveying the EPT members may have helped illustrate the challenges in a more objective fashion. Lastly, the wording of certain questions in the post-drill evaluation may have caused some confusion among respondents and affected their answers.
Conclusion
The vertical evacuation of a hospital is a complicated and potentially dangerous procedure that requires extensive staff training and drilling. A drill evaluating a kidney transplant unit’s preparation for evacuation demonstrated several significant challenges, including staff reluctance, poor training retention, and lack of leadership. A disconnect between the staff’s perception of the drill and the evaluators also became apparent. These challenges demonstrate the need for further training and drills to clarify the shortcomings of current trainings and protocols as well as the necessity for the continuous development of decisive and thorough evacuation protocols.
Supplementary Materials
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X18001097
