Introduction
During any incident that results in patient surge, such as a natural or manmade disaster, hospitals should provide essential medical care to the victims.Reference Lynn, Gurr and Memon1, 2 A well-established preparedness program is a prerequisite for an effective response of health and medical systems, including hospitals, to disasters.3-Reference Djalali, Castren, Hosseinijenab, Khatib, Ohlen and Kurland6
Hospital preparedness encompasses those actions, programs, and systems developed and implemented before a major incident to improve the capability and capacity of the hospital to respond to disasters and emergencies.7 Hospital response includes immediate and ongoing activities, programs, and services to provide essential medical care for casualties of an incident, and also minimizes negative effects of the incident on critical services of the hospital.3, 7-10
Even for a well-prepared hospital, giving an effective response to a disaster is a complex challenge.2 The assessment of hospital disaster preparedness and response performance is a way to find and remove possible gaps and weaknesses in the hospital disaster management functions and effectiveness.Reference Djalali, Castren and Khankeh9, Reference Adini, Goldberg and Laor11, Reference Kaji, Langford and Lewis12 However, a valid and standardized evaluation methodology is required to assess both the level of preparedness and the efficacy of response performance, and the correlation between them.Reference Djalali, Castren, Hosseinijenab, Khatib, Ohlen and Kurland6, Reference Adini, Goldberg and Laor11-Reference McCarthy, Brewster and Hsu14
Various indicators and methods have been created by researchers and authorities to evaluate either preparedness level or response performance of a hospital disaster management system,2, 3, Reference Djalali, Castren, Hosseinijenab, Khatib, Ohlen and Kurland6, Reference Adini, Goldberg and Laor11, Reference Jenkins, Kelen and Sauer13, Reference Higgins, Wainright and Liu15-Reference Niska and Shimizu20 but there is no consensus on valid and standardized methods and tools. Also, it is unclear whether the evaluation tools can reliably predict the effectiveness of a hospital function during an actual disaster.Reference Kaji, Langford and Lewis12, Reference Lazar, Cagliuso and Gebbie21, Reference Kollek and Cwinn22
Even though some studies have evaluated the correlation between level of hospital preparedness and response in a simulated disaster, only few functional elements or groups of hazards have been assessed underlining, once again, the lack of internationally-accepted standards for preparedness and response performance measures.Reference Kaji, Langford and Lewis12, Reference Adini, Goldberg and Cohen23-Reference Adini, Goldberg and Cohen25
The Hospital Safety Index (HSI), an all-hazards checklist developed by the World Health Organization (WHO), is a standardized, internationally-accepted method.3 It evaluates the functional capacity, which is defined as the level of preparedness of hospital staff for disasters, as well as the level of the implementation of a hospital disaster plan.3 A modification of a standardized Swedish methodReference Ruter, Nilsson and Vilkström18, Reference Ruter, Ortenwall and Vikström26, Reference Lundberg, Jonsson and Vikström27 has been used by Centro di Ricerca Interdipartimentale in Medicina d'Emergenza e dei Disastri ed Informatica applicata alla pratica Medicina (CRIMEDIM) in full-scale disaster exercises and standardized with respect to a new scoring method.28 In both the Swedish and CRIMEDIM methods, the main focus is on the command and control performance of a hospital during a simulated or real disaster.
The aim of this pilot study was to test the association between the level of preparedness and the level of response performance during a full-scale hospital exercise. The results of this study could be used to design and conduct a correlative study of preparedness indicators as predictors for response efficacy of hospital functions during disasters.
Methods
Setting
To conduct this study, a hospital in the Piedmont region of Italy was chosen. It is a general hospital with 300 beds, including 20 beds in the emergency department, 10 beds in the intensive care unit, and 36 beds in the surgery department. The hospital has eight surgery operating rooms. In a normal situation, the hospital occupancy rate is, on average, 75%.
A full-scale exercise was conducted in the same province of the hospital during the residential course of the European Master in Disaster Medicine (EMDM).29 The exercise scenario was an explosion of a gas station. The victims were triaged at the scene and then transported to the hospital. The hospital triage of victims was conducted using the Simple Triage and Rapid Transport (START) method.
The hospital was evaluated twice; a preparedness evaluation was done three days before the exercise, and a response evaluation was conducted during the exercise, on May 23, 2013.
Evaluation of Preparedness
The evaluation of the selected hospital, in terms of disaster preparedness, was conducted on May 20, 2013. It was done by a group of three evaluators who were experts in emergency and disaster medicine. They were medical doctors who also had master's degrees in disaster medicine. The Chief Doctor of the Emergency Medicine Department, who was a member of the Hospital Disaster Management Committee, provided all requested information for the evaluation group.
The checklist for the functional capacity module of the HSI was used for preparedness evaluation (Appendix).3 This module consists of 61 elements that evaluate five parts of hospital preparedness:
1. Organization of the Hospital Disaster Committee and the Emergency Operations Center
2. Operational plan for internal or/and external disasters
3. Contingency plans for medical treatment in disasters
4. Plans for the operation, preventive maintenance, and restoration of critical services and
5. Availability of medicines, supplies, instruments, and equipment in emergencies.
Accordance to the HSI evaluation guideline, the level of each element was determined, by the evaluators in consensus, as high, average, or low.3 The value of each level was also scored using the HSI evaluation guideline as 1, 0.5, or 0, respectively.3
Evaluation of Response Performance
Performance is a measure of how well an activity is done and can be measured in terms of accuracy, time, and quality.Reference Djalali, Castren, Hosseinijenab, Khatib, Ohlen and Kurland6, 30 In this study, the response performance of the “command and control” function of the hospital was evaluated by a group of three evaluators who were medical doctors and experts in disaster medicine.
To evaluate the response performance, nine semiquantitative performance indicators with respect to command and control functions, were used (Table 1).
Response Performance Indicators on the Basis of the CRIMEDIM Method, a Modification of the Method Developed by a Swedish TeamReference Ruter, Nilsson and Vilkström18, Reference Ruter, Ortenwall and Vikström26, Reference Lundberg, Jonsson and Vikström27
Abbreviation: CRIMEDIM, Centro di Ricerca Interdipartimentale in Medicina d'Emergenza e dei Disastri ed Informatica applicata alla pratica Medicina.
The main source of these performance indicators was The Center for Teaching and Research in Disaster Medicine and Traumatology at the University of Linköping, Sweden,31 but the scoring method was modified by CRIMEDIM in scoring method. The performance indicator was scored with respect to quality and time, as 0, 1, 2, or 3 as follows:
0: Incorrect regardless of time;
1: Incomplete and not on time;
2: Correctly performed but not on time OR Incomplete on time; and
3: Correctly performed on time.
For each action, there is a standardized time (Table 1); however, the quality scoring was based on the evaluators’ consensus. The indicators were entered into an online system, the Disaster Simulation Suite,32 which allowed the evaluators to use tablets and mobile devices to enter the data directly into the integrated exercise management platform. The evaluators were familiar with the system.
Ethical Issues
The evaluation of the hospital was with permission from the authorities. Also, this study was performed in accordance with WHO's recommendation that the participating hospital's name and exact location are to be treated as confidential and not discussed with outside parties.3
Results
In this study, the preparedness score of the chosen hospital was 36 out of 61 (59%). The response performance score of the hospital during the exercise was 19 out of 27 (70%).
During the exercise, first responders triaged the victims at the scene and sent 61 casualties, with various triage priorities, to this hospital. The hospital staff performed START triage while they received the casualties, which was 90% correct for the yellow group (Priority II) and 100% for the green group (Priority III). Only one red patient and one black patient were recognized by the hospital staff.
The average time of first triage for the yellow and green groups was two and eight minutes, respectively. Also, the average time for bed/place allocation in the emergency department for yellow and green groups was one and five minutes, respectively.
On the basis of preparedness evaluation, 49% of the hospital elements were at a high level of preparedness, 15% of the elements were at a moderate level, and 36% were at a low level of preparedness.
Considering different submodules of the functional capacity, preparedness level (out of 20% for each submodule) ranged from 1.8% to 19% (Figure 1). The lowest preparedness level was assigned to “organization of the hospital disaster committee and the emergency operations center,” and the highest level of preparedness was seen in “availability of medicines, supplies, instruments, and other equipment for use in emergency.”
Preparedness Level, as Measured by Functional Capacity, of the Evaluated Hospital Comparing to a Standard Safe Hospital.
Submodule 1: Organization of the Hospital Disaster Committee and the Emergency Operations Centre
Submodule 2: Operational plan for internal or/and external disasters
Submodule 3: Contingency plans for medical treatment in disasters
Submodule 4: Plans for the operation, preventive maintenance, and restoration of critical service
Submodule 5: Availability of medicines, supplies, instruments, and other equipment for use in emergency
On the basis of response evaluation, three out of nine activities were performed both correctly and on time (score 3), as presented in Table 2. Also, five activities were conducted correctly but not on time (Table 2). However, a press release was not issued by the hospital command staff during the exercise.
Response Performance, as Measured by CRIMEDIM Score, of the Evaluated Hospital During the Full-scale Exercise with the Value of Each Activity Defined as: 0, Incorrect Regardless of Time; 1, Incomplete and Not on Time; 2, Correctly Performed but Not on Time OR Incomplete on Time; and 3, Correctly Performed on Time
Abbreviation: CRIMEDIM, Centro di Ricerca Interdipartimentale in Medicina d'Emergenza e dei Disastri ed Informatica applicata alla pratica Medicina.
Discussion
In the evaluated hospital, the general preparedness condition and the response performance were close, but different (59% and 70%, respectively).
Previous studies have shown different results. Some report a significant relationship between hospital disaster preparedness and response indicators, but they evaluated only procedures or infrastructure elements of hospitals with respect to response to pandemic flu.Reference Adini, Goldberg and Cohen23, Reference Adini, Goldberg and Cohen25 Others, with more comprehensive evaluation, have shown no significant relationship between preparedness and response performance.Reference Kaji, Langford and Lewis12, Reference Valesky, Roblin and Patel24
Similar studies in a group of hospitals, using the method and tools of the current study, may identify and measure any association between preparedness and response indicators. Also, a comprehensive evaluation of the response performance, including all functions of hospitals during disasters, is suggested.
If reliable preparedness elements predict performance of hospital response to disasters, that will help the health authorities to enhance readiness of hospitals for disasters, which will probably diminish mortality and morbidity of the victims.
In this study, preparedness of the hospital was not high enough (59%). This result is consistent with previous studies that report no adequate disaster preparedness in Italy hospitals.Reference Giacomet, Tarallo and De Marco33-35 Those studies didn't consider the functionality of measured preparedness during the response phase, but the current study evaluated both preparedness and its functionality, as command and control performance, during a simulated disaster. Outcome-based studies are suggested in the field of hospital disaster management.
In the evaluated hospital, the lowest level of preparedness was seen in the organization of the Hospital Disaster Committee and the Emergency Operations Center. It is supported by a recent study in the L'Aquila earthquake area (Italy).35 On the other hand, availability of medical resources for use in an emergency was at standard level, similar to other high-income countries that have sufficient medical resources to use during disasters.Reference Djalali, Castren and Khankeh9, Reference Kaji and Lewis17, 36
Although the current study is a pilot study, and the main gap or strength in the elements of hospital preparedness in Italy was not examined, the result is based on an international, standardized tool and should be considered by the hospital authorities.
This study showed that the response performance of the evaluated hospital, with respect to command and control, is 70%. Other studies have focused on managerial performance of hospitals during disaster exercises.Reference Djalali, Castren, Hosseinijenab, Khatib, Ohlen and Kurland6, Reference Ruter, Nilsson and Vilkström18, Reference Gryth, Radestad and Nilsson37 Studies by Swedish researchers, with methods and indicators similar to the current study, support the current results.Reference Ruter, Nilsson and Vilkström18, Reference Gryth, Radestad and Nilsson37 Another study showed “fair” performance of command function of hospitals during disaster exercises, but it was conducted using different indicators modified for the hospital incident command system.Reference Djalali, Castren, Hosseinijenab, Khatib, Ohlen and Kurland6
Compared with previous studies, the current study was conducted during a full-scale exercise and also used a group of more precise indicators (a modified method with four options for each indicator). However, other functions of a hospital, in respect of response to disasters, should be considered in future studies.
In this study, the hospital response performance was high with respect to triage accuracy and time.
Hospital preparedness with respect to triage was evaluated in two submodules: procedures for triage in Submodule 2 and resources for triage in Submodule 5 (Appendix).
Both preparedness for triage resources and response performance (triage accuracy and rapidity) were high in this study. This is supported by a previous study that showed a relationship between stockpiles and response performance during a flu drill.Reference Adini, Goldberg and Cohen25
This study was a pilot study involving only one hospital. Although the triage method was standardized and the triage performance was high, the relationship between hospital preparedness and response performance with respect to triage should be taken under consideration by future studies involving a high number of hospitals.
Limitations
The main limitation of this study was the sample size of only one hospital. However, it was a pilot study, and it revealed some executive and scientific gaps and issues for the researchers. Also, two standardized, international tools were used in this study to simultaneously evaluate both preparedness and response functions. The results of this pilot study form a basis for future studies.
The second limitation was the response performance indicators, which were limited to command and control actions. However, the command and control function is the core of the whole response to disasters. Also, the study was conducted during a full-scale exercise. This experience may form a basis for future studies with more comprehensive context, including other hospital functions.
Conclusion
This pilot study showed that it is possible to use standardized evaluations tools, simultaneously, to assess the relationship between preparedness elements and response performance measures. The hospital preparedness condition and response performance were in different levels on the basis of two international evaluation methods. An experimental study including a group of hospitals, also using more comprehensive evaluation tools, should be done to evaluate the correlation between the level of preparedness and the response performance of a hospital, and the impact of hospital disaster planning on the outcome of disasters victims.
Appendix The Functional Capacity Variables as Grouped in Five Submodules
