Introduction
A disaster represents “an occurrence, whether or not due to natural causes, that causes loss of life, injury, distress or danger to people, or loss of, or damage to property.” 1 Australia traditionally has faced disasters, from widespread flooding and devastating bushfires to tropical cyclones, but the threat of terrorism targeting Australian nationals in South East Asia and on the Australian mainland is becoming increasingly likely.Reference Smith 2 , Reference Caldicott and Edwards 3
Despite an increase in Australia's level of disaster preparedness following terrorist events with high media coverage in Tokyo, New York, London, and Bali, the overwhelming consensus is that Australia is not well prepared to cope with a disaster.Reference Smith 2 - Reference Edwards, Caldicott and Aitken 6 The most important obstacles to disaster preparedness in Australia include: insufficient funding; inadequate educational programs for all personnel (both medical and non-medical), especially those in receiving hospitals; a lack of available technology including appropriate personal protective equipment; and the absence of national preparedness guidelines.Reference Barbera, Yeatts and MacIntyre 7
The Commonwealth Government Disaster Response Plan 8 was developed as an agreement for a coordinated response providing physical assistance to all Australian states and territories during a disaster from any cause. While this document stipulates when and how the Australian government will respond, currently there are only generalized minimum standards of disaster preparedness for hospitals. 9 Without detailed guidelines as to the level of preparedness expected of a facility, disaster response coordinators are unable to apply for, obtain, or allocate resources to the areas in which they are needed. For workable guidelines to be developed, they must be based on realistic capabilities for each area and individual hospital. A recent literature review found that only one Australian hospital has published the results of research into their current preparedness levels.Reference Bartley, Stella and Walsh 10
The objective of this study was to establish baseline levels of hospital staff disaster knowledge and preparedness, and willingness to respond in-hospital to an external disaster, with the aim of identifying learning areas to be addressed in appropriate teaching exercises. By determining existing knowledge and preparedness, stringent guidelines based on realistic expectations of individual hospital capabilities can be developed to facilitate a more disaster-ready Australia.
Methods
Setting and Participants
This study was conducted at St Vincent's Hospital in Darlinghurst, Australia, a public, tertiary, teaching hospital with a 35-bed Emergency Department (ED) equipped with standard disaster equipment, chemical, biological and radiological (CBR) equipment, and a decontamination facility. The hospital has not yet responded to an external disaster. St Vincent's Hospital has declared to New South Wales Health its capability to deploy two health response teams should a disaster occur in which the hospital will not be receiving disaster patients. The hospital has a detailed external disaster response plan that incorporates the entire campus including St Vincent's Private Hospital.
Participants for this study were recruited during staff meetings, in-service programs, education days, and by written request during an eight-week period. Meetings were timed to capture staff working all days of the week; where this was not possible, hard copies of the questionnaire with a return envelope were delivered to prospective subjects. Specific key stakeholders targeted included the hospital's incident commander and section chiefs. Participants were recruited from departments that would be required to respond to an external disaster as per the St Vincent's Hospital Emergency Operations (Surge) Plan, including the Executive, Administration, and Security Departments, as well as those involved in routine patient care (both ED and non-ED staff).
This study was approved by the Human Research and Ethics Committees at St Vincent's Hospital, Darlinghurst, and the University of New South Wales. Anonymity was maintained and return of the questionnaire was considered to be consent for participation.
Data Collection
A questionnaire was constructed using models targeting knowledge and perception of preparedness by Bartley, Stella and Walsh,Reference Bartley, Stella and Walsh 10 and Considine and Mitchell,Reference Considine and Mitchell 11 respectively (Figure 1). The questionnaire was used to gather demographic information including the respondent's department of work and number of years at St Vincent's, as well as experience in disaster training, simulation, and incidents. It consisted of the following 10 yes/no tick box and short-answer disaster-preparedness knowledge questions:
-
1. Is an external disaster a Code Brown?
-
2. Does St Vincent's Hospital have an external disaster plan?
-
3. If yes, is there a copy of this plan in your department?
-
4. Can you locate the plan within your department?
-
5. Can you locate the plan on the St Vincent's local computer network?
-
6. Does the Incident Control Team assemble in a designated area in a disaster?
-
7. If yes, what is the location?
-
8. Does a back-up communication network exist within the hospital?
-
9. Does the disaster plan contain job cards explaining staff roles?
-
10. If yes, what is your designated role in a disaster response?
In addition, four preparedness perception and willingness questions were included in the survey.
Respondents’ perceptions of personal and departmental preparedness were evaluated utilizing a five point Likert scale (0 = Not at all; 1 = Not really; 2 = Not sure; 3 = Well; and 4 = Very well). Respondents’ willingness to participate in conventional and CBR incidents also was evaluated utilizing a Likert scale (0 = Not at all; 1 = Not really; 2 = Unsure; 3 = Somewhat; and 4 = Very). Emergency Department physicians and the hospital's Trauma Service manager piloted the questionnaire and provided feedback. A revised questionnaire subsequently was distributed to participants individually or via the respective department head or unit manager.
The questionnaire was distributed to 200 staff members employed at St Vincent's Hospital, including Executive, Administration, Security, Pharmacy, Physiotherapy and Medical Imaging Departments as well as non-ED and ED nursing and ED medical staff, during an 8-week period in July and August of 2010.
Analysis
Each correct answer to the 10 disaster knowledge questions received a score of one. As the St Vincent's Hospital Emergency Operations (Surge) Plan was undergoing revision at the time the questionnaire was completed by staff, no paper copies of the plan were available in the departments. For this reason, results from Question 4 (Can you locate the plan within your department?) were excluded from study analysis. Data from the knowledge questions were entered into SPSS v18.0 (Statistical Package for the Social Sciences, Chicago, Illinois) to produce descriptive statistics, which were examined for disaster-specific education level and prior participation in simulation drills.
Data from the Likert scale-rated questions were entered into Microsoft Excel 2007 (Microsoft Inc., Redmond, Washington) and examined for frequency distribution. Excel also was used to perform a multiple linear regression analysis. Results with a P value ≤.05 were considered to be statistically significant.
For ease of analysis, data from all non-ED nursing staff (i.e., from the Intensive Care Unit and Outpatient Departments) were combined into one group, while data from all administration staff (i.e., from the Emergency and Outpatient Departments) were also combined.
Results
Of the 200 questionnaires distributed, 140 were completed and returned, representing a 70% response rate. The majority of respondents from any one group were ED nurses (n = 42, 30%) but overall, more non-ED staff (n = 81, 57.9%) were represented in the study population compared with ED staff (n = 59, 42.1%) (Table 1).
Demographic Data of Study Participants
Abbreviation: ED, Emergency Department.
Disaster Education and Experience
Of the 140 respondents, 83 (59.3%) reported having had previous disaster education. Of these, 33 (50.8%) had been employed at St Vincent's Hospital >5 years. The staff of the Physiotherapy Department were the least trained with 10 (83.3%) having had no disaster training. This was followed by non-ED nursing staff (n = 14, 66.7%), and staff from administration (n = 8, 61.5%). The security officers were the most trained with 100% (n = 6) having had some education specific to disasters.
The time elapsed since the last disaster-specific education session varied greatly among respondents, with an overall median time of 1-2 years (range 0-10 years, Table 2). Staff with prior disaster education had statistically significant higher scores in the disaster preparedness knowledge section compared with those who had no prior disaster education (4.74; SD = 0.99 vs. 3.56; SD = 0.61, P = <.05). Those respondents with prior disaster education also perceived themselves to be better prepared than those who had no disaster-specific education (1.74 vs. 1.38, respectively, P = .05). However, previous disaster-specific education had no significant effect on the respondents’ perceptions of departmental preparedness (P = .47) or willingness to participate in the event of a disaster (P = .94).
Time Lapsed Since Study Participants’ Last Disaster-Specific Training
Abbreviation: ED, Emergency Department.
A total of 53 (37.9%) respondents reported having attended a disaster simulation drill, either real-time or table-top; 45 (84.9%) of these individuals also reported having had previous education specific to disasters. Of those staff members who had participated in a simulation drill, 28 (52.8%) had been employed by St Vincent's Hospital for >5 years, 13 (24.5%) were ED nurses, 11 (20.8%) were ED medical staff, and eight (15.1%) were from administration. Only two staff members (3.8%) from the Physiotherapy Department had disaster simulation experience. All four executive staff members (100%) who completed the questionnaire had attended a simulation drill in the past. When compared with respondents who had no disaster simulation experience, staff members with simulation experience had a statistically significant higher disaster preparedness knowledge score (4.93; SD = 1 vs. 3.56; SD = 0.62; P = < .05). Simulation experience also was associated with a statistically significant greater perception of personal preparedness compared with those without simulation experience (2.06 vs. 1.38, respectively, P = < .05), but had no statistically significant effect on respondents’ perceptions of departmental preparedness (P = .55) or their willingness to participate in an event (P = .19).
Of the 140 respondents, 18 (12.9%) individuals from five different departments previously had been involved in a response to an actual disaster, (Table 3). Prior actual disaster response experience had no effect statistically on respondents’ disaster preparedness knowledge score (P = .70), perceived personal (P = .83) or departmental (P = .10) preparedness, or willingness to respond to a disaster (P = .58).
Characteristics of Study Participants Who Had Responded to an Actual Disaster
Abbreviation: ED, Emergency Department.
Disaster Knowledge
Emergency Department staff had an average score of 3.57 (SD = 0.54), higher than Administration (2.62, SD = 0.73) and Physiotherapy (2.82, SD = 0.73). Non-ED nurses (3.78, SD = 0.63), Pharmacy Department (3.98, SD = 0.64) and Executive Department (3.63, SD = 0.09) staff averaged scores in a similar range to staff from the ED. Overall, Security staff performed the best in the preparedness knowledge section with the average score of 5.86 (SD = 0.87; P = < .05). Medical Imaging staff scored the worst, with an average score of 1.88 (SD = 0.8; P = < .05). Compared with non-ED staff, ED staff scored more correctly in all but one of the 10 preparedness knowledge questions (Figure 1). Both prior education and participation in a simulation drill were associated with an increased score in the preparedness knowledge section (P = .05). However, there was no statistically significant difference between the scores of staff who had prior disaster experience and those without experience during a disaster (2.67, SD = 1.13 vs. 3.56, SD = 0.62, respectively; P = .08).
Disaster Knowledge Test Results. Q1: Is an external disaster a Code Brown? Q2: Does St Vincent's Hospital have an external disaster plan? Q3: If yes, is there a copy of this plan in your department? Q4: Can you locate the plan within your department? Q5: Can you locate the plan on the St Vincent's local computer network ? Q6: Does the Incident Control Team assemble in a designated area in a disaster? Q7: If yes, what is the location? Q8: Does a back-up communication network exist within the hospital? Q9: Does the disaster plan contain job cards explaining staff roles? Q10: If yes, what is your designated role in a disaster response? Abbreviation: Q, question number.
Eighty-five (59%) of the 140 respondents correctly identified that an external disaster is termed a “Code Brown” (Question 1). Twenty-nine (69%) of the ED nurses correctly answered this question compared with only eight (47.1%) of ED medical staff. All four (100%) of the executive staff respondents, but only three (37.5%) of the medical imaging staff answered this question correctly.
Questions 2 through 5 referred to the hospital's disaster plan and its location within the departments and on the hospital's local computer network. A total of 132 (94.3%) staff completing the questionnaire knew that St Vincent's Hospital has an external disaster plan, and 64 (48.6%) could locate the plan on the intranet.
In response to Question 6 regarding the Incident Control Team and its assembly location, 82 (58.6%) of the respondents knew that the Incident Control Team assembled in a designated area during the disaster response, but only 11 (7.9%) could correctly identify the location of the Incident Control Centre (Question 7). All four of the Executive staff respondents knew their assembly location.
In response to Question 8 concerning the presence of a back-up communication system within the hospital, 110 (78.6%) of all the respondents knew that back-up systems are present at St Vincent's. By department, the staff that performed the best in response to this question were from Security (n = 6, 100%). In contrast, staff from the Medical Imaging Department scored the worst on this question, with only three (37.5%) staff members answering this question correctly.
The final two questions referred to individual staff members’ specific roles within the disaster response. Sixty-four (45.7%) of the respondents knew that the hospital disaster plan included job cards detailing the role of each staff member. Of the 16 (11.4%) staff members who correctly identified their designated role in a disaster, 56.3% (n = 9) had been employed by St Vincent's Hospital >5 years.
Disaster Preparedness
The majority of respondents (104, 74.3%) reported a neutral to negative sense (“not at all,” “not really,” or “not sure”) of their personal preparedness (Figure 2a). All six security staff (100%) rated themselves as very well prepared. Two members (50%) of the executive staff reported positive confidence levels in their individual preparedness. In contrast, 11 (91.7%) of the physiotherapy staff, and 15 (71.4%) non-ED nursing staff members responded negatively (“not at all,” “not really”) regarding their level of preparedness. Those respondents who had prior disaster education or who had participated in a disaster simulation exercise reported significantly higher levels of perceived individual preparedness (P = .05; P = <.05, respectively) than did staff with no prior disaster education or simulation experience. There was no association with having had actual disaster experience and perceived level of disaster preparedness (P = .80).
Respondents’ Perceptions of Their Own (a) and of Their Department's (b) Level of Preparedness
With regard to participants’ perception of their department's level of disaster preparedness, the majority of respondents (n = 71, 50.7%) were “not sure” of the level of preparedness of their department (Figure 2b). A total of 23 (16.4%) respondents reported negative (“not at all,” “not really,” or “not sure”) perceptions of their department's preparedness level; the remaining 42 staff (30%) considered their department to be “well” or “very well” prepared for a disaster. Security staff rated their department most positively, with four respondents (66.7%) indicating their department was “well” or “very well” prepared. This was followed by ratings from staff of the Pharmacy Department with nine (52.9%) positive responses. The department that was perceived to be least prepared by its staff was the Physiotherapy Department, with four (33.3%) of its staff members responding negatively. The respondents’ level of disaster education, disaster simulation, or actual disaster experience had no effects on their perception of their department's level of preparedness (P = .47; P = .55; P = .10, respectively).
Willingness to Participate
The majority of respondents reported some (“very” or “somewhat”) willingness to participate in either a conventional incident involving burns or physical trauma (n = 118, 84.3%; Figure 3a), or a chemical, biological, or radiological (CBR) incident (n = 88, 62.9%; Figure 3b). By department, staff from the Security Department and ED medical staff were the most willing to participate in a conventional disaster, with 100% (n = 6, n = 17, respectively) responding positively. All six Security Department staff participants and 14 (82.4%) of the ED medical staff also indicated that they would be “very” or “somewhat” willing to respond in-hospital to an external CBR incident. The staff least willing to participate in a conventional incident were three Administration staff participants (23.1%), and the staff least likely to respond to a CBR incident were two (25%) staff from the Medical Imaging Department. Prior disaster education, disaster simulation, or actual disaster experience had no effect on a respondent's willingness to participate in either a conventional incident (P = .94; P = .19; P = .58, respectively) or a CBR incident (P = .17; P = .72; P = .57, respectively).
Respondents’ Willingness to Participate During a Disaster Caused by a Conventional Incident (a) and a Chemical, Biological, or Radiological Incident (b)
Discussion
There is very little published data on individual hospital preparedness and capabilities in Australia, with the majority of studies focusing on prehospital emergency services response.Reference Caldicott and Edwards 3 , Reference Edwards, Caldicott, Eliseo and Pearce 5 Overall, the staff surveyed in this study compare favorably with those surveyed by Bartley, Stella and Walsh in Geelong, Victoria, Australia.Reference Bartley, Stella and Walsh 10 Nonetheless, Australian hospitals continue to lag behind international disaster preparedness standards.Reference Brennan, Sage and Simpson 12 , Reference Hsu, Jenckes and Catlett 13 Current Australian disaster standards require that the “frequency of training in hospitals should be such as to ensure appropriate levels of preparedness and response are maintained.”Reference Hsu, Jenckes and Catlett 13 The lack of prescriptive national guidelines for hospital preparedness may be attributable to the variable levels of preparedness across state-run hospitals. Nationally enforced standards of disaster preparedness are required to determine the level of knowledge expected of hospital staff. Without such regulations, budget restraints will force disaster preparedness to a low priority, and improvement will not occur. Furthermore, without guidelines specifying what is expected of hospitals and their staff, measuring individual hospital capabilities is redundant as only arbitrary observations can be made, and inter-hospital comparisons are difficult.
The results of this study describe existing baseline levels of disaster education and training, disaster preparedness knowledge, willingness to respond, and perceptions of preparedness of medical and non-medical personnel working in an Australian urban trauma center likely to participate during the in-hospital response to a major incident in the Sydney central business district. The high response rate yielded a sufficient sample size to enable reliable statistical inference. Despite the increased focus on disaster preparedness documented in Australia since 2001,Reference Smith 2 - Reference Edwards, Caldicott and Aitken 6 hospital staff remain relatively untrained and inexperienced.
Unsurprisingly, those respondents who had received disaster-specific education scored higher marks in the knowledge section, and reported a higher self-perception of preparedness, similar to findings from studies conducted both in AustraliaReference Bartley, Stella and Walsh 10 , Reference Bartley, Fisher and Stella 14 , Reference Duong 15 and internationally.Reference Brennan, Sage and Simpson 12 - Reference Hsu, Jenckes and Catlett 13 , Reference Lenaghan, Smith and Gangahar 16 - Reference Chapman and Arbon 19 It is interesting to note that having prior disaster education and a greater sense of individual preparedness had no effect on the individual's willingness to respond to a disaster. This is similar to the findings of Alexander and Wynia who suggested that despite the fact that only 21% of physicians surveyed felt prepared to respond to a CBR disaster, 80% were willing to respond.Reference Alexander and Wynia 17 Predictably, participants in this study were much less willing to participate in a disaster involving CBR weapons compared to a conventional incident. The most commonly cited reason for this in the present study was related to the associated increased personal risk.
This study reaffirms the positive influences of disaster-specific education and simulation drills on disaster knowledge and individual perceptions of preparedness. These results strongly suggest that education and participation in simulation drills in the future can help to improve staff disaster knowledge, and ultimately, patient outcomes.
The St Vincent's Hospital Emergency Operations (Surge) Plan emphasizes the need for a hospital-wide approach to disasters in order to achieve the best possible patient outcomes as measured by morbidity and mortality. This study differs from those already published in that it investigates the disaster training, knowledge, preparedness, and willingness of medical and non-medical staff, working in both ED and non-ED settings. The results demonstrate that, compared with non-medical personnel, medical staff perceive themselves to be more prepared for a disaster, a finding that, in part, could be related to familiarity with some of the tasks required during a disaster. Regular, job-specific training and greater involvement in simulation drills for non-medical staff could help to rectify this gap.
Staff from the Executive and Security Departments scored better than did staff in other departments in the knowledge and preparedness sections, which likely demonstrates the presence of skewed levels of knowledge according to the hierarchy of the chain of command. Not only does this represent inadequate education for front-line staff, but also a failure of information dissemination down the chain of command.
Very few staff were able to identify their individual role during the disaster response. This finding demonstrates the disconnect between the general knowledge of staff that results from their prior disaster education and simulation experience with the situational knowledge that is required to respond within the hospital context. As a result, future educational programs and simulation exercises not only should encompass general disaster knowledge, but also site-specific learning and application of theoretical principles.
This study was used to demonstrate baseline disaster knowledge and preparedness of staff at St Vincent's Hospital prior to the implementation of a training program and simulation exercise. Since then, some important steps have been taken to improve the level of preparedness. Subsequently, the hospital has tested its disaster plan in an Emergotrain exercise, which highlighted existing problems including issues with communication, the “Code Brown” notification process, familiarity with the chain of command, and overloading of individual departments. With this feedback, detailed plans for all stakeholder departments have been drafted and department-specific educational programs have commenced. In addition, the disaster preparedness profile within the hospital has increased to include a Medical Grand Rounds presentation on disaster preparedness, and a Major Incident Medical Management and Support (MIMMS) course for team members.
Prescriptive national guidelines outlining the minimum acceptable frequency of training for hospital staff, as well as the frequency of review and evaluation of the disaster plan through simulation exercises are suggested. Following systems currently used in EnglandReference Brennan, Sage and Simpson 12 and America,Reference Hsu, Jenckes and Catlett 13 it is recommended that all Australian hospitals regularly test their disaster plans based on accurate hazard-vulnerability analysis to address the most likely threats, both natural and man-made. Once the disaster plans are in place, regular staff education using lectures, videos, table-top exercises, and real-time community-wide drills should be initiated in a timely and cost-effective way. Regular auditing by groups, such as Emergency Management Australia, may assist hospitals to maintain adequate levels of readiness as measured through perceived preparedness and staff knowledge tests.
Limitations
Some limitations are recognized in the methodology of this study. The opportunistic recruitment process (convenience sample) is subject to selection bias. Unfortunately, not all major departments that play a role in the St Vincent's Hospital disaster response were able to participate in this study. Particularly missing from this study are Intensive Care Unit medical staff, anesthetists, surgeons, and members of the Mental Health Team. The small sample size yielded high enough power to be significant overall, but this is diminished in the small group analysis.
The self-reported nature of the data is open to a socially desirable response bias. Poor disaster knowledge prior to the administration of the questionnaire meant that some staff may have misinterpreted questions that were scored as incorrect.
As most questionnaires in this study were completed by staff while on duty, time constraints meant that a more detailed questionnaire addressing theoretical knowledge was not possible. For example, knowing where to find the plan does not mean that respondents have read or understood the plan. Ideally, a distinction would be made between those who had simply skimmed the plan and those who had read the plan thoroughly. Similarly, the knowledge that backup communication methods exist within the hospital does not mean that staff members know when or how to use these alternative means. Ideally, a post-intervention questionnaire assessing the knowledge, preparedness, and willingness of staff would be administered after department-specific disaster education. This was not part of this study due to time constraints.
Conclusions
This study demonstrates that front-line medical and non-medical hospital staff remain poorly trained and relatively unprepared to respond to a major disaster. While this is disappointing, Australia has very little exposure to disasters, and therefore, cannot reasonably be expected to compare favorably to health care providers in those countries in which disasters occur frequently.
Despite perceiving themselves as unprepared, the majority of staff participating in this study expressed a willingness to participate during a disaster from both a CBR and a conventional incident. In view of these findings, it is imperative that staff receive the adequate knowledge and skills necessary to improve outcomes following a disaster. Regular, department-specific training should be made available to all hospital staff, medical and non-medical, both in the Emergency Department and other areas, through a combination of traditional lecture-style education and participation in simulation drills. Prescriptive national guidelines for levels of hospital preparedness will enable hospital and state disaster coordinators to create a more disaster-ready Australia.
Abbreviations
- CBR:
-
chemical, biological or radiological (weapons)
- ED:
-
Emergency Department
