Outbreaks of communicable respiratory illnesses can lead to epidemics with significant public health effects, depending on the severity of illness and the impact on health care systems. Most recently, the emergence of novel swine-origin influenza A (H1N1) virus in humans has required heightened infection control measures to limit the spread of the infection.Reference Dawood, Jain and Finelli1 Concern about pandemic influenza with a high mortality, similar to the 1918 to 1919 influenza pandemic, has led to preparedness efforts on many levels, even before the current novel influenza A (H1N1) pandemic.Reference Monto and Whitley2
During a widespread respiratory epidemic it is expected that a large proportion of affected individuals will seek medical attention at hospitals, resulting in severe consequences for the health care infrastructure. At the same time, institution of a combination of efforts aimed at limiting spread of the severe acute respiratory syndrome (SARS) virus in hospitals is thought to have prevented further transmission.Reference Loeb3Reference Svoboda, Henry and Shulman4Reference Srinivasan, McDonald and Jernigan5 In addition to enhanced infection control measures, use of an emergency department screening tool and assigning patients to different hospital areas has been part of a set of interventions used to contain the SARS outbreak in several countries.Reference Foo, Tham and Seow6Reference Loutfy, Wallington and Rutledge7Reference Wu, Yang and Wu8 However, evidence-based guidance on successful procedures and practices is lacking.
Determining hospital preparedness for the complex process of responding to an infectious disease event is a difficult task. Simply assessing inventory and personnel resources fails to take into account the wide range of variables that may impact the effort. “Real” events, either hospital-specific or drawn from the experiences of others, usually provide little quantitative or qualitative data from which to draw evidence-based lessons.
The goal of this observational study was to test the ability of hospitals to perform mass screening, isolation, and triage according to a formal (mass screening, isolation, and triage [MSIT]) protocol developed for use by health care facilities responding to an outbreak of a communicable respiratory illness. The main objective of this protocol was to separate patients who are symptomatic for the communicable illness from those who are not through rapid screening and isolation before the patients enter the health care facility to minimize exposure time of potentially infected patients and health care staff to a transmissible infectious agent. The efficacy of the use of the MSIT protocol by hospital staff in correctly identifying and placing drill subjects into a cohort according to case status was tested. The ability of hospital staff to adhere to infection control recommendations for the scenario was also evaluated.
METHODS
Study Setting
Three urban, acute care hospitals in central Brooklyn, New York, participated in this drill: a tertiary care level 1 trauma center with 627 beds and 115 820 annual emergency department (ED) visits that serves mostly low-income, uninsured patients; a university referral center with 376 beds and 60 000 annual ED visits; and a community hospital with 284 beds and 20 127 annual ED visits. The hospitals serve a community of approximately 12 contiguous zip codes in 4 neighborhoods in which more than 1 million people reside. Minorities comprise over 70% of the service-area population.
Development of the MSIT Protocol
A formal protocol for MSIT in hospitals during a respiratory communicable disease outbreak was developed by reviewing pandemic influenza plans and existing protocols for mass screening for other infectious outbreaks (eg, SARS).Reference Ong9 This study was carried out before the 2009 influenza pandemic; therefore, the most current pandemic planning assumptions at that time (spring 2008) were used for planning purposes. A working group, including experts in infectious diseases, infection control, emergency planning, and public health developed the MSIT protocol. While the protocol is adaptable to different scenarios, we chose pandemic influenza for this exercise. Following peer review at a public meeting with regional stakeholders, the working group evaluated and incorporated the suggestions into the final MSIT protocol.
As illustrated in Figure 1, the MSIT protocol specifies a multitiered approach that allows for rapid initial screening for influenza in a mass screening station immediately outside the hospital ED. Patients are separated by influenza case status in physically separate areas (cohort areas) of the hospital ED, with staff dedicated to each area. They are directed to either triage station 1 for case-patients (Figure 1, dark purple) or triage station 2 for patients who do not meet the case definition (Figure 1, light purple). Patients who do not meet the case definition for influenza but have “non-event related” symptoms remain separated from the cohort area of case-patients to avoid exposure to patients with potential influenza. All patients, regardless of infection status and severity of illness, are expected to receive a full evaluation before leaving the ED.
Respiratory Epidemic Hospital Isolation/Evaluation Flowchart.
aMass screening using case definition set in mass screening, isolation, and triage (MSIT) protocol.
bCriteria or specific scoring systems used to determine severity of illness and cut-off point between minor and major illness will depend on characteristics of outbreak and resources.
cVictims with nonsevere (minor) illness not requiring emergency department (ED) resources should be managed in an overflow area.
dSeparate physical space in ED for case-patients; ED resources will be limited in a large outbreak and need to be reserved for patients with severe (major) illness.
eSeparate physical space in ED for victims who do not meet the case definition for influenza.
Drill Description
For this drill, the influenza case definition was adapted from the US Department of Health and Human Services (HHS) pandemic influenza plan: temperature higher than 38°C plus sore throat, cough, or dyspnea; in addition, symptoms of diarrhea (only in children <5 years) and body aches/soreness were included.1011 Patients with a temperature higher than 38°C plus at least 1 of the cited symptoms were considered “symptomatic” for influenza and met the case definition; all others were “asymptomatic” for influenza and did not meet the case definition.
The full-scale exercise was conducted using 345 healthy volunteers (recruited from a wide range of community organizations, with the majority living within a 5-mile radius of the 3 hospitals) to act as arriving patients and/or family members. Volunteer victims were each assigned signs and symptoms with which they were to present to the hospital and that defined their influenza status. Hospital staff members who participated in the drill itself were either on duty at the time of the drill or were recruited by hospital leadership to participate. Stations were operated almost exclusively by ED physicians and registered nurses. Given the drill scenario (ie, nonacute event unfolding over time), the timely recruitment of hospital personnel was felt to reflect “real world” response.
A representative pandemic influenza scenario was developed and presented in stepwise fashion to each of the hospitals through mock communiqués from the New York City (NYC) Department of Health and Mental Hygiene (NYC DOHMH) (see the eAppendix. To emphasize the critical need for transmission precautions in the scenario, antiviral agents and vaccination effective against the pandemic influenza virus were assumed to be unavailable. Key recommendations provided to the hospitals from the NYC DOHMH in health alerts were the preparation for use of the MSIT protocol, which was made available to all hospitals (Figure 1, Figure 2), and, finally, activation of the protocol at their facility. The amount and type of resources used by each facility for the drill were not dictated by the drill coordinators and varied by hospital. On the day of the drill, volunteer patients were distributed to participating hospitals during a 3-hour period. The number of volunteer patients dispatched to each hospital was calculated to simulate the expected additional burden to each ED based on their usual patient load (CDC FluSurge 2.0 software).
Mass Screening/Triage Form.
Study End Points
The primary study end point was correct determination of the drill victims' preset case status by hospital personnel staffing a designated mass screening station using the event-specific mass screening form (Figure 2). A post hoc analysis compared the percentage of correct determinations of case status among hospitals. In addition, the ability to maintain strict, physical separation of cases based on preset patient profiles between the screening and the triage stations was assessed and compared among hospitals. The use of hand hygiene products, donning and removing of gowns, placement of gloves, and placement of face masks, when appropriate, by both staff and case-patients, was also assessed and compared both among hospitals and between screening and triage stations at each hospital. Also measured were distancing a minimum of 3 feet between case-patients and wait times at either of the stations.
An attempt was made to ascertain if compliance with these measures improved or deteriorated over time during the drill. For this purpose, victim presentations were divided by sequence into tertiles reflecting 3 time periods of patient presentation to the mass screening and triage stations: early, middle, and late in the drill. Further post hoc attempts also were made to correlate performance of one outcome measure with another, such as correct disposition assignment with hand hygiene.
Evaluation and Data Collection
Based on their experience with infection control and/or emergency preparedness, 24 drill evaluators were selected from various institutions. They received the evaluation materials and forms 1 month before the event and underwent a 1-day training session on the nature of the exercise, MSIT protocol, public health recommendations, and the use of the forms. Evaluation forms were created and tested in a pilot study to capture critical areas of infection control and compliance with the MSIT protocol for each individual victim encounter in addition to observations of general preparedness of the facility. General preparedness and individual patient encounters were evaluated separately for the following critical stations: security checkpoint outside the hospital, mass screening station, triage station 1, and triage station 2.
General facility preparedness variables for each specific station are shown in the eAppendix. The individual victim encounters included assessment of the following variable categories: successful identification of potentially infectious patients by the MSIT protocol, correct separation of patients into the respective, pre-designated physical spaces of the ED, and compliance with appropriate infection control precautions during patient's encounter with all staff. The evaluation variables were specific to each area of the hospital and chosen based on the relevance to the hospital area of the expected tasks to be performed. The evaluators were distributed to all critical areas of the participating hospitals. Forms were designed as easy-to-use checklists, with the option for additional comments.
Statistical Analysis
SAS Release 9.1.3. (SAS Institute) was used to generate statistics. For general area observation, the overall rate of compliance (ie, across all relevant aspects of the MSIT protocol) was computed separately for each hospital, station, and time point. For patient observations, the rate of compliance with variables relevant to the MSIT protocol during each patient encounter was computed separately for each hospital, station, and time tertiles (early, middle, and late). Since the time of arrival of each patient at each location was not recorded, the patient identification (ID) number was used as a proxy for timing; this was done by creating tertiles of patients at each site based on ID number, the assumption being that patients with lower ID numbers would likely be processed earlier than those with higher numbers.
Generalized Fisher exact tests were used to compare compliance rates among both sites and time tertiles. The prevalence of correct influenza case status determination in the mass screening area was computed for each hospital and time tertiles. Officer behaviors that were assessed at multiple locations within each site (hand hygiene, wearing mask, and proper mask use) were compared across locations, separately for each site and time tertile. Generalized Fisher exact tests were used to compare compliance rates with certain victim variables (proper mask use and color coding) at multiple stations within each site. For each patient, the number of locations at which the patient exhibited the correct variable was divided by the number of locations at which a relevant assessment was made to generate a proportion. Exact logistic regression with likelihood ratio tests was used to estimate and compare the mean of this proportion among all hospitals and time periods. Correlation of officer hand hygiene in the mass screening area with correct case status determination at the same station and with patient's subsequent mask use was measured by the phi coefficient (which resembles the Pearson correlation coefficient) and tested using the Fisher exact test.
RESULTS
A total of 345 patients participated in the drill at all 3 participating hospitals. A complete set of evaluator assessments was collected on 304 patients. Incomplete assessments were randomly distributed among facilities, and there was no statistically significant difference between baseline variables of patients with missing data compared to those with complete assessments.
Determination of Influenza Case Status
Of 304 patients who had mass screening forms available for analysis, 24 (8%) had no recorded disposition on the form, and 10 did not complete the mass screening process because the drill was stopped. As a result, there were 280 patients with screening forms for analysis, with no significant differences in the percentage of missing forms among institutions. The percentage of missing forms was similar among participating facilities. For each preset influenza case status of each patient (Table 1), the case status was determined correctly in 207 of 280 cases overall (74%). Differences in the rate of correct determination were not statistically significant among the 3 hospitals, with 84%, 74.5%, and 70.4% for hospital 1, 2, and 3, respectively, with no significant effect of the patients' sequence. Of 73 patients with incorrect case determination by hospital staff, 46 (63%) were labeled as non-cases when they should have been labeled as case-patients, thereby compromising hospital areas designated for patients without pandemic influenza (ie, those with a preset influenza case status of positive were determined to be case status negative by a mass screening officer).
Rate of Agreement Between Determined and Expected Case Status Disposition as Determined in Mass Screening Facility (n = 280)a
MSIT Protocol Compliance
Several of the observed variables were central to the MSIT protocol for screening and cohort placement: color coding of patients according to case status, covering patients' nose and mouth with a face mask, keeping wait times at the triage station to less than 5 minutes, and maintaining at least 3 feet distance to other patients. The staff implemented the use of face masks for patients as recommended by the MSIT protocol well and did so similarly at all 3 facilities (Table 2). Statistically significant differences were found among hospitals for color coding, maintenance of distances, and wait times between and for patients.
MSIT Protocol Variables
To determine if individual mass screening officers' performance in one infection control task was associated with similar performance in another infection control task, we correlated officer hand hygiene and the fitting of victim face masks. No correlation was found between the 2 tasks (no phi coefficient calculated due to low variability in overall high-compliance rates with fitting face masks at all hospitals).
Station-Specific Performance
We assessed appropriate infection control measures in addition to the correct application of the MSIT protocol at each of the 4 stations where patients came into contact with hospital staff: the security checkpoint outside the hospital, the mass screening station at the entrance to the hospital, triage station 1, and triage station 2. The observed differences in infection control procedures among facilities at the following 3 stations: mass screening station, triage station 1, and triage station 2 are shown in Tables 3-6, respectively. Hospital 2 consistently performed more poorly than the others. Strikingly, certain protocol elements were followed well at some facilities and poorly at others (eg, hand hygiene, cleaning of horizontal surfaces, and use of gowns). Staff use of face mask and color coding of victims was at or close to 100% for all hospitals.
Infection Control and MSIT Preparedness at the Mass Screening Station by Hospital
Infection Control and MSIT Preparedness at Triage Station 1 (for Case-Patients)
Infection Control and MSIT Preparedness at Triage Station 2 (for Noncase Patients)
Relationship Between Infection Control Compliance and Hospital Station
Compliance with infection control recommendations as a function of station within each hospital is shown in Table 6. Staff face mask use was assessed for security personnel at the security checkpoint outside the facilities in addition to mass screening and triage officers. Other infection control procedures were evaluated only for mass screening and triage officers. Significant intrafacility differences in officer mask use and hand hygiene were present for hospital 2. While general staff use of masks was excellent across most stations at the 3 hospitals, compliance with hand hygiene ranged from 17% to 92% and was particularly poor in areas where patients were thought to be noninfectious. There was no effect of patient sequence on compliance rates at any of the hospitals.
General Preparedness and Incident Command
In addition, general station assessments were collected for each hospital station (ie, security checkpoint, mass screening, triage stations, and the hallway) at the beginning of the drill and in 30-minute intervals until the end of the drill. In these assessments each variable was assigned an equal share of the full-compliance rate. General preparedness at the beginning of the drill at each station for the 3 facilities is shown in Figure 3. Over time (30-minute intervals until the end of the drill at 120 minutes), compliance rates remained constant, and little variability was noted in the specific items with which stations were compliant. Qualitative observations of the incident command noted that the hospital with the highest compliance with infection control recommendations (hospital 1) had assigned the director of the hospital epidemiology department to be chief medical officer during the drill. Evaluators also commented that in hospital 1 a constant and authoritative presence of infection control staff was at the front line, while it was thought that little oversight and involvement in situation management was shown by infection control staff in the hospital with the worst compliance (hospital 2).
General Preparedness Observations at Hospitals at Time 0 Using Station-Specific Variables for Each Station, With a Maximum Cumulative Score of 100% (see eAppendix).
DISCUSSION
This study presents evidence that preparation for infectious epidemics can be effectively drilled at hospitals using a scenario-specific protocol and quantitative outcome variables. While real-life events such as the 2009 pandemic provide important lessons, they cannot replace the need for ongoing training to maintain knowledge and skills among hospital staff. Assessment tools such as the one proposed by us allow for identification of weaknesses and continuous reassessment, which may improve the response to infectious outbreaks.
In this drill, we found inconsistent application of basic infection control procedures. The drill procedure itself appeared to be easily understood by staff; however, patients frequently were miscategorized based on their signs and symptoms for reasons that are not completely clear from this drill analysis. The personnel and infrastructure resources necessary to create this process were rapidly deployed without delay.
Some measurable activities such as the mandated use of masks were consistently followed while compliance with other activities, such as hand hygiene between cases, was variable. Hand hygiene, a core infection control practice, was particularly neglected when patients were categorized as noncase and presumably noninfectious. Compliance with a core group of infection control measures ranged from near 100% at one institution to approximately 50% at another. While it is not certain, it appears likely that a transmissible disease would be poorly contained at an institution that had poor compliance with infection control measures.
On the day of the drill ambient temperatures were approximately 95°F, and humidity was high. This made working conditions outside the hospital difficult. Institutions need to consider environmental factors in their planning for disasters. Spacing between patients was difficult, especially during periods when the flow of patients increased. Realistic expectations regarding crowd control and flow need to be considered by institutions. One somewhat unexpected challenge was the difficulty in keeping family groups together irrespective of whom and how many in that group were symptomatic. It is recommended that in such a chaotic environment it is best for children to remain with their parents.12 In an effort to stringently segregate case-patients and noncase-patients, hospital staff occasionally split up family groups.
Compliance with protocol elements appeared to be no more effective at the triage station for patients who met the case definition and were considered infectious when compared to the screening stations set up outside the hospital. Interestingly, compliance with one aspect of infection control did not necessarily mean that compliance was equally good for other practices. The suggestion is that infection control practices are not a “bundle” but must be looked at and invariably trained as individual specific practices and monitored during the event. Compliance remained unchanged during the time of the drill. This finding was encouraging, as it was thought that personnel compliance would decline with fatigue.
The reasons for variability among hospitals on compliance with protocol were unclear. Screening protocols were not new to the hospitals and had been introduced at several staff meetings ahead of time, although not practiced in the past. Clear epidemiologic case definitions were provided on paper instruments. Expected infection control practices did not deviate from standard practices for patients requiring isolation due to pathogens transmitted by respiratory secretions. We can speculate that willingness of administrative staff and supervisory staff to commit their attention and energy to the process contributed to staff inattention at certain hospitals. While the drill was not set up to capture these data quantitatively, it was observed that where infection control staff members took an active leadership role in monitoring and managing the flow of patients and constantly were updating and advising staff on the proper use of personal protective equipment that compliance with infection control practices not surprisingly was much more successful. It would be interesting to determine in a follow-up study if performance in all hospitals is better following the 2009 influenza pandemic; however, all participating hospitals were drilled at the same time with access to the same amount of information and perceived risk alone would not explain differences among institutions.
The 2009 influenza pandemic also offers an opportunity to study one of the assumptions we have made based on the biology of influenza and the most current scientific evidence of transmission at the time; currently, there is no high-quality evidence for mass screening as an effective tool in improving patient outcome during an influenza pandemic. In addition, our study had to use a hypothetical case definition, again based on the current criteria for influenza-like illness suggested at the time. These criteria obviously would be different during a real epidemic or pandemic, as they can only be known once initial epidemiologic data become available. However, this factor has no significant effect on the variables we studied. The respiratory epidemic protocol and assessment tool developed for this study can also be easily adapted for other biologic agents.
As a training modality, drills have been widely adopted by hospitals in the United States to exercise disaster responses and are, in fact, a requirement of hospital accreditation by the Joint Commission on Accreditation of Healthcare Organizations.13 Few usable data have come from drill experiences, and there is no clear coordinated effort to use drills to learn what does and does not work. While a few studies have evaluated use of personal protective equipment and potential pathogen transmission in hospital settings, we believe that our study is novel because it demonstrates the ability of a full-scale exercise to quantify and compare preparedness variables related to infection control as well as emergency management across hospitals.Reference Phin, Rylands, Allan, Edwards, Enstone and Nguyen-Van-Tam14Reference Swaminathan, Martin and Gamon15 This aspect may help establish benchmarks and identify specific deficiencies at institutions. If many or all hospitals are conducting drills, it seems prudent to standardize approaches and collect data in a systematic way. National organizations should set benchmarks in the way that infection control and other quality measures are assessed. Only then will we learn to respond more effectively if such situations arise. Much work needs to be done in this area.
Funding and Support: This publication was supported by grant 4 U3REP070025-01-03 from the US Department of Health and Human Services (HHS). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of HHS.
