Since 2000, natural disasters have resulted in more than 5600 deaths in the United States.1 Disaster-related mortality can be prevented by understanding the common circumstances that result in death. Although the frequency of disaster-related deaths is recorded, circumstances surrounding deaths rarely are. Ultimately, disaster-related mortality surveillance must aim for timely assessment of the distribution and determinants of disaster-related deaths to establish priorities for action and guide relief efforts during disasters. This goal can be achieved by systematic measurement of the impact of disasters on people and using those data to provide evidence-based guidelines for prevention and preparedness activities.
In spite of federal funding for emergency preparedness activities being available at the state level, few states have established active disaster-related mortality surveillance systems.Reference Zane, Bayleyegn and Hellsten2, Reference McKinney, Houser and Meyer-Arendt3 In contrast, the American Red Cross (ARC) tracks disaster-related mortality in all 50 states.4 For the ARC, the main goal of tracking disaster-related mortality is to identify deaths so that they can provide condolence services to surviving family members. Examples of condolence services include crisis counseling, assistance with funeral-related expenses, locating emergency housing, identification or referral of resources to assist with recovery, and addressing disaster-related health needs. In 1987, in collaboration with the Centers for Disease Control and Prevention (CDC), the ARC instituted an active disaster-related mortality surveillance system designed to methodically record these deaths.4 However, no clearly established operations protocol exists for disseminating results or using them to guide preparedness efforts. Therefore, the CDC and the ARC determined that an evaluation of the system was warranted.
In September 2008, Hurricane Ike made landfall in Galveston, Texas, as a category 2 storm. It was the third most destructive hurricane to make landfall in the United States. It affected 34 counties in Texas, resulting in evacuations of 1.9 million residents and leaving 4.5 million without power for weeks. The Texas Department of State Health Services (DSHS) and the ARC independently collected mortality data for several weeks after Hurricane Ike made landfall in Texas. By using the DSHS system as the gold standard, we evaluated the key attributes, strengths, and limitations of the ARC disaster-related mortality surveillance system implemented during Hurricane Ike in Texas and provided recommendations for system improvement, which can benefit states that do not have an active disaster-related mortality surveillance system.
Methods
We used CDC's updated guidelines for surveillance system evaluationReference German, Lee, Horan, Milstein, Pertowski and Waller5 to assess the ARC system. The guidelines include measures of simplicity, flexibility, acceptability, timeliness, and stability, as well as data quality, sensitivity, positive predictive value (PPV), and representativeness. We reviewed the mortality surveillance form used by the ARC to assess its simplicity and conducted interviews with stakeholders (ie, ARC leadership and volunteers deployed to Texas, DSHS mortality surveillance staff involved during Hurricane Ike mortality surveillance, and medical examiners in Texas) to assess flexibility, acceptability, timeliness, and stability of the system. We linked the data from the DSHS and the ARC systems and used the DSHS system for comparison to assess data quality, sensitivity, PPV, representativeness, and timeliness. On the basis of our findings, we provided recommendations to improve the ARC system's performance.
Red Cross System Description
The ARC is primarily funded by public donations6 and is a supporting agency under Emergency Support Functions 6 and 8 in the National Response Framework.7 Approximately 600 local ARC chapters are located throughout the 50 states and US territories. Disaster Health Services (DHS) is the activity within the ARC responsible for the system. The majority of DHS staff are volunteers, with the exception of 5 reserve-ready nurses and 2 full-time paid staff who work at the ARC DHS headquarters in Washington, DC. The ARC DHS volunteers are skilled health professionals (86% are nurses).
Within the ARC organizational structure, the DHS activity is responsible primarily for assessment, treatment, and referral of disaster-related health needs. It also leads the provision of condolence services for families who have lost members in a disaster, as well as conducting morbidity and mortality surveillance. During a disaster response, condolence teams are formed with 2 to 3 ARC volunteers comprising health, mental health, and caseworkers. Their task is to ascertain all the disaster-related deaths, visit and provide services to the affected families, and complete a mortality surveillance form. This process begins immediately through the local chapter after an ARC response is initiated and terminates after the relief operation transitions back to the local chapter. Relief operations and surveillance are always initiated at the local chapter level and supported by the national headquarters, depending on the scope and scale of the disaster. For widespread disasters requiring additional resources, the national headquarters sends their staff and identifies volunteers nationally to support the local chapter response activities. During the Hurricane Ike disaster response, 4 condolence teams with 2 to 3 members per team (a health or mental health professional and a caseworker) collected mortality data during September 13 through October 7, 2008.
Death Ascertainment
Information flow in the ARC Hurricane Ike mortality surveillance system was linear. After the disaster response was initiated, ARC staff actively searched for reports of disaster-related deaths. Information sources included funeral home directors, disaster-relief shelters, the Federal Emergency Management Agency, the disaster mortuary operational response team, emergency operations centers, hospitals, and media reports. On verification of a death, volunteers contacted medical examiners and justices of the peace (who serve as coroners in Texas) to confirm and obtain more information, especially if the death was disaster related. Information on the disaster-related deaths was recorded on a 1-page CDC-ARC mortality surveillance form. At the end of the disaster response, copies of completed mortality surveillance forms were sent to the ARC DHS headquarters in Washington, DC, which in turn sent copies to CDC's National Center for Environmental Health. These are standard procedures for all disaster-related mortality conducted by the ARC.
Case Definition of Disaster-Related Deaths
On the ARC mortality surveillance form, disaster-related deaths are classified into 2 categories: directly and indirectly related deaths. A directly related death is defined as a death caused by the environmental force of the disaster (eg, wind or flood) or by the direct consequences of these forces (eg, structural collapse). An indirectly related death is defined as a situation in which the disaster led to unsafe conditions (eg, hazardous roads) or caused a loss or disruption of usual services that contributed to the death (eg, loss of electrical services).
Texas Department of State Health Services System (Gold Standard)
Texas DSHS piloted their active disaster-related mortality surveillance system during Hurricane Ike.Reference Zane, Bayleyegn and Hellsten2 The flow of information in the DSHS system is similar to that of the ARC system, whereby triggering sources are used to identify deaths and detailed data on a confirmed death are obtained from medical examiners and justices of the peace by using a 1-page form developed by DSHS.8 The major differences between the 2 systems are that DSHS system's pre identified and trained staff and written guidelines are in place, including a clear case definition. A 2009 CDC evaluation of the DSHS system recommended that it be used to identify disaster-related mortality because, as an active surveillance system, it identified more deaths, as compared with a text-string search of disaster terms (eg, “Ike,” “hurricane,” and “carbon monoxide poisoning”) in the Texas vital statistics database.Reference Choudhary, Zane and Beasley9
In the DSHS system, a case was defined as any death, directly, indirectly, or possibly associated with the hurricane among evacuees, residents, or rescue personnel related to the hurricane in declared disaster counties, counties along the Texas Gulf coast, or counties known to have evacuation shelters occurring approximately 5 days before hurricane landfall and continuing for approximately 4 weeks after landfall. Deaths classified as directly related included any death caused by the physical forces of the hurricane (eg, wind, rain, or floods), or by direct consequences of these forces (eg, structural collapse or flying debris). Deaths classified as indirectly related were any deaths caused by unsafe or unhealthy conditions that occur because of the anticipation or actual occurrence of the hurricane. These conditions include loss or disruption of usual services (ie, utilities, transportation, environmental protection, medical care, or police/fire), personal loss, and lifestyle disruption (eg, temporary displacement or property damage). Deaths that occurred from natural causes were considered indirectly related if physical or mental stress before, during, or after the storm resulted in exacerbation of preexisting medical conditions and contributed to death. Deaths classified as possibly related were deaths in the targeted areas in which the cause or manner of death was undetermined or pending or information indicated that the storm might have caused or exacerbated a situation leading to death.
Stakeholders
Engaging stakeholders is essential for success of a surveillance system evaluation because it ensures that relevant questions are addressed and resulting recommendations will be adopted. We engaged 2 categories of stakeholders, data users and data providers. Data users are both local and national ARC chapters, CDC, and Texas DSHS. All 3 organizations have preparedness staff who can use information regarding circumstances of death to direct their community preparedness activities and communicate these findings to the public. Data providers are the medical examiners and justices of the peace who provide detailed information on deaths.
Results
Usefulness and Flexibility
The CDC guidelines for surveillance system evaluation define usefulness as the ability of the system to contribute to prevention and control of adverse health-related events, including an improved understanding of the public health implications of such events. Flexibility of the system describes its ability to adapt to changing information needs or operating conditions with limited additional time, personnel, or allocated funds.Reference German, Lee, Horan, Milstein, Pertowski and Waller5 The data collected enabled the ARC to identify deaths rapidly and systematically so that volunteers could provide timely condolence services. However, these surveillance data were not used to their full potential to implement timely interventions, preparedness planning, and mitigation strategies. Information was not disseminated to the ARC chapters in Texas or Texas DSHS and local public health departments.
The system demonstrates flexibility in its adaptability and scalability. It is adaptable because procedures are the same regardless of the disaster's type or size. It is also scalable whereby all responses begin at the local chapter level with increasing support from the national headquarters, depending on the disaster size. Simplicity of the system adds to its flexibility, with fewer components requiring modification for adapting to new operating procedures or information needs.
Simplicity
Simplicity of the system refers to both its structure and ease of operation.Reference German, Lee, Horan, Milstein, Pertowski and Waller5 A strength of the ARC mortality surveillance system implemented during Hurricane Ike is that deaths were reported on a simple, standardized, 1-page mortality form. In addition, all ARC chapters use this form. Weaknesses include lack of written guidelines and instructions on how to complete the form. Also, no clear case definition of a disaster-related death exists aside from what is on the mortality form. Notably, time and place of death are missing from the case definition. This deficit is further complicated by the fact that Texas ARC volunteers did not receive formal or just-in-time training on completion of the mortality form.
Acceptability
Acceptability indicates the willingness of individuals and organizations to participate in the surveillance system.Reference German, Lee, Horan, Milstein, Pertowski and Waller5 For any system to function optimally, it must be acceptable to users. This system involves volunteers collecting mortality data as well as medical examiners and justices of the peace who provide detailed mortality data. Our interviews with volunteers involved in the Hurricane Ike response and the most recent response to Tropical Storm Hermine in Texas (2010) identified problems that limited the volunteers’ acceptance of the ARC system. Volunteers stated that absence of written instructions for completion of specific fields made achieving complete reporting difficult. Medical examiners also identified items on the form that were confusing and not aligned with the state's electronic death reporting system.
Data Quality
Data quality reflects the completeness and validity of the data recorded in the surveillance system.Reference German, Lee, Horan, Milstein, Pertowski and Waller5 Problems with data quality were partly attributable to acceptability problems. Two key aspects of data quality to consider are completeness and validity. Completion rates were high for the majority of demographic variables (90%-97%), with the exception of race/ ethnicity, which the medical examiners noted as frequently difficult to determine during their visual inspection. Conversely, such variables as relatedness to the disaster (directly/ indirectly), cause and location of death, circumstances of death, and reporting source had lower completion rates (61%-87%) (Figure 1). Incomplete reporting reflects the need for standardized and routine training on the importance of complete reporting of all data fields on the mortality form.
Completeness of Data Reporting in the American Red Cross Disaster-Related Mortality Surveillance System During Hurricane Ike — Texas 2008.
Figure 2 illustrates the percentage of concordance between the DSHS and ARC systems on key variables. With the exception of relatedness to the disaster variable, agreement levels were high. Thus, information collected by the ARC system was valid, and the problem lies in incomplete reporting on all data fields in the mortality form. The low agreement level for relatedness to the disaster variable was likely caused by differences in case definition between the 2 systems. In the DSHS system, deaths that occurred from natural causes were considered indirectly related if physical or mental stress before, during, or after the storm resulted in exacerbation of preexisting medical conditions and contributed to death. On the other hand, the ARC system did not include such conditions in their case definition. In addition, unlike the DSHS system, the ARC system did not have a possibly related death category.
Percentage of Concordance Between the American Red Cross and the Texas Department of State Health Services Data on Key Variables Captured During Hurricane Ike Surveillance—Texas 2008.
Sensitivity and Positive Predictive Value
Sensitivity refers to the proportion of cases identified by the surveillance system. Assessment of sensitivity requires determination of the true frequency of the condition under surveillance through access to accurate data that are external to the system (a gold standard). In this study, the DSHS surveillance system was used as the gold standard. The DSHS surveillance system identified 74 deaths, and the ARC system identified 38 deaths. The sensitivity of the ARC system to identify deaths related to Hurricane Ike accurately was 47% (sensitivity = number of cases identified by both systems [true positives]/number of cases identified by the standard [true positives + false negatives]; 35/74 = 47%). We calculated a PPV of 92%, meaning that 92% of the deaths identified in the ARC system as being related to the hurricane were indeed related to Hurricane Ike. This percentage is less than 100% because 3 deaths were reported in the ARC system and missed by the gold standard.
Representativeness
Representativeness reflects the ability of the system to accurately describe the occurrence of deaths over time and the distribution in the population by place and person.Reference German, Lee, Horan, Milstein, Pertowski and Waller5 Thirty-nine cases identified by the DSHS system were not captured in the ARC system. Of the 3 aspects of representativeness of the ARC data, we first examined relatedness of deaths to the disaster. Of the 39 deaths that were not captured by the ARC system, only 3 were classified in the DSHS system as being directly related to the disaster; 27 deaths were indirectly related; and 9 deaths were possibly related to the disaster. The possibly related category was not included in the ARC system; therefore, why these 9 deaths were not captured is clear. By examining the cause and circumstances of the 27 deaths that were classified as being indirectly related in the DSHS system, we determined that 7 deaths were attributable to causes not captured anywhere in the ARC system. For example, 5 were suicide deaths, and ARC did not capture suicide-related data. Nevertheless, ARC missed 20 indirectly related deaths that were attributable to causes captured by the ARC system (eg, carbon monoxide poisoning-related deaths). Indirect deaths are at times more difficult to identify, especially if one lacks training in identifying them.
The second aspect of representativeness that we examined was geographic distribution of deaths. Figure 3 is a map of deaths captured by the DSHS and ARC systems during Hurricane Ike. Eight counties existed in which only DSHS identified any deaths. The ARC system missed a total of 10 deaths in these 8 counties, 5 of which were not directly along the path of the hurricane. Possibly, the reason the ARC missed these deaths is because it did not conduct surveillance in counties that were not directly along the hurricane's path; however, they did identify deaths in 2 counties along the Louisiana border that were not along the path and did not identify any deaths in 2 counties that were directly along the path. Therefore, it is not entirely clear why the ARC system did not identify deaths in these 8 counties.
Distribution of Deaths Captured by the American Red Cross and Texas Department of State Health Services (DSHS) Systems During Hurricane Ike, by County—Texas 2008.
Finally, we evaluated representativeness of death identification as they occurred over time. Figure 4 is a timeline of deaths captured by the ARC and DSHS systems. Hurricane Ike made landfall on September 13. With rare exceptions, ARC captured deaths throughout the same period that the DSHS system identified deaths. Deaths occurring before Ike made landfall were related to preparedness activities (eg, cutting tree branches before the storm to keep them away from roofs).
Timeline of Hurricane Ike-Related Deaths Captured by the American Red Cross vs Texas Department of State Health Services (DSHS) Disaster Mortality Surveillance Systems—Texas 2008.
Timeliness
Timeliness is the speed between steps in a public health surveillance system.Reference German, Lee, Horan, Milstein, Pertowski and Waller5 It is one of the most important attributes of a surveillance system because it allows for early detection and response, which is the ultimate goal of surveillance. Two important aspects of timeliness should be considered. The first is timeliness of capturing disaster-related deaths. To assess this, we need information regarding the date of death and the date it was reported in the system to identify time lags that might be improved. The reporting date was available for only 3 deaths in the ARC system; therefore, we were unable to assess timeliness of reporting. The second important aspect of timeliness is in communicating results back to stakeholders. This aspect of timeliness is deficient in the ARC system as it stands. Months often occur before CDC receives the paper forms from the ARC, which decreases timely data entry and analysis. In addition, CDC generates reports for multiple years of data and not for each disaster.4 Finally, no clearly established protocol exists for communicating results to the stakeholders.
Stability
Stability refers to the ability of the system to collect, manage, and provide data without failure and to be operational when needed.Reference German, Lee, Horan, Milstein, Pertowski and Waller5 The ARC disaster-related mortality surveillance system is a stable system through the support of CDC's National Center for Environmental Health and the ARC DHS efforts. Data are collected regardless of type or size of the disaster (ranging from an apartment complex fire to severe hurricanes). Because it is a paper-based system, no information technology requirements for maintaining it exist, and no downtimes are necessary for system maintenance. However, because the system is operated by volunteers, staff turnover can undermine the system's stability. Volunteer turnover might have contributed to problems in quality and completeness of collected data in regard to Hurricane Ike.
Comment
The ARC is a key supporting agency during disasters. It seeks timely access to mortality data, driven by its goal of providing condolence services to relatives. The system has strengths, including simplicity, flexibility, and stability. However, its main weakness has been low sensitivity (47%), because 27 indirectly related deaths (eg, carbon monoxide poisoning and suicide) were captured by the state's pilot system but not in the ARC system. One possible explanation for the missing deaths might be that the relatives declined condolence services, in which case whether a mortality form was completed is unclear. Certainly the dependency on cooperation from state agencies to confirm deaths and provide relatives’ contact information, lack of a case definition, and absence of written guidelines affected the quality and representativeness of data in the ARC system. In addition, a feedback method to stakeholders was not clearly defined; specifically, no plans exist for analysis and dissemination of findings.
On the basis of this evaluation, we developed recommendations for system improvements. The first and most important recommendation is to develop written guidelines for mortality surveillance that include a case definition of a disaster-related death.Reference Combs, Quenemoen, Parrish and Davis10 The guidelines should include simple written instructions that accompany the data collection form. Training volunteers on how to apply the guidelines and complete the mortality form is essential. This can be in the form of annual Internet-based or just-in-time training for field-based volunteers. The just-in-time training can also help address the problem of volunteer turnover and its effect on data quality. In addition, the ARC has made a large commitment to addressing volunteer development and stability through its volunteer connection program. This program is a web-based, organization wide system that will streamline the way volunteers are attracted, matched, trained, deployed, and retained. This program holds great promise for addressing the issue of volunteer turnover. We also recommend developing an electronic reporting system whereby volunteers enter the data collected on paper forms into an online database that can be shared with partners (eg, CDC and local and state health departments). This procedure can increase timeliness of reporting and ease of data analysis and dissemination of results.
For the ARC to accomplish its condolence mission, the organization should continue to strengthen collaborations with local emergency managers and health departments and develop processes to identify and share information on disaster-related deaths across large geographic areas and jurisdictions. We had the unique ability to examine the ARC system during an event in which active surveillance was implemented and deaths were confirmed in vital records. To strengthen collaborations with state and local health departments we recommend (1) signing a memorandum of understanding that provides guidelines for communication and data sharing between the agencies; (2) providing training to familiarize the ARC and state and local health departments with the functions performed and methods used by each organization; (3) strengthening communication channels by including the chapter's ARC DHS advisor on monthly preparedness calls and daily situation reports during disaster response and including the state or local health department contacts on the distribution list for ARC external partner reports; (4) providing post disaster training on lessons learned; and (5) identifying uses of the mortality data such as generating evidence-based recommendations for future disasters and ensuring distribution of recommendations to all stakeholders. Because state health departments often do not have a formally established disaster-related mortality surveillance system, we recommend that the ARC, CDC, and states leverage the existing systems to assist local health authorities in accurately capturing the circumstances of deaths that can be used to guide preparedness and prevention efforts.
Although the performance of the ARC system should be similar across all chapters, their ability to capture the data are determined by the willingness of the coroners to release the information. Involving medical examiners and justices of the peace is essential because they provide the detailed mortality data. If they recognize the importance of timely, quality disaster-related mortality data for decision making, they might be more willing to share information. Their feedback on the mortality form should be sought and their comments considered. Finally, we recommend ongoing assessment and revalidation of the system by using CDC's updated guidelines for surveillance system evaluations.Reference German, Lee, Horan, Milstein, Pertowski and Waller5 This process can be accomplished through collaborations with CDC and state and local health departments. Both organizations have trained staff capable of providing needed oversight and ongoing monitoring of the key attributes to ensure consistent improvement in the system.
Conclusion
Surveillance system evaluations provide evidence-based recommendations that can be used to improve and support public health activities. The ARC Hurricane Ike mortality surveillance system evaluation indicated that timeliness, sensitivity, and data quality were areas of weakness that should be addressed. Data analysis and report generation can be enhanced by increasing collaborations with federal and state health departments. In addition, collaborations can facilitate other surveillance and response activities (eg, shelter-based morbidity surveillance).
A positive outcome of this evaluation has been increased collaborations among the ARC national headquarters, Texas chapters, and DSHS. We are exploring ways to facilitate, within the legal framework regarding confidentiality, timely data sharing on mortality and shelter morbidity surveillance, strengthening communication channels by participating in joint conference calls, joint training to familiarize the Red Cross and state health departments of the functions performed and methods used by each entity, and joint presentations at conferences (eg, the 56th Annual Texas Vital Statistics Conference, Austin 2010, and the Public Health Preparedness Summit, Atlanta, Georgia, 2011). This Texas experience provides a model for collaboration between the ARC and health departments in other states. By collaborating with the ARC, states can enhance their capacity to track, respond, and assess disaster-related mortality during an event.
About the Authors
Centers for Disease Control and Prevention, Atlanta, Georgia (Drs Farag and Bayleyegn and Mss Rey and Noe); The Texas Department of State Health Services, Austin (Dr Farag, Ms Rey, and Mr Zane); and the Disaster Health Services, American Red Cross, Washington, DC (Ms Wood).
Support
This study was supported by the Centers for Disease Control and Prevention and the Texas Department of State Health Services.
Disclaimer
The findings and conclusions in this report are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Acknowledgments
Sharon Stanley, PhD, RN, RS, Chief Nurse, American Red Cross, provided the opportunity to conduct this surveillance evaluation; the American Red Cross staff and Disaster Health Services leadership volunteers Jeanne Spears, RN; Tami Dittburner, RN, MSN; and Debra Williams, RN, MSN, CS, assisted in conducting this evaluation; American Red Cross volunteers conducted the condolence visits; Tarek Rashed, PhD, and Tracy Haywood supplied mapping assistance; and Eric Miller, PhD, reviewed and edited the manuscript.
Published online: December 7, 2012. doi:10.1001/dmp.2012.54
