Hostname: page-component-74656f9884-rdgjk Total loading time: 0 Render date: 2026-11-07T08:03:39.991Z Has data issue: false hasContentIssue false
  • English
  • Français

Evaluation of Oklahoma’s Electronic Death Registration System and Event Fatality Markers for Disaster-Related Mortality Surveillance – Oklahoma USA, May 2013

Published online by Cambridge University Press:  03 May 2019

Anindita N. Issa ,
Kelly Baker ,
Derek Pate ,
Royal Law ,
Tesfaye Bayleyegn  and
Rebecca S. Noe
Anindita N. Issa*
Affiliation:
Centers for Disease Control and Prevention, Chamblee, Georgia, USA
Kelly Baker
Affiliation:
Oklahoma State Department of Health, Oklahoma City, Oklahoma, USA
Derek Pate
Affiliation:
Oklahoma State Department of Health, Oklahoma City, Oklahoma, USA
Royal Law
Affiliation:
Centers for Disease Control and Prevention, Chamblee, Georgia, USA
Tesfaye Bayleyegn
Affiliation:
Centers for Disease Control and Prevention, Chamblee, Georgia, USA
Rebecca S. Noe
Affiliation:
Centers for Disease Control and Prevention, Chamblee, Georgia, USA
*
Correspondence: Anindita N. Issa, MD, Centers for Disease Control and Prevention, National Center for Environmental Health, 4770 Buford Hwy, NE, MS F-60, Chamblee, Georgia 30341 USA E-mail: aissa@cdc.gov
Rights & Permissions [Opens in a new window]

Abstract

Introduction:

Official counts of deaths attributed to disasters are often under-reported, thus adversely affecting public health messaging designed to prevent further mortality. During the Oklahoma (USA) May 2013 tornadoes, Oklahoma State Health Department Division of Vital Records (VR; Oklahoma City, Oklahoma USA) piloted a flagging procedure to track tornado-attributed deaths within its Electronic Death Registration System (EDRS). To determine if the EDRS was capturing all tornado-attributed deaths, the Centers for Disease Control and Prevention (CDC; Atlanta, Georgia USA) evaluated three event fatality markers (EFM), which are used to collate information about deaths for immediate response and retrospective research efforts.

Methods:

Oklahoma identified 48 tornado-attributed deaths through a retrospective review of hospital morbidity and mortality records. The Centers for Disease Control and Prevention (CDC; Atlanta, Georgia USA) analyzed the sensitivity, timeliness, and validity for three EFMs, which included: (1) a tornado-specific flag on the death record; (2) a tornado-related term in the death certificate; and (3) X37, the International Classification of Diseases, 10th Revision (ICD-10) code in the death record for Victim of a Cataclysmic Storm, which includes tornadoes.

Results:

The flag was the most sensitive EFM (89.6%; 43/48), followed by the tornado term (75.0%; 36/48), and the X37 code (56.2%; 27/48). The most-timely EFM was the flag, which took 2.0 median days to report (range 0-10 days), followed by the tornado term (median 3.5 days; range 1-21), and the X37 code (median >10 days; range 2-122). Over one-half (52.1%; 25/48) of the tornado-attributed deaths were missing at least one EFM. Twenty-six percent (11/43) of flagged records had no tornado term, and 44.1% (19/43) had no X37 code. Eleven percent (4/36) of records with a tornado term did not have a flag.

Conclusion:

The tornado-specific flag was the most sensitive and timely EFM. Using the flag to collate death records and identify additional deaths without the tornado term and X37 code may improve immediate response and retrospective investigations. Moreover, each of the EFMs can serve as quality controls for the others to maximize capture of all disaster-attributed deaths from vital statistics records in the EDRS.

Issa AN, Baker K, Pate D, Law R, Bayleyegn T, Noe RS. Evaluation of Oklahoma’s Electronic Death Registration System and event fatality markers for disaster-related mortality surveillance – Oklahoma USA, May 2013. Prehosp Disaster Med. 2019;34(2):125–131

Keywords

disasterEDRSflagmortalitytornado

Information

Type
Original Research
Information
Prehospital and Disaster Medicine , Volume 34 , Issue 2 , April 2019 , pp. 125 - 131
Copyright
© World Association for Disaster and Emergency Medicine 2019 

Introduction

On May 19, 20, and 31, 2013, central Oklahoma (USA) experienced three powerful and lethal tornadoes. The tornadoes destroyed houses and businesses, injured over 500 people, and killed 48 people. The last tornado on May 31 caused severe, deadly flash flooding. It is critical to accurately identify tornado-related mortality data to guide evidence-based public health messaging.Reference Brown, Archer, Krueger and Mallonee1–Reference Morton and Levy3 During the 2013 tornadoes, Oklahoma initiated a novel method to track tornado-related deaths. The new system was assessed for sensitivity, validity, reliability, timeliness, usefulness, and acceptability in improving disaster response.Reference Brown, Archer, Krueger and Mallonee1–Reference Morton and Levy3

During the 2013 tornadoes, the Oklahoma Vital Records (VR; Oklahoma City, Oklahoma USA) piloted an ad hoc event fatality flag, a categorical variable with a drop-down menu, in its Electronic Death Registration System (EDRS). In addition to the ad hoc flag, two other event fatality markers (EFMs) tracked tornado-related deaths within the EDRS: (1) the tornado-related term (ie, “tornado” or “twister”) in the literal text sections of the death certificate; and (2) the International Classification of Diseases, 10th Revision (ICD-10) mortality code X37 designating Victim of a Cataclysmic Storm, which includes tornadoes. Ideally, the most robust disaster-mortality dataset would have all three EFMs present in each disaster-related death record (Figure 1).

Figure 1.

Key Features of the Event Fatality Markers in the Electronic Death Record during Tornadoes – Oklahoma, May 2013.

Abbreviations: EDRS, Electronic Death Registration System; ICD-10, International Classification of Diseases, Tenth Revision; NCHS, National Center for Health Statistics; OCME, Oklahoma Office of the Chief Medical Examiner; VR, Oklahoma State Health Department Division of Vital Records.Note: The X37 code designates Victim of a Cataclysmic Storm and includes tornadoes, along with other storm systems like hurricanes. The NCHS assigns ICD-10 codes to registered death records received from state vital records offices. The NCHS’s computerized systems use a complex algorithm to search the death record for keywords that prompt assignment of certain ICD-10 codes. To add the X37 code in the death records for tornado-attributed deaths, NCHS’s algorithm searched for tornado-related terms, like “tornado” or “twister,” in the literal text and underlying cause of death sections in the death certificate. Typically, the ME added these terms to the literal text of the How Injury Occurred section of the death certificate. If such tornado terms were not included, the death record would likely not receive the X37 code. In May 2013, VR instructed the ME to add the flag to the death record in EDRS as well. However, NCHS’s algorithm cannot detect this flag, which is only part of the electronic death record and not part of the official death certificate. Therefore, the ICD-10 coding algorithm does not depend on the tornado flag.

Death registration systems ultimately provide mortality data to guide evidence-based public health messaging. Over the past decade, most states in the US have adopted EDRS, web-based applications for filing death records that have replaced paper-based processes.Reference Heim4 The EDRS facilitates an online platform for users to enter mortality data into the death certificate and has vastly improved accuracy of mortality data, timeliness of reporting deaths, and the availability of mortality data as a surveillance tool.Reference Heim4, Reference Baker5

However, despite these improvements, EDRS surveillance of disaster-related mortality in particular has significant growth potential. Currently, no national standard exists for disaster-related mortality surveillance in the US.Reference Rocha, Fromknecht, Redman, Brady, Hodge and Noe6–Reference Hanzlick and Parrish8 State, territorial, and local jurisdictions across the US use various methods of tracking disaster-related deaths, with a range of results regarding the accuracy of disaster-related death counts and the timing of reporting mortality data.Reference Brunkard, Namulanda and Ratard9–Reference Zane, Bayleyegn and Hellsten14

During the May 2013 tornadoes in Oklahoma, the ad hoc flag in EDRS tracked tornado-related deaths within VR, assisted VR in rapidly responding to media inquiries about tornado-related fatalities, ensured VR death counts coincided with the official death counts of the Office of Chief Medical Examiner (OCME; Oklahoma City, Oklahoma USA), and closed the decedents’ birth certificate records promptly to prevent post-disaster identity fraud.Reference Baker5, 15 Shortly after the first tornado, VR notified OCME of the newly-added flag variable in EDRS and requested that all deaths related to the tornado be marked with the flag. The tornado term and X37 ICD-10 code were available for researchers to use to collate deaths related to the tornadoes for retrospective analysis.

Disaster-related epidemiological data can help identify risk factors that lead to adverse health outcomes in a population. Naturally, the most-dire health outcome is death, and tracking disaster-related mortality can measure the impact and severity of a disaster on a population and provide information to help avert further morbidity and mortality by identifying needs and allocating resources.Reference Brown, Archer, Krueger and Mallonee1–Reference Morton and Levy3, 16

Disaster management, which occurs in a cyclical manner, consists of four phases: preparedness, response, recovery, and mitigation.16 Even if a disaster has not recently occurred, the preparedness phase is essentially a default phase. The uses of disaster-related mortality data vary depending on the current stage in the disaster management cycle. During the response phase, disaster mortality data can be used for situational awareness to identify main causes and circumstances of death to inform immediate public health interventions. These data must be accurate and near real-time. As activities transition away from the immediate response, retrospective research analyses are vital to future planning and mitigation efforts. These data must still be accurate, but the urgency for real-time access naturally decreases.

The EFMs may seem redundant, but each has a unique role. Together, all three create the most robust disaster mortality data set, for both the immediate phases following a disaster event during response and recovery efforts, as well as for subsequent phases focused more on retrospective research.

The objective of this evaluation is to assess the ability of the three EFMs (flag, tornado term, and X37 code) to track disaster-related deaths. Sensitivity, validity, reliability, and timeliness of the EFMs were analyzed. Usefulness and acceptability of the EDRS overall and of the three EFMs were also evaluated.

Methods

Medical examiners (MEs) in Oklahoma are required to process all un-natural deaths, which includes disaster-related deaths. The MEs and funeral home directors enter mortality data into the death record module in EDRS. The ME then certifies the death certificate, and subsequently VR registers it. After it is registered, VR sends the death data file to the National Center for Health Statistics (NCHS; Hyattsville, Maryland USA), where ICD-10 codes are assigned (Figure 2). The EDRS provides all mortality data for this evaluation. Usefulness and acceptability were assessed through information gleaned from stakeholders’ discussions.

Figure 2.

Mortality Data Flow during Tornadoes – Oklahoma, May 2013.

Abbreviations: EDRS, Electronic Death Registration System; ICD-10, International Classification of Diseases, Tenth Revision; NCHS, National Center for Health Statistics; OCME, Oklahoma Office of the Chief Medical Examiner.Note: Shortly after the first of three tornadoes affecting central Oklahoma in May 2013, VR piloted an ad hoc flagging procedure to track tornado-related deaths within their EDRS. The VR notified the OCME to start flagging tornado-related deaths. After post-mortem assessment, the OCME flagged tornado-related deaths and included terms such as “tornado” in the death certificate. After the OCME completed the preliminary death certificate in EDRS, VR finalized the death certificate and sent a death data file containing the tornado term to the NCHS. The NCHS used computerized systems to apply a complex algorithm to assign ICD-10 codes to each death record. This algorithm searched for key words like “tornado” to assign the ICD-10 code for tornadoes (X37 code).

Oklahoma State Department of Health Injury Prevention Service (IPS; Oklahoma City, Oklahoma USA) identified 48 tornado-attributed deaths from a comprehensive retrospective review of the OCME database, EDRS mortality records, and hospital discharge databases. The IPS defined cases as any death with a completed death certificate attributed to the May 19, 20, or 31, 2013 tornadoes or related flash flooding events with the dates of death falling between May 19 and August 1, 2013 (the date of the last delayed death). This included the deaths of out-of-state residents. The IPS included all directly related deaths and only flood-related indirect deaths; IPS defined directly related deaths as those caused by physical forces of the tornado or direct consequences of these forces such as structural collapse or flying debris.17–19 Indirect deaths are those caused by secondary hazards such as drowning from the ensuing floods.17–19 However, IPS excluded certain indirect deaths, such as those related to chronic disease exacerbation, acute disease from storm hazards, and repair injuries. Additionally, IPS excluded a reported death of an individual whose body was never recovered from the floodwaters, as there was no associated death certificate. In a retrospective study, VR matched the 48 deaths found by IPS with corresponding death records in EDRS to create a de-identified database that included different EDRS variables.

Key Data Elements and Data Quality

Evaluating Sensitivity, Validity, Reliability, and Timeliness

Sensitivity of each EFM was calculated as the total death records identified by each EFM divided by the 48 tornado-related deaths.

To evaluate validity, the use of each of the three EFMs as a cross-check for the other two was examined.

Reliability was evaluated by calculating Fleiss’ Kappa, as well as the average percentages of agreement. Additionally, Cohen’s Kappa for different pairwise combinations of the three EFMs was calculated.

To assess timeliness, four time variables were used: date of death, start date of the record in EDRS, date of death certificate certification, and date of death certificate registration. As EDRS did not provide time stamps when assigning each EFM to the record, proxies to estimate the EFM time stamps were used. For estimating when the OCME added the flag to the record, the proxy of the date the record was started in EDRS was used. For estimating when the OCME added the tornado term to the record, the proxy of the date of certification was used. These two proxies are likely close estimates as the OCME likely added the flag soon after they started the record in EDRS. Similarly, the OCME likely added the tornado term just before they certified the record. However, there is no close proxy for when NCHS returned the death record to VR with the X37 code. It can be said, however, that when VR registered the death record, it was ready to be sent to NCHS for X37 coding. So, the date of registration was used to approximate when the record was ready to be sent to NCHS for X37 coding.

Timeliness was approximated by calculating median and range in days for each EFM as follows: (1) Flag: death to start of record in EDRS; (2) Tornado Term: death to certification of the death certificate; and (3) X37 Code: death to registration of the death certificate.

Evaluating Usefulness and Acceptability

During a three-day site visit to Oklahoma in September 2016, open-ended discussions were held on mortality tracking during the May 2013 tornadoes with the following stakeholders from Oklahoma: the VR Division, the OCME, the State Epidemiologist, the Emergency Operations Center (EOC; Oklahoma City, Oklahoma USA), and a former president of the Oklahoma Funeral Directors Association (Oklahoma City, Oklahoma USA). These discussions provided key information to evaluate system attributes, like usefulness and acceptability, which were defined according to the Updated Guidelines for Evaluating Public Health Surveillance Systems.20

Results

Sensitivity

Out of the three EFMs, the ad hoc flag had the highest sensitivity at 89.6% (43/48), followed by the tornado term (75.0%; 36/48), and X37 code (56.2%; 27/48; Table 1). Out of the eight deaths resulting from drowning, many as a result of the deceased having taken cover in drainage ditches during flash flooding associated with the May 31 tornado, all eight records contained a flag. One of the eight contained a tornado term, and one of eight contained an X37 code.

Table 1.

Notes on System Attributes of Each Event Fatality Marker and of the Overall EDRS during Tornadoes – Oklahoma, May 2013.

Abbreviations: EDRS, Electronic Death Registration System; EFM, event fatality marker; ICD-10, International Classification of Diseases, Tenth Revision; IPS, Oklahoma State Department of Health Injury Prevention Service; NCHS, National Center for Health Statistics; OCME, Oklahoma Office of the Chief Medical Examiner.

Validity

Twenty-three of the 48 records (47.9%) contained all three EFMs. In other words, 52.1% (25/48) of death records were missing at least one EFM. Twenty of 47 (42.6%) records contained a flag but were missing either the tornado term and/or the X37 code. Thirteen of 47 (27.7%) records contained a tornado term but were missing either the flag and/or the X37 code. Four of 47 (8.5%) records contained an X37 code but were missing either the flag or the tornado term (Table 1).

Reliability

Average pairwise agreement for different pairwise combinations of the three EFMs and their Cohen’s Kappa values are in Figure 3. The tornado term paired with the X37 code had the highest average percent agreement 77.1% (95% CI, 62%-87%), and had the highest Cohen’s Kappa value (0.51; 95% CI, 0.26-0.76), which showed moderate agreement. The flag paired with either the tornado term or X37 had lower percentages agreement, and corresponding Cohen’s Kappa values showed no agreement. In addition, Fleiss’ Kappa value of 0.142 demonstrated poor agreement between all three EFMs (Figure 3).

Figure 3.

Percent Agreement for Different Pairwise Combinations of Event Fatality Markers during Tornadoes – Oklahoma, May 2013.

Timeliness

Timeliness of the EDRS was demonstrated using the three time intervals described previously. Death to the start of the record in EDRS was timely at a median of 2.0 days (range: 0-10 days). Death to certification of the death certificate was also timely at a median 3.5 days (range: 1-21 days). Death to registration was less timely at a median of 10 days (range: 2-122 days). The death records involved only a few delayed outliers, as demonstrated by the wide range of days, namely the 21 days and 122 days.

The availability of preliminary and final death certificate data was overall timely with a few delayed outliers. The time interval from death to start of the record in EDRS (which approximated when OCME added the flag) was the shortest interval (Table 1).

Usefulness

Although OCME entered flagging data into EDRS, OCME did not use flag data or tornado term data for compiling mortality reports for dissemination during the May 2013 tornadoes. Instead, as the morgue (the OCME) received bodies of tornado-related deaths, OCME added the deaths to an on-going line list, which was used to update public health and response agencies. During this time, EDRS had the capability of generating real-time, automated mortality reports using the flag, which could have provided enhanced support during the response effort. However, VR used the flag only internally, allowing VR to identify and collate the tornado-related deaths rapidly in response to potential inquiries about fatalities, and to compare its overall and daily mortality counts to that of the media, OCME daily reports, and state EOC situational reports (Table 1).

As described before, IPS conducted a comprehensive retrospective review of the OCME database, EDRS mortality records, and hospital discharge databases to find the 48 tornado-related deaths. However, IPS did not use the flag to collate tornado-related death records for its retrospective study.

The OCME did, however, use the EDRS (though not any EFM specifically) to correct an inaccurate death count. During the May 2013 tornadoes, as the morgue received decedent bodies, OCME compiled an on-going daily line list of the tornado-related deaths to report to government and response agencies. However, with extremely high mortality counts, protracted disaster events, or a highly chaotic atmosphere in the immediate aftermath of a disaster, keeping track of the death count using this method could be exceedingly difficult. During the chaos after the May 20 tornado, various local response personnel self-dispatched to the tornado-affected areas, resulting in double-counting decedents. As there was no set protocol to verify the on-going death count during the May 2013 tornadoes, the inaccurate, inflated death count unnecessarily triggered the mobilization of additional mortuary services. The OCME corrected the inaccurate death count using EDRS and canceled the additional services, saving the state millions of dollars.

Acceptability

The simplicity and flexibility of the ad hoc flag, simplicity and stability of EDRS, and EDRS’s mandated use by MEs contributed to acceptability of EDRS, in general, and of its ad hoc flag (Table 1). The EDRS’s interface is user-friendly and involves simple data entry into free-text boxes and selecting options from drop-down menus. The flagging procedure consists of clicking on the correct mass-fatality event variable from a simple drop-down menu incorporated into the existing death certificate interface that MEs were already using to certify deaths. The MEs reported that entering the tornado term into the death certificate module in EDRS was also an inherently simple procedure as it involved typing the appropriate terms into a free-text box. Additionally, online tutorials and prompt customer service helped streamline end-user experience with EDRS. The system backed-up data in multiple locations, and EDRS was financially stable. Having only two agencies (OCME and VR) involved in the flagging procedure also contributed to the simplicity of the system. The VR could easily add new flag drop-down menu response choices as additional tornadoes occurred, and then notify the ME to begin using them, thus highlighting the flexibility of the flagging system. After the tornadoes, public health researchers easily searched EDRS text fields and response options, including the tornado term and X37, to find tornado-attributed deaths.

Stakeholders reported that the lack of a written flagging protocol or guide for how to use the flag and tornado term presented problems. For example, they had no specific, agreed-upon tornado-related terms to include on the decedents’ death certificates. Also, the definition of tornado-attributed deaths was not clear. Though the ICD-10 mortality coding protocol is complex, ensuring that the death record included a tornado-related term would have increased the possibility of receiving the X37 ICD-10 code, thus simplifying identification of deaths for retrospective studies.

Discussion

Impact

In this evaluation, sensitivity, validity, reliability, and timeliness of the EFMs were assessed, as well as usefulness and acceptability of the overall EDRS and the individual EFMs. This approach highlights the EFMs collective roles in disaster-related mortality surveillance. The EDRS overall proved to be useful and acceptable with excellent service quality and a simple, user-friendly interface. The tornado term and X37 ICD-10 code were valuable for collating deaths for retrospective research.

The ad hoc tornado flag, however, proved to be the most favorable EFM to use for tracking disaster-related deaths in future disaster responses. The flag was simple, flexible, and acceptable. It was the most sensitive and most timely of the three EFMs. The flag had the highest sensitivity, not only for the deaths directly related to the tornado, but also for the indirectly-related deaths due to drowning. It also identified the highest percentage of records missing at least one other EFM, and thus, was the most optimal cross-check of the three EFMs. Additionally, the flag had a role in facilitating intra-agency communication during the tornadoes – particularly between the OCME and VR.

Novel uses for the flag in future disasters include the following: (1) tracking of disaster-related deaths for situational awareness in an accurate and timely manner; (2) informing policy makers of public health messaging to prevent further mortality; (3) cross-checking records for presence of disaster-related term and ICD-10 code; and (4) collating disaster-related deaths for retrospective research.

It should be emphasized that death certifiers should not view the flag as an alternative for including the disaster-related term in the death certificate literal text. The disaster-related term is necessary for appropriate assignment of a disaster-related ICD-10 code at NCHS.

Linkage Ability

In 2015, well after the 2013 tornadoes, OCME had launched its own disaster-related death flag within its new internal, electronic case management system, which is not inter-operable with EDRS for a number of legal, political, budgetary, and operational reasons. During the May 2013 tornadoes, the ME staff copied and pasted data from their system into the corresponding field in the EDRS death certificate module. This practice persisted at the time of the stakeholder discussions. Future linkage analyses can include the OCME case management system.

Next Steps

The poor agreement in the pairwise combinations of the flag with the other two EFMs and poor agreement between all three EFMs underscores the need for a flagging protocol and supplemental training on how to document and track deaths related to a disaster. Drafting a written protocol for EDRS users, designating how to use the flag, and tornado terms for these purposes would help simplify and streamline the disaster mortality surveillance system. The protocol can include options for harmonizing the flag in the OCME case management system with the EDRS flag.

As the OCME and VR are the two main agencies monitoring and disseminating disaster mortality data, inter-operability between the OCME case management system and EDRS would also streamline the system and enhance timeliness and data quality of disaster mortality data. Piloting a one-way transfer of OCME data to EDRS with safeguards to protect sensitive legal information in the OCME system would be a way to initiate inter-operability between the two systems.

Oklahoma may consider performing additional sensitivity, reliability, and timeliness analyses on the use of EFMs in future disasters to further identify gaps and challenges in EFM use. The integration of time stamps into EDRS for certain steps in the mortality data flow process could improve timeliness data analyses in future disaster settings. Examples of these additional variables could include: (1) when OCME assigned the flag to the death record; (2) when VR sent the death file to NCHS; and (3) when VR received the file with ICD-10 codes from NCHS.

Finally, performing further analyses in collaboration with NCHS to explore ways to improve the percent agreement between the disaster term in the death certificate and the disaster ICD-10 code can also enhance disaster mortality surveillance.

Limitations

Because EDRS does not include exact time stamps for when the three EFMs were applied, approximations had to be used. Therefore, analysis of timeliness is an estimate. Certain indirect deaths were excluded from this study. Inclusion of these deaths would have affected the sensitivity, timeliness, and reliability results. Reliability analyses assume that the people who assign the EFMs are independent of each other. These analyses are included despite knowing that the same person (the ME) assigns both the flag and the tornado term to the record.

Conclusion

Overall, the Oklahoma EDRS and its three EFMs (tornado flag, tornado term, and X37 ICD-10 code) successfully tracked deaths attributed to the May 2013 tornadoes. Suggested improvements in the areas of sensitivity, validity, reliability, timeliness, usefulness, and acceptability will improve disaster response using EDRS and help inform mortality prevention public health messaging. The VR and OCME can use the EDRS disaster flag, the most sensitive and timely of the three EFMs, during all phases of the disaster response to collate death records for immediate response needs or retrospective research. The flag also has great potential for facilitating cross-agency communication during a response. Developing a written disaster mortality surveillance protocol to harmonize the OCME flag with the EDRS flag and delineate uses of the EFMs may improve future disaster mortality surveillance efforts.

Footnotes

Conflicts of interest: none

References

Brown, S, Archer, P, Krueger, E, Mallonee, S. Tornado-related deaths and injuries in Oklahoma due to the 3 May 1999 tornadoes. WAF. 2002;17(3):343–353.Google Scholar
Centers for Disease Control and Prevention (CDC). Tornado-related fatalities — five states, Southeastern United States, April 25–28, 2011, 2012. MMWR. 2012;61(28):529–533.Google Scholar
Morton, M, Levy, JL. Challenges in disaster data collection during recent disasters. Prehosp Disaster Med. 2011;26(3):196–201.CrossRefGoogle ScholarPubMed
Heim, RT. Electronic Death Registration. PowerPoint presentation. August 2010. https://www.cdc.gov/nchs/ppt/nchs2010/26_trasatti.pdf. Accessed June 10, 2017.Google Scholar
Baker, K. Partnering to Enhance Electronic Death Registration for Disaster Related Deaths. PowerPoint presentation. May 2015. http://www.cste.org/resource/resmgr/DisasterEpi/Baker_Disaster_Epi_Conferenc.pdf. Accessed January 10, 2017.Google Scholar
Rocha, LA, Fromknecht, CQ, Redman, SD, Brady, JE, Hodge, SE, Noe, RS. Medicolegal death scene investigations after natural disaster- and weather-related events: a review of the literature. Acad Forensic Pathol. 2017;7(2):221–239.CrossRefGoogle ScholarPubMed
Hanzlick, R, Combs, D. Medical examiner and coroner systems: history and trends. JAMA. 1998;279(11):870–874.CrossRefGoogle ScholarPubMed
Hanzlick, R, Parrish, RG. The role of medical examiners and coroners in public health surveillance and epidemiologic research. Ann Rev Public Health. 1996;17:383–409.CrossRefGoogle ScholarPubMed
Brunkard, J, Namulanda, G, Ratard, R. Hurricane Katrina deaths, Louisiana, 2005. Disaster Med Public Health Prep. 2009;2(4):215–223.CrossRefGoogle Scholar
Choudhary, E, Zane, DF, Beasley, C, et al. Evaluation of active mortality surveillance system data for monitoring hurricane-related deaths - Texas, 2008. Prehosp Disaster Med. 2012;27(4):392–397.CrossRefGoogle ScholarPubMed
Howland, RE, Li, W, Madsen, AM, et al. Evaluating the use of an electronic death registration system for mortality surveillance during and after Hurricane Sandy: New York City, 2012. Am J Public Health. 2015;105(11):55–62.CrossRefGoogle ScholarPubMed
Johnson, MG, Brown, S, Archer, P, Wendelboe, A, Magzamen, S, Bradley, KK. Identifying heat-related deaths by using medical examiner and vital statistics data: surveillance analysis and descriptive epidemiology– Oklahoma, 1999–2011. Environ Res. 2016;150:30–37.CrossRefGoogle Scholar
Olayinka, OO, Bayleyegn, TM, Noe, RS, Lewis, LS, Arrisi, V, Wolkin, AF. Evaluation of real-time mortality surveillance based on media reports. Disaster Med Public Health Prep. 2017;11(4):460–466.CrossRefGoogle ScholarPubMed
Zane, DF, Bayleyegn, T, Hellsten, J, et al. Tracking deaths related to Hurricane Ike, Texas, 2008. Disaster Med Public Health Prep. 2011;5(1):23–28.CrossRefGoogle Scholar
Personal communication with first author: Kelly Baker, MPH. Oklahoma City, Oklahoma USA: Centers for Disease Control and Prevention; September 19, 2016.Google Scholar
Centers for Disease Control and Prevention (CDC). Disaster Preparedness and Response Training: Complete Course. Participant workbook, 1st edition. Atlanta, Georgia USA: CDC; 2014.Google Scholar
Centers for Disease Control and Prevention (CDC). Death Scene Investigation After Natural Disaster or Other Weather-Related Events Toolkit. First edition. Atlanta, Georgia USA: CDC; 2017.Google Scholar
Coombs, DL, Quenemoen, LE, Parrish, RG, Davis, JH. Assessing disaster-attributed mortality: development and application of a definition and classification matrix. Int J Epidemiol. 1999;28(6):1124–1129.CrossRefGoogle Scholar
Centers for Disease Control and Prevention (CDC). National Center for Health Statistics. A Reference Guide for Certification of Deaths in the Event of a Natural, Human-Induced, or Chemical/Radiological Disaster. Hyattsville, Maryland USA: CDC; 2017.Google Scholar
Centers for Disease Control and Prevention (CDC). Updated Guidelines for Evaluating Public Health Surveillance Systems: Recommendations from the Guidelines Working Group. MMWR Recomm Rep. 2001;50(RR-13):1–35.Google Scholar
Figure 0

Figure 1. Key Features of the Event Fatality Markers in the Electronic Death Record during Tornadoes – Oklahoma, May 2013.

Abbreviations: EDRS, Electronic Death Registration System; ICD-10, International Classification of Diseases, Tenth Revision; NCHS, National Center for Health Statistics; OCME, Oklahoma Office of the Chief Medical Examiner; VR, Oklahoma State Health Department Division of Vital Records.Note: The X37 code designates Victim of a Cataclysmic Storm and includes tornadoes, along with other storm systems like hurricanes. The NCHS assigns ICD-10 codes to registered death records received from state vital records offices. The NCHS’s computerized systems use a complex algorithm to search the death record for keywords that prompt assignment of certain ICD-10 codes. To add the X37 code in the death records for tornado-attributed deaths, NCHS’s algorithm searched for tornado-related terms, like “tornado” or “twister,” in the literal text and underlying cause of death sections in the death certificate. Typically, the ME added these terms to the literal text of the How Injury Occurred section of the death certificate. If such tornado terms were not included, the death record would likely not receive the X37 code. In May 2013, VR instructed the ME to add the flag to the death record in EDRS as well. However, NCHS’s algorithm cannot detect this flag, which is only part of the electronic death record and not part of the official death certificate. Therefore, the ICD-10 coding algorithm does not depend on the tornado flag.
Figure 1

Figure 2. Mortality Data Flow during Tornadoes – Oklahoma, May 2013.

Abbreviations: EDRS, Electronic Death Registration System; ICD-10, International Classification of Diseases, Tenth Revision; NCHS, National Center for Health Statistics; OCME, Oklahoma Office of the Chief Medical Examiner.Note: Shortly after the first of three tornadoes affecting central Oklahoma in May 2013, VR piloted an ad hoc flagging procedure to track tornado-related deaths within their EDRS. The VR notified the OCME to start flagging tornado-related deaths. After post-mortem assessment, the OCME flagged tornado-related deaths and included terms such as “tornado” in the death certificate. After the OCME completed the preliminary death certificate in EDRS, VR finalized the death certificate and sent a death data file containing the tornado term to the NCHS. The NCHS used computerized systems to apply a complex algorithm to assign ICD-10 codes to each death record. This algorithm searched for key words like “tornado” to assign the ICD-10 code for tornadoes (X37 code).
Figure 2

Table 1. Notes on System Attributes of Each Event Fatality Marker and of the Overall EDRS during Tornadoes – Oklahoma, May 2013.

Figure 3

Figure 3. Percent Agreement for Different Pairwise Combinations of Event Fatality Markers during Tornadoes – Oklahoma, May 2013.