Hurricane Sandy was one of the most devastating storms ever to impact the Northeastern United States. Although it had slowed to a posttropical cyclone just before making landfall on October 29, 2012, the storm’s unprecedented size caused 53 deaths and damaged over 300 000 housing units in New York State alone. 1 , Reference Blake, Kimberlain and Berg 2
Studies have documented the impact that hurricanes have on individuals’ mental health. Following Hurricane Katrina, many individuals living in affected areas reported or screened positive for mental distress in the form of posttraumatic stress disorder (PTSD), depression, or other anxiety and mood disorders.Reference Abramson, Stehling-Ariza and Garfield 3 Moreover, studies have shown that mental distress prevalence remains high months or even years following a disaster,Reference Galea, Brewin and Gruber 4 and is highly associated with the level of exposure, or the severity and or number of traumas a person experiences during the event.Reference Cerda, Bordelois and Galea 5 For these reasons, mental health should be added to the list of syndromes tracked by state public health tracking systems.
In addition, the physical force and resulting destruction of hurricanes are a major cause of injuries, often surging the number of ED visits.Reference Miller, Kearney and Proescholdbell 6 However, the increased risk for hurricane-related injuries is not limited to the immediate storm period or to direct exposure to the storm.Reference Shultz, Russell and Espinel 7 Traumatic injuries can be dwarfed in scale by the hazards posed by storm preparation and evacuation, reentry to damaged regions, initial cleanup efforts, and long-term reconstruction.Reference Sullivent, West and Noe 8 , Reference Uscher-Pines, Vernick and Curriero 9 In densely populated and highly developed coastal urban centers like the New York City metropolitan region, the risk of injury is magnified even further.Reference Lane, Charles-Guzman and Wheeler 10
Syndromic surveillance is a public health detection system used to monitor disease trends and to track early signals of mass health events, such as bioterrorism and infectious disease outbreaks.Reference Henning 11 Emergency department (ED) syndromic surveillance is recognized as a valuable tool for informing public health activities during and immediately following a disaster. Data typically consist of daily visit reports from hospital EDs of basic patient data and chief complaints. In contrast to a formal diagnosis, a chief complaint is the initial health complaint that motivated the patient’s visit to the ED. Chief complaints are typically recorded as free text rather than a predefined category or code. Although they lack the accuracy of a final diagnosis, chief complaints provide an opportunity for capturing the health impacts of disaster events in near-real time, and facilitate continuous surveillance from the immediate impact period through long-term recovery.
Timely tracking of disease patterns is crucial to disaster planning, response, and recovery. At the time that Hurricane Sandy hit, the New York State Department of Health (NYSDOH) Division of Epidemiology was already using its syndromic surveillance system to track a number of disaster-related disease syndromes, including asthma, carbon monoxide poisoning, gastro-intestinal disease, and hypothermia. However, no syndromic surveillance filters were then available to monitor and track trends in mental health or injury, 2 health conditions that are significantly impacted by hurricanes.Reference Shultz, Russell and Espinel 7 Since mental health and injury are important and sensitive indicators of the short- and long-term impacts of disasters, creating syndromes for these 2 health outcomes was a high priority after Hurricane Sandy. Furthermore, determining how wide a net to cast to effectively capture the critical range of mental health and injury subcategories had not yet been clearly defined. With the storm providing a window of opportunity for new disease surveillance, the aim of this project was to create comprehensive sets of mental health and injury syndrome filter keywords, and to apply them to the evaluation of short-, medium-, and long-term trends in post-Hurricane Sandy mental health and injury outcomes in New York State.
Methods
Each year the NYSDOH syndromic surveillance system receives data on roughly 4 million ED visits from 140 New York hospitals. (This number excludes New York City hospitals, which report directly to the New York City Department of Health and Mental Hygiene.) These reports, which are sent every 24 hours, contain visit date, chief complaint, and basic patient demographics (date of birth, sex, residential ZIP code, and occasionally race). The system uses keyword search lists to monitor daily counts of various illness categories in each county. Syndromes that are currently tracked on a daily basis include fever, gastrointestinal illness, asthma, rash, and neurological complaints, among others.
Since mental health and injury rates were not being tracked by the syndromic surveillance department at the time of Hurricane Sandy, we generated lists of keyword search terms to identify mental health and injury related ED visits to study the storm’s short- and long-term impacts on ED visits for these categories. Access to these data has been granted by the NYSDOH Institutional Review Board.
We began this process with a comprehensive literature search. Using CDC-defined categories (BioSense) as a key reference, we conducted iterative keyword queries into New York State’s syndromic surveillance database of ED visit chief complaints. Syndromic categories are defined as the broad health classifications, in this case “injury” or “mental health.” Unlike simple syndromes like asthma or carbon monoxide poisoning, both of these are considered compound categories since they are made up of multiple subcategories. Subcategories are defined as more specific classifications within each category, such as “falls” under the injury category or “mood disorders” under mental health. Table 1 lists CDC classifications for the mental health and injury subcategories, the corresponding ICD codes, and the initial sub-categories developed for use by NYSDOH. The injury subcategories were compared to the CDC Reportable Mechanism Categories for Injury-Related Hospitalizations/ED Visits (http://www.cdc.gov/injury/wisqars/cost_help/mechanism_injury.html), and the mental health subcategories were compared to the annual average number and rate of ambulatory care visits for mental health disorders among adults aged 18 or older (http://www.cdc.gov/mmwr/preview/mmwrhtml/su6003a1.htm).
Comparison of CDC Categories, ICD-9 Codes, and NYSDOH Categories
This list was then compared to lists from the New York City and New Jersey health departments for validation and consequently refined. The final lists of keywords were incorporated in two 20-page specifications documents (not provided in this paper). Figure 1 summarizes the general process.
Schematic diagram of keyword list generation process
Creating Compound Categories
Rather than being a single condition, the term “mental health” is a heuristic for a wide variety of behavioral, genetic, and neurological disorders and diseases. Similarly, “injuries” may occur intentionally or unintentionally and can affect any part of the body. Thus, while the existing syndromes may contain a limited number of search terms—eg, the asthma keyword list is composed of 8 search terms—the mental health list has 9 subcategories and the injury list has 7 subcategories (Table 2). Each has over 100 core terms. This fact has both advantages and disadvantages. The obvious disadvantage is the upfront investment in time and effort required to generate and validate lists with numerous, overlapping subcategories. For example, the high degree of correlation of some syndromes across multiple subcategories can make it difficult to isolate a single condition. In the case of mental health visits, a chief complaint containing DEPRESSION will often also contain complaints of anxiety, suicide/self-harm, psychosis, substance abuse, and bipolar complaints. For injury visits, a general term like HIT may appear in complaints related to accidental contusions (HIT HEAD), vehicular accidents (HIT BY CAR) or intentional injury (ASSAULTED, HIT WITH BASEBALL BAT), among others.
Final List of Mental Health and Injury Subcategories
Working With Free Text Fields
Another consideration when generating keyword lists is the free-form nature of chief complaint text fields, especially when compared with standardized diagnosis codes. Up until the implementation of public health syndromic surveillance systems, the chief complaint in the ED record was only intended for internal hospital staff use during the time of the ED visit. The aim was to capture the nature of the ailment from the patient’s perspective, and thereby provide hospital staff with a starting point for the clinical inquiry. As such, not only was there was little need for standardized terms, but capturing the patient’s complaint in her own words could provide potentially valuable contextual information for clinicians.
In an automated syndromic surveillance system, however, the free text format of chief complaints can lead to a number of classification issues. For example, a lack of consistency—both among an individual hospital’s staff and across hospitals—in describing similar events and symptoms can add numerous additional search terms to the keyword list. In the mental health list, MHE, MHA, and PSYCH EVAL are all synonymous for “mental health evaluation,” while ETOH and ALCH can both indicate alcohol intoxication. Among injury categories, CAR, VEH, MVA, MVC, CRASH, and COLLISION can all indicate a motor vehicle accident. Also, alternate spellings, abbreviations, misspellings, and typos need to be incorporated into the search terms. Common examples include COCANE for COCAINE, MOTOR CYCLE, MOTERCYCLE and MOTER CYCLE for a motorcylce [accident], and WND for WOUND.
Without a primary diagnosis, the presence of multiple complaints in a single text field presents a classification challenge for syndromic surveillance units. Some units create a hierarchy of syndromes and limit classification to a single syndrome accordingly. Other departments, including NYSDOH, allow single ED visits to be listed in multiple syndromes. Thus the single complaint ETOH FELL BROKE LEG may be classified as an alcohol (ie, mental health) related visit and or a leg injury. The vernacular and contextual descriptions that are likely among the most useful to ED staff may defy keyword categorization completely. THINKS HE CAN FLY speaks volumes to hospital staff but contains little in the way of searchable terms. Finally, an inability to detect nonmandatory missing information can result in an overabundance of false negatives when a term is not present. Terms like WORKERS COMP and codes for police escorts are usable search terms only if they are listed consistently.
Code Development and Initial Validation
An iterative search process was used to identify as complete a range of keywords as possible and to exclude or modify search terms that resulted in false positives. Daily hospital records covering the 5-year study period were first imported into SAS v. 9.3. Using the shortest possible search term segment (eg, INJ for INJURY), searches were conducted in the chief complaint field and sorted by frequency. Common abbreviations and misspellings were identified through this process and added to the keyword list.
Next, searches were conducted for secondary terms or terms that would plausibly occur with a target keyword, and again sorted by frequency of chief complaint. For example, a search for a term such as LEG would produce a wide range of synonyms for INJURY (FALL, TRIP, TWIST, FRAC, etc.) that would be searched for in turn. Furthermore, reviewing results by frequency also worked to identify exclusion terms, eg, the specificity of BURN increased when BURNING, HEARRTBURN, URIN, AND SUN were excluded from the search. Likewise, the use of NAIL required that INGROW, TOE, FINGER, and POLISH were excluded. In cases where unrecognized terms or numeric codes were used consistently by particular hospitals, definitions were confirmed by knowledgeable hospital staff members.
The final requirements document for each keyword list included the primary keyword, alternate spellings, exclusionary terms, examples/explanations, and the number of records that contained the term. An excerpt from the injury requirements document is shown in Table 3.
Excerpt From the Injury Requirements Document
Discussion
The advantage of such a system is that each list can be implemented as either a single, large, catch-all category or as individual subcategories (eg, vehicular accidents or work-related injuries), depending on the needs of the disaster event and or end users. The full list of keywords can therefore be monitored in the case of such an event. Conversely, if an individual county requests ongoing surveillance of a particular health outcome such as teen suicide or falls among elderly, the individual subcategories can be easily incorporated into custom surveillance activities.
There was not always consensus in the literature about how best to categorize subcategories in either mental health or injury. For example, CDC and others include cognitive and memory disorders, including dementia, in their search terms for “mental health.” While this makes heuristic sense from a public health standpoint, it would be more accurate to include those terms into a separate Neurological syndrome. Additionally, we eventually excluded an 11th mental health subcategory on “fainting and disorientation” (a common symptom of anxiety) since the symptom is so broad and picks up too many false positives to be of much use. For both mental health and injury we suggest comparing such chief complaints to the ICD codes listed in the corresponding hospital admission records dataset to validate the keywords.
Psychological trauma may lead to behavioral issues, substance abuse, and or violent behavior. Currently each keyword is only listed once. Another issue in the mental health category is the fact that the Addiction and Substance Abuse subcategory does not include synthetic drugs or many slang terms. Currently a separate syndrome exists for synthetic drugs. Cursory searches for slang terms did not yield significant results. The 2013 NYC list does not include terms from this category. For general use (outside of the Sandy studies) it may work better as a separate syndrome.
As with the Mental Health list, the Injury surveillance syndrome is intended to capture the full range of injuries reported in NYS emergency departments. Thus, while it was initially created to support the current Hurricane Sandy study, the subcategories and keywords address injuries beyond the scope of that study’s aims. For example, the Sports and Recreation subcategory was created as part of this effort but was not considered to be a high priority for tracking storm-related injuries. It should also be noted that terms related to surgery or follow-up care (SUTURE REMOVAL, SURGICAL WOUND, etc.) were excluded from the main lists since the project’s aim was to study initial injury events. Since these terms have been listed, they could easily be incorporated into a search list if needed.
Implications
Despite the challenges inherent in working with syndromic surveillance datasets, including compound categories and free text fields, tracking mental health and injury trends during and after disaster events can provide valuable insight into the impact and trends of mental health and injury outcomes on a community for the purposes of real-time surveillance, monitoring these health outcomes over time, and mitigation and future disaster planning. Furthermore, collaboration in the development of keyword lists can lead to standardized, comparable surveillance practices across large regions impacted by the same or similar events. Although the new mental health and injury syndromes that were created for this project are not currently being tracked by the NYSDOH syndromic surveillance system on a daily basis, they can be deployed at a moment’s notice as the need arises.
Future Research
Several follow-up studies are planned. First, we intend to validate these 2 new filters by comparing them with finalized ICD coded ED data. Similar validation studies have been conducted by NYSDOH and the New Jersey health department for carbon monoxide poisoning and extreme weather keyword lists, respectively.Reference Chen, Schmit, Chang and Miller 12 , Reference Hamby, Tsai and Gu 13 Both these studies have shown strong correlations between ICD and syndromic surveillance keyword lists. Next, studies are being considered to compare injury and mental health chief complaints across different recent disaster events in New York State, including the 2009 blackout, Hurricane Irene, and the 2014 Buffalo blizzard. Finally, we plan to conduct regional comparisons with the New York City and New Jersey health departments. Findings from these studies will be reported in separate papers.
Acknowledgments
The following people contributed to this paper: Hwa-Gan Chang (New York State Department of Health), Stella Tsai (New Jersey Department of Health), and Ramona Lall and Hilary Parton (New York City Department of Health and Mental Hygiene).
Funding
FOA Title: CDC RFA-TP13-001, Centers for Disease Control and Prevention Public Health Preparedness and Response Research to Aid Recovery from Hurricane Sandy. Project Title: Assessing Multiple Health Endpoints, Vulnerability and Risk factors in the aftermath of Hurricane Sandy in New York State. NOA: 1U01 TP000566-01.
