Introduction
Quantitative measurements of the medical severity of multiple-casualty events (MCEs) is an important facet of disaster metrics, which, in themselves, are postulated to be at the core of evidence-based disaster medicine.Reference Debacker 1 - Reference Bayram, Zuabi and Subbarao 4 The importance of a quantitative scale that measures the severity of the medical impact of MCEs is multifaceted. First, it directly influences disaster response, allocating scaled resources based on an objective severity level. Second, it would directly affect preparedness efforts by providing historical numerical values that can inform hazard-vulnerability analysis. Third, a quantitative scale is also important for comparative research, by measuring the medical impacts of different types of MCEs given certain parameters, and by discriminating severity within specific types of MCEs, be it man-made or natural. Fourth, a quantitative scale with assigned scoring rubrics for each parameter would help standardize data collection.
The first proposal to classify the medical severity of MCEs occurred in August of 1979 at the International Conference on Disaster Medicine in Cape Town, South Africa. 5 During the conference, it was proposed that disaster severity be classified according to seven parameters (cause, duration, radius, number of casualties, nature of injuries, rescue time, and effect on the surrounding community). 5 Subsequently, a working group on disaster medicine was developed and adopted this classification,Reference Rutherford and de Boer 6 which fell short of developing an actual quantitative scale as numerical values were not assigned to any of the seven parameters. It was not until 1990 that de Boer et al assigned numerical values to these parameters and presented the Disaster Severity Scale (DSS; Table 1), which ranged from a lowest possible score of one to a highest score of 13.Reference de Boer 7 - Reference de Boer 8
Disaster Severity Scale, de Boer (1990)
In 2005, Ferro applied the scale to major and minor MCEs of various causes occurring in Italy during the last century. Ferro observed that natural events scored the highest on the DSS and none of the man-made events scored higher than eight points on the DSS. Aside from Ferro's isolated preliminary application, there has been no statistical analysis of the reliability or dimensionality (number of salient themes) of the DSS,Reference Ferro 9 which this study aimed to do through its application to 144 discrete earthquake events. A description of the technical terms used is provided in Table 2.
Technical Terms with Descriptions
Methods
Individual earthquakes from 2003 through 2013 were identified based on their listing in both the National Oceanic and Atmospheric Administration's (NOAA; Washington DC, USA) Significant Earthquake Database 10 and the US Geological Survey's (USGS; Reston, Virginia USA) “Did You Feel It” database. 11 For the cause parameter, earthquakes were assigned a score of one for “natural,” as specified by de Boer's DSS. In addition, for the duration of the disaster, the initial earthquake shockwaves were assumed to last less than one hour and assigned a score of zero according to de Boer's DSS. The radius parameter was estimated by the distance from the epicenter of the earthquake to the farthest reported Modified Mercalli Intensity (MMI) of V or more (Table 3) since higher MMI scores are associated with the occurrence of physical injuries.Reference Shoaf, Nguyen, Sareen and Bourque 12 - Reference Peek-Asa, Ramirez, Seligson and Shaof 14
Modified Mercalli Intensity Scale
The number of casualties parameter was scored based on the number of dead and injured listed in the NOAA's “Significant Earthquake Database.” Earthquakes that had fewer than two casualties (injured or dead) were excluded, as they did not represent MCEs, and those with two to 25 casualties were considered minor and given a score of zero. Two experts in the field of disaster medicine, and authors of this study (JDB and SZ), independently reviewed and assigned scores for the remaining three parameters: nature of injuries, rescue time, and effect on the surrounding community. Differences were reconciled via consensus. For statistical analysis, SPSS Statistics for Windows Version 22.0 (IBM Corp; Armonk, New York USA) was used. Cronbach's alpha, a measure of internal consistency, was used to calculate the reliability of the DSS. Principle Component Analysis, a measure of dimensionality (number of salient themes or constructs) in a data set, was used to evaluate the dimensionality of the DSS.
Results
A total of 144 earthquakes were scored based on the methodology outlined previously. The mean total score was 6.04, the mode score = 5, minimum score = 4, maximum score = 11, and standard deviation = 2.23. Detailed descriptive statistics for each of the seven parameters are shown in Table 4.
Descriptive Statistics
Abbreviation: DSS, disaster severity scale.
Three parameters in de Boer's DSS (cause, duration, and radius) had zero variance (ie, all of the 144 earthquakes had the same score in each of the three parameters). These scores were: one for the cause, due to being classified as natural disasters; zero for the duration, since the initial shockwaves on all earthquakes were assumed to last less than one hour; and two for radius, since all earthquakes were estimated to have a radius greater than 10 km. Because of the null contribution to statistical variance, these three parameters in the DSS had to be removed in order to run further statistical analysis regarding reliability and dimensionality. For the remaining four parameters, Cronbach's alpha score, a coefficient of internal consistency, was calculated to be 0.89. This value did not increase significantly if any of the four parameters were deleted (Table 5).
Item-total Statistics
Principle Component Analysis of the 4-parameter abbreviated DSS scale revealed one major component with an eigenvalue of 3.17, contributing 0.793 of the variance in the data. The other three components all had eigenvalues < 0.45 (Table 6).
Total Variance Explained
Table 7 shows the correlation between each of the four parameters and the component extracted.
Discussion
Quantifying the medical-severity impact of various MCEs is one of the most important aspects of disaster medicine. Other than an attempt by Rutherford, which was subsequently expanded by de Boer, there has not been a serious effort to develop a scale that measures the acute medical severity of various MCEs. Furthermore, aside from the isolated preliminary application of the DSS by Ferro et al, there has been no analytical consideration of de Boer's 7-parameter DSS for reliability or dimensionality. Based on the findings through an examination of 144 earthquake events, the 7-parameter DSS in its current format does not hold up statistically. For the dataset, three identified parameters, namely cause, duration, and radius, did not contribute at all to the variance of the DSS. Accordingly, they may undermine the reliability, and hence, the validity, of the scale itself, at least with respect to earthquakes. On closer analysis, it appears that these three parameters are therefore better indicators of risk severity than they are of medical-impact severity.
Cronbach's alpha is a measure of internal consistency (ie, how closely related a set of items are as a group when measuring an underlying construct), in this case, the medical severity of MCEs.Reference Cronbach 15 - Reference Takavol and Dennick 18 Statistical analysis of the 4-parameter abbreviated DSS showed an excellent internal reliability among the four parameters with Cronbach's alpha of 0.89 (Table 5). Looking at these four parameters (number of casualties, severity of injuries, rescue time, and effect on surrounding communities), they are all conceptually related and would be expected to rise and fall in unison. As the number of injured and dead increases, the time needed for rescue by prehospital medical services, the severity of injuries, and the effect on the surrounding communities are all likely to increase.
In every set of observational data in a scale or index, it is also important to test how many dimensions this data set really measures.Reference Hyvarinen, Karhunen and Oja 19 - Reference Rummel 24 For example, in the case of the DSS, do the data measure the medical severity of MCEs, or do they also measure some other important aspect? Principle Component Analysis is one of the most commonly used statistical methods to test for dimensionality, and components that have eigenvalues greater than one (1.0) indicate themes or constructs that should be taken into consideration. Principle Component Analysis of the abbreviated 4-parameter DSS (Table 6) shows uni-dimensional characteristics, with only one component having an eigenvalue greater than one at 3.17. This component, assumed to be the medical severity of MCEs, is well represented by each of the four parameters, which measure a common theme of “medical impact” (Tables 6 and 7).
However, even in its 4-parameter abbreviated format, the DSS is flawed in its capacity to discriminate and differentiate between various MCEs. For example, the DSS parameter number of casualties does not discriminate above 500. All of the MCEs resulting in more than 500 casualties will receive a score of two on this parameter. Accordingly, in this study's dataset, the 2010 Haiti earthquake with more than 500,000 casualties will receive a score of two, identical to the 2011 New Zealand Christchurch earthquake that resulted in 1,863 total casualties. Intuitively, the resulting medical severity differed significantly between these two earthquakes, but is not reflected in the scoring rubric of this parameter. Similar arguments can be used in relation to the three other parameters and the range of the scale itself. The range of the DSS (one to 13) is too narrow, which severely restricts the discrimination of medical severity and limits its utility for comparative interpretation. To highlight this, the total score on the DSS for the 2009 L'Aquila earthquake (Italy; 1,295 injured or dead) scored 11 points out of 13, an equivalent score to the far more catastrophic 2010 Haiti earthquake with more than 500,000 casualties.
It is very important to note that the present discussion revolves around a scale that measures severity of the medical impact of MCEs. The intended scale does not measure other important aspects of impact from various “disasters,” such as: environmental (eg, the BP (BP plc; London, England) oil spill in the Gulf of Mexico in 2010); infrastructure (eg, Hurricane Sandy in New York USA in 2012); psychological (eg, the September 11 terrorist attacks in the US in 2001); or financial (eg, Wall Street market crash in New York in 2009). Measuring the severity of impact from a major cyber attack on the banking system, for example, requires another scale composed of different parameters. Such an attack could cripple an entire nation, but may have no immediate, direct physical casualties, and a scale like the one discussed in this study would not be applicable.
Limitations
This study had several limitations. First, no single comprehensive database exists that provides all the scores needed on all parameters of the DSS. For instance, calculating the furthest radius of the earthquake where casualties occurred was challenging, since this is not recorded in any single database. The radius parameter was estimated based on a reported MMI of V or more, which is supported by the literature.Reference Shoaf, Nguyen, Sareen and Bourque 12 Second, the margin of error for the statistical analysis is dependent on the cumulative potential margin of errors on the scores reported on each parameter, when applicable. Third, three parameters in the DSS (nature of injuries, rescue time, and effect on community) are not documented in any database and had to be estimated by two experts, who were not blinded to the study objectives as they were the first two authors of this study. To mitigate this potential bias, the scoring was performed independently and earthquakes were listed chronologically rather than by severity to avoid direct comparison of scores. Fourth, this was a prospective study with unknown values on three parameters. Prospective data collection, albeit not feasible at the present time as it requires global consensus and infrastructure, would eliminate much of evaluator/scorer bias noted previously. Finally, due to a shortcoming of the 7-parameter DSS itself, three parameters contributed zero variance to the data and had to be removed before conducting further statistical analysis.
Conclusion
Jan de Boer's DSS in its 7-parameter format fails to demonstrate reliability and uni-dimensionality when applied to a dataset of 144 earthquakes subjected to analysis. A modified 4-parameter scale more directly assesses medical severity; however, it remains flawed due to range restriction on both individual parameter and total scale levels. There is significant utility in further research to develop a revised scale that in its complete set of parameters is reliable, uni-dimensional, and capable of better fine discrimination in its measurement of the medical severity of MCEs.
Acknowledgments
The authors would like to acknowledge Dr. Daniel Barnett for editing the manuscript and Matthew Toerper for his technical assistance.
