Hostname: page-component-74656f9884-c52kr Total loading time: 0 Render date: 2026-11-07T08:10:31.405Z Has data issue: false hasContentIssue false
  • English
  • Français

Development and Validation of a New Tool to Improve the Accuracy of the Hospital Mass-Casualty Incident Response Plan Activation: The PEMAAF Score

Published online by Cambridge University Press:  24 November 2023

Claudia Ruffini Open the ORCID record for Claudia Ruffini [Opens in a new window] ,
Monica Trentin ,
Alberto Corona ,
Marta Caviglia Open the ORCID record for Marta Caviglia [Opens in a new window] ,
Giuseppe Maria Sechi ,
Maurizio Migliari ,
Riccardo Stucchi ,
Luca Ragazzoni Open the ORCID record for Luca Ragazzoni [Opens in a new window]  and
Roberto Fumagalli
Claudia Ruffini*
Affiliation:
Anesthesia and Intensive Care Unit, Luigi Sacco University Hospital, ASST Fatebenefratelli-Sacco Milan, Italy
Monica Trentin
Affiliation:
CRIMEDIM – Center for Research and Training in Disaster Medicine, Humanitarian Aid and Global Health, Università del Piemonte Orientale, Novara, Italy
Alberto Corona
Affiliation:
Department of Anesthesia and Intensive Care and Accident & Emergency, ASST Valcamonica, Breno, Lombardia, Italy
Marta Caviglia
Affiliation:
CRIMEDIM – Center for Research and Training in Disaster Medicine, Humanitarian Aid and Global Health, Università del Piemonte Orientale, Novara, Italy
Giuseppe Maria Sechi
Affiliation:
Agenzia Regionale Emergenza Urgenza (AREU), Milan, Italy
Maurizio Migliari
Affiliation:
Agenzia Regionale Emergenza Urgenza (AREU), Milan, Italy
Riccardo Stucchi
Affiliation:
SSD AAT 118 Milano, Agenzia Regionale Emergenza Urgenza (AREU), Accident & Emergency Department, ASST Grande Ospedale Metropolitano Niguarda, Milan, Italy
Luca Ragazzoni
Affiliation:
CRIMEDIM – Center for Research and Training in Disaster Medicine, Humanitarian Aid and Global Health, Università del Piemonte Orientale, Novara, Italy
Roberto Fumagalli
Affiliation:
SSD AAT 118 Milano, Agenzia Regionale Emergenza Urgenza (AREU), Accident & Emergency Department, ASST Grande Ospedale Metropolitano Niguarda, Milan, Italy Department of Medicine and Surgery, University of Milano-Bicocca, Monza, Italy; Department of Anesthesia and Intensive Care, ASST Grande Ospedale Metropolitano Niguarda, Milan, Italy
*
Correspondence: Claudia Ruffini, MD Anesthesia and Intensive Care Unit Luigi Sacco University Hospital Via GB Grassi 74- 20157 Milano, Italy E-mail: claudia.ruffini@asst-fbf-sacco.it
Rights & Permissions [Opens in a new window]

Abstract

Introduction:

Effective response to a mass-casualty incident (MCI) entails the activation of hospital MCI plans. Unfortunately, there are no tools available in the literature to support hospital responders in predicting the proper level of MCI plan activation. This manuscript describes the scientific-based approach used to develop, test, and validate the PEMAAF score (Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria], a tool able to predict the required level of hospital MCI plan activation and to facilitate a coordinated activation of a multi-hospital network.

Methods:

Three study phases were performed within the Metropolitan City of Milan, Italy: (1) retrospective analysis of past MCI after action reports (AARs); (2) PEMAAF score development; and (3) PEMAAF score validation. The validation phase entailed a multi-step process including two retrospective analyses of past MCIs using the score, a focus group discussion (FGD), and a prospective simulation-based study. Sensitivity and specificity of the score were analyzed using a regression model, Spearman’s Rho test, and receiver operating characteristic/ROC analysis curves.

Results:

Results of the retrospective analysis and FGD were used to refine the PEMAAF score, which included six items–Proximity, Event, Multitude, Emergency Department (ED) Overcrowding, Temporary Ward Reduction Capacity, and Time Shift Slot–allowing for the identification of three priority levels (score of 5-6: green alert; score of 7-9: yellow alert; and score of 10-12: red alert). When prospectively analyzed, the PEMAAF score determined most frequent hospital MCI plan activation (>10) during night and holiday shifts, with a score of 11 being associated with a higher sensitivity system and a score of 12 with higher specificity.

Conclusions:

The PEMAAF score allowed for a balanced and adequately distributed response in case of MCI, prompting hospital MCI plan activation according to real needs, taking into consideration the whole hospital response network.

Keywords

MCI hospital plan activationPEMAAF scoreregional trauma network tool

Information

Type
Original Research
Information
Prehospital and Disaster Medicine , Volume 38 , Issue 6 , December 2023 , pp. 725 - 734
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press on behalf of the World Association for Disaster and Emergency Medicine

Introduction

During a mass-casualty incident (MCI), emergency medical resources are rapidly overwhelmed by the number and severity of casualties. Effective response to such events entails the activation of MCI plans. 1,2 A hospital MCI plan features different levels of activation, from a “standby level” to upper levels that require withholding routine activities to effectively manage the large influx of patients by re-organizing staff, stuff, and structure. Reference Zarka, Furman and Polyakov3 Since activating a hospital MCI plan might result in the cancellation of elective diagnostic and therapeutic procedures, as well as elective surgery and ambulatory activities, thus impacting the provision of basic health services for the population, medical responders and hospital managers should carefully weigh the decision of activating the plan. Reference Hugelius, Becker and Adolfsson4

In the current literature, many after action reports (AARs) of hospital responses documented the increasing need to avoid an excessive mobilization of hospital resources compared to the actual needs, preserving routine hospital activities, and avoiding unjustified expenses. Reference Hugelius, Becker and Adolfsson4–Reference Gilles, Wijeysundera and Harrison8 The unpredictability and uncertainty that characterize MCIs certainly impact the decision-making process of first responders, who are often exposed to unconfirmed or contradictory information, especially in the first phase of the disaster response. Reference Hugelius, Becker and Adolfsson4,Reference Glick and Barbara6 This blurred scenario, together with the constant stress and time pressure, can lead responders to either rush into activating an inappropriate level of response Reference Sollid, Rimstad and Rehn9,Reference Carli and Telion10 or to wait too long before activating the MCI plan. Reference Ardagh, Richardson and Robinson11–Reference Sabah, Alsharqawi and Haddad13

Unfortunately, in the literature, there are no tools available to support hospital medical first responders in performing the initial assessment of the severity of an MCI, or to guide the activation of the hospital MCI plan, indicating the proper level of response.

In urban settings, where several hospitals with different levels of care are usually part of a dynamic response network coordinated by a local or regional emergency communication center or dispatch center (DC), coordination and communication among the different health facilities become crucial during an MCI. The lack of a tool able to guide a coordinated response within an urban hospital network further complicates the overall MCI management.

The Metropolitan City of Milan, the second most populous metropolitan city in Italy with its 133 municipalities and 3.25 million inhabitants (2019), has recently been affected by several MCIs. Reference Faccincani, Stucchi and Carlucci14,15 The AARs of those events reported that all the hospitals present in the area activated the higher function level of their MCI plan, as if each hospital was the only one involved in the response.

This manuscript aims to describe the scientific-based approach used to develop, test, and validate a new tool, named the PEMAAF score (Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]), that could assist hospital managers and hospital medical first responders in the MCI management by predicting the required level of hospital MCI plan activation after an initial assessment, and could also assist crisis units or DC personnel by guiding a proper coordinated activation of an urban multi-hospital network.

Methods

Study Phases

To develop the PEMAAF score, the authors adapted the methodology used in the analysis of user requirements for information system design, which included the gathering of relevant information, the identification of user needs, envisioning, evaluating, and testing. Reference Maguire and Bevan16 These different steps have been summarized and hereafter described into three distinct phases, encompassing a research phase for information gathering and identification of user needs, the conceptualizing and development of the score, and its validation through a series of evaluating and testing activities.

Study Setting

In the Metropolitan City of Milan, there are 35 hospitals serving a population of 3.25 million inhabitants. The majority of these hospitals are located in Milan, its capital city. In case of an MCI, the Metropolitan Emergency Medical Services (EMS) DC, (SOREU Metropolitana - Regional Emergency Agency Operations Rooms [Sale Operative Regionali - Agenzia Regionale Emergenza Urgenza]), notifies the hospital emergency departments (EDs) closer to the event. The regional law for major trauma network referral pathway classifies the 35 hospitals into four different levels: (1) specialized trauma centers (CTS); (2) local trauma centers with neurosurgery (CTZNCH); (3) local trauma centers without neurosurgery (CTZ); and (4) trauma emergency rooms (PST).

Within this referral pathway, MCI casualties are referred to the different health care facilities according to a series of pre-registered information collected in a confidential document deposited with the regional authorities, which include the number of staffed operating rooms available in the hospital and the number of triage codes that can be managed during the first three hours of the MCI.

Research Phase

The authors retrospectively analyzed the AARs documented by the Luigi Sacco University Hospital (Sacco Hospital; Milan, Italy) after the response to five MCIs, four fires and a railway accident, that occurred from 2016 through 2018. The Sacco Hospital, which is one of the 35 hospitals within the Metropolitan City of Milan, is a 500-bed trauma center classified as a CTZ according to the regional trauma network classification. According to its MCI plan (lastly revised in 2015 for the World Exposition held in Milan), the activation follows an “On-Off” mechanism after a direct notification from DC. The retrospective analysis of the AARs led to the identification of six key variables:

  1. 1. Distance from the hospital (proximity indicator);

  2. 2. Type of event;

  3. 3. Number of people involved;

  4. 4. Emergency department room (EDR) overcrowding rate;

  5. 5. Temporary ward reduced capacity, either scheduled (eg, summer holidays) or unscheduled (eg, maintenance); and

  6. 6. Time shift slot.

The authors all agreed that these six variables represented the main critical information that needed to be known at the hospital level to optimize the MCI management of the five events analyzed. Reference Dean and Nai17–Reference Ghanchi19

This agreement was also supported by their own previous field experience and by the current scientific literature.

Score Development Phase

For each of the variables identified, the authors elaborated a series of possible options (eg, the proximity indicator the two options identified where either “inside the city of Milan” or “outside the city of Milan”). For each option, a specific value from one-to-three or from one-to-four was given to obtain the weighted decision matrix depicted in Figure 1. This matrix was named the PEMAAF score as the Italian acronym of the six variables identified and which constituted the matrix. As shown in Figure 1, the first version of PEMAAF score could range from five to 12 points. Three different levels were then identified and labeled by a color code (green, yellow, and red) representing the growing severity of the event and the complexity of the hospital response. A PEMAAF score below six corresponds to a “green alert,” which might suggest a reconnaissance of the situation in the ED. A PEMAAF score from seven through nine calls for a “yellow alert,” which might entail a series of procedures aiming to prepare the ED to manage the MCI casualties, such as early discharge of stable patients or patient transfers into hospital wards, as established by the hospital MCI plan. This second level of activation might allow hospitals to initiate preparatory activities without activating the whole MCI plan, thus safeguarding routine hospital activities. A PEMAAF score of ten or higher corresponds with a “red alert,” which might suggest the full activation of the hospital MCI plan, including the interruption of routine hospital activities, cancellation of elective procedure, and mobilization of additional resources.

Figure 1.

PEMAAF Score (First Version).

Abbreviations: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; SOREU, Regional Emergency Agency Operations Rooms [Sale Operative Regionali - Agenzia Regionale Emergenza Urgenza].

Score Validation Phase

To validate the PEMAAF score and analyze its reliability, a multi-step process was followed:

  1. 1. Retrospective analysis of past MCIs, using the score;

  2. 2. Focus group discussion (FGD) to evaluate results and refine the score;

  3. 3. Second retrospective analysis using the score modified by the FGD; and

  4. 4. Final prospective study.

First Retrospective Analysis

In the first retrospective analysis, the authors used real data derived from the Sacco Hospital AARs to mimic the decision-making process and apply the PEMAAF score. Results were then compared to real decisions undertaken by responders during the five MCIs described by the Sacco Hospital AARs and presented to the DC in a dedicated FGD (Table 1).

Table 1.

Retrospective Analysis of the Response to MCI at the Sacco Hospital 2016-2018: Real-Time Hospital Management Decision versus Response Suggested by the PEMAAF Score

Abbreviations: MCI, mass-casualty incident; PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria].

Focus Group Discussion

The objective of the FGD was to review the application of the score, to identify gaps, and to establish corrective measures to ultimately refine the PEMAAF score. During the discussion, participants also reviewed data concerning nine MCIs declared on a regional basis, during the period 2016-2018, analyzing the type of event, the number and type of EMS vehicles involved (ambulance/car/helicopter), the location, time of the day, and the number of hospitals alerted. The PEMAAF score was also applied to the decision-making process of the nine additional MCIs and results were compared to real decisions of responders.

Second Retrospective Analysis

In the second retrospective analysis, the revised PEMAAF score (Figure 2) was applied to one of the five MCIs, the “Pioltello railway accident,” for validation. A dedicated panel of experts comprising the authors of this paper recruited to analyze results, decided to reproduce the abovementioned scenario in four different sites, situated at important railway junctions, respectively located in the north-east, north-west, south of Milan, and in the center of Milan. The 12 hospitals involved represent a reliable sample of all levels of trauma care alerted by the DC.

Figure 2.

PEMAAF Score (Revised).

Abbreviations: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; CTS, specialized trauma center; CTZNCH, local trauma centers with neurosurgery; CTZ, local trauma centers without neurosurgery; PST, trauma emergency rooms; MCI, mass-casualty incident; SOREU, Regional Emergency Agency Operations Rooms [Sale Operative Regionali - Agenzia Regionale Emergenza Urgenza].

To validate the score, each of the six variables of the PEMAAF were considered as epidemiological units. The statistical analysis was based on simulated activations of the system. Firstly, two indexes were devised in order to validate the predictive power of the PEMAAF score during the process (Figure 3).

Figure 3.

PEMAAF Score Indexes.

Abbreviation: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria].

These indexes can identify intervals linked to the saturation of the resources made available by the activated hospitals:

  • > 1.0: over-saturation of the response system (especially if restricted to red and yellow codes);

  • 0.75-1.0: saturation of the response system;

  • 0.5-0.75: functionality adequacy of the response system;

  • 0.25-0.5: excessive activation of the response system; and

  • <0.25: improper activation of the response system.

In a following step, a minimum number of further replicas was established to create a statistically significant sample to validate the PEMAAF score, by using the G Power V3.0 Software program (Faul, Erdfelder, Lang, Buchner; Germany). Considering that the PEMAAF score consists of six variables (categorical, dichotomous, or polytomous with hierarchical order), for each simulation, 72 specific answers to the score’s questions were collected for each hospital. The following points were considered as critical: (1) the epidemiological unit (ie, the information per single variable); and (2) the bivariate correlation between what is suggested by the PEMAAF score and the effective activation of its hospital MCI plan. Assuming a probability of error α <0.05 and the probability of error β of 95%, the total of the minimum sample of observations collected was between 162 (slope correlation of 0.25) and 262 (slope correlation 0.20). Reference Erdfelder, Faul, Buchner and Lang20 The “curve estimation” method was used for the statistical analysis and identification of the scores, and, in particular, the regression with the Italic S correlation was identified as the “optimum.” The confirmatory test was based on the determination of Spearman’s Rho. The analysis was later extended to the 12 hospitals alerted for the MCI in Pioltello, Italy.

Prospective Study

In a subsequent step, the 12 hospitals were involved in four simulations to test the PEMAAF sensitivity and specificity, respectively held in 2019 on August 13, October 18, December 13, and on January 10, 2020. The four simulations were conducted via phone involving the DC and a member of the strategic management (strategic decision-making level), and the physician designated as coordinator of the hospital MCI plan (operational decision-making level), which were selected by each one of the hospitals involved in the activity. The first two simulations used data retrieved from the AAR of the MCI in Pioltello, a scenario well-known by the responders. In contrast, two additional scenarios had been designed by the authors for the simulation performed in December (an explosion in a department store facing Piazza Duomo in Milan) and in January (explosion of gas cylinders in the EXPO 2015 area), thus forcing responders to deal with an unknown scenario.

For the first two simulations, the hospital strategic management office provided to the DC only their PEMAAF reference on the second “A” item (temporary ward reduction capacity) by phone (affirmative or negative answer). The phone call was replicated at different time slots (11:00am, 6:00pm, 12:00pm, and 7:00am of the following Saturday morning) to calculate and test the PEMAAF score in the different time shift slots and the corresponding personnel. The last two simulations performed in December 2019 and January 2020 involved four different phases: (1) first alert call from the DC to the hospitals according to the hospital alert list and real-time filling out of the PEMAAF score by the MCI coordinator; (2) second call from the DC to the different alerted hospitals asking for confirmation of the level of alert chosen by the hospital health management (code suggested by the PEMAAF score or different code chosen according to hospital internal protocols and guidelines), communication of the hospital surge capacity by color code, and operating rooms and relevant from the MCI scene; (3) application of the SORT triage to the victims on the scene and distribution of patients according to the mapping obtained from the score (performed by the DC); and (4) third call from the DC to hospitals to communicate the number, color code, and injury mix of patients sent to the EDRs. During the simulated event, a one-way communication protocol (from DC to hospitals) was adopted, and patient influx to hospitals from the scene considered the EDR capacity declared in the confidential document by each hospital in case of MCI plan activation. In a last step, the PEMAAF score model had been verified and evaluated with respect to specificity and sensitivity through a regression model and using the statistical sensitivity test represented by Spearman’s Rho test or receiver operating characteristic/ROC analysis curves. The SPSS 20.1 (SPSS Statistic Software; IBM Corp.; Armonk, New York USA) version has been used for the statistical analysis with an established significance level of P <.05. The gap arising from the inverse relationship between specificity and sensitivity required a corrective tool, which led to the introduction of an additional level of alert, the “orange level,” associated with a PEMAAF score between 10 and 11. The purpose of this additional level was to allow hospitals to reach a very advanced stage of preparation with respect to the conservation of the discretionary activities in progress and guarantees better flexibility of the score in terms of resources to be activated (ready to cope) and capacity to cope with the influx of patients.

Results

First Retrospective Analysis

Table 1 compares the real decision undertaken by the Sacco Hospital during the five MCIs (Pioltello railways accident and four fires) against the action proposed by the application of the PEMAAF score. As shown, Sacco Hospital activated the MCI plan three times. Results indicated that the application of the PEMAAF score would have reduced the activation in two cases out of the three. Specifically, the activation of the hospital MCI plan was deemed as necessary only for the fire MCI that occurred at the Fire 4, while a “yellow alert” would have been enough in managing the fires in Fire 1 and Fire 2.

Focus Group Discussion

Figure 2 shows the final PEMAAF score obtained after revision performed by the expert panel. During the FGD, three points of re-evaluation emerged, concerning: (1) the “proximity” item, or the determination of two different kilometric cut-offs, considering the size of the Milan area; (2) the “event” item, aligning it to the level of care capacity/specialist resources for the treatment of trauma available in each hospital codified by levels of competence considering the complexity of traumatic injuries; and (3) the “ED overcrowding” item, integrating it with the software used by EDRs on a routinary basis.

Second Retrospective Analysis

The application of this updated version of the PEMAAF score to the Pioltello MCI confirmed its balancing effect on the hospital MCI plan activation and on the hospital general alert level, when compared with the decisions taken by the hospital management without the aid of the score, as shown in Table 2.

Table 2.

“Pioltello” MCI Retrospective Analysis: Real Hospital Response versus PEMAAF Score Application

Abbreviations: MCI, mass-casualty incident; PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; CTS, specialized trauma center; CTZNCH, local trauma centers with neurosurgery; CTZ, local trauma centers without neurosurgery; PST, trauma emergency rooms.

a Hospital level according to organization of an integrated system for the trauma treatment (in Italian indicated with the acronym S.I.A.T Trauma Network).

Prospective Study

Four simulations and consequent assessment were performed from August 2019 through January 2020. For each simulation, 72 answers were collected to the score’s questions for each single hospital, thus obtaining a total number of 288 observations. As shown in Figure 4, the PEMAAF score was affected by time, suggesting that the hospital MCI plan was more frequently activated during night shift and the holiday shift.

Figure 4.

PEMAAF Score Daily Trend According to Time Shift.

Abbreviations: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; CTS, specialized trauma center; CTZNCH, local trauma centers with neurosurgery; CTZ, local trauma centers without neurosurgery; PST, trauma emergency rooms.

Figure 5 shows the trend of the PEMAAF score in the four different levels of trauma care during morning shift and during nighttime. Hospitals within the CTS and CTZNCH reported a high PEMAAF score (> 10), which would enable for full activation of their hospital MCI plan during the simulated MCI. Hospitals classified as CTZ and PST reported lower levels of PEMAAF score (< 10) in the four simulations, thus showing a more conservative approach. Of note, no major changes in the different time slots and during holidays were observed.

Figure 5.

PEMAAF Score Detailed Trends According to Morning and Night Shifts (Seasonal).

Abbreviations: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; CTS, specialized trauma center; CTZNCH, local trauma centers with neurosurgery; CTZ, local trauma centers without neurosurgery; PST, trauma emergency rooms.

Specificity and sensitivity of the PEMAAF score model are illustrated in Figure 6. A PEMAAF score of 11 was associated with a sensitivity of 86% and specificity of 63%, while a PEMAAF score of 12 was associated with a sensitivity of 58.5 % and specificity of 86%. Therefore, a PEMAAF of 11 favored a system that functions at a higher sensitivity, by alerting hospitals that might not receive any casualty, thus encouraging a prudential decision that is made in the general interest but that could negatively impact the planned activities. On the contrary, a PEMAAF of 12 favored a system that functions at a higher specificity, by activating few hospitals MCI plans, thus committing less resources but also incurring in the risk of a sub-activation of the response.

Figure 6.

Specificity and Sensitivity of the PEMAAF Score Model.

Abbreviation: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria].

Discussion

This manuscript provides a comprehensive description of the multi-step approach used to design, test, and validate a new tool to support emergency medical systems during MCIs, by predicting the required level of hospital MCI plan activation after an initial assessment. Indeed, the PEMAAF score proposed in the Metropolitan City of Milan provides a snapshot of key data and characteristics of an MCI, thus estimating the adequate hospital response level and therefore supporting hospital managers, incident commanders, and hospital medical responders in the decision to appropriately activate the hospital MCI plan. The retrospective analyses clearly demonstrated how the use of the PEMAAF score would have correctly reduced the activation of the hospital MCI plan according to the real need dictated by MCIs that occurred in Milan. The potential of the tool has then been further emphasized by the simulations performed in the months of August and December, which represent the most challenging and busiest months in Milan.

By including a series of indicators (Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, and Time Shift Slot) widely identified in the literature as crucial to determine the effectiveness of the re-organization procedure following an MCI, Reference Zarka, Furman and Polyakov3,Reference Dean and Nai17–Reference Ghanchi19 the PEMAAF score also allows to avoid unjustified activations of the hospital MCI plan, thus safeguarding elective and routinary hospital activities, thus not hampering the continuity of care and assistance. Reference Ardagh, Richardson and Robinson11

The PEMAAF score enabled the MCI management within the Metropolitan City of Milan to reach a much higher plateau of innovativeness, by enhancing communication and response strategies between the DC and the regional trauma network, through the establishment of a common language and shared operating procedure. Remarkably, the application of the PEMAAF score allows hospitals to act as a network, in which the hospital MCI response is no longer managed by very single hospital as standalone unit, rather guided in a coordinated manner. The importance of a standardized and coordinated response to MCIs, based on shared terminology and common understanding, has already been highlighted in available peer-review literature. Reference Dean and Nai17,Reference Erdfelder, Faul, Buchner and Lang20–Reference Amram, Schuurman, Hedley and Hameed24

Additionally, the implementation of the PEMAAF score enabled coordination centers and DCs to provide for a hub and a capacity for the hospital network, and to contain the compressive effect on ordinary activities and flows to the hospital EDRs on the territory, thus protecting the need for the non-MCI health, which is incessant by definition. Reference Ardagh, Richardson and Robinson11,Reference Davis, Poste, Hicks, Polk, Rymer and Jacoby25

Yet, the implementation of such a tool requires the continuous training of all operators involved in the MCI response, especially given the current limited knowledge in terms of MCI preparedness and response in the Italian context. Reference Paganini, Borrelli and Cattani26 To this regard, simulations and exercises can effectively foster this awareness, 27 ensuring that the operators’ training and their confidence is constantly monitored and that any shortcomings are identified and contained. Reference Sollid, Rimstad and Rehn9

Finally, it should be emphasized that the multi-step methodology adopted to design the PEMAAF score followed a sound scientific approach based on an existing framework Reference Faccincani, Stucchi and Carlucci14–Reference Boyle, Able and Raut28 and relied on the participation of all the local stakeholders (regional trauma network and the DC), which have been involved from the very first step of the process and actively contributed to refine and validate it. Moreover, incorporation of validated indicators allows its application to a broader scale (eg, at a regional or national level). Reference Boyle, Able and Raut28 As such, the tool has also the potential to be adapted and implemented in different contexts than where it has been conceived and developed. Reference El Sayed29,Reference Hitti, Ali Cheaito, El Sayed, Kellermann and Kazzi30

Limitations

Since the study was conducted in an Italian urban area, result validity needs to be verified in other environments.

Additionally, artificial intelligence algorithms were not incorporated, which have the potential to enhance the accuracy of the scoring system.

It is worth mentioning that the PEMAAF score has never been tested through a blind exercise (ie, in the case where hospitals had not been warned); such an event was expected to be carried out in the first months of 2020, and it had then been canceled due to COVID-19.

Conclusions

This manuscript offers a comprehensive overview of the multi-step process employed for developing, testing, and validating a novel tool designed to assist the emergency medical system during MCIs. The tool predicts the necessary activation level of hospital MCI plan, following an initial assessment.

Specifically, the PEMAAF score, tested in the Metropolitan City of Milan, presents crucial data and MCI characteristics, enabling the estimation of an appropriate hospital response level.

This, in turn, aids hospital administrators, incident commanders, and medical responders in making informed decisions regarding the activation of the hospital’s MCI plan.

The PEMAAF score also serves the purpose of preventing unwarranted activations of the hospital MCI plan. Consequently, it helps to safeguard regular hospital activities, ensuring the uninterrupted delivery of care and assistance.

Acknowledgments

This manuscript is the result of a study conducted in the framework of the Advanced Master of Science in Disaster Medicine (EMDM - European Master in Disaster Medicine), jointly organized by CRIMEDIM - Center for Research and Training in Disaster Medicine, Humanitarian Aid and Global Health of the Università del Piemonte Orientale (UPO) and REGEDIM - Research Group on Emergency and Disaster Medicine of the Vrije Universiteit Brussel (VUB).

Conflicts of interest

All the authors declare none.

Footnotes

Note: Authors Riccardo Stucchi, Luca Ragazzoni, and Roberto Fumagalli contributed equally.

References

Mass Casualty Systems: strategies and guidelines for building health sector capacity. World Health Organization (WHO) web site. https://www.who.int/hac/techguidance/tools/mcm_guidelines_en.pdf. Accessed October 1, 2021.Google Scholar
Health service continuity planning for public health emergencies. A handbook for health facilities. World Health Organization web site. https://apps.who.int/iris/bitstream/handle/10665/344796/9789240033337-eng.pdf. Accessed October 5, 2021.Google Scholar
Zarka, S, Furman, E, Polyakov, O. Hospital operation during a disaster-hospital multi component emergency center (HMCEC). Disaster Med Public Health Prep. 2019;15(1):92–98.CrossRefGoogle Scholar
Hugelius, K, Becker, J, Adolfsson, A. Five challenges when managing mass casualty or disaster situations: a review study. Int J Environ Res Public Health. 2020;17(9):3068–3080.CrossRefGoogle ScholarPubMed
Savoia, E, Agboola, F, Biddinger, PD. Use of after-action report (AARs) to promote organizational and systems learning in emergency preparedness. Int J Environ Res Public Health. 2012;9(8):2949–2963.CrossRefGoogle ScholarPubMed
Glick, AJ, Barbara, JA. Moving from situational awareness to decisions making during disaster response: transition to decision making. J Emerg Manag. 2013;11(6):423–432.CrossRefGoogle ScholarPubMed
Healy, S, Tyrrell, M. Importance of debriefing following critical incidents. Emerg Nurs. 2013;20(10):32–37.CrossRefGoogle ScholarPubMed
Gilles, MA, Wijeysundera, DN, Harrison, EM. Counting the cost of cancelled surgery: a system wide approach is needed. Br J Anaesth. 2018;4(121):691–694.CrossRefGoogle Scholar
Sollid, SJM, Rimstad, R, Rehn, M, et al. Oslo Government District Bombing and Utøya Island Shooting July 22, 2011: the immediate prehospital Emergency Medical Service response. Scand J Trauma Resusc Emerg Med. 2012;20:3.CrossRefGoogle ScholarPubMed
Carli, PT, Telion, C. Paris City Disaster: response to the recent terror attacks and lessons learned. Curr Trauma Rep. 2018;4:96–102.CrossRefGoogle Scholar
Ardagh, MW, Richardson, SK, Robinson, V, et al. The initial health-system response to the earthquake in Christchurch, New Zealand, February 2011. Lancet. 2012;379(9831):2109–2115.CrossRefGoogle Scholar
El Sayed, M, Chami, AF, Hitti, E. Developing a hospital disaster preparedness plan for mass casualty incidents: lessons learned from the Downtown Beirut Bombing. Disaster Med Public Health Prep. 2018;12(3):379–385.CrossRefGoogle ScholarPubMed
Sabah, SA, Alsharqawi, N, Haddad, EA. The aftermath of the Kuwait mosque bombing: a retrospective cohort analysis and lessons learned. Int J Surg. 2018;56:15–20.CrossRefGoogle ScholarPubMed
Faccincani, R, Stucchi, R, Carlucci, M, et al. Evaluation of interaction between emergency medical system and hospital network during a train derailment in Milano. Disaster Med Public Health Prep. 2022;16(2):829–834.CrossRefGoogle ScholarPubMed
ANSA web site. Latest News. https://www.ansa.it/english/news/2018/12/28/1-badly-hurt-in-fire-in-nursing-home_e82a475c-5586-4906-a648-dc0ecedbe5e1.html;https://www.ansa.it/english/news/2018/02/15/13-year-old-dies-after-milan-apartment-block-fire-2_cbb8494b-9354-4f6d-9509-d12df0d763ce.html. Accessed October 1, 2021.Google Scholar
Maguire, M, Bevan, N. User Requirements Analysis: A Review of Supporting Methods. In Proceedings of the IFIP 17th World Computer Congress - TC13 Stream on Usability: Gaining a Competitive Edge. Kluwer, BV, NLD, 133-148. 2002.CrossRefGoogle Scholar
Dean, MD, Nai, SK. Mass-casualty triage: distribution of victims to multiple hospitals using the SAVE model. Eur J Oper Res. 2014;238(1):363–373.CrossRefGoogle Scholar
Lennquist, S, Dobson, R. The Prehospital Response. In: Lennquist, S, eds. Medical Response to Major Incidents and Disasters. Berlin, Germany: Springer, Berlin-Heidelberg; 2012: 33–61.CrossRefGoogle Scholar
Ghanchi, A. Insights into French emergency planning, response, and resilience procedures from a hospital managerial perspective following the Paris terrorist attacks of Friday, November13, 2015. Disaster Med Public Health Prep. 2016;10(5):789–794.CrossRefGoogle Scholar
Erdfelder, E, Faul, F, Buchner, A, Lang, AG. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods. 2009;41(4):1149–1160.Google Scholar
Skryaabina, E, Betts, N, Reedy, G, Riley, P, Amlôt, R. UK healthcare staff experiences and perceptions of a mass casualty incident response: a mixed-methods study. Emerg Med J. 2021;38(10):756–764.CrossRefGoogle Scholar
Teasdale, G, Jennett, B. Assessment of coma and impaired consciousness–a practical scale. The Lancet. 1974;304(7872):81–84.CrossRefGoogle Scholar
Apgar, V. A Proposal for a new method of evaluation of the newborn infant. [Originally published in July 1953]. Anesth Analg. 2015;120(5):1056–1059.CrossRefGoogle ScholarPubMed
Amram, O, Schuurman, N, Hedley, N, Hameed, SM. A web-based model to support patient-to-hospital allocation in mass casualty incidents. J Trauma Acute Care Surg. 2012;72(5):100–105.CrossRefGoogle ScholarPubMed
Davis, DP, Poste, JC, Hicks, T, Polk, D, Rymer, TE, Jacoby, I. Hospital bed surge capacity in the event of a mass-casualty incident. Prehosp Disaster Med. 2005;20(3):169–176.CrossRefGoogle ScholarPubMed
Paganini, M, Borrelli, F, Cattani, J, et al. Assessment of disaster preparedness among emergency departments in Italian hospitals: a cautious warning for disaster risk reduction and management capacity. Scand J of Trauma Resusc Emerg Med. 2016;24(1):101.CrossRefGoogle ScholarPubMed
Exercises. Ready website. https://www.ready.gov/exercises. Accessed October 5, 2021.Google Scholar
Boyle, A, Able, G, Raut, P, et al. Comparison of the International Overcrowding Measure in Emergency Departments (ICMED) and the National Emergency Department Overcrowding Score (NEDOCS) to measure emergency department overcrowding: pilot study. Emerg Med J. 2016;33(5):307–312.CrossRefGoogle ScholarPubMed
El Sayed, MJ. Emergency response to mass casualty incidents in Lebanon. Disaster Med Public Health Prep. 2013;7(4):433–438.CrossRefGoogle ScholarPubMed
Hitti, E, Ali Cheaito, M, El Sayed, MK, Kellermann, AL, Kazzi, AA. Mass casualty management in the emergency department-lessons learned in Beirut, Lebanon –Part I. Med J Emerg Med Acute Care. 2020;1(2):14.Google Scholar
Figure 0

Figure 1. PEMAAF Score (First Version).Abbreviations: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; SOREU, Regional Emergency Agency Operations Rooms [Sale Operative Regionali - Agenzia Regionale Emergenza Urgenza].

Figure 1

Table 1. Retrospective Analysis of the Response to MCI at the Sacco Hospital 2016-2018: Real-Time Hospital Management Decision versus Response Suggested by the PEMAAF Score

Figure 2

Figure 2. PEMAAF Score (Revised).Abbreviations: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; CTS, specialized trauma center; CTZNCH, local trauma centers with neurosurgery; CTZ, local trauma centers without neurosurgery; PST, trauma emergency rooms; MCI, mass-casualty incident; SOREU, Regional Emergency Agency Operations Rooms [Sale Operative Regionali - Agenzia Regionale Emergenza Urgenza].

Figure 3

Figure 3. PEMAAF Score Indexes.Abbreviation: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria].

Figure 4

Table 2. “Pioltello” MCI Retrospective Analysis: Real Hospital Response versus PEMAAF Score Application

Figure 5

Figure 4. PEMAAF Score Daily Trend According to Time Shift.Abbreviations: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; CTS, specialized trauma center; CTZNCH, local trauma centers with neurosurgery; CTZ, local trauma centers without neurosurgery; PST, trauma emergency rooms.

Figure 6

Figure 5. PEMAAF Score Detailed Trends According to Morning and Night Shifts (Seasonal).Abbreviations: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria]; CTS, specialized trauma center; CTZNCH, local trauma centers with neurosurgery; CTZ, local trauma centers without neurosurgery; PST, trauma emergency rooms.

Figure 7

Figure 6. Specificity and Sensitivity of the PEMAAF Score Model.Abbreviation: PEMAAF, Proximity, Event, Multitude, Overcrowding, Temporary Ward Reduction Capacity, Time Shift Slot [Prossimità, Evento, Moltitudine, Affollamento, Accorpamento, Fascia Oraria].