Background
Public health emergencies (PHEs), including outbreaks of infectious diseases, unexplained illnesses, foodborne illnesses, or other sudden incidents with the potential to have serious health consequences, are unforeseen events that pose a significant threat to the health of the population.Reference Wang, Sun, Zou, Ning and Du1
Since 2009, the World Health Organization (WHO) has declared 6 international public health emergencies, encompassing events such as 2009’s swine influenza virus (H1N1), 2014’s resurgence of the poliovirus, August 2014’s spread of Ebola virus in West Africa, 2016’s outbreak of Zika virus across 24 countries, 2019’s Ebola virus outbreak in the Democratic Republic of the Congo, and 2019’s Coronavirus Disease 2019 (COVID-2019).Reference Xin, Shao, Wang, Xu and Li2 These events represent major emerging infectious diseases (MEIDs) and pose a significant threat to global health.Reference Metcalf and Lessler3
MEIDs exhibit traits such as abrupt onset, elevated transmissibility, and unpredictable clinical outcomes, resulting in substantial adverse impacts on individuals’ well-being, public health, and the functioning of economic and societal systems.Reference Lunt and Heenan4, Reference Wang, Ye and Liu5
Colleges and universities play a crucial role in society and face challenges in effectively responding to crises such as the COVID-19 pandemic, highlighting the necessity for emergency preparedness measures in higher education institutions.Reference Hamer, White and Jenkins6 In recent years, while research has been conducted to develop an evaluation index system for major epidemics to assess the emergency preparedness of the general public,Reference Wei, Liu and Zhou7 health institutes,Reference Wang, Sun, Zou, Ning and Du1 and health care providersReference Bai, Gan and Yang8, Reference Wu, Wu and Li9 during MEIDs, limited attention has been given to the evaluation index system for emergency preparedness at the higher education level.
Universities and colleges play a crucial role in responding to MEIDs, given their high population density, significant public attention, difficulty in control, and high risk of epidemic outbreaks.Reference Quattrone, Borghini, Emdin and Nuti10, Reference Edmunds, Bowater and Brainard11 These characteristics present significant challenges in preparedness and response to MEIDs, underscoring the importance of conducting a scientific assessment of higher education institutions’ response capabilities to such diseases.
This study aims to establish a framework for evaluating emergency response capabilities in universities during MEIDs using the Delphi method and the analytic hierarchy process (AHP), while also offering universities a systematic guide to identify areas of insufficient preparation and potential intervention opportunities during an epidemic.
Methods
This research was approved by the Ethics Committee of Shanghai University of Medicine and Health Sciences, China (Approval No. 2022-lwfb-02). It is based on the Harden model and involves the participation of 15 experts from 10 universities and 5 disease control and prevention centers in Shanghai, who will engage in 2 rounds of Delphi consultations.Reference McPherson, Reese and Wendler12, Reference Black, Matthews and Millington13 A comprehensive evaluation framework for emergency response in higher education institutions has been developed, employing the Analytic Hierarchy Process (AHP)7 for the weighted analysis of various assessment indicators. The Delphi study followed the guidelines outlined in the Conducting and Reporting of Delphi Studies (CREDES) framework.Reference Jünger, Payne, Brine, Radbruch and Brearley14 The Delphi process is visually represented in Figure 1.
Flowchart of the study. Abbreviation: AHP, analytic hierarchy process.
Construction the Initial Evaluation Indicators
Drawing upon the Haddon matrix,Reference Barnett, Balicer, Blodgett, Fews, Parker and Links15 this study has established an evaluation framework specifically tailored to assess emergency response capabilities within higher education institutions. This framework incorporates a comprehensive array of evaluation indicators that span the entire incident lifecycle, systematically divided into 3 principal phases: prevention and preparation, response and handling, and recovery and rehabilitation. The indicators within this framework are organized around three pivotal factors—environment, population, and equipment—each meticulously designed to provide a thorough assessment of the emergency response capabilities of colleges and universities. The formulation of these indicators is based on a comprehensive analysis of our previous survey, a literature review,Reference Quattrone, Borghini, Emdin and Nuti10, Reference Murphy, Brown, Shankar and Lichtveld16–Reference Cao, Shan, Gong, Kuang and Gao18 and opinions from research group. Consequently, a preliminary evaluation index system has been developed. This initial system comprises 3 primary indicators, 10 secondary indicators, and 36 tertiary indicators.
Expert Selection
The selection of Delphi panel members was conducted through purposive sampling utilizing specific criteria, including experience in public health management, infectious disease prevention and control, or epidemic response management in higher education, a minimum of 8 years of work experience, possession of a professional title qualification of intermediate or above, attainment of a bachelor’s degree or higher, and voluntary willingness to participate in the investigation and commit to two rounds of consultation.
Design of the Questionnaire
The expert consultation questionnaire was structured into 4 distinct sections: (1) Preface, offering a concise summary of the research objectives, content, and completion criteria; (2) general information pertaining to the experts, encompassing age, educational attainment, professional designation, work experience, and areas of research specialization; (3) the expert consultation form serves as the primary assessment component within the evaluation index system, incorporating all evaluation indicators. Experts assessed the significance of each indicator using the Likert 5-level scoring method, with ratings ranging from 5 to 1 representing varying degrees of importance. Additionally, experts were prompted to provide personal insights and recommendations to improve the overall comprehensiveness of the evaluation index system.Reference Toussaint, Vincent and Whipple19 (4) Expert familiarity with the study’s content and assessment of its index.
Consultation and Feedback Cycle of Delphi
The researchers individually contacted the experts through email, WeChat, or smartphones from September to November 2022. The experts did not engage in direct communication with one another. Subsequently, the researchers aggregated the viewpoints, recommendations, and responses offered by each expert and then amended the consultation questionnaire for a follow-up round in order to enhance the precision of the evaluation index system.Reference Hasson, Keeney and McKenna20 Furthermore, the criteria for index inclusion were determined based on specific thresholds: an average importance score of each index evaluated by the expert panel greater than 3.5, a coefficient of variation less than 0.25, and a full mark rate which was defined as the proportion of experts assigning “5 = very important” exceeding 20%. For items that failed any threshold or were flagged as unclear, redundant, or misaligned with the Haddon framework, the research team conducted group discussions and, considering experts’ free-text comments, theoretical relevance, feasibility, and clarity, reached consensus to revise, merge, reclassify, or delete the item.Reference Shen, Yang, Jin, Hou, Shang and Zhang21 After 2 rounds of expert consultation, the evaluation index system was finalized.
Statistical Analysis
Excel 2010 and SPSS 25.0 were employed for statistical analysis, utilizing frequency and percentage to characterize the personal information of the experts, while the questionnaire response rate served as an indicator of their level of enthusiasm. The authority coefficient (Cr) of the expert was determined by averaging the expert’s judgment basis (Ca) and degree of familiarity (Cs) with the research content, as expressed by the formula Cr = (Ca + Cs)/2. The level of variability in the experts’ viewpoints was evaluated through the utilization of Kendall’s coefficient of concordance (Kendall’s W) and coefficient of variation (CV), with the significance of each index elucidated in relation to the mean ± standard deviation. A significance level of P < 0.05 was employed to ascertain statistical significance for the observed variances. The weight coefficients for each index were derived by constructing a judgment matrix utilizing the yaahp12.9 software, developed by Shanxi Yuanshi Software Technology Co., Ltd., and applying the analytic hierarchy process (AHP).Reference Wei, Liu and Zhou7
Containment of Quality
To maintain the scientific rigor and precision of the research findings, stringent criteria were established for the selection of experts. It is posited that the inclusion of experts from homogenous backgrounds may introduce a level of bias into the study outcomes.Reference Garnett, Crane, West, Brown and Michie22 In this study, the selected experts demonstrated dedication to various research areas. In the event of incorrect or incomplete questionnaires, prompt verification was sought from the experts. Questionnaires with more than 10% missing data were deemed invalid and subsequently removed from the analysis. Data entry was conducted by 2 researchers utilizing Excel 2010 software.
Results
Socio-demographic Information from Experts
The current study recruited a cohort of 15 experts based in Shanghai with varying levels of professional experience ranging from 8 to 38 years. Each expert possessed expertise in infectious disease prevention and control, emergency management, public health, or health policy. The socio-demographic characteristics of the experts are outlined in Table 1.
The socio-demographic information of the experts
Experts’ Enthusiasm and Authority Coefficient
The level of enthusiasm exhibited by experts can be gauged through the response rate of the questionnaire. It is commonly accepted that experts demonstrate a significant level of enthusiasm when the response rate exceeds 70%.Reference Dai, Wei, Chen and Ju23 In the initial round, 16 questionnaires were disseminated, yielding a return of 15 valid responses, resulting in a response rate of 93.75%. Subsequently, in the second round, all 15 questionnaires were returned, for a response rate of 100% (Table 2). Researchers assert that an expert consultation authority coefficient exceeding 0.7 is indicative of reliability.Reference Liu, Su, Li, Zhu, Ma and Zhu24 Following meticulous computation, the authority coefficients (Cr) for the 2 rounds of expert consultation were determined to be 0.82 and 0.80, respectively, signifying a high level of expertise among the participating experts.
Recovery of the questionnaire and suggestions offered
The Level of Concentration and Coordination among Experts’ Opinions
During the initial round of consultation, the importance scores of all items ranged between 3.00 and 4.67, with a full mark rate exceeding 20%. The coefficient of variation (CV) was less than 0.25, except for 16 indexes for which the CV exceeded 0.25. Kendall’s W value was calculated at 0.32 (χReference Xin, Shao, Wang, Xu and Li2 = 217.669, P < 0.001). In the subsequent round of consultation, the importance scores of all items ranged from 4.13 to 5.00, with a full mark rate exceeding 20% and a CV below 0.25. Kendall’s W value was 0.161 (χReference Xin, Shao, Wang, Xu and Li2 = 113.798, P < 0.001) (see Table 3).
The result of expert opinions’ coordination degree
How the Evaluation Framework Is Formed
This study involved two rounds of consultation during which adjustments to indicators were made based on screening criteria, expert opinions, and discussions within the research group. In the initial round, 7 indexes were modified, 9 indexes were deleted, 11 indexes were added, and 2 indexes were adjusted. In the subsequent round, the positions of the 2 indexes were adjusted. As a result, a comprehensive index system was developed to assess the emergency response capacity for epidemics in higher education institutions, consisting of 3 main categories: prevention and preparedness, response and handling, recovery and rehabilitation, each linked to 11 subcategories and 34 specific indicators (see Table 4).
The index system for evaluating the emergency response capability for epidemics in colleges and universities
Full mark rate: Total number of experts who gave 5 points to the importance of each item/Total number of experts who evaluated each item. Abbreviation: SD: standard deviation.
The Indicator Weight
The concept of the index weight pertains to the proportional significance of a specific index within the comprehensive assessment framework.Reference Wu, Wu and Li9 The weights assigned to individual indicators were determined using the AHP, a method capable of quantifying experts’ subjective assessments and facilitating multiobjective decision-making analysis. This approach ensures that optimal outcomes are aligned with expert opinions.Reference Wei, Liu and Zhou7 The consistency ratio (CR) values for all indicator levels were found to be less than 0.10, indicating that the judgment matrix adhered to an acceptable level of consistency. Within the initial tier of indicators, prevention and preparedness held the greatest weight at 0.426, followed by recovery and rehabilitation at 0.326. Conversely, response and handling were assigned the lowest weights. A comprehensive breakdown of the index weights for all the tiers of the indicators can be found in Table 4.
Discussion
Although numerous studies have examined infectious disease management in educational environments,Reference Quattrone, Borghini, Emdin and Nuti10, Reference Zhang25 few have proposed a structured, validated assessment system specifically designed to evaluate universities’ emergency response capacity. Epidemics in university settings can rapidly jeopardize student and staff safety and disrupt institutional operations.Reference Sokol, Heinze and Doan26 Therefore, it is crucial to develop an assessment index specifically tailored to measure the epidemic response capabilities of universities, aligned with their unique developmental trajectories. The evaluation of epidemic response capacity involves a holistic assessment of an institution’s ability to effectively manage outbreaks. The primary goal of such evaluations is to identify vulnerabilities and critical barriers encountered during emergency responses, thereby facilitating continuous improvement in preparedness and response strategies. By synthesizing lessons learned from past epidemic responses,Reference Shamsir, Krauss, Ismail, Ab Jalil, Johar and Abdul Rahman27 this study developed an evaluation framework grounded in the Haddon matrix and adapted for higher education to support holistic appraisal of epidemic response capacity across pre-event, event, and postevent phases. Using a 2-round Delphi process with 15 domain experts from multiple departments in Shanghai, the study refined a 48-item (3 primary, 11 secondary, 34 tertiary) index covering prevention and preparedness, response, and recovery. While the expert panel was geographically localized, disciplinary diversity was leveraged to broaden perspectives. Content validity was supported by a comprehensive literature reviewReference Quattrone, Borghini, Emdin and Nuti10, Reference Murphy, Brown, Shankar and Lichtveld16–Reference Cao, Shan, Gong, Kuang and Gao18, Reference Skurka, Quick, Reynolds-Tylus, Short and Bryan28 and initial survey input. Expert authority coefficients were 0.82 and 0.80, and Kendall’s W was statistically significant across both Delphi rounds, indicating robust expert consensus. Reliability and clarity were further reinforced through iterative item revision. Weights for each dimension and indicator were derived using the Analytic Hierarchy Process (AHP), enabling evidence-based prioritization within the index. This framework provides universities with a systematic method for identifying areas of inadequate preparedness and opportunities for timely intervention. It is intended to identify institutional vulnerabilities and priority areas for improvement; it does not constitute a comparative effectiveness evaluation of existing systems.
According to expert consensus in our study, prevention and preparedness constitute the highest-priority domain for MEID readiness in higher education (weight = 0.426). During the pre-event phase, preventive and preparatory actions constitute the first line of defense against the spread of infectious diseases in university settings.Reference Madden29 Within this domain, experts emphasized emergency personnel training, early warning and monitoring, health literacy and behavior promotion, and emergency management, with tertiary indicators prioritizing emergency safety knowledge dissemination, the establishment of university emergency command systems, and the implementation of primary prevention measures, such as vaccination campaigns. Our design provides an evidence-based prioritization framework rather than a comparative evaluation of existing systems; accordingly, we qualify our conclusion to state that these elements should be prioritized to strengthen institutional readiness. The COVID-19 pandemic highlighted the importance of preparedness across instructional continuity, workforce protection, and health risk mitigation. While we do not estimate causal effects, the prioritized components we identify are consistent with prior literature associating surveillance capacity and integrated command structures with more effective emergency responses in institutional settings,Reference Shamsir, Krauss, Ismail, Ab Jalil, Johar and Abdul Rahman27, Reference Raftery, Hossain and Palmer30 as well as with preparedness benefits spanning instructional continuity in higher education and workforce protection in clinical environments.Reference Turner, Albolino and Morabito31, Reference Bacus, Dayagbil, Monacillo and Bustos32 These findings suggest practical directions for universities to: (1) implement robust emergency command systems; (2) improve dissemination of safety and emergency information; (3) build training and response capabilities; and (4) bolster proactive epidemic management.
To implement these priorities effectively, leadership operates in 2 interdependent capacities within our framework: as an emergency management capability and as an organizational governance mechanism. First, leadership and coordination capabilities underpin planning, operations, partnerships, and logistics by establishing clear chains of command and role matrices, activating campus emergency command systems, creating liaison mechanisms with public health and clinical partners, and allocating resources with surge staffing. Second, at the organizational level, establishing emergency management as a primary function through a formally empowered command structure ensures the authority necessary for swift decision-making, effective cross-unit integration, and the continuity of essential functions during crises.Reference Yang, Mamun and Salameh33, Reference Hashim, Tlemsani, Matthews, Mason-Jones and Ndrecaj34 Furthermore, integrating diverse multicultural perspectives across both the operational and governance functions of leadership is essential to ensure that emergency plans adequately address the needs of all stakeholders.Reference Abukhalaf and Von Meding35
In the phase of occurrence, the promptness and efficacy of response measures are essential for mitigating the spread of an epidemic.Reference Meng, Yang and Duan36, Reference Hsu, Chien, Yeh and Chou37 Factors such as organizational decision-making and information monitoring are key components during this stage, with actions such as activating university organizational command systems and emergency medical treatment systems and establishing emergency response oversight mechanisms being of particular importance. This highlights the importance placed by experts on timely and efficient organizational leadership and information surveillance in the context of epidemics. Consequently, it is recommended that universities implement strong command structures and information monitoring protocols to facilitate swift responses and accurate decision-making in the event of an epidemic.
In the present study, the recovery and rehabilitation capability received a weight coefficient of 0.326, positioning it as the second highest among the primary indicators, trailing only behind the preparedness and prevention capability. This highlights the considerable importance and esteem that experts place on this particular capacity. Within this category, the weight assigned to “self-improvement and enhancement of pertinent plans and system measures” is the most substantial, indicating that postevent assessment, introspection, and enhancement are the foremost priorities of the system. Numerous scholarly investigations highlight the diverse challenges that must be confronted during the recovery stage of MEIDs, such as the necessity for psychological counseling,Reference Liao, Zhang, Wang, Jiang, Li and Zhang38 postevent evaluation, and reassessment of emergency protocols,Reference Moore, Geller and Clark39 all of which align with the outcomes of our research. This study highlights the importance of postepidemic recovery in emergency management, emphasizing that it is a critical component alongside preventive measures and event response. The cyclical nature of emergency management suggests that the effectiveness of response efforts is heavily influenced by the recovery phase, with implications for future preparedness strategies.
A sophisticated and inclusive assessment system for emergency response capabilities can effectively evaluate the progression of an epidemic from its initial stages through its conclusion, pinpointing deficiencies in current emergency response protocols. This process enables ongoing enhancements and the development of a more responsive, efficient, and precise epidemic emergency response capability system within academic institutions,Reference Loder, Minadeo and Jimenez40 thereby safeguarding the well-being of students and promoting the stable growth of campus environments. Finally, we acknowledge that the framework’s applicability across diverse cultural and socio-economic contexts—and the role of technology in strengthening preparedness—requires explicit, multi-site validation and integration throughout design, implementation, and evaluation.
Limitations
This study has several limitations.
-
1. The expert panel was recruited from higher education institutions within the Shanghai region, which may affect the generalizability of the findings due to geographical, cultural, and socio-economic factors and may also introduce panel homogeneity. While the observed agreement as reflected in Kendall’s W and relatively low dispersion (CV) suggests convergence, these indices cannot fully disentangle true consensus from potential similarity in expert backgrounds and context.
-
2. The panel’s geographic localization may limit multicultural generalizability because its composition may not fully capture cultural and linguistic diversity across higher education institutions or the perspectives of key stakeholder groups, such as international student services and disability support. These considerations also extend to socio-economic diversity across institutions and regions, which may influence resource capacity, risk communication, and access to essential services.
-
3. Although the proposed evaluation framework achieved expert recognition, its practical applicability and feasibility require validation through specific case studies and pilot implementation.
-
4. Given the evolving nature and heterogeneity of epidemics, future work should develop mechanisms to dynamically adjust and recalibrate the evaluation system, such as periodic indicator review, context-specific weighting, and sensitivity analyses, to address different forms of major emerging infectious diseases.
-
5. Variation in financial incentives, cultural features, and government programs—specifically funding and reimbursement structures; risk perception, trust in authorities, and linguistic diversity; as well as public-health mandates, reporting, and cross-agency coordination—may alter the observed patterns across cities, particularly in other countries.
Future research should validate the applicability of this framework across diverse regions and multiple university types, including comprehensive universities, medical universities, and other specialized institutions, ideally through multi-site Delphi rounds and field testing, and should broaden stakeholder representation to enhance cultural and linguistic inclusivity, and evaluate technology-enabled components (e.g., real-time surveillance dashboards, multilingual alerting, tele-counseling) and their contribution to preparedness and response, especially in resource-limited settings.
Conclusions
In conclusion, this study integrates the Haddon matrix with the Delphi method and AHP to develop a university-specific framework for evaluating emergency response capability. The framework spans prevention and preparedness, response, and recovery, organizing actions across the full outbreak lifecycle. Although the approach exhibits methodological rigor and acceptable internal consistency, its external validity, particularly across cultural, governance, and socio-economic settings, remains to be established; accordingly, we treat cross-regional applicability as a hypothesis to be empirically tested rather than a demonstrated generality. Future research should prioritize multi-site field studies across distinct regions and university types, include varied outbreak profiles, and assess technology-enabled components to refine indicators, optimize weighting, and enhance practical applicability.
