Introduction
The COVID-19 pandemic severely disrupted health care systems globally, affecting access, quality, and cost of care. Shortages of personal protective equipment (PPE), ventilators, and hospital beds significantly hindered health care access, limiting hospitals’ ability to treat patients. Lockdowns, social distancing measures, and overwhelmed health care facilities further restricted access, particularly for underserved and marginalized communities.Reference Kaye, Okeagu and Pham1 The quality of care declined as hospitals faced severe staff shortages and burnout, forcing many facilities to postpone elective procedures and routine medical visits—delays that exacerbated chronic health conditions.Reference Schneider2 Health care costs skyrocketed as hospitals struggled with revenue losses and increased operational expenses. The American Hospital Association estimated that from March to June 2020, U.S. hospitals and health systems experienced losses totaling approximately $202.6 billion.Reference Kaye, Okeagu and Pham1 These systemic challenges underscore the need for health care systems to enhance resiliency and improve emergency preparedness for future pandemics.
Unlike the civilian health care system, which involves multiple stakeholders across state and federal levels, the Military Health System (MHS) operates under a centralized governance structure. Managed by the Defense Health Agency (DHA), the MHS benefits from standardized policies, rapid decision-making, and coordinated health care delivery across different departments of the military.Reference Koehlmoos, Korona-Bailey and Janvrin3 This governance model also facilitated rapid deployment of medical personnel and resources to support civilian hospitals and public-private collaborations, such as Operation Warp Speed, aiding vaccine development and distribution.Reference Mani, Pomer and Korona-Bailey4 Additionally, the MHS ensures military medical readiness while delivering health care services to 9.5 million beneficiaries, including active-duty personnel, retirees, and dependents.Reference Adirim5 Among the largest health systems in the U.S., the MHS offers valuable lessons for improving quality, effectiveness, and accessibility in response to public health crises.6 Overall, the MHS played a vital role in the U.S. civilian health care sector response to the COVID-19 pandemic, with the most pronounced collaborative efforts seen in human resources, health care delivery, and medicine and technology.Reference Koehlmoos, Korona-Bailey and Janvrin3
To understand and evaluate the MHS’s response to the COVID-19 pandemic, Congress mandated a series of internal reports and after-action reviews (AARs) under National Defense Authorization Act (NDAA) 2021 §731, which were ultimately compiled into a final Report to Congress.7 The study team previously analyzed sixteen of these internal Department of Defense (DoD) reports, including the Report to Congress, to evaluate the MHS COVID-19 response. This review identified 189 findings across eight categories of the DOTMLPF-P framework—doctrine, organization, training, materiel, leadership, personnel, facilities, and policy.Reference Pomer, Munigala and Coles8 While this review identified a broad range of recommendations for the MHS’s pandemic response, it did not prioritize the recommendations and did not assess implementation importance and feasibility. The findings also highlighted key systemic gaps within the MHS, including breakdowns in communication, unclear operational roles, training disruptions, and workforce shortages.Reference Koehlmoos, Korona-Bailey and Janvrin3, Reference Pomer, Munigala and Coles8 These issues were augmented by the ongoing organizational transition of the MHS, which contributed to inconsistent policy implementation and reduced system agility during the crisis.Reference Pomer, Munigala and Coles8 Despite its robust capabilities, the MHS struggled with personnel reductions due to slow hiring processes and non-competitive wages, further straining the workforce.Reference Mani, Pomer and Pritchett9 These challenges were compounded by high levels of provider burnout, mental health strain, and rapid shifts in service delivery such as telehealth expansion, which raised concerns about deferred care and uneven workload distribution.Reference Korona-Bailey, Janvrin and Shaw10, Reference Gilder, Banaag and Madsen11 While the MHS played a vital support role for civilian systems, these internal challenges and coordination gaps underscore the need for strategic reforms to strengthen future preparedness.Reference Koehlmoos, Korona-Bailey and Janvrin3
To address the limitations of the AARs, the study team conducted a Modified Delphi consensus process to evaluate and prioritize the most critical recommendations from the AARs based on expert judgment of their importance and feasibility. The goal was not to re-evaluate the MHS pandemic response but to synthesize the broad set of recommendations from previous reports into a prioritized, expert-informed framework to support future pandemic preparedness within the MHS and inform disaster planning across other complex health systems. This project aimed to compile recommendations and engage expert opinion to rate these recommendations on the importance and feasibility of each to enable a harmonized approach for future preparedness.
Methods
The study team utilized a Modified Delphi process to prioritize a list of previously developed recommendations for future pandemic planning in the MHS. The Modified Delphi process is based on the RAND/UCLA Appropriateness methodReference Fitch, Bernstein and Aguilar12 and includes a format through which experts and other stakeholders will engage in thoughtful and conceptual discussion to generate ideas for research and recommendations for practice.Reference Linstone and Turoff13–Reference Jones and Hunter15 The Delphi technique uses a series of feedback responses from an expert panel and statistical analysis techniques to interpret the data. It uses quasi-anonymous voting to avoid the effects of dominant individuals or the tendency to conform to a particular viewpoint. The process allows for examination of underlying assumptions that lead to different judgments among group members and to generate consensus while educating group members about different and interrelated aspects of each proposed recommendation.
Recruitment
Thirty panelists were recruited for participation in the Delphi process. Panelists were subject matter experts with experience in the MHS in disaster medicine, public health, emergency management, or civilian-military partnerships. The study team recruited potential panel members via email, outlining the purpose of the Delphi process and the time commitment involved.
Delphi Panel Process
The process consisted of a pre-work phase followed by 4 rounds of panelist engagement: a round of interviews, 2 rating rounds, and a final round to present results.
Pre-work : The study team conducted a review of Congressional and DoD after-action reports of the pandemic response.Reference Pomer, Munigala and Coles8 The documents reviewed, in particular the COVID-19 MHS Report to Congress7 and DoD Office of the Inspector General (OIG) reports,16 listed potential recommendations for improving pandemic-planning capabilities based on specific issues identified in the after-action reviews. The study team extracted these recommendations and compiled them into one list for panelists to review and comment on in subsequent Delphi process rounds.
Interview Round : From July to September 2024, the study team conducted a series of semi-structured interviews with the panelists to assess initial thoughts on the preliminary list of recommendations, including clarifying the wording and value of each recommendation, and asking for updates on whether the DoD was currently addressing the recommendation. Interviewees received the list of recommendations for review beforehand. Interviews were conducted using Microsoft Teams video conferencing software. Interviewers took detailed notes, and interviews were not audio or video recorded.
Initial Rating Round : After compiling feedback from the qualitative interviews, a new recommendation list was sent to panelists for rating using REDCap software in September 2024. Five categories were added to the original 8, for a total of 13 categories (Supplement Table 1). Attendees rated each recommendation separately on importance and feasibility using a Likert scale of 1 to 9, where 1 = Not Important/Not Feasible and 9 = Extremely Important/Extremely Feasible. Importance was defined as “the significance of the recommendation for addressing future pandemics.” Feasibility was defined as “the potential to implement the recommendation, taking into consideration financial, time, technological, legal, political, administrative, cultural, or ethical factors.” The goal was to use the Delphi process to identify recommendations that were of both high importance and high feasibility, thus recommendations of uncertain importance were not advanced to the next round. All voting was quasi-anonymous: only study staff could see identities for analysis. Analysis of each set of ratings used the median panel rating less the highest and lowest scores, and a measure of disagreement. Consensus on recommendation ratings was determined using strict criteria, where high consensus was a median vote in the high tertile (rating score 7-9) and no more than 10% of votes in the low tertile (rating score 1-3); low consensus was the median vote in the low tertile (rating score 1-3) and no more than 10% of votes in high tertile (rating score 7-9); an uncertain consensus was any vote that did not fall within the high or low consensus definitions.
Consensus Rating Round : The study team facilitated a virtual conference in November 2024, where panelists engaged in group discussion and quasi-anonymous consensus voting on the recommendations rated as of uncertain importance after the previous round of voting. Panelists discussed the importance and feasibility of these recommendations and re-voted on the importance and feasibility of each recommendation using the same 9-point scale utilized in the previous round.
Final Briefing Round : The study team held a final virtual conference in January 2025 to present the final ratings and gather final comments, including dissents, as well as to learn of any new efforts to address the recommendations in the DoD.
Ethics Approval and Consent to Participate
This study was reviewed and determined to be exempt by both the Mass General Brigham Institutional Review Board (2024P000535) and the Uniformed Services University of the Health Sciences Human Research Protections Program (DBS.2024.732). Informed consent was obtained verbally prior to the start of the interview stage. Participation was voluntary, so continued participation constituted continuing implied consent.
Results
Our panel included a total of 30 panelists. Panelists represented a diverse group of MHS senior leaders and subject matter experts across military and civilian roles (Table 1), bringing varied perspectives in policy, academia, operational leadership, and clinical practice. This professional diversity facilitated robust discussions and helped ensure recommendations addressed broad operational needs and strategic priorities. Table 1 presents the professional demographics of the panelists during the COVID-19 pandemic period (2020-2023). Ten of the 30 panelists were active-duty military during the pandemic period, while the remaining 20 panelists were in civilian positions. All 30 panelists completed the interview round, 21 completed the first round of asynchronous rating, 15 participated in the second round of consensus rating, and 14 attended the final consensus conference. Panelists were not required to attend all rounds and could attend later rounds even after missing one or more earlier rounds. Several panelists missed one or more rounds due to other commitments. Additionally, the timing of our consensus rating round (November 13, 2024) and final consensus conference (January 15, 2025) fell between the US federal elections and administration transition. As many panelists were federal employees, their employment and job responsibilities were impacted by these changes in the federal government, thus impacting their ability to participate in this Delphi process.
Participant demographics during the COVID-19 pandemic period (2020-2023)
The Interview Rating Round began with 102 recommendations derived from the documents evaluated in the pre-work stage (Figure 1). Following the Interview Round, panelists collectively refined the list, removing 20 recommendations due to redundancy or completed actions, editing 25 recommendations for clarity, and adding 12 new recommendations to address previously unrecognized gaps from panelist suggestions. After the Initial Rating Round, 75 recommendations were categorized as high importance (20 high importance/high feasibility and 55 high importance/uncertain feasibility), while the remaining 19 recommendations were rated as being of uncertain importance. Recommendations with high importance/high feasibility were advanced to the final list. Recommendations with high importance/uncertain feasibility were not advanced to the final list; these recommendations were not discussed further.
Summary of Delphi rounds.
The Consensus Rating Round involved discussion of and re-voting of the 19 recommendations of uncertain importance. After discussion, 3 recommendations were removed from the list as panelists determined they were no longer relevant or had already been undertaken. Of the remaining 16 recommendations re-voted on, 2 were recategorized as low importance, 5 remained as uncertain importance, and 9 were reclassified as high importance. Of these 9 high-importance recommendations, 5 were also voted to be high feasibility, thus these 5 were added to the final list of high-importance and high-feasibility recommendations for a total of 25 recommendations (Table 2). Examples illustrating shifts in recommendation ratings include the establishment of standardized criteria for Health Protection Condition (HPCON) levels, which moved from uncertain to high importance after panelists recognized its potential to streamline installation-level decisions during crises.
Highest-ranked recommendations by category
These recommendations fell into 9 categories: 1) MHS Support to Civil Authorities (six); 2) Public Health and Public Health Emergency Management (PHEM) (six); 3) Policy (including TRICARE) (three); 4) Systems: Supply Chain (three); 5) Personnel (two); 6) Preparedness and Scenario Planning (two); 7) OCONUS (Outside the Continental US) (one); 8) Telehealth (one); and 9) Vaccination and Testing (one). Representative examples include:
-
• MHS Support to Civil Authorities: Recommendations emphasized enhanced interagency communication, including the development of tabletop exercises integrating warfighting scenarios alongside civilian pandemic responses.
-
• Public Health and Emergency Management: Recommendations highlighted clarifying authority roles, notably establishing clear guidelines for commanders in declaring public health emergencies.
-
• Systems: Supply Chain: Recommendations advocated ongoing evaluation and prioritization strategies for pandemic-specific supplies like PPE and ventilators.
-
• Personnel: Recommendations emphasized systematic programs to monitor and address health care worker burnout, ensuring sustained workforce readiness during prolonged crises.
Limitations
There are several limitations of note for this study. Recruitment for the study was limited to individuals with current or prior experience at the senior policy-making level of the MHS, particularly during the COVID-19 pandemic. The exclusion of other stakeholder groups, including frontline workers (e.g., physicians, nurses), may have influenced the importance and feasibility ratings. These stakeholders could perceive domains and priorities differently from senior leader panelists. The limited inclusivity of perspectives might have unintentionally reduced cross-disciplinary viewpoints and introduced blind spots.
The study used virtual rounds, which may have limited the richness of dialogue and the opportunity for participants to clarify the intent behind the recommendations compared with in-person rounds. In contrast, in-person interactions may foster more nuanced deliberations and consensus-building, in part because, during in-person rounds, participants may have additional time outside of the formal panel processes to discuss findings and opinions.
Participant time conflicts and constraints across multiple rounds may have led to incomplete feedback, potentially affecting the final set of recommendations.
Strengths
The Delphi process is a structured, reproducible approach to expert consensus that serves as an efficient and unbiased model for policy recommendations on a variety of topics within the DHA and MHS. Its use in this study proved particularly valuable for navigating the complexity of public health emergency preparedness and response—an area that often spans multiple agencies, disciplines, and levels of operational authority. A key strength of this project is the synthesis of previously dispersed recommendations into a unified, prioritized framework. This is understood to be the first comprehensive effort to review, evaluate, and rate pandemic preparedness recommendations across the full spectrum of MHS operations. While many after-action reports exist, they are often siloed within specific commands or departments. By consolidating insights from across the MHS and incorporating perspectives from both military and non-DoD stakeholders, this study offers a rare cross-cutting view of institutional challenges and opportunities for reform.
The panel’s diversity—including individuals with operational, policy, public health, and clinical experience—adds to the robustness of the findings. This broad representation increases the generalizability of the recommendations, suggesting that the lessons learned may be applicable not only to future military public health emergencies but also to large civilian health systems confronting similar challenges. This study distills and evaluates expert-informed recommendations from across the MHS, presenting a replicable model that strengthens military health preparedness and informs policy improvements in civilian health systems.
Discussion
The 25 highest-rated recommendations highlight key areas identified by subject matter experts to enhance MHS planning for future pandemic preparedness. These recommendations are broadly relevant to any type of pandemic or public health emergency preparedness efforts for both the MHS and large civilian health care systems, as recommendations very specific to the COVID-19 pandemic were eliminated in the early stages of the Delphi process as already addressed or ranked as of low importance. These final recommendations reflected 4 broad themes of important considerations for future pandemic responses: communication, common operating picture, up-to-date policies, and clearly defined roles and responsibilities.
Communication
Many of the top-rated recommendations emphasized the need for stronger collaboration between military and civilian domains. Six of the 25 highest-rated recommendations fell within the “MHS Support to Civil Authorities” category, reinforcing the importance of integrated interagency planning. The COVID-19 pandemic demonstrated the necessity of strengthening these communication pathways to facilitate rapid information-sharing and coordinated action. During the COVID-19 pandemic, the DoD engaged in multi-level partnerships, ranging from federal collaborations with the Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH) to local-level efforts with public health agencies.Reference Mani, Pomer and Korona-Bailey4 Yet, fragmentation in communication due to operational structures and policy differences revealed significant coordination challenges between military and civilian health sectors.17
Common Operating Picture (COP)
While strengthening communication pathways is crucial, ensuring all agencies share a real-time understanding of emerging threats further enhances coordination. This underscores the need for a common operating picture (COP)—a unified framework that facilitates situational awareness and decision-making across military and civilian health systems.17 Five of the 25 recommendations in this Delphi process emphasized the need for interoperable information systems and standardized communication protocols to establish a COP, particularly within the domains of Public Health and Public Health Emergency Management (PH/PHEM), Preparedness/Scenario Planning, and Outside the Continental United States (OCONUS). The COVID-19 pandemic exposed critical deficiencies in biosurveillance and data integration, highlighting a lack of a cohesive surveillance system that connects military and civilian data sources.Reference Mani, Pomer and Korona-Bailey4 Although the DoD maintains robust biosurveillance capabilities internally, these systems often operate in silos, limiting timely data exchange and coordinated response.Reference Mani, Pomer and Korona-Bailey4 Future preparedness strategies should focus on building a comprehensive global health monitoring framework that fully integrates military and civilian biosurveillance networks.17 This approach would support early detection of emerging infectious diseases, enhance threat assessments, and support a shift from reactive crisis response to proactive public health intervention.Reference Mani, Pomer and Korona-Bailey4
Current Policy
A comprehensive global health monitoring framework is only as effective as the policies that sustain it. Without proactive and timely policy reforms, gaps in preparedness, coordination, and response infrastructure will persist—ultimately undermining crisis response. This risk is exemplified by the Walker Dip phenomenon, where lessons learned from past crises are forgotten, leading to repeated missteps in future emergencies.Reference Walker18 Addressing the Walker Dip is critical to maintaining institutional memory and transforming reactive systems into proactive ones. Failing to do so leads to unnecessary delays, preventable casualties, and diminished national resilience.Reference Goralnick, Elster and Woodson19 To mitigate these risks and break the cycle of repeated vulnerabilities, panelists recommend targeted policy updates in 4 key areas, including health care access (through updated TRICARE Policy), supply chain resilience, telehealth integration, and vaccination and testing. By embedding these lessons into updated policy frameworks, the MHS can not only strengthen its own operational readiness but also contribute to a more agile and coordinated national health security system.
Roles and Responsibilities
Policy alone is insufficient without clearly defined operational roles. The National Response Framework (NRF) and Incident Command System (ICS) establish a standardized structure that allows DoD and civilian emergency management systems to coordinate seamlessly.20 Panelists emphasized the need for clear delineation of roles in 3 key areas: personnel, supply chain management, and PH/PHEM. Clearly defining roles and responsibilities ensures that installation commanders can act decisively, ensures personnel can be effectively deployed across military and civilian health care systems during an emergency, and allows supply chains to function without disruption, preventing critical shortages and delays. Well-defined roles and responsibilities are also key to workforce sustainability. Ambiguity in roles contributed to high levels of burnout and staff attrition across the MHS and other health systems during the COVID-19 pandemic.Reference Pomer, Munigala and Coles8, Reference Mani, Pomer and Pritchett9 By aligning DoD operations with NRF and ICS structures, installation commanders and public health officials can coordinate resources, streamline response efforts, and maintain operational readiness. Implementing these reforms ensures that the lessons of the COVID-19 pandemic are integrated and strengthens national resilience against future public health emergencies.
Key Lessons and Actionable Strategies
The findings from this Delphi process coalesced around four interrelated priorities for strengthening pandemic preparedness in the Military Health System (MHS): enhanced communication across military and civilian agencies, development of a shared COP, timely policy updates, and clearly defined operational roles. While the 25 final recommendations were identified in the context of the MHS, the overarching themes reflect the pressure points also observed in civilian health systems during the COVID-19 pandemic. This alignment suggests that the recommendations—particularly those focused on interoperable data systems, public health coordination, and workforce readiness—can inform preparedness efforts in other large health care systems. Although consensus was achieved on only a subset of recommendations rated as both highly important and feasible, panelists broadly agreed on the relevance of all proposed actions. Prioritizing the most actionable strategies enables meaningful progress under constrained resources. Collectively, these expert-informed recommendations offer not only a roadmap for enhancing MHS operational readiness and resilience but also a foundation for broader health care system reform, supporting national preparedness for future public health emergencies.
It is important to note that although only 25 recommendations reached consensus as high importance and high feasibility, there was general agreement among panelists that all recommendations had value in being evaluated and implemented. However, given limited resources, there was an understanding that certain recommendations should be prioritized more immediately than others, which helped drive consensus. These Delphi-driven recommendations from DoD-affiliated experts can lead to improvements in DoD policy and, by providing documentation of lessons learned for senior leaders and policymakers to look back on, ultimately reinforce operational readiness and national security resilience.
Although the individual recommendations in this Delphi process may be specific to the MHS, the findings from these recommendations offer strategic value to broader health care systems for future public health emergency planning. For instance, communication breakdowns and supply chain fragility were not unique to the DoD. Civilian health care systems—particularly large hospital networks and public health agencies—can adapt MHS-aligned strategies such as logistics forecasting tools, workforce mental health support initiatives, and scenario-based emergency preparedness training to strengthen their own resilience. Integrating structured mental health services for health care workers, alongside cross-training programs that build surge capacity and flexibility, can help address staff burnout and readiness challenges that were widespread across both military and civilian sectors during the COVID-19 pandemic. Making these cross-sector connections ensures the relevance and applicability of these recommendations beyond the military context.
To enhance the practical value of these recommendations, future efforts should build on this work by translating general recommendations into actionable strategies. For example, recommendations around improving interagency coordination could specify the development of shared incident management dashboards or integrated contact tracing platforms. Providing this level of specificity would allow senior leaders to more directly incorporate these insights into operational planning and policy updates.
Conclusion
The expert-backed results of this Delphi process illustrate similar patterns of preparedness recommendations as in civilian health care sectors, particularly preparedness in terms of public health measures, coordination with other health care systems and authorities, and preparedness for sufficient personnel and materiel. By using a modified Delphi process to allow subject matter experts to review recommendations that came out of early DoD reviews of the MHS response to the COVID-19 pandemic, the study team created a list of findings that can serve as the strategic foundation for refining future pandemic response frameworks within the MHS.
As a nationally integrated and centrally governed system, the MHS played a critical leadership role during the COVID-19 pandemic by supporting civilian institutions and demonstrating the power of coordinated, large-scale health response. The results of this Delphi process offer not only an opportunity to strengthen internal MHS readiness but also provide actionable strategies that can inform and elevate civilian health system preparedness. By incorporating these lessons into future policy, training, and planning efforts, the MHS can continue to serve as a national model for coordinated crisis response—demonstrating how centralized systems can act swiftly, support civilian partners, and ensure access to care across diverse populations during public health emergencies. Together, these insights lay the groundwork for a more prepared, connected, and resilient national health response system.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/dmp.2026.10315.
Data availability statement
The datasets generated and analyzed during the current study are not publicly available due to the presence of personal identifiers but are available from the corresponding author upon reasonable request.
Acknowledgments
The study team would like to acknowledge and thank the panelists who participated in the Delphi process; whether they participated in 1 phase or all 4, this project would not have been possible without their willingness to share their time and their perspectives.
Author contribution
AP: Study concept and design, interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, and approval of the final draft of the manuscript. VM: Study concept and design, interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, and approval of the final draft of the manuscript. AW: Interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, and approval of the final draft of the manuscript. CC: Study concept and design, interpretation of the data, critical revision of the manuscript for important intellectual content, and approval of the final draft of the manuscript. JSW: Study concept and design, interpretation of the data, critical revision of the manuscript for important intellectual content, and approval of the final draft of the manuscript. TPK: Study concept and design, interpretation of the data, critical revision of the manuscript for important intellectual content, and approval of the final draft of the manuscript. EG: Study concept and design, interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, and approval of the final draft of the manuscript.
Disclaimer
The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views, assertions, opinions, or policies of the Uniformed Services University of the Health Sciences (USUHS), the Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc. (HJF), the Department of Defense (DoD), or the Departments of the Army, Navy, or Air Force. Mention of trade names, commercial products, or organizations does not imply endorsement by the U.S. Government.
Funding statement
This study was funded through the Military Health System Response to COVID-19 (MiHReC) project by the United States Defense Health Agency, Grant #HU001-21-2-0089. The funding agency played no role in the design, analysis, or interpretation of findings.
Competing interests
The authors declare that they have no competing interests.
