Hostname: page-component-7b7567c8c-2n9c9 Total loading time: 0 Render date: 2026-11-15T20:49:09.386Z Has data issue: false hasContentIssue false
  • English
  • Français

A Framework for Training Public Health Practitioners in Crisis Decision-Making

Published online by Cambridge University Press:  18 November 2015

Harvey KaymanOpen the ORCID record for Harvey Kayman[Opens in a new window] and
Tea Logar
Harvey Kayman*
Affiliation:
University of California School of Public Health, Richmond, California
Tea Logar
Affiliation:
University of California San Francisco.
*
Correspondence and reprint requests to Harvey Kayman, MD, MPH, 1301 Quarry Court #404, Richmond, CA 94801 (e-mail: Hkay20@berkeley.edu).
Rights & Permissions [Opens in a new window]

Abstract

Three sets of issues tend to be overlooked in public health emergency preparedness and response, which can be addressed with new training protocols. The first issue is procedural and concerns the often intuitive (as opposed to deliberative) nature of effective crisis decision-making. The second issue is substantive and pertains to the incorporation and prioritization of ethical, political, and logistical concerns in public health emergency guidelines. The third issue is affective and concerns human feelings and human frailty, which can derail the most well designed and best practiced procedural and substantive approaches to emergency response. This article offers an outline for a decision-making framework for public health emergencies that addresses and incorporates these issues within relevant guidelines and training. (Disaster Med Public Health Preparedness. 2016;10:165–173)

Keywords

public health decision-makingdual process (intuition/deliberation)emergency responseethicsincident command system (ICS)

Information

Type
Concepts in Disaster Medicine
Information
Disaster Medicine and Public Health Preparedness , Volume 10 , Issue 1 , February 2016, pp. 165 - 173
Copyright
Copyright © Society for Disaster Medicine and Public Health, Inc. 2015 

Public health emergencies often involve dynamic and chaotic circumstances, and the need to adjust to constantly changing conditions requires a flexible framework to guide the decisions of those in charge. During a crisis, various factors (such as ethical, political, and logistical) need to be addressed simultaneously, and making decisions about prioritization of such issues is a considerable challenge, especially because health officials are rarely trained to address these concerns (based on the research survey by Kayman et alReference Kayman, Salter and Mittal 1 cited below).

Additionally, we propose that official decisions are affected by decision-makers’ conflicting experiences, intuitions, emotions, and personal ethical values. Existing emergency planning frameworks tend to emphasize the importance of rational linear processes in crisis decision-making 2 - 6 ; however, a strictly deliberative approach tends to inadvertently minimize the vital contribution of intuition to successful problem-solvingReference Kahneman 7 - Reference Kahneman and Klein 11 during crisis. Guidance documents about allocation of scarce resources like vaccines 12 and ventilatorsReference White, Katz, Luce and Lo 13 during a pandemic are examples of recommendations that seldom address the interactions between dual process decision-making, ethics, and emotion.Reference Singer, Martin, Giacomini and Purdy 14 - Reference Barnett, Taylor and Hodge 16 We propose that unpredictable and chaotic circumstances may be dealt with more efficiently with comprehensible decision-making guidelines that take these factors into account.

A Need for Guidelines

A research survey conducted by a University of California Berkeley School of Public Health team showed in 2011 that only 13.4% of California local public health officials who responded to the survey had meaningful decision-making training.Reference Kayman, Salter and Mittal 1 The surveyed officials had almost no training in decision-making methods, processes, or criteria; did not appear to have a standard process for crisis decision-making; and did not know about or use methods to prioritize criteria as a basis for judging what course of action to take. In this article, we will propose a set of comprehensible and flexible decision-making guidelines to address these problems. A majority of survey responders (77.6%) requested comprehensive, adaptable linear and nonlinear systems with checklists, and many would like decision-making tools to address and adequately prioritize the logistical, political, and ethical concerns that present themselves in a chaotic emergency response environment.

This article offers organized but flexible guidelines that can serve public health officials and their collaborators both in day-to-day management conditions as well as in times of extreme urgency and severe emotional stress. We propose that decision-makers’ understanding and respecting the contributions of each team member’s intuition in the deliberative portion of the dual-process model will improve awareness of individual and team thought processes and ultimately the quality and effectiveness of the implementations of decisions. We further propose that documenting the processes of decision-making can be helpful in justifying how decisions were made in complex, urgent, and uncertain situations, if justification turns out to be necessary.

Discussion

Key Issues in Crisis Decision-Making

There are 3 important sets of issues that tend to be left out when it comes to public health emergency decision-making guidelines; one issue is procedural, the second is substantive, and the third is affective. The first issue concerns the crisis decision-making process, which turns out to be importantly different from the kind of decision-making usually employed in “normal” circumstances. This fact is often overlooked in public health emergency guidelines. The second overlooked issue concerns the content of decision-making guidelines, which should include ethical, political, and logistical concerns. The third overlooked or perhaps repressed issue concerns affect, which opens up the sensitive issue of unstable human emotions, which may be considered too soft, too unpredictable, or too scaryReference Docter 17 for front line leaders or commanders to deal with. Frequently, this third overlooked issue often determines successful or unsuccessful implementation of emergency response and recovery plans (Katrina, Sandy, etc). Because public health officials are rarely trained to incorporate, address, or appropriately prioritize these concerns, we argue that these 3 sets of issues should be addressed in crisis decision-making training to better implement response and recovery.

The Importance of Training

The Incident Command System (ICS) is now recognized as a proven organizational model to guide officials in charge of response to crises. However, emergency planning frameworks and the ICS 18 have traditionally ignored decision-making processes. Other government guidance documents suggest that only rational linear approaches matter in response to urgent unstable complex situations with uncertain conditions. 19 - 22 Such approaches tend to be based on optimal decision-making processes in noncrisis situations, meaning that they incorporate a seemingly rational, deliberative approach, wherein the decision-maker examines a range of alternative responses to a problem, weighs the pros and cons intuitively, and ultimately chooses what seems like the most suitable option.Reference Tversky and Kahneman 23

The 2014 Federal Emergency Management Agency course “IS 241.B Decision-Making and Problem Solving” is an excellent guide to the rational linear aspects of decision-making. The course’s problem solving matrix is comprehensive, concrete, and cogent and suitable for review by a broad spectrum of public officials. However, the chapter on ethics does not identify any important ethical issues nor does the course provide any guidance on priority setting. As long as decision-makers are fully rational, and working together seamlessly,Reference Simon 24 IS 241.B is a fine guidance document. 25

We believe that incorporating metacognition and crisis decision-making processes into the ICS structure will help public officials reach “good enough”Reference Simon 24 , Reference Haidt 26 decisions in a dynamic, time-urgent, and uncertain environment, issues that show up frequently in public health crises. Issues that are now addressed with ad hoc processes can be improved by recognizing the power of overt attention to procedural, substantive, and affective concerns.

In the remainder of this section, we present the procedural, substantive, and affective issues that we propose be integrated into crisis decision-making training and guidelines.

Procedural

The Illusion of Deliberative Decision-Making

Routine, everyday decision-making rarely actually involves a fully deliberative, conscious, linear process; instead, people tend to unconsciously call up an intuitive response that involves the use of experience and heuristics.Reference Khatri and Alvin 8 , Reference Gigerenzer and Gaissmaier 10 , Reference Kahneman and Klein 11 In 2001, HaidtReference Graham, Haidt and Koleva 27 reported multidisciplinary investigations into the source of moral judgment and concluded that affective evaluation, which is unconscious and intuitive, occurs so quickly that it is often mistaken for mere perception. We make intuitive judgments all the time and are more often than not unaware of their impact on our ultimate decisions, because they happen without explicit awareness of the cues that generate them.Reference Kahneman and Klein 11 The so-called dual-process model that integrates this knowledge is now accepted by many disciplines.Reference Hogarth 28 Using this model when making decisions, we simultaneously employ 2 different systems or processes: system 1 consists in automatic operations and produces intuitive judgments rapidly, whereas system 2 involves conscious, slow, voluntary, and controlled processes.Reference Kahneman 7 , Reference Haidt 26

In crisis situations where stakes are high and circumstances are constantly changing, making the decision-making process more deliberate and conscious may improve the quality and reliability of decisions. In several published articles investigating decision-making processes, the authors concluded that experienced, high-ranking officials tend to reach optimal decisions when they rely on heuristics and unconscious clues (system 1) in the process of reaching the initial decision, and only then employ deliberate, rational processes (system 2) when working out a strategy to implement their plan. When experts rely on their intuitive responses, this does not mean that they possess some distinct faculty that most people have no access to; it simply means that they have sufficient experience that enables them to immediately and virtually unconsciously recognize relevant parameters in a given situation and are capable of instantly thinking of the optimal procedure to approach the challenge.Reference Kahneman and Klein 11

Previously published research shows that people are often unjustifiably confident of their intuitive skills, and are in such cases frequently outperformed by algorithms.Reference Tversky and Kahneman 9 , Reference Tversky and Kahneman 23 Such findings, however, have been inconsistent and in certain circumstances, expert intuition has been shown superior to the conclusions produced by a set algorithm. Kahneman and KleinReference Kahneman and Klein 11 propose 2 necessary conditions that must be fulfilled in order for an intuitive judgment to be considered genuinely reliable. The first condition concerns environment validity: when relevant parameters in a given field are too complex, long-term, or random to warrant reliable forecast, decisions should be reached via a reasoned, carefully deliberated approach. Environments where regularities can be observed and confirmed, however, offer solid ground for intuitive judgments (eg, firefighting, nursing). The other necessary condition concerns the experts themselves: they need to have had adequate opportunity to learn the relevant cues. This means that, no matter how reliable the environment, experts’ intuitive judgment should not be trusted unless they have had plenty of validated experiences that relevantly informed their decisions.

Kahneman and KleinReference Kahneman and Klein 11 argue that high-validity environments can still contain uncertainties. As long as they also involve statistical regularity they can be considered “valid” if sufficiently reliable. Crisis situations may be singular overall, but they still comprise discrete events, each of which tends to be far less rare than their sum. This means that an experienced official may still have had adequate exposure to the relevant clues and can therefore rely on her intuitive response in a given situation. Using judgment in crisis situations requires recognizing the tension of traps such as fluency and recognition on the one hand and the benefits of years of experience on the other.Reference Kahneman and Klein 11

Moreover, expert intuitions should be considered not only because of their reliability, but also because they save critical time in crisis situations. Recognizing the benefits of a dual process response using an integrated intuitive and deliberative approach will expose experienced decision-makers to the advantages of being conscious of their thinking processes. When decision-makers are asked in surveys or focus groups why they chose a particular option, they usually provide a rational explanation to account for their choices, which makes the immediate process seem deliberative. Yet, the given explanation may be more the result of rationalization after the fact than the result of true reasoning: intuition is often relied upon to a greater extent than some would like to believe.Reference Gigerenzer and Gaissmaier 10 , Reference Graham, Haidt and Koleva 27

This fact about rationalization brings up a worrisome issue: because we trust that our decisions are the result of a reasoned process rather than an intuition, we are less likely to be convinced by a fellow team member who sees things differently and deems our decision incorrect. Unfortunately, intuitive judgments are not always based on experience, skill, or expertise; they can, and often do, result from various cognitive biases, overconfidence, and flawed heuristics.Reference Kahneman 7 We propose that, in crisis situations, the decisions should be reached by expert teams rather than by a single expert in their own field through the use of various tools (eg, Team STEPPS and Crew Resource Management), 22 , Reference Weaver, Rosen and Diaz Granados 29 - Reference Briese 31 (B Park, personal communication, Crew Resource Management Basics. US Air Force, 2014). Team members can keep each other in check and can make sure that intuitive decisions are only implemented when both necessary conditions—environment validity and adequate experience—are fulfilled.

Emergency preparedness guidelines should acknowledge the fact that people usually respond to crisis situations without being fully aware of what is guiding their initial response, and incorporate correspondingly flexible procedures into crisis decision-making training. Below, we present a list of cognitive, moral, and emotional “traps” and biases that can greatly affect our intuitive judgment in crisis situations. When decision-makers reflect on alternative courses of action in a crisis situation, they should always consider to what extent their decisions are influenced by their intuitive responses, which are heavily but unconsciously based on moral guidance heuristics.Reference Hogarth 28 However, because we do not always have access to our own intuitions and biases, teammates should check each other’s (including the chief decision-maker’s) responses for signs of bias or other sorts of unexamined principles and address them accordingly. We propose that becoming aware of our intuitive responses enables us to reexamine our positions and to more easily reconcile differing opinions.

Cognitive Biases and Heuristics

The list of various cognitive biases and heuristics that often guide our behavior seems practically inexhaustible. However, some heuristics are especially relevant in situations that feature chaos and uncertainty, as Tversky and KahnemanReference Tversky and Kahneman 9 have found in their research. These heuristics are representativeness (overestimating the importance of similarity between a specific event and the prototype of a certain type of event), availability (estimating the size or frequency of an event by relying on how easy you can think of instances of such events), and adjustment (implicitly suggested reference points affect how we adjust our estimates).

Additionally, SchwenkReference Schwenk 32 has argued that some so-called “cognitive simplification processes” are especially relevant to crisis decision-making: prior hypothesis bias, illusion of control, devaluation of partially described alternatives, and a number of other biases affect how we reach decisions in critical situations. Other biases utilized frequently include regret avoidance,Reference Connolly and Reb 33 loss aversion, and confirmation bias as well as the recognition trap.Reference Kahneman 7 We propose that decision-makers attempt to recognize the effect of these heuristics when making decisions under stressful conditions.

Moral Intuitions

Graham et alReference Hogarth 28 have gathered evidence about the sources of moral guidance heuristics, noting especially Shweder’s 3 universal moral foundational principles, the ethics of autonomy, community, and divinity,Reference Hogarth 28 (p 6 ) which influenced the development of their moral foundations theory. According to Graham and Haidt,Reference Hogarth 28 at least 5 innate moral foundations guide our moral development and often form the basic orientation of our decision-making. Each individual develops a different ranking of these foundations, based on their upbringing, community, and genetic predispositions.Reference Haidt 26 These foundations are as follows:

  • ∙ Care/harm

  • ∙ Fairness/cheating

  • ∙ Loyalty/betrayal

  • ∙ Authority/subversion

  • ∙ Sanctity/degradation

Graham and Haidt acknowledged that other moral guidance foundational principles are likely and formulated a system with 5 criteria to judge if issues qualify, and others disagree that any moral foundations exist at all.Reference Jacob and Dupoux 34 Liberty/oppression is suggested as a sixth foundation. 35 We propose a seventh foundation, “self-interest,” which meets all 5 criteria.

When trying to choose a course of action, we often make moral judgments unaware of the influence of these foundations. Because these moral considerations are likely to unconsciously guide our decision-making in crisis situations, it is important that we are aware of our commitment to 1 or 2 of them as absolute truth in group decision-making because others may be guided by different foundational principles with a similar conviction that theirs are absolute truth.

Substantive: Ethical, Political, and Logistical Concerns

In public health crises, human and material resources can become scarce, and consequently tough decisions may need to be made regarding resource allocation. While health sciences can help us determine how best to distribute resources during a crisis situation, they cannot tell us how to make initial allocation decisions just and fair. Existing structural approaches to problem solving in crises usually incorporate insufficient (if any) directions on how to prioritize ethical, political, and logistical concerns. 12 , 36 - 40 Subsequently, emergency response policy makers, researchers, and public officials struggle with the issue of deciding what specific ethical criteria to include in planning guidelines and trainings, as well as in actual emergency response.Reference Barnett, Taylor and Hodge 16 , Reference Kinlaw, Levine and Barret 41 - 44

Failing to consider the ethical ramifications of public health decisions may undermine many of the goals of public health, and the costs can be severe. Thompson et alReference Thompson, Faith and Upshur 45 argue and events in Los Angeles county and elsewhere demonstrate that neglecting ethical concerns erodes trust, erodes morale of public health and health care personnel, leads to degradation of professional responsibilities, and can stigmatize populations. 5 , Reference Thompson, Faith and Upshur 45 , 46 Because of the scope of public health police power in crisis situations, political and public health authorities can curtail individual freedoms, create unfair distribution of benefits and burdens, or neglect some of the most vulnerable populations,Reference Cauchemez, Ferguson and Wachtel 47 (p476) as a result of the standard emergency response commitment to the utilitarian doctrine, namely, the greatest good for the greatest number.Reference Driver 48 , 49

Along with ethical concerns, personnel must also consider significant logistical and political issues to maintain credibility and implement effective solutions. Logistical concerns, such as feasibility and utility, press in during times of crisis, whereas political concerns, including public pressure and buy in, may limit or direct the nature of an official response. 50

Other logistical issues arise when a variety of agencies and institutions are called upon to collaborate under conditions of intense duress. Public health personnel must quickly find common ground with other public officials and private sector emergency responders so they can consider a wide enough array of options, make plans, and implement their decisions.Reference Noble and Letsky 51 , Reference Kapucu and Garayev 52 When partners who know little about each other are thrown together by catastrophic events, they need mechanisms to get to a “shared mental model” so they can agree on a path to achieve those goals.Reference Van den Bossche, Gijselaers and Segers 53

The authors of the influential article “Mapping the Terrain”Reference Childress, Faden and Gaare 54 and othersReference White, Katz, Luce and Lo 13 , Reference Levin, Cadigan and Biddinger 55 - Reference Manuell, Co and Ellison 57 suggest mechanisms to address conflicting priorities. In reference to crisis situations, their techniques seem to be cumbersome and lack sufficient granularity to differentiate the subtle differences between conflicting values. I hope this framework will stimulate interest so that new tools can be developed to address priority setting for ethical, logistical, and political concerns in emergency response.

Affective: Emotional Responses to Stress

Public health emergencies and crisis situations can be stressful for everyone, especially health officials responsible for decision-making where health, life, and property are at risk. The cognitive systems most affected by crisis situations, according to research conducted by NASA, are perception, attention, memory, simple decision-making, complex problem solving, and response execution.Reference Bourne and Yaroush 58

There is variability among individuals when it comes to how they perform in stressful situations. The continuum of performance under stress ranges from no effect (the person handles the emergency situation as he or she would in the absence of stress) through varying degrees of degradation (the person makes errors or inadequately slow responses) to outright panic (resulting in primitive ineffective responses, or complete paralysis).Reference Bourne and Yaroush 58

McCauley et alReference McCauley, Minsky and Viswanath 59 suggest that during crisis situations a proportion of responders, especially in stigmatized groups, will experience stressful primary (media) and secondary (social stigma) negative appraisals that lead them to involuntarily “disengage” followed by inappropriate arousal, rumination, and intrusive thoughts and impulsive actions. Even well-trained official responders can disengage with cognitive coping skills giving way to more primitive self-destructive responses. Finding our way to resolving these “soft” problems might help avoid the “second” catastrophe of crisis management that has consistently derailed effective emergency response and recovery, ie, “flawed” human nature. This is no easy task. The destructive power of emotionally based mismanagement can be mitigated by strong, confident leaders and teams who use practiced TeamSTEPPS techniques.Reference King, Battles and Baker 30 , Reference Riley, Bowen and Scullard 60 , Reference Riley 61 If dealing with emotions is repressed by leaders rather than faced by leaders, otherwise clearly rational responses can fail.

In the following section, we present a decision-making framework that guides the prioritization process of all the relevant considerations. We provide comprehensive but not exhaustive lists of ethical, political, and logistical considerations that we suggest be taken into account when deciding on a course of action in a given crisis situation.

The Framework

Prioritization Process

The proposed framework employs a decision-making process that can be used to prioritize several possible courses of action in order to establish the optimal intervention in a given crisis situation. This framework is based on the highly successful OODA loop developed during the Korean War by Colonel John Boyd.Reference Boyd 62 , Reference Angerman 63 A dual-process model includes both intuition for pairwise comparisons and deliberation for choosing alternative interventions.

As presented in Tables 1 through 3, the order of steps in the process is as follows:

  1. 1. Observe (Table 1a)

  2. 2. Orient (Table 1b)

  3. 3. Decide (Table 2a)

  4. 4. Act (Table 2b)

Table 1

Decision-Making Framework Steps to Follow: Observe and Orient

Table 2

Decision-Making Framework Steps to Follow: Decide and Act

a Details of D 2b:

List your 3 final interventions.

Consult Table 3 “Relevant Considerations.”

Determine a general weight (discuss intuitive weighting assignments if group deciding) for each ethical, political, and logistical consideration based on your estimate of importance (5 for highest weight, 1 for lowest).

Determine a specific score for each proposed intervention and for each consideration based on the current situation by answering questions in Table 3. (Assign 3 for highest weight, 1 for lowest to the intervention that corresponds most to a given consideration.)

Reflect on why you assigned a specific weight and score, and reevaluate whether the underlying reason is a reliable decision-making guide in such a situation.

Set up a matrix; enter ethical, political, and logistical considerations in the left column and each course of action in the top row; assign weights and scores, and enter scores for each consideration under each course of action. Multiply the weight for each criterion by the score for each choice with that criterion and enter the number in parentheses next to the score number. 72 , Reference Bahill 73

Relevant considerations are listed in Table 3.

Table 3

Decision-Making Framework: Relevant ConsiderationsFootnote a

a The table lists ethical, political, and logistical considerations to guide the decision-making process in public health emergency and crisis situations. Answer the questions by assigning a score from 1 to 3 for each proposed intervention based on your estimation of the consequences each intervention is likely to generate. When addressing these questions, it is important to keep reexamining and reevaluating these answers based on new information.Reference Angerman 63

The matrix represents a deliberative decision-making model that takes into account the influence of intuitions and biases on the decision-making process in crisis situations: while the process itself is rational and deliberative, the decision-maker is encouraged to take into account the less deliberate cognitive and emotional processes and evaluate their importance and reliability in a given context.

Conclusion

Public health emergencies can be extremely stressful events for those who have to make decisions on appropriate measures and interventions in crisis situations. The chaos, unpredictability, and constant changes of circumstances make it especially hard to take all important considerations into account when making decisions that can have grave long-term consequences for the affected communities and individuals. High-ranking decision-makers often base their decisions on heuristics and biases they may not be aware of, such as deeply held moral values or responses based on past experience. While such intuitive responses can be valuable in situations that don’t leave much time for systematic, deliberative examination, they can be also grounds for irrational and counterproductive decisions; for example, considerations as “greatest good for the greatest number,” minimizing harm to the population, and pressure from political officials and community may overshadow respect for autonomy and human rights, legality, and avoidance of coercion in politically charged situations. Consulting relevant considerations within the framework we propose can enable decision-makers to become aware of their intuitions, heuristics, and biases; their own emotions; and the emotions of others and to address these considerations with a team of other experts.

Currently, decision-makers lack training that would teach them about important considerations, how to think about them and apply them.Reference Kayman, Salter and Mittal 1 We suspect that many public health officials lack awareness of their own biases and heuristics, which can drive a spectrum of decisions ranging from perfect decisions to completely wrong decisions. For the majority of decisions made by officials, the correctness of their decisions may never be known because the correctness of decisions is context and population dependent. Comparisons to other jurisdictions may or may not help. However, if public health officials approach decision-making systematically, with awareness of what guides them, documenting all relevant considerations in a systematic manner, then the process itself can be fully defensible. The idea behind our proposed framework is that public health officials will make better decisions in times of crisis if they follow a matrix that attempts to bring to surface their intuitive responses so they can examine them appropriately. We suspect that conscious and systematic consideration of our guiding principles will benefit the public health officials’ ability to reach optimal decisions in times of crisis. Because our ideas are based on a novel integration of 4 domains (application of decision theory, harnessing the potency of prioritization of ethical principles, incorporating the recognition of the impact of individual and group emotion, and integrating the experiences of the incident command system with OODA loop processes), we would like to test our hypotheses by building training modules for public health officials to evaluate what works and what needs improvement.

References

1. Kayman, H, Salter, S, Mittal, M, et al. School closure decisions made by local health department officials during the 2009 h1n1 influenza outbreak. Disaster Med Public Health Prep. 2015;9:464-471.CrossRefGoogle ScholarPubMed
2. FEMA. Threat and Hazard Identification and Risk Assessment Guide. Comprehensive Preparedness Guide (CPG) 201. 2nd ed. http://www.fema.gov/media-library-data/8ca0a9e54dc8b037a55b402b2a269e94/CPG201_htirag_2nd_edition.pdf. Published August 23, 2013. Accessed February 23, 2015.Google Scholar
3. FEMA. National Disaster Recovery Framework Strengthening Disaster Recovery for the Nation. http://www.fema.gov/media-library-data/20130726-1820-25045-5325/508_ndrf.pdf. Published September 2011. Accessed February 24, 2015.Google Scholar
4. National Preparedness. FEMA website. https://www.fema.gov/national-preparedness. Last updated February 6, 2015. Accessed February 24, 2015.Google Scholar
5. Association of State and Territorial Health Officials. Public Health Emergency Preparedness Six Years of achievement. http://cdm266901.cdmhost.com/cdm/singleitem/collection/p266901coll4/id/1749. Published 2008. Accessed March 27, 2013.Google Scholar
6. Novation. Emergency Disaster Plan Template. https://www.novationco.com/media/safety/DisasterPrepPlanTemplate.doc. Accessed May 20, 2014.Google Scholar
7. Kahneman, D. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux; 2011.Google Scholar
8. Khatri, N, Alvin, H. The role of intuition in strategic decision making. Hum Relat. 2000;53:57-86.CrossRefGoogle Scholar
9. Tversky, A, Kahneman, D. Judgment under uncertainty: heuristics and biases. Science. 1974;185:1124-1131.CrossRefGoogle ScholarPubMed
10. Gigerenzer, G, Gaissmaier, W. Heuristic decision making. Annu Rev Psychol. 2011;62:451-482.CrossRefGoogle ScholarPubMed
11. Kahneman, D, Klein, G. Conditions for intuitive expertise: a failure to disagree. Am Psychol. 2009;64:515-526.CrossRefGoogle ScholarPubMed
12. California Department of Public Health. Prioritizing Populations for Pandemic Influenza Vaccine in California. http://www.cdph.ca.gov/HealthInfo/discond/Documents/CDPH%20Interim%20Guidance-Prioritizing%20Populations%20for%20Pandemic%20Influenza%20Vaccine%20in%20California.pdf. Published August 2013. Accessed March 9, 2014.Google Scholar
13. White, D, Katz, M, Luce, J, Lo, B. Who should receive life support during a public health emergency? Using ethical principles to improve allocation decisions. Ann Intern Med. 2009;150:132-138.CrossRefGoogle ScholarPubMed
14. Singer, P, Martin, DK, Giacomini, M, Purdy, L. Priority setting for new technologies in medicine: qualitative case study. BMJ. 2000;321:1316-1318.CrossRefGoogle Scholar
15. Mbah, M, Gilligan, C. Resource Allocation for Epidemic Control in Metapopulations. PLoS One. 2011;6(9):e24577. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3172228.CrossRefGoogle Scholar
16. Barnett, DJ, Taylor, HA, Hodge, JG, et al. Resource allocation on the frontlines of public health preparedness and response: report of a summit on legal and ethical issues. Public Health Rep. 2009;124:295-303.CrossRefGoogle ScholarPubMed
17. Docter, P. Inside Out [DVD]. Walt Disney Pictures, distributor. Emeryville, CA: Studio Pixar Animation Studios; 2015.Google Scholar
18. California Emergency Medical Services Authority. Hospital Incident Command System Guidebook. CA.gov website. http://www.emsa.ca.gov/HICS/default.asp. Published August 2006. Accessed April 28, 2013.Google Scholar
19. Homeland Security. National Preparedness Report 2012. Homeland Security. 2012.Google Scholar
20. Association of Schools and Programs of Public Health. Public health preparedness and response core competency model. http://www.asph.org/document.cfm?page=1081. Version current December 17, 2010. Accessed October 28, 2013.Google Scholar
21. Centers for Disease Control and Prevention. Public health preparedness capabilities: national standards for state and local planning. CDC website. http://www.cdc.gov/phpr/capabilities. Last updated August 25, 2015. Accessed October 28, 2013.Google Scholar
22. Guidelines for field triage of injured patients: recommendations of the National Expert Panel on Field Triage, 2011. MMWR Recomm Rep. 2012;61(RR01):1-21.Google Scholar
23. Tversky, A, Kahneman, D. Rational choice and the framing of decisions. J Bus. 1986;59:S251-S278.CrossRefGoogle Scholar
24. Simon, HA. Behavioral model of rational choice. Q J Econ. 1955;69:99-118.CrossRefGoogle Scholar
25. FEMA. IS-241.B: Decision Making and Problem Solving. Student Manual, 1–134. FEMA website. http://training.fema.gov/is/courseoverview.aspx?code=is-241.b. Published March 31, 2014. Accessed August 11, 2015.Google Scholar
26. Hogarth, RM. Intuition: a challenge for psychological research on decision making. Psychol Inq. 2010;21:338-353.CrossRefGoogle Scholar
27. Haidt, J. The emotional dog and its rational tail: a social intuitionist approach to moral judgment. Psychol Rev. 2001;108:814-834.CrossRefGoogle ScholarPubMed
28. Graham, J, Haidt, J, Koleva, S, et al. Moral foundations theory: the pragmatic validity of moral pluralism. Adv Exp Soc Psychol. 2012;64:55-130.Google Scholar
29. Weaver, S, Rosen, M, Diaz Granados, D. Does teamwork improve performance in the operating room? A multilevel evaluation. Jt Comm J Qual Patient Saf. 2010;36:133-142.Google ScholarPubMed
30. King, H, Battles, J, Baker, D. TeamSTEPPSTM: Team Strategies and Tools to Enhance Performance and Patient Safety. NCBI webSite. http://www.ncbi.nlm.nih.gov/books/NBK43686/. Accessed August 12, 2015.Google Scholar
31. Briese, GL. Crew Resource Management: A Positive Change for the Fire Service. International Association of Fire Chiefs website. http://www.iafc.org/files/1SAFEhealthSHS/pubs_CRMmanual.pdf. Published August 2002. Accessed April 3, 2013.Google Scholar
32. Schwenk, CR. Cognitive simplification processes in strategic decision-making: insights from behavioral decision theory and cognitive psychology. Strateg Manage J. 1984;5:111-120.CrossRefGoogle Scholar
33. Connolly, T, Reb, J. Omission bias in vaccination decisions: Where’s the ‘omission’? Where’s the ‘bias’? Organ Behav Hum Decis Process. 2003;91:186-202.CrossRefGoogle Scholar
34. Jacob, P, Dupoux, E. Universal moral grammar: a critical appraisal. Trends Cogn Sci (Regul Ed). 2007;11:373-378.Google Scholar
35. Moralfoundations.org. Website. http://www.moralfoundations.org. Accessed February 24, 2015.Google Scholar
36. Centers for Disease Control and Prevention. CDC Influenza Pandemic Operation Plan (OPLAN). Washington, DC: US Department of Health and Human Services; 2006.Google Scholar
37. Department of Homeland Security, Federal Emergency Management Agency. National Preparedness Report. http://www.fema.gov/media-library-data/20130726-1833-25045-2705/national_preparedness_report_20120330_v2_1.pdf. Published March 30, 2012. Accessed April 17, 2013.Google Scholar
38. Drazen, JM, Campion, EW, Robin, EJ, et al. Ebola in West Africa at one year — from ignorance to fear to roadblocks. N Engl J Med. 2015;372:263-264.CrossRefGoogle ScholarPubMed
39. Gonsalves, G, Staley, P. Panic, caranoia, and public health—the AIDS epidemic’s lessons for Ebola. N Engl J Med. 2014;371:2348-2349.CrossRefGoogle ScholarPubMed
40. US Government Accountability Office. Influenza Pandemic: Lessons from the H1N1 Pandemic Should Be Incorporated into Future Planning. GAO website. http://www.gao.gov/products/GAO-11-632. Published June 27, 2011. Accessed April 4, 2013.Google Scholar
41. Kinlaw, K, Levine, R, Barret, D. Ethical guidelines in pandemic influenza: recommendations of the Ethics Subcommittee of the Advisory Committee to the Director, Centers for Disease Control and Prevention. Disaster Med Public Health Prep. 2009;3(suppl 2):S185-S192.CrossRefGoogle Scholar
42. Kass, NE, Otto, J, O’Brien, D, et al. Ethics and severe pandemic influenza: maintaining essential functions through a fair and considered response. Biosecurity Bioterrorism Biodefense Strategy Practice Science. 2008;6(3):227-236.CrossRefGoogle ScholarPubMed
43. Vawter, DE, Garrett, JE, Gervais, KG, et al. Dueling Ethical Frameworks for Allocating Health Resources. Am J Bioeth. 2010;10(4):54-56.CrossRefGoogle ScholarPubMed
44. California OES. Meeting the Needs of Vulnerable People in Times of Disaster: A Guide for Emergency Managers. http://www.caloes.ca.gov/AccessFunctionalNeedsSite/Documents/Vulnerable%20Populations.pdf. Published May 2000. Accessed October 25, 2012.Google Scholar
45. Thompson, AK, Faith, K, Upshur, RE, et al. Pandemic influenza preparedness: an ethical framework to guide decision-making. BMC Med Ethic. 2006;7:12. doi:10.1186/1472-6939-7-12.CrossRefGoogle ScholarPubMed
46. LA County Department of Public Health. How prepared are Los Angeles County residents for an emergency? http://publichealth.lacounty.gov/wwwfiles/ph/hae/ha/EmergencyPreparedness05.pdf. Published May 2007. Accessed March 27, 2013.Google Scholar
47. Cauchemez, S, Ferguson, NM, Wachtel, C, et al. Closure of schools during an influenza pandemic. Lancet Infect Dis. 2009;9:473-481.CrossRefGoogle ScholarPubMed
48. Driver, J. The history of Utilitarianism. The Stanford Encyclopedia of Philosophy website. http://plato.stanford.edu/archives/sum2009/entries/utilitarianism-history/. Published March 27, 2009. Accessed June 18, 2014.Google Scholar
49. Forest Service. The Greatest Good. A Forest Service Centennial Film. http://www.fs.fed.us/greatestgood/press/mediakit/facts/pinchot.shtml. Accessed June 18, 2014.Google Scholar
50. Center for Infectious Disease Research and Policy. K-12 School Closure Guidance. CIDRAP website. http://www.cidrap.umn.edu/practice/k-12-school-closure-guidance-az. Published May 4, 2009. Accessed July 7, 2013.Google Scholar
51. Noble, D, Letsky, M. Cognitive-based metrics to evaluate collaboration effectiveness. Paper presented at: RTO SAS Symposium Analysis of the Military Effectiveness of Future C2 Concepts and Systems; April 23-25, 2002; The Hague, The Netherlands. http://www.au.af.mil/au/awc/awcgate/navy/onr_metrics_on_collab.pdf. Accessed October 30, 2013.Google Scholar
52. Kapucu, N, Garayev, V. Collaborative decision-making in emergency and disaster management. Int J Public Adm. 2011;34:366-375.CrossRefGoogle Scholar
53. Van den Bossche, P, Gijselaers, W, Segers, M, et al. Team learning: building shared mental models. Instr Sci. 2011;39:283-301.CrossRefGoogle Scholar
54. Childress, JF, Faden, R, Gaare, R, et al. Public health ethics: mapping the terrain. J Law Med Ethics. 2002;30:170-178.CrossRefGoogle ScholarPubMed
55. Levin, D, Cadigan, RO, Biddinger, PD, et al. Altered standards of care during an influenza pandemic: identifying ethical, legal, and practical principles to guide decision making. Disaster Med Public Health Prep. 2009;3(suppl 2):S132-S140.CrossRefGoogle ScholarPubMed
56. Daniels, N. Accountability for reasonableness. BMJ. 2000;321:1300-1301.CrossRefGoogle ScholarPubMed
57. Manuell, M, Co, M, Ellison, R. Pandemic influenza: implications for preparation and delivery of critical care services. J Intensive Care Med. 2011;26:347-367.CrossRefGoogle ScholarPubMed
58. Bourne, L, Yaroush, R. Stress and cognition: a cognitive psychological perspective. NASA/CR—2003–212282. http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20040034070.pdf. Published September 2003. Accessed October 15, 2015.Google Scholar
59. McCauley, M, Minsky, S, Viswanath, K. The H1N1 pandemic: media frames, stigmatization and coping. BMC Public Health. 2013;13:1116-1132.CrossRefGoogle ScholarPubMed
60. Riley, W, Bowen, P, Scullard, M, et al. Methodologies to Study and Improve Team Performance, Decision-Making and Training in a Department Operations Center. http://www.academyhealth.org/files/phsr/3-Bowen.pdf. Published 2012. Accessed January 29, 2013.Google Scholar
61. Riley, W. Moving Beyond HSEEP. Improving Team Performance Instructor Manual Focus on Team Function and Communications To Create High Reliability Teams For Public Health Preparedness Instructor Manual. St. Paul, MN: Minnesota Department of Health & University of Minnesota School of Public Health; 2012.Google Scholar
62. Boyd, J. The OODA or Boyd Loop in performance, learning, leadership, & knowledge. http://www.nwlink.com/~donclark/leadership/ooda.html. Accessed November 8, 2012.Google Scholar
63. Angerman, WS. Coming Full Circle With Boyd’s Ooda Loop Ideas: An Analysis of Innovative Diffusion and Evolution [thesis]. Wright-Patterson Air Force Base, OH: Air Force Institute of Technology; 2004. http://www.dtic.mil/dtic/tr/fulltext/u2/a425228.pdf. Accessed April 18, 2013.Google Scholar
64. Klein, G, Snowden, D, Pin, CL. Anticipatory thinking. In Mosier KL, Fischer UM, eds. Informed by Knowledge: Expert Performance in Complex Situations. New York: Psychology Press; 2011:235-245.Google Scholar
65. Dalkey, N. The Delphi Method: An Experimental Study of Group Opinion. RM-5888-PR. http://www.rand.org/content/dam/rand/pubs/research_memoranda/2005/RM5888.pdf. Published June 1969. Accessed April 18, 2013.Google Scholar
66. Jeste, DV, et al. Expert consensus on characteristics of wisdom: a Delphi Method study. Gerontologist. 2010;50:668-680.CrossRefGoogle Scholar
67. Manketlow, J. The Mind Tool Guide to Group Decision Making. https://ejerciciosmentales.files.wordpress.com/2010/08/groupdecisionmakingguide1.pdf. Published 2009. Accessed April 18, 2013.Google Scholar
68. Gosenheimer, C, Rust, B, Thayer-Hart, N. Project Prioritization: A Structured Approach To Working On What Matters Most. http://oqi.wisc.edu/resourcelibrary/uploads/resources/Project_Prioritization_Guide_v_1.pdf. Published 2012. Accessed May 24, 2014.Google Scholar
69. Dalal, S, Khodyakov, D, Srinivasan, R, et al. ExpertLens: a system for eliciting opinions from a large pool of non-collocated experts with diverse knowledge. Technol Forecast Soc Change. 2011;78:1426-1444.CrossRefGoogle Scholar
70. AHRQ. Pocket Guide: TeamSTEPPS. Strategies & Tools to Enhance Performance and Patient Safety. Agency for Healthcare Research and Quality website. http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/instructor/essentials/pocketguide.html#contents. Revised December 2013. Accessed June 3, 2014.Google Scholar
71. Olson, D. Multi-attribute utility theory. cbafiles.unl.edu/public/cbainternal/facStaffUploads/mcdm.PPT. Published May 15, 1996. Accessed October 22, 2015.Google Scholar
72. MAUT. Multiattribute utility theory: concepts, application, examples [slide presentation]. cbafiles.unl.edu/public/cbainternal/facstaffuploads/mcdm.ppt. Accessed February 25, 2015.Google Scholar
73. Bahill, T. The Decision Analysis and Resolution Process. http://www.sie.arizona.edu/sysengr/slides. Accessed January 9, 2013.Google Scholar
Figure 0

Table 1Decision-Making Framework Steps to Follow: Observe and Orient

Figure 1

Table 2Decision-Making Framework Steps to Follow: Decide and Act

Figure 2

Table 3Decision-Making Framework: Relevant Considerationsa