Decision-Making from Bedside to Bench

As with so many areas of human neuroscience, clinical observations provided early clues about the key neural substrates of decision-making. There is no escaping the influence of the “capricious and vacillating” post-frontal-injury Phineas Gage (Harlow, Reference Harlow1999), revived for modern audiences by the anatomical detective work of Hanna Damasio and colleagues (Damasio, Grabowski, Frank, Galaburda, & Damasio, Reference Damasio, Grabowski, Frank, Galaburda and Damasio1994). More rigorous case reports, such as of the patient EVR (Eslinger & Damasio, Reference Eslinger and Damasio1985), provided a structure to decades of clinical observation in patients with frontal injury (see Loewenstein, Weber, Hsee, & Welch, Reference Loewenstein, Weber, Hsee and Welch2001 for review) by identifying a specific deficit in decision-making, and by focusing on the role of ventral frontal regions, including ventromedial prefrontal cortex (vmPFC) and orbitofrontal cortex (OFC). Group studies of patients with damage to these areas followed, accompanied by the development of novel experimental tasks aiming to specify the component processes of decision-making (Bechara, Damasio, Tranel, & Damasio, Reference Bechara, Damasio, Tranel and Damasio1997; Rogers et al., Reference Rogers, Everitt, Baldacchino, Blackshaw, Swainson, Wynne and Robbins1999).

Reward Learning as a Window on Decision-Making

Those early efforts presaged the direction of the subsequent 15 years of human experimental work: One line of research considered decision-making in the context of trial-and-error reward learning (Murray, Wise, & Rhodes, Reference Murray, Wise and Rhodes2011). This has been a very productive approach, making contact with a long history of research on conditioning in both rodents and non-human primates, and developing in parallel with important progress on the role of dopamine in such learning (Montague & Berns, Reference Montague and Berns2002; Schultz, Reference Schultz1998). In addition to providing novel insights into the functions of OFC (Fellows & Farah, Reference Fellows and Farah2003; Fellows & Farah, Reference Fellows and Farah2005a; O'Doherty, Dayan, Friston, Critchley, & Dolan, Reference O'Doherty, Dayan, Friston, Critchley and Dolan2003; Rolls, Reference Rolls2004; Rolls, Hornak, Wade, & McGrath, Reference Rolls, Hornak, Wade and McGrath1994), this influenced both clinical and research thinking about the effects of Parkinson disease (PD) and its dopaminergic treatment on various aspects of decision-making (Cools, Altamirano, & D'Esposito, Reference Cools, Altamirano and D'Esposito2006; Swainson et al., Reference Swainson, Rogers, Sahakian, Summers, Polkey and Robbins2000). This has included the recognition of heightened risk taking in a subset of PD patients (so-called dopamine dysregulation syndrome), linked to the effects of dopamine agonist treatment (Dagher & Robbins, Reference Dagher and Robbins2009; Evans & Lees, Reference Evans and Lees2004).

Learning paradigms in humans with OFC and vmPFC damage have provided more specific accounts of certain aspects of “frontal” behavior. Damage to these regions can sometimes lead to perseveration, but this is probably better understood as a more general difficulty adapting behavior to the specific context: While such patients may perseverate despite negative feedback in some contexts, such as in simple reversal learning tasks (Fellows & Farah, Reference Fellows and Farah2003; Rolls et al., Reference Rolls, Hornak, Wade and McGrath1994), they show increased (maladaptive) “switching” behavior in other paradigms where feedback must be integrated over multiple trials (Tsuchida, Doll, & Fellows, Reference Tsuchida, Doll and Fellows2010).

The Economist's View of Decision-Making

In addition to exploiting reward learning frameworks, cognitive neuroscientists have also looked to economics and decision psychology for inspiration, finding a rich set of paradigms and models to apply to study brain-behavior relationships. The latter traditions provide formal definitions of the concept of “value” (also termed “utility”), a core construct in analyzing economic decision-making (Rangel, Camerer, & Montague, Reference Rangel, Camerer and Montague2008). Functional neuroimaging in healthy human subjects, and electrophysiological work in non-human primates has demonstrated that activity in vmPFC, OFC, and ventral striatum (in addition to other regions, depending on the paradigm) varies parametrically with the subjective value of stimuli (Kable & Glimcher, Reference Kable and Glimcher2007; Padoa-Schioppa & Cai, Reference Padoa-Schioppa and Cai2011; Tom, Fox, Trepel, & Poldrack, Reference Tom, Fox, Trepel and Poldrack2007). Economics research has also provided paradigms for studying how risk, ambiguity, delay and effort can modify value representations (for reviews of this field, dubbed “neuroeconomics,” see Loewenstein, Rick, & Cohen, Reference Loewenstein, Rick and Cohen2008; Padoa-Schioppa, Reference Padoa-Schioppa2011; Sharp, Monterosso, & Montague, Reference Sharp, Monterosso and Montague2012).

Judgment Under Certainty

Neuropsychological work has confirmed that the value information represented in vmPFC/OFC is critical for decision-making: patients with damage to this brain region make more erratic choices even in very simple preference judgment paradigms that require only comparisons of the subjective value of decision options, so-called judgment under certainty (Camille, Griffiths, Vo, Fellows, & Kable, Reference Camille, Griffiths, Vo, Fellows and Kable2011; Fellows & Farah, Reference Fellows and Farah2007; Henri-Bhargava, Simioni, & Fellows, Reference Henri-Bhargava, Simioni and Fellows2012). A common example is choosing between ice cream flavors at your favorite soda shop—a choice that is no easier if you have tried all the flavors before, and so have complete knowledge of each potential outcome. When values are similar, comparing them is difficult, and intact vmPFC/OFC seems to be critical for making such decisions: Healthy subjects are relatively consistent in their preferences in laboratory measures of decisions like this, presumably reflecting an underlying orderliness to their value assessments, while those with vmPFC/OFC damage flip-flop more often, suggesting a compromised valuation process.

Risky Decisions

There also have been advances in understanding how focal brain injury affects simple judgments under uncertainty. In formal economic terms, this refers to choices made under conditions of risk, but where the risks are known: for example, deciding between a sure 10 dollar win or gambling on a coin toss where “heads” will net you 20 dollars, “tails” nothing. The choices in this example are formally equivalent: economists would calculate expected utility (probability × reward) for each option, which here is 10$ in both cases. Regular human beings tend to be irrational; most would have a strong preference for the sure thing in this choice set, demonstrating the risk aversion that is a classic feature of human decision-making (Kahneman & Tversky, Reference Kahneman and Tversky1979). The neural substrates of this risk aversion have been studied. An early effort in this regard was the Cambridge Gambling Task, which provides subjects with known odds, and asks them to place bets (Rogers et al., Reference Rogers, Everitt, Baldacchino, Blackshaw, Swainson, Wynne and Robbins1999). Patients with focal damage affecting the vmPFC/OFC or the insula were impaired at this kind of decision-making under risk, albeit in different ways: vmPFC damage was associated with a tendency to place higher bets across the board, suggesting less risk aversion, while the bets of those with insula damage were less influenced by the particular odds of winning on a given trial (Clark et al., Reference Clark, Bechara, Damasio, Aitken, Sahakian and Robbins2008). The amygdala has also been implicated in decision-making under uncertainty (Hsu, Bhatt, Adolphs, Tranel, & Camerer, Reference Hsu, Bhatt, Adolphs, Tranel and Camerer2005).

Risky choices can occur in isolation, or in sequence: the gambler's fallacy, in which random events are believed to be linked (i.e., if the coin toss was heads 3 times in a row, many (wrongly) believe it will be tails on the next toss at a probability greater than 0.5). Whether choices in successive gambles rely on the same brain regions as single gambles is still unknown, but real life gambling often involves repeated experiences with feedback, a clear example of how learning and decision-making are intrinsically linked. The paper by Simioni et al. in this symposium suggests that this link may be important in fully understanding “impulsive” choices in PD patients taking dopaminergic medications.

Delay Discounting

Delay aversion is a second, also widely studied decision-making phenomenon that has been proposed as a basis for impulsivity. People are typically willing to pay a penalty for immediate gratification: 10$ now is worth more to many people than 20$ in a month, a phenomenon also described as “myopia for the future.” Of interest, the loss of subjective value with increasing delay is present in other animals, and can be described by a hyperbolic discounting function, the steepness of which varies considerably across individuals (Ainslie, Reference Ainslie2001). The brain substrates of this phenomenon are a matter of debate: OFC/vmPFC seems to be involved, but perhaps in relation to the basic process of comparing relative value that is at the core of any choice, rather than for any delay-specific reason. Conflicting findings have come from both fMRI and neuropsychological work (Fellows & Farah, Reference Fellows and Farah2005b; Kable & Glimcher, Reference Kable and Glimcher2007; McClure, Laibson, Loewenstein, & Cohen, Reference McClure, Laibson, Loewenstein and Cohen2004; Sellitto, Ciaramelli, & di Pellegrino, Reference Sellitto, Ciaramelli and di Pellegrino2010). Simioni et al., in this issue, show that temporal discounting, at least as measured in a conventional hypothetical task, is not sensitive to dopaminergic medication in PD patients.

Decisions in a Social Context

While these phenomena continue to be studied, other researchers have seized on the possibilities provided by economics to study decision-making in the interactions between decision-makers. Transactions involving buyers and sellers have the potential to shed light on social behavior. Economic games, such as the Ultimatum game, provide a highly structured (and much-studied) framework for this. The study by Shamay-Tsoory et al., in this symposium, illustrates the potential of such frameworks, making a link between the ability to take the perspective of another person and the willingness to accept “unfair” offers in the Ultimatum game. That study also shows that even this highly restricted social decision context likely taps a variety of abilities, from theory-of-mind to the emotions triggered by fair or unfair offers from another person to core value considerations such as those outlined above. The study by Ciaramelli et al. (in this issue) echoes this concept, arguing that moral decisions can be decomposed into “intentions” and “outcomes,” and showing that OFC damage reduces how much intentions are weighed in the balance. Again, this highlights the potential complexity of decisions in the social sphere, and also demonstrates that this complexity can be effectively studied. This is clearly rich terrain, with the potential to yield well-specified models of the brain basis of social interactions that may have tremendous clinical impact.

Integrating Neuroeconomic and Reward Learning Research

Economic frameworks have provided an interesting angle on the neural basis of decision-making, and have provided useful simplifying tools, including computational approaches (Krajbich, Lu, Camerer, & Rangel, Reference Krajbich, Lu, Camerer and Rangel2012). The link between what these methods have revealed, and the literature on reward learning is still under construction. One reason that a more complete model of the brain basis of decision-making has been elusive is that this complex behavior is probably supported by several partially redundant systems: The adaptive importance of reward and punishment information to any organism makes it likely that the human brain is equipped with multiple systems capable of solving decision problems. Do these compete within the brain for control over behavior, or do they somehow combine to produce an integrated value assessment sensitive to the demands of a given context, encoded in some neural “common currency” (Montague & Berns, Reference Montague and Berns2002; Murray et al., Reference Murray, Wise and Rhodes2011)?

While the exact mechanisms by which outcomes are valued and weighed against each other remain to be fully specified, it is clear that this is must be a forward-looking process. Decisions are taken in anticipation of outcomes; the predicted outcome is then somehow compared to the actual outcome, providing the basic contrast required for reinforcement learning (Sutton & Barton, Reference Sutton and Barton1998). Humans may update more than the value representation of the experienced choice: when the outcomes of the unchosen options are also revealed, these can also influence future decisions (as in regret, Camille et al., Reference Camille, Coricelli, Sallet, Pradat-Diehl, Duhamel and Sirigu2004). In social choice settings (inter-personal transactions), the outcome of a decision also leads to updating of beliefs about the other person, including assessment of their intentions, making a link to theory-of-mind (Shamay-Tsoory et al.; Ciaramelli et al., both articles in this issue).

Decision-Making and Emotion

A second important question is whether the brain regions implicated in economic choice are dedicated to solving such problems, or serve a more general role in weighing the value of behavioral options: It is striking that the brain regions that have, at least so far, been most strongly implicated in economic decision-making, such as OFC and amygdala, have also been implicated in social and emotional processing of various kinds (Adolphs et al., Reference Adolphs, Tranel, Hamann, Young, Calder, Phelps and Damasio1999; Beer, John, Scabini, & Knight, Reference Beer, John, Scabini and Knight2006; Heberlein, Padon, Gillihan, Farah, & Fellows, Reference Heberlein, Padon, Gillihan, Farah and Fellows2008; Zald & Andreotti, Reference Zald and Andreotti2010). The study by Bertoux et al. in this symposium adds further data to this still unresolved question: Are the social and emotional functions of these regions in common with their roles in decision-making, are they simply anatomically co-incident, or perhaps some of both?

This brief overview, and the papers that follow, provide a sense of where the neuroscience of human decision-making stands today, and a basis for thinking about the many directions that remain to be fully pursued. From a basic research point of view, there is clearly much more to be done to pin down the specific component processes subserved by the brain regions that seem most important for decision-making. There is a particular need to think more about how regions outside the PFC may be involved, and to consider how these regions interact. Economic and reward learning paradigms clearly have much to offer in further understanding the functions of brain regions, such as the insula and vmPFC, that have been a challenge to date.

From Bench Back to Bedside

The behavioral measures emerging from this research field are already finding their way into the clinic (Sharp et al., Reference Sharp, Monterosso and Montague2012; Zald & Andreotti, Reference Zald and Andreotti2010). This is welcome, in that they have the potential to shed new light on complex symptoms such as poor judgment, impulsivity and apathy, in more refined and specific ways. We can hope that clinical uptake of such tasks will be iterative, as research provides more specificity about both the processes that go into decision-making, and their brain substrates. The Iowa Gambling task already has been used fairly widely in a range of conditions, including many psychiatric disorders. However, it illustrates the challenges of measuring executive functions more generally: it is a complex task, capturing several processes to varying degree and sensitive to dysfunction in several brain systems for different reasons (Bechara, Damasio, Tranel, & Anderson, Reference Bechara, Damasio, Tranel and Anderson1998; Fellows & Farah, Reference Fellows and Farah2005a; Maia & McClelland, Reference Maia and McClelland2004).

The study by Bertoux et al. in this symposium is a good example of the effort to apply advances in decision-making research to the clinic. They include a quite specific decision-making task (reversal learning) in a battery aimed at detecting vmPFC and OFC dysfunction in patients with behavioral variant frontotemporal dementia. Their finding that reversal learning performance relates to grey matter volume reduction in OFC in this neurodegenerative disorder converges with studies in patients with focal PFC injury (Fellows & Farah, Reference Fellows and Farah2003; Tsuchida et al., Reference Tsuchida, Doll and Fellows2010), arguing that component process measures of decision-making can provide localizing information.

Further progress in translating decision-making measures to the clinic will require not only continued focusing of behavioral measures, and careful attention to the critical issue of norming (see Cohen & Insel, Reference Cohen and Insel2008, for more discussion of this point), but also more work on the clinical relevance of deficits on these new tasks. There has long been suggestive evidence of correlation between clinical symptomatology and specific deficits in patients with OFC damage (Bechara et al., Reference Bechara, Damasio, Tranel and Damasio1997; Rolls et al., Reference Rolls, Hornak, Wade and McGrath1994), but there is more to be done to fully establish the ecological validity of even the more established learning and decision-making measures, and much more for newer approaches. Clinicians are often called upon to address the decision-making capacity of patients with compromised cognition; we are clearly some way from being able to apply any of these experimental decision-making measures to address such issues.

Decision-making research is beginning to influence clinical assessment and diagnosis, albeit still in a preliminary way. Is it ready to influence treatment? As outlined earlier, there has been some impact in PD, with more awareness of the potential (adverse) effects of dopaminergic treatments on reward-related behaviors. However, the same decision-making/reward learning research framework may be useful in rationally applying dopaminergic treatments for certain aspects of apathy, a frequent feature of many neurological disorders. An improved understanding of decision-making may also provide novel rehabilitation strategies: Reward processing and decision-making are likely not monolithic constructs. Some of the research reviewed earlier, and the work in this symposium, highlights the many routes to choice. When brain injury erodes one of these routes, others could be emphasized to preserve function. Thus, value-based choice could be shored up by more use of heuristics (Gigerenzer & Todd, Reference Gigerenzer and Todd1999), or satisficing (good enough) decision strategies could be emphasized over maximizing (find the best) approaches (Fellows, Reference Fellows2006; Schwartz et al., Reference Schwartz, Ward, Monterosso, Lyubomirsky, White and Lehman2002). If learning through trial and error is impaired, explicit, rule-based learning could be substituted.

Social decision contexts may be particularly challenging for patients with impairments in learning or judgment: having social partners make their intentions and their feedback more explicit, spelling out their positive or negative reactions to the patient's behavior more strongly than through the usual subtle cues may be helpful. Translational decision-making research could direct how and when these approaches should be applied in neurological and perhaps also psychiatric patient populations (Sharp et al., Reference Sharp, Monterosso and Montague2012). Finally, simply helping patients and families understand behavioral disturbances in terms of decision-making deficits may be very helpful as they struggle to make sense of what has changed; of why, in Dr Harlow's words: “Gage was no longer Gage” (Harlow, 1868/1999). This, at least, we are ready to do now.