The pursuit of pleasurable, rewarding experiences is an impetus for everyday human behavior throughout our life span. Our behaviors early in life are motivated by immediate rewards that satisfy primary needs (e.g., the search for food to satisfy hunger). As we grow, we come to place value on more long-term rewards; for instance, we progress through rigorous schooling and training in hopes of embarking on a path to a successful career. However, reward seeking can also be maladaptive and lead to risky and often poor decision making, as illustrated by excessive gambling, underage drinking, or consumption of drugs. Humans are more vulnerable to such risky behaviors during adolescence, a period where the specific neural structures and associated connectivity linked to advantageous goal-directed behavior are not yet fully developed. Thus, there is a great significance in understanding the neural basis of reward processing and decision making, particularly across development, when faulty reward processing may result in poor choices.
This review will center on recent findings from human neuroimaging research, as neuroimaging techniques have proven to be vital resources in examining the structural and functional changes across development in the putative neural circuitry implicated in reward processing and decision making. Notably, much of this work has built upon seminal research in animals, which is beyond the scope of the current paper, but thoroughly discussed elsewhere (e.g., Robbins & Everitt, Reference Robbins and Everitt1996; Schultz & Dickinson, Reference Schultz and Dickinson2000). This review will therefore take advantage of recent investigations of the human brain using neuroimaging techniques to characterize what is known about the development of the reward system and its effects on adolescent behavior. First, the contributions of frontal corticobasal ganglia networks to reward processing in humans will be discussed in detail. Second, these anatomical circuits will be considered from a developmental perspective, concentrating on adolescence, a period during which the brain undergoes vast changes. Third, and finally, some examples of what can happen when atypical reward circuitry development occurs will be discussed, particularly how these disturbances may affect decision making in clinical developmental populations.
Reward-Related Processing in Adults
Anatomical and functional considerations
Various structures have been implicated in particular facets of the brain's response to reward (Kelley & Berridge, Reference Kelley and Berridge2002; Murray, Reference Murray2007; Robbins & Everitt, Reference Robbins and Everitt1996; Rolls, Reference Rolls2000; Schultz, Tremblay, & Hollerman, Reference Schultz, Tremblay and Hollerman2000). Central to basic reward processing that contributes to decision making is the role of subcortical regions, such as the basal ganglia, and its cortical targets. These structures form different frontal corticobasal ganglia connections commonly referred to as corticostriatal “loops,” which influence executive function and goal-directed behavior (Balleine, Delgado, & Hikosaka, Reference Balleine, Delgado and Hikosaka2007; Middleton & Strick, Reference Middleton and Strick2000a, Reference Middleton and Strick2000b; Wickens, Budd, Hyland, & Arbuthnott, Reference Wickens, Budd, Hyland and Arbuthnott2007).
The basal ganglia consist of several different structures including the striatum, the globus pallidus, the subthalamic nucleus, and the substantia nigra. The striatum is the input unit of the basal ganglia and receives afferents from various cortical sites as well as different limbic regions, resulting in various corticostriatal loops (Haber, Reference Haber2003). The striatum can be further subdivided into a dorsal and ventral component (see Figure 1). Within the dorsal striatum lie the caudate nucleus and putamen, which connect primarily to more motor and cognitive regions of the prefrontal cortex (PFC). In contrast, the ventral striatum is connected to more ventral regions of the PFC thought to be involved in emotion and motivation (Groenewegen & Uylings, Reference Groenewegen and Uylings2000). The ventral striatum primarily features the nucleus accumbens (NAcc), although it also includes ventral portions of the putamen and the caudate nucleus (Fudge & Haber, Reference Fudge and Haber2002; Voorn, Vanderschuren, Groenewegen, Robbins, & Pennartz, Reference Voorn, Vanderschuren, Groenewegen, Robbins and Pennartz2004). The globus pallidus, subthalamic nucleus, and substantia nigra (pars reticulata) all comprise different subnuclei within the basal ganglia that contribute to the processing of different types of information through the basal ganglia and to cortical outputs (via the thalamus) to influence behavior. A different section of the substantia nigra, the pars compacta division (located in the midbrain), contains dopaminergic neurons that project to the dorsal striatum. Another midbrain nucleus that sends dopaminergic projections to more ventral striatal regions is the ventral tegmental area, located in the midbrain. Dopamine, an important neurotransmitter in the brain's response to rewards, is thought to exert a modulatory role over information integrated in the striatum (for a review, see Schultz, Reference Schultz2007).
A coronal section depicting the human striatum and a basic division of its subsections. The dorsal striatum comprises the caudate nucleus (orange) and putamen (red). The ventral striatum primarily includes the nucleus accumbens and portions of the ventral caudate and putamen (green).
Various cortical sites that are either directly or indirectly connected to the striatum have been implicated in reward processing. Chief among them is the orbitofrontal cortex (OFC), a potential site for the integration of sensory and affective information (for a review, see Rolls, Reference Rolls2000) that contribute to the formation of a reward representation (Kringelbach, Reference Kringelbach2005; O'Doherty, Reference O'Doherty2004). The dorsolateral PFC (DLPFC) has also been implicated in reward processing and decision making (Watanabe, Reference Watanabe1996), but with a more regulatory role potentially exerting cognitive control over rewarding alternatives (Miller & Cohen, Reference Miller and Cohen2001). In addition, the anterior cingulate (Rushworth, Walton, Kennerley, & Bannerman, Reference Rushworth, Walton, Kennerley and Bannerman2004; Tomlin et al., Reference Tomlin, Kayali, King-Casas, Anen, Camerer and Quartz2006) and insular cortex (Huettel, Reference Huettel2006; Paulus, Lovero, Wittmann, & Leland, Reference Paulus, Lovero, Wittmann and Leland2008; Preuschoff, Quartz, & Bossaerts, Reference Preuschoff, Quartz and Bossaerts2008) have been linked to different aspects of reward processing and decision making. Given the reciprocal connections between some cortical sites and the striatum, the basal ganglia and prefrontal regions that comprise various corticostriatal circuits are in a prime position to influence reward-processing, decision-making, and goal-directed behavior.
Although elegant animal paradigms and findings have informed our basic understanding of the involvement of dopamine in reward processing and the role of corticostriatal circuits in goal-directed behavior (for reviews, see Balleine & Dickinson, Reference Balleine and Dickinson1998; Robbins & Everitt, Reference Robbins and Everitt1996; Rolls, Reference Rolls2000; Schultz et al., Reference Schultz, Tremblay and Hollerman2000; Wise, Reference Wise2004), neuroimaging techniques have more recently provided a new tool to expand this knowledge to the complex domain of human reward processing and decision making (Delgado, Reference Delgado2007; Knutson & Cooper, Reference Knutson and Cooper2005; Montague & Berns, Reference Montague and Berns2002; O'Doherty, Reference O'Doherty2004). Early neuroimaging studies took advantage of the ability of positron emission tomography (PET) to measure dopamine release via radioactive tracer isotopes (e.g., raclopride). For instance, one particular study asked participants to play a video game for monetary rewards (Koepp et al., Reference Koepp, Gunn, Lawrence, Cunningham, Dagher and Jones1998). The authors observed a correlation between endogenous dopamine release in both dorsal and ventral striatum (as measured by displacement of raclopride from dopamine receptors) and game performance (which included reward gains). Subsequent PET studies have also observed similar results during tasks investigating responses to monetary rewards in other types of games, food rewards (Volkow et al., Reference Volkow, Wang, Fowler, Logan, Jayne and Franceschi2002) and even symbolic feedback (Thut et al., Reference Thut, Schultz, Roelcke, Nienhusmeier, Missimer and Maguire1997).
These early neuroimaging studies confirmed and enhanced findings from animal work, but were not without technical limitations. Because radioactive tracer isotopes require time to travel through the bloodstream and become metabolized in the brain, PET studies have poor temporal resolution. They require designs in which individual events or stimuli are not separable, making it difficult to determine exactly what observed metabolic activity is a response to and reflects. The newer technique of functional magnetic resonance imaging (fMRI) began to garner wide use, as it allows for much higher temporal resolution than PET (e.g., can assess the neural response to individual events with durations as short as a half a second). fMRI typically measures changes in the blood oxygen level dependent (BOLD) responses to determine which brain regions are involved (active) during a particular task. A correlate for the underlying neural activity, evidence from simultaneous physiological and BOLD signal recordings suggests that fMRI activation reflects inputs into a particular region (Logothetis, Pauls, Augath, Trinath, & Oeltermann, Reference Logothetis, Pauls, Augath, Trinath and Oeltermann2001).
Reward anticipation and delivery
Using fMRI, researchers began to examine the neural response underlying simple processes such as the anticipation and the delivery of rewards. For instance, Knutson et al. (Knutson, Adams, Fong, & Hammer, 2001; Knutson, Westdorp, Kaiser, & Hammer, 2000) investigated the neural correlates of reward anticipation by presenting participants with the monetary value of a particular trial (elicited by a visual cue), of which attainment was contingent on a rapid motor response. This allowed the authors to examine the expectation of reward elicited by the visual cue. The authors observed that BOLD signals in the ventral striatum parametrically varied with the magnitude of the expected reward; that is, the greater the expectation of a potential reward (e.g., $5) the greater the BOLD response in the ventral striatum. Other corroborating findings were observed with primary rewards such as juice (O'Doherty, Deichmann, Critchley, & Dolan, Reference O'Doherty, Deichmann, Critchley and Dolan2002) and even cocaine, where ventral striatum BOLD signals correlate with subjective reports of craving (Breiter et al., Reference Breiter, Gollub, Weisskoff, Kennedy, Makris and Berke1997). In addition to the ventral striatum, different paradigms have reported OFC and regions such as the amygdala (Breiter, Aharon, Kahneman, Dale, & Shizgal, Reference Breiter, Aharon, Kahneman, Dale and Shizgal2001; Kirsch et al., Reference Kirsch, Schienle, Stark, Sammer, Blecker and Walter2003; O'Doherty et al., Reference O'Doherty, Deichmann, Critchley and Dolan2002) during reward expectation. In sum, these initial investigations on the neural basis of reward processing implicated corticostriatal circuits, particularly the ventral striatum during the anticipation or expectation of potential rewards.
Given that corticostriatal circuitry is involved in reward expectation, it follows that this circuitry would also be involved during processing of rewarding outcomes. Delgado, Nystrom, Fissell, Noll, and Fiez (Reference Delgado, Nystrom, Fissell, Noll and Fiez2000) used event-related fMRI to examine the striatal response to monetary rewards and punishments in humans. Participants played a computerized card game in which they had to guess whether an unknown card was higher or lower than five and then received a monetary incentive for a correct guess (reward trials), monetary penalty for an incorrect guess (punishment trials), or neutral feedback irrespective of guess (neutral trials). A contrast between the different types of outcomes revealed greatest changes in BOLD response in both dorsal and ventral striatum (Figure 2). Specifically, the striatal BOLD response was greater for reward trials, whereas a decrease was observed following the delivery of a punishment. A follow-up study observed that, particularly in the caudate nucleus in the dorsal striatum, the BOLD signal scaled parametrically with respect to both valence (i.e., reward and punishment) and magnitude (i.e., large and small) of a trial (Delgado, Locke, Stenger, & Fiez, Reference Delgado, Locke, Stenger and Fiez2003), although this scaling may also reflect the context in which the rewards and punishments are experienced (Delgado, Stenger, & Fiez, Reference Delgado, Stenger and Fiez2004; Elliott, Friston, & Dolan, Reference Elliott, Friston and Dolan2000; Nieuwenhuis et al., Reference Nieuwenhuis, Heslenfeld, von Geusau, Mars, Holroyd and Yeung2005). Further, as discussed later, the striatum response to outcomes is highly sensitive to not only context but behavioral contingency (O'Doherty, Reference O'Doherty2004; Tricomi, Delgado & Fiez, Reference Tricomi, Delgado and Fiez2004) suggesting a greater role for the striatum in outcome processing during learning and goal-directed behavior, rather than just a response to or detection of rewards.
Striatum responses during the outcome phase of a card-guessing game. Participants guessed the value of a card for monetary rewards and penalties. The horizontal slice shows activation of the caudate nucleus bilaterally identified by a contrast of positive and negative outcomes. The time-series graph depicts the averaged hemodynamic response for each condition (reward, punishment, neutral) across the trial. An initial rise is observed for all conditions at the onset of the trial, followed by a parametric ranking according to value, with a sustained response to reward outcomes and a decrease for punishment outcomes. Adapted from “Tracking the Hemodynamic Responses to Reward and Punishment in the Striatum,” by M. R. Delgado, L. E. Nystrom, C. Fissell, D. C. Noll, and J. A. Fiez, 2000, Journal of Neurophysiology, 84, 3072–3077. Copyright 2000 by The American Physiological Society. Adapted with permission.
Different cortical sites have also been implicated in processing reward outcomes during delivery. A region in the mesial PFC (roughly corresponding to Brodmann's Area 10), for example, has been observed during the delivery of monetary rewards (Knutson, Fong, Bennett, Adams, & Hommer, Reference Knutson, Fong, Bennett, Adams and Hommer2003). The OFC has also been reported to differentially respond to monetary gains and losses (Elliott et al., Reference Elliott, Friston and Dolan2000; O'Doherty, Kringelbach, Rolls, Hornak, & Andrews, Reference O'Doherty, Kringelbach, Rolls, Hornak and Andrews2001). It is of interest that the OFC has also been linked to the representation of different rewards (for review, see Kringelbach, Reference Kringelbach2005) such as beautiful faces (O'Doherty et al., Reference O'Doherty, Winston, Critchley, Perrett, Burt and Dolan2003) and pleasant odors (Anderson et al., Reference Anderson, Christoff, Stappen, Panitz, Ghahremani and Glover2003; Gottfried, O'Doherty, & Dolan, Reference Gottfried, O'Doherty and Dolan2003). Further, BOLD signals in OFC decrease after devaluation of a stimulus (e.g., satiety to a particular reinforcer), suggesting that these outcome representations are also dependent on the current motivational state of the organism (O'Doherty et al., Reference O'Doherty, Rolls, Francis, Bowtell, McGlone and Kobal2000), and this representation of value is reflected during decision making (Plassmann, O'Doherty, & Rangel, Reference Plassmann, O'Doherty and Rangel2007; Plassmann, O'Doherty, Shiv, & Rangel, Reference Plassmann, O'Doherty, Shiv and Rangel2008). Thus, research suggests that processing of primary and secondary rewards relies on corticostriatal circuitry involved in goal-directed behavior.
Reward-related learning
An efficient actor strives to maximize the available rewards in the environment by learning what choices provide the best outcomes. It is not surprising, therefore, that corticostriatal circuits involved in reward processing are sensitive to contingencies, predictability and learning in general. One study by Berns et al. (Reference Berns, McClure, Pagnoni and Montague2001) demonstrated this phenomenon by showing that activity in corticostriatal circuits was modulated by the predictability of rewards. The authors administered liquid rewards (juice or water) to thirsty participants either at predictable or unpredictable time intervals. Activity in the ventral striatum and OFC was greatest when the rewards were unpredictable, and did not correlate with subjective preferences for a specific liquid, suggesting a role for corticostriatal circuits in predicting potential rewards in the environment.
This study builds upon dopaminergic theories of reinforcement learning (see Dayan & Balleine, Reference Dayan and Balleine2002; Schultz, Reference Schultz2007), and is consistent with the idea that specific corticostriatal systems are involved in a general valuation process to guide behavior (Montague & Berns, Reference Montague and Berns2002). As previously mentioned, one of the primary inputs into the striatum is dopamine. The widespread dopaminergic innervation of corticostriatal circuits exerts a modulatory influence on goal-directed behavior. With respect to reward processing, one leading theory derived from physiological recordings of nonhuman primates suggests that dopamine bestows a prediction error learning signal that aids goal-directed behavior (Schultz, Dayan, & Montague, Reference Schultz, Dayan and Montague1997). Several characteristics of dopamine neuron firing provide evidence for this learning signal: (a) dopamine neurons fire upon receipt of unpredictable rewards, (b) dopamine neurons fire to the earliest predictor of a reward. That is, if an animal learns through conditioning that a cue (such as a light or a tone) predicts a reward, dopamine neurons will no longer fire upon reward receipt, but instead signal a potential reward upon presentation of the conditioned cue; (c) the withdrawal of an expected reward leads to a depression in firing of dopamine neurons. Dopaminergic influence therefore involves more than just responding to reward onset, instead signaling mismatches between expected and received rewards leading to a prediction error signal that indicates the need to adjust reward expectations (Schultz, Reference Schultz2007).
Given that the human striatum is a major projection site of midbrain dopamine neurons, and fMRI activation may reflect the inputs to particular regions (Logothetis et al., Reference Logothetis, Pauls, Augath, Trinath and Oeltermann2001), researchers have sought neural evidence of a prediction error learning signal in the human striatum. Such studies have tested whether activity in the striatum was related to positive (unexpectedly receiving a reward) and negative (omission of an expected reward) temporal prediction errors (McClure, Berns, & Montague, Reference McClure, Berns and Montague2003; O'Doherty et al., Reference O'Doherty, Winston, Critchley, Perrett, Burt and Dolan2003; Pagnoni, Zink, Montague, & Berns, Reference Pagnoni, Zink, Montague and Berns2002). In one such study, the authors used a classical conditioning paradigm with juice rewards and periodically induced both positive and negative prediction errors by delivering juice at an unexpected later time (McClure et al., Reference McClure, Berns and Montague2003). They found that higher BOLD responses in the putamen correlated with positive prediction errors, whereas decreased putamen signals reflected negative prediction errors.
Within the striatum, fMRI studies started to investigate how different striatal components contributed to reward-related learning. Research in nonhuman animals provided insight into the idea of a ventromedial to dorsolateral gradient mediating the flow of information during affective learning (Voorn et al., Reference Voorn, Vanderschuren, Groenewegen, Robbins and Pennartz2004), particularly with respect to action selection during decision making (Balleine et al., Reference Balleine, Delgado and Hikosaka2007). This distinction is also observed in human reward-related learning, with the simpler dorsal–ventral dichotomy. For instance, Tricomi and colleagues (2004) sought to determine if the differential dorsal striatal (caudate nucleus) response to reward versus punishment outcomes previously reported (Delgado et al., Reference Delgado, Nystrom, Fissell, Noll and Fiez2000) was due to the outcomes' hedonic nature (i.e., the pleasure of the reward) or if it was driven by the requirement that subjects perform a particular action to earn rewards (e.g., the contingency between action and outcome). The authors conducted two experiments using an oddball paradigm in which reward outcomes were randomly presented oddballs that followed a motor requirement. It is important that there were two motor conditions triggered by a visual cue. In the first (noncontingent condition), participants were instructed that their button presses had no effect on the valence of the outcome (reward or punishment), whereas in the second (contingent condition), they were told that they had control over the outcome via their button presses. The authors found no responses in the dorsal striatum during the noncontingent condition aimed at testing for the hedonic value of a monetary reward. Instead, the dorsal striatum response was sensitive to the perceived action–reward contingency in the second condition, suggesting that the dorsal striatum, particularly the caudate nucleus, responds to reinforcement of an action rather than a reward per se, a result a corroborated by different paradigms (Elliott, Newman, Longe, & William Deakin, Reference Elliott, Newman, Longe and William Deakin2004; Zink, Pagnoni, Martin-Skurski, Chappelow, & Berns, Reference Zink, Pagnoni, Martin-Skurski, Chappelow and Berns2004).
An elegant study by O'Doherty and colleagues (Reference O'Doherty, Dayan, Schultz, Deichmann, Friston and Dolan2004) provides an expanded view of the functional divisions of the striatum in the form of the actor–critic model (Barto, Reference Barto, Houk, Davis and Beiser1995). O'Doherty and colleagues (Reference O'Doherty, Dayan, Schultz, Deichmann, Friston and Dolan2004) hypothesized that the ventral striatum is the “critic,” which learns to predict rewards, whereas the dorsal striatum is the “actor,” which learns to choose the action associated with rewarding outcomes. Participants learned to associate particular cues with a higher probability of receiving a juice reward both during a Pavlovian or passive learning paradigm and an instrumental or active learning session. They observed ventral striatum activity correlating with prediction errors in both Pavlovian and instrumental tasks supporting a general “critic” role, whereas dorsal striatum activity correlated with prediction errors during the instrumental task only, suggesting an “actor” role.
The role of the dorsal striatum, particularly the caudate nucleus, in instrumental learning tasks has been reported in a variety of studies investigating more cognitive-based category or feedback learning (Filoteo et al., Reference Filoteo, Maddox, Simmons, Ing, Cagigas and Matthews2005; Poldrack et al., Reference Poldrack, Temple, Protopapas, Nagarajan, Tallal and Merzenich2001; Seger & Cincotta, Reference Seger and Cincotta2005) to monetary rewards (Delgado, Miller, Inati, & Phelps, Reference Delgado, Miller, Inati and Phelps2005; Haruno & Kawato, Reference Haruno and Kawato2006; Haruno et al., Reference Haruno, Kuroda, Doya, Toyama, Kimura and Samejima2004) with considerable overlap in terms of anatomy and function between the two types of feedback (Tricomi, Delgado, McCandliss, McClelland, & Fiez, Reference Tricomi, Delgado, McCandliss, McClelland and Fiez2006). Notably, the hemodynamic response in the caudate nucleus is greater when uncertainty or the need to learn because of uncertain probabilities is higher, and decreases when action–outcome associations become fully learned (Delgado, Miller, et al., 2005) in accordance with previous results on the striatum and predictability (Berns et al., Reference Berns, McClure, Pagnoni and Montague2001) and dopaminergic signals during probability and uncertainty (Fiorillo, Tobler, & Schultz, Reference Fiorillo, Tobler and Schultz2003).
Dorsal and ventral striatum differences may be apparent in some of these paradigms, although it is clear they both contribute to reward-related learning. This is illustrated in a recent study where advantageous decision making requires participants to learn which of four different decks of cards yields the best probability of getting a winning card (Schonberg, Daw, Joel, & O'Doherty, Reference Schonberg, Daw, Joel and O'Doherty2007). Better behavioral performance correlated with stronger prediction error signals in both dorsal and ventral striatum.
Although the striatum is clearly a key player in reward-related learning paradigms, particularly when mismatches between expected and attained reward occur, involvement of the PFC is often observed in conjunction. One hypothesis supported by neurophysiological recordings is that different learning rates subserve reward-related learning in corticostriatal circuits, with the striatum being involved in initial learning of associations or contingencies, with its connectivity helping shape representations in the PFC (Pasupathy & Miller, Reference Pasupathy and Miller2005). Both ventral striatum and OFC are observed in a reward-learning task, for instance, with the ventral striatum activity shifting to the earliest predictor of the reward (Galvan et al., Reference Galvan, Hare, Davidson, Spicer, Glover and Casey2005). The locus within the PFC often fluctuates, however, depending on the paradigm, the stimulus used, or the type of learning. Ventromedial PFC and OFC activity are often observed in simple conditioning studies (O'Doherty, Reference O'Doherty2004), for example, while dorsal prefrontal regions (e.g., frontopolar cortex) are observed during more cognitive tasks, such as when exploration is necessary to ensure better decisions (Daw, O'Doherty, Dayan, Seymour, & Dolan, Reference Daw, O'Doherty, Dayan, Seymour and Dolan2006). Cortical activation is also susceptible to changes in contingencies as stimuli may be devalued (Gottfried et al., Reference Gottfried, O'Doherty and Dolan2003) or reversed (Cools, Barker, Sahakian, & Robbins, Reference Cools, Barker, Sahakian and Robbins2001) depending on the context of the task or motivational state of the organism. Taken together, these results strongly suggest a role for corticostriatal circuits in reward-related learning.
Reward and decision making
The next frontier of research in human reward processes has been the intricate world of decision making, which can range from simple (e.g., do I want an apple or a piece of chocolate for dessert?) to complex choices (e.g., should I choose the apple because it is better for me in the long haul or pick the chocolate which I desire the most now?) and can vary according to a multitude of variables such as social factors (e.g., should I eat the chocolate or share it with my significant other?). The study of decision making has bridged together different disciplines from economics (Camerer, Loewensteinm & Prelec, Reference Camerer, Loewenstein and Prelec2005; Glimcher & Rustichini, Reference Glimcher and Rustichini2004) to social psychology (Cacioppo et al., Reference Cacioppo, Amaral, Blanchard, Cameron, Carter and Crews2007; Ochsner & Lieberman, Reference Ochsner and Lieberman2001). Understanding the neural basis of decision making involves some of the previously discussed basic questions of valuation signals (Montague & Berns, Reference Montague and Berns2002), as selection of one reward over another requires a mechanism for computing and assigning value to different rewards for comparison and selection purposes. Therefore, determining value and preference are important early steps in the decision-making process. Research in nonhuman primates has implicated the OFC in representing the value of a stimulus (Critchley & Rolls, Reference Critchley and Rolls1996; Rolls, Sienkiewicz, & Yaxley, Reference Rolls, Sienkiewicz and Yaxley1989; Tremblay & Schultz, Reference Tremblay and Schultz1999) specifically with regard to relative preferences (Tremblay & Schultz, Reference Tremblay and Schultz1999). Similarly, corresponding evidence from human fMRI studies also implicates the OFC and ventromedial PFC in the representation of value (for a review, see O'Doherty, Reference O'Doherty2004) and preferences (McClure, Li, et al., Reference McClure, Li, Tomlin, Cypert, Montague and Montague2004).
More recently, research on decision making has evolved to investigate matters of risk. That is, what neural mechanisms may be involved in making a risky compared to a safe decision. In addition, are these mechanisms modulated by a decision between an immediate reward when a subjectively better, but temporally delayed reward is available? Although there are numerous types of decisions that involve risk, the most commonly studied with fMRI are financial or economic decisions. This work on financial risk taking has developed from the burgeoning field of neuroeconomics, which is described and reviewed elsewhere (Glimcher & Rustichini, Reference Glimcher and Rustichini2004; Loewenstein, Rick, & Cohen, Reference Loewenstein, Rick and Cohen2008; Sanfey, Reference Sanfey2007). Some examples include studies using neural activity to predict a potential decision during financial decision-making tasks (Hampton & O'Doherty, Reference Hampton and O'Doherty2007; Kuhnen & Knutson, Reference Kuhnen and Knutson2005). Kuhnen and Knutson (Reference Kuhnen and Knutson2005), for instance, observed that risky choices were preceded by increased ventral striatum activation, whereas choices that did not involve risk were preceded by anterior insula activation. In addition, Hampton and O'Doherty (Reference Hampton and O'Doherty2007) found that combined activity in the anterior cingulate cortex (ACC), ventral striatum, and medial PFC from the previous trial predicted the next decision in a probabilistic reward reversal task. It is of interest that a recent study reported that participants who received anodal, excitatory stimulation (transcranial magnetic stimulation) of the right DLPFC coupled with cathodal, inhibitory stimulation of the left DLPFC showed decreased high-risk choices in a gambling task (Fecteau et al., Reference Fecteau, Pascual-Leone, Zald, Liguori, Theoret and Boggio2007), suggesting a potential regulatory role or cognitive control over decisions by the DLPFC (Miller & Cohen, Reference Miller and Cohen2001). Finally, the insula has also been involved in the assessment of risk (Huettel, Reference Huettel2006; Paulus et al., Reference Paulus, Lovero, Wittmann and Leland2008), although its general role in decision making remains unclear, with suggestions that the insula may calculate a “risk” prediction error during risk evaluation (Preuschoff et al., Reference Preuschoff, Quartz and Bossaerts2008).
Corticostriatal circuits have also been linked to intertemporal decision making, or the preference demonstrated by some for small, immediate rewards, compared to larger but delayed alternatives. Studies with frontal patients suggest that ventromedial PFC is important for considering future alternatives, rather than perseverating in the immediacy of a reward (Bechara, Tranel, & Damasio, Reference Bechara, Tranel and Damasio2000). Although some argue that subcortical regions such as the striatum are responsible for more immediate decisions, and prefrontal centers focus on long-term consequences (Li, McClure, King-Casas, & Montague, Reference Li, McClure, King-Casas and Montague2006; McClure, Laibson, Loewenstein, & Cohen, Reference McClure, Laibson, Loewenstein and Cohen2004), others believe this corticostriatal loop works in unison to represent subjective value irrespective of time but varying according to individual differences (Kable & Glimcher, Reference Kable and Glimcher2007).
Finally, decision-making research is extending to the domain of social interactions and investigating how neural and behavioral responses are modulated by social factors. This is displayed by decisions such as learning to trust someone during a monetary exchange (Berg, Dickhaut, & McCabe, Reference Berg, Dickhaut and McCabe1995), which elicits striatal activation during the acquisition of reputation (King-Casas et al., Reference King-Casas, Tomlin, Anen, Camerer, Quartz and Montague2005) that is further modulated by previous knowledge about moral traits of others (Delgado, Frank, & Phelps, Reference Delgado, Frank and Phelps2005), which may lead participants to want to make vengeful decisions and exert revenge on noncooperators (de Quervain et al., Reference de Quervain, Fischbacher, Treyer, Schellhammer, Schnyder and Buck2004). The study of neuroeconomics and social neuroscience is in its infancy but has many implications for understanding reward processing, risky decision-making, and goal-directed behavior that takes into account self and other.
Reward-Related Processing During Development
The dopamine-rich, cognitive corticostriatal circuits are centrally involved in reward processing. More recently, investigators have begun to use neuroimaging techniques to understand how such circuits develop across our life span, how they mature with learning, and how evolving changes in circuits processing rewards influence decision-making and goal-directed behavior, at times in a maladaptive way. Adolescence, for example, is a time characterized by developmental changes in both brain and behavior. The maturing of neural structures and the refinements of the connections between them leads to modifications in their functionality, typically accompanied by behavioral changes, such as increases in risk taking (Casey, Galvan, & Hare, Reference Casey, Galvan and Hare2005; Casey, Jones, & Hare, Reference Casey, Jones and Hare2008). One interesting idea that has been suggested (Durston & Casey, Reference Durston and Casey2006; Sowell, Thompson, Holmes, Jernigan, & Toga, Reference Sowell, Thompson, Holmes, Jernigan and Toga1999) is that the concordant protracted development of corticostriatal circuits previously reviewed may play an underlying role in increased risk-taking behavior during adolescence. This section will review select neuroimaging research that has investigated the maturation of structure and function in the brain during adolescence that may underlie risk-taking preferences exhibited by some adolescents.
Structural neuroimaging
Although much of the human brain is structurally developed by the time one reaches adolescence, it is during this time that dynamic and important neural refinements are made. Research indicates that much of the brain develops before birth and in the very early stages of life, reaching close to 90–95% of its adult weight around ages 5–6, with very little change in total cerebral volume after this point (Casey, Giedd, & Thomas, Reference Casey, Giedd and Thomas2000; Giedd et al., Reference Giedd, Snell, Lange, Rajapakse, Casey and Kozuch1996; Reiss, Abrams, Singer, Ross, & Denckla, Reference Reiss, Abrams, Singer, Ross and Denckla1996). However, the advent of MRI has allowed for a wealth of in vivo volumetric examinations of the developing brain from childhood through adolescence and young adulthood. Some of these investigations suggest that although total brain volume does not significantly change, there are significant changes in development of cortical gray and white matter, as well as for subcortical structures implicated in reward circuitry. For instance, linear increases in cortical white matter along with linear decreases in cortical gray matter are observed over the course of development from childhood through adolescence and adulthood (Caviness, Kennedy, Richelme, Rademacher, & Filipek, Reference Caviness, Kennedy, Richelme, Rademacher and Filipek1996; Giedd et al., Reference Giedd, Snell, Lange, Rajapakse, Casey and Kozuch1996; Jernigan, Trauner, Hesselink, & Tallal, Reference Jernigan, Trauner, Hesselink and Tallal1991; Pfefferbaum et al., Reference Pfefferbaum, Mathalon, Sullivan, Rawles, Zipursky and Lim1994; Reiss et al., Reference Reiss, Abrams, Singer, Ross and Denckla1996). White matter increases are indicative of an increase in myelination of axons, whereas gray matter decreases suggest decreases in synaptic density.
Certain subcortical gray matter structures have also been found to show developmental differences. For example, limbic system structures such as the amygdala and parts of the basal ganglia (globus pallidus, caudate nucleus, putamen) are reportedly larger in younger populations than typical adult levels (Caviness et al., Reference Caviness, Kennedy, Richelme, Rademacher and Filipek1996; Jernigan et al., Reference Jernigan, Trauner, Hesselink and Tallal1991), and the volumes of these structures decrease through adolescence and into young adulthood (Sowell et al., Reference Sowell, Thompson, Holmes, Jernigan and Toga1999). In a larger developmental study of children and adolescents (ages 4–18 years), Giedd et al. (Reference Giedd, Snell, Lange, Rajapakse, Casey and Kozuch1996) showed similar patterns of results while also finding interesting effects of gender and asymmetry for both cortical and subcortical regions, which included larger cerebellar, putamen, and globus pallidus volume in younger males and larger caudate volume in younger females. However, with increasing age, the caudate and putamen were seen to decrease in volume in males but not females, whereas globus pallidus volume did not significantly change across the sample in either gender. The authors posited that the patterns of development found particularly in striatal regions might have implications for understanding developmental disabilities, such as attention-deficit/hyperactivity disorder (ADHD).
More recent longitudinal studies, however, have shown a different pattern of development for cortical gray matter (Giedd, Reference Giedd2004; Giedd et al., Reference Giedd, Blumenthal, Jeffries, Castellanos, Liu and Zijdenbos1999). Specifically, changes were nonlinear with respect to age, increasing during preadolescent periods, peaking, and starting to decrease in adolescence and through postadolescence in all four lobes (described as an “inverted-U” pattern by Giedd, Reference Giedd2004). It is worth noting that although the trajectories and peaks of these patterns vary slightly across the different lobes, the general pattern seems consistent. Decreases in cortical gray matter postadolescence may reflect changes in neuronal density and synaptic pruning, contributing to the refinement of neuronal connections. It is of interest that according to Giedd (Reference Giedd2004), the general pattern of this process in gray matter occurs earliest in more primitive areas necessary for basic motor/sensory processes and latest in those areas subserving higher cognitive/executive functions needed later in life (e.g., DLPFC). The authors suggest here that the late development of DLPFC may play an integral role in the behavioral changes in adolescence. Later development of dorsal, medial, and lateral frontal areas has also been seen in previous work comparing adolescents and young adults, with the latter exhibiting more maturation (Sowell et al., Reference Sowell, Thompson, Holmes, Jernigan and Toga1999).
In concordance with MRI, another imaging technique that has been able to provide more insight into the development and connectivity of corticostriatal circuitry, as well as other regions of the brain, is diffusion tensor imaging (DTI). Briefly, DTI can measure the diffusion of water in white matter tracts in the brain, which is affected by myelin and fiber orientation. Water typically diffuses in parallel to white matter tracts, which is known as anisotropic diffusion, and this property can aid in observing the structure of white matter tracts and connectivity between brain regions (Casey et al., Reference Casey, Galvan and Hare2005; Klingberg et al., Reference Klingberg, Hedehus, Temple, Salz, Gabrieli and Moseley2000; Klingberg, Vaidya, Gabrieli, Moseley, & Hedehus, Reference Klingberg, Vaidya, Gabrieli, Moseley and Hedehus1999; Liston et al., Reference Liston, Watts, Tottenham, Davidson, Niogi and Ulug2006; Pierpaoli, Jezzard, Basser, Barnett, & Di Chiro, Reference Pierpaoli, Jezzard, Basser, Barnett and Di Chiro1996). DTI can also serve many other clinical uses that are outside the scope of this paper (for a review, see Assaf & Pasternak, Reference Assaf and Pasternak2008).
Liston et al. (Reference Liston, Watts, Tottenham, Davidson, Niogi and Ulug2006) used DTI to investigate the relation between corticostriatal connectivity and performance on a go/no-go task (which measures response inhibition) in children and adults. The prime interest here was in cognitive control of response behavior as indicated by reaction time to target and nontarget trial types. The authors were interested in projections from the PFC (left, right, and ventromedial) to the striatum. The results were supportive of the hypothesis that as corticostriatal connectivity matures, recruitment of this circuitry for response inhibition becomes more efficient. DTI analyses suggested that increases in myelination seemed to positively correlate with age in the tracts of interest connecting PFC and striatum. This increased myelination, in turn, also correlated with faster reaction times across all subjects, with a particularly strong correlation observed in the right ventral PFC tract to the striatum.
This brief review of structural neuroimaging findings shows that although much of the brain is grossly developed early on in life, important changes to structures comprising corticostriatal reward circuitry continue to occur through adolescence and some even into adulthood. The increasing myelination of cortical (especially prefrontal) areas coinciding with decreased synaptic density in cortical gray matter signifies a refinement of connectivity in the brain. Such changes might be viewed as an effort to increase efficient communication between regions.
Functional neuroimaging
Continued maturation and refinement of corticostriatal circuitry has inspired much research probing its functional consequences. fMRI allows the unique opportunity to examine the functional changes in this circuitry through the course of development. This section will consider research focusing on the functional changes occurring during development as they relate to executive function, reward learning, and risky decision making.
PFC and executive functions
The PFC is generally involved in higher cognitive and executive functions, such as cognitive control of behavior and affect, as well as in decision making (Casey et al., Reference Casey, Giedd and Thomas2000; Cohen et al., Reference Cohen, Forman, Braver, Casey, Servan-Schreiber and Noll1994; Luna et al., Reference Luna, Thulborn, Munoz, Merriam, Garver and Minshew2001; Miller & Cohen, Reference Miller and Cohen2001). Not surprisingly, the first studies of developmental neuroimaging targeted executive functions subserved by PFC centers. One such study examined neural activity during a working memory task in children 9–11 years of age (Casey et al., Reference Casey, Cohen, Jezzard, Turner, Noll and Trainor1995). The authors reported two PFC regions, the DLPFC and the ACC, a region also known to be involved in cognitive control and conflict monitoring (Bush, Luu, & Posner, Reference Bush, Luu and Posner2000), that were active during working memory trials, replicating results of a similar investigation in adults (Cohen et al., Reference Cohen, Forman, Braver, Casey, Servan-Schreiber and Noll1994). Of interest, in children, activity in both regions was more diffuse than that observed in adults, coinciding with the ongoing myelination/synaptic pruning occurring in those areas as previously described.
The PFC is also involved in response inhibition, the ability to inhibit a prepotent response, which may be deficient in adolescents. This behavior has frequently been assessed in the developmental neuroimaging literature using go/no-go paradigms, where participants either respond (go) or inhibit their responses (no-go), and better performance (ability to inhibit responding to nontargets) is correlated with increased PFC function (Casey et al., Reference Casey, Giedd and Thomas2000; Luna & Sweeney, Reference Luna and Sweeney2004). Many of these studies further indicate that children make more errors than adults, accompanied by more diffuse PFC activity, despite a clear overlap in general PFC activation compared to adults (Casey et al., Reference Casey, Castellanos, Giedd, Marsh, Hamburger and Schubert1997; Durston et al., Reference Durston, Thomas, Yang, Ulug, Zimmerman and Casey2002, Reference Durston, Davidson, Tottenham, Galvan, Spicer and Fossella2006; Tamm, Menon, & Reiss, Reference Tamm, Menon and Reiss2002). The observed diffuse prefrontal activity in these studies is in line with findings from longitudinal structural work (Giedd et al., Reference Giedd, Blumenthal, Jeffries, Castellanos, Liu and Zijdenbos1999, 2004); taken together, the structural and functional research seem to suggest that as the PFC matures and connections refine, activation will be more focal when trying to exert cognitive control.
Reward anticipation and delivery
Adolescence is typically characterized by increases in impulsivity, risk-taking behavior, and a general impaired ability to consider long-term goals or consequences of behavior, compared to adults (O'Donoghue & Rabin, Reference O'Donoghue, Rabin and Gruber2001; Reyna & Farley, Reference Reyna and Farley2006; Spear, Reference Spear2000). As noted in a review by Spear (Reference Spear2000), increased risk-taking behaviors during comparable developmental periods in other species (e.g., rats) have been observed as important in the context of increased approach behavior, novelty seeking, and social affiliation. These behaviors may be important in developing independence. As such, risk taking in humans may have some similar positive benefits, such as boosting one's self esteem or aiding in social development (Spear, Reference Spear2000). However, risk taking can also lead to severe negative consequences such as drug addiction and other maladaptive behaviors (Reyna & Farley, Reference Reyna and Farley2006). The continuing development of the human PFC and its ability to exert inhibitory control of motivated and impulsive behaviors (in which areas of the striatum have been implicated) might underlie the increased risky decision making exhibited by adolescents (Chambers, Taylor, & Potenza, Reference Chambers, Taylor and Potenza2003). Researchers have more recently begun to delve more deeply into the neural underpinnings of reward-related and decision making processes in adolescents.
An investigation by Bjork et al. (Reference Bjork, Knutson, Fong, Caggiano, Bennett and Hommer2004) posited the idea that as the reward system is still developing, it is somewhat deficient during adolescence and requires more intense stimulation that leads to risk-seeking behavior. Comparing adolescents (12–17 years of age) and young adults (21–28 years of age), Bjork and colleagues (2004) examined the activity in the ventral striatum to anticipation of monetary rewards based on prior findings with adults (Knutson et al., Reference Knutson, Westdorp, Kaiser and Hommer2000, Reference Knutson, Adams, Fong and Hommer2001). In their paradigm, participants' reaction time to respond to various probabilistic reward cues was used as an index of motivation and outcome anticipation. Behaviorally and subjectively, both groups were similar as evidenced by reaction times and reports of more happiness and excitement with increasing potential gains, and greater unhappiness and fearfulness with increasing potential losses. Differences were seen in reward-related circuitry, however, as ventral striatum BOLD responses to anticipation of rewards were larger in young adults compared to adolescents. These findings led the authors to hypothesize that adolescents necessitate more intense and salient stimulation than young adults.
In a study investigating neural responses to reward outcomes across development, May et al. (Reference May, Delgado, Dahl, Stenger, Ryan and Fiez2004) used a gambling paradigm previously found to recruit striatal responses during reward-related processing (Delgado et al., Reference Delgado, Nystrom, Fissell, Noll and Fiez2000). Results indicated that children and adolescents (ages 8–18 years) demonstrated similar patterns of increased ventral striatum activity to reward outcomes as reported in adults (Delgado et al., Reference Delgado, Nystrom, Fissell, Noll and Fiez2000), although BOLD signals to rewards in the adolescent ventral striatum were more sustained (see Figure 3). This sustained activation in the younger population was also observed in different foci in the OFC, suggesting an increased sensitivity to rewards throughout corticostriatal circuits that may influence learning and decision making.
Responses of the adolescent left ventral striatum (Talairach coordinates [x, y, z]: −10, 7, −5), coupled with orbitofrontal activation, during the outcome phase of a card-guessing game previously used in adults (see Figure 2). Adolescents show a similar, but more sustained, response to adults when contrasting positive (green line) and negative (red line) outcomes (Y axis denotes percent BOLD signal change). Adapted from “Event-Related Functional Magnetic Resonance Imaging of Reward-Related Brain Circuitry in Children and Adolescents,” by J. C. May, M. R. Delgado, R. E. Dahl, V. A. Stenger, N. D. Ryan, J. A. Fiez, et al. 2004, Biological Psychiatry, 55, 359–366. Copyright 2004 by Elsevier. Adapted with permission.
Building on these initial papers, another investigation of responses to reward delivery (Ernst et al., Reference Ernst, Nelson, Jazbec, McClure, Monk and Leibenluft2005), hypothesized that differences in neural responses to rewards, rather than to anticipation of rewards, may underlie the increased risk-seeking behavior exhibited by adolescents. Ernst and colleagues (Reference Ernst, Nelson, McClure, Monk, Munson and Eshel2005) looked at performance on the Wheel of Fortune task (Ernst et al., Reference Ernst, Nelson, McClure, Monk, Munson and Eshel2004), designed to probe monetary decision-making behavior under risk, in adolescents (9–17 years of age) and adults (20–40 years of age). Activity in the ventral striatum was observed bilaterally in both groups for trials with large incentives when contrasting gains and nongains. Subjectively, both groups were happier to win larger rewards, but this effect was stronger in adolescents and correlated with right ventral striatum BOLD responses. Signals in the amygdala, a structure involved in the brain's response to emotion and fear in particular (LeDoux, Reference LeDoux1996, Reference LeDoux2000) were also observed showing similar patterns (greater signals for gains compared to nongains) and greater responses in adults. It is worthy to note, however, that amygdala responses to monetary outcomes are not easily replicated and often vary across paradigms. Regardless, Ernst and colleagues' (Reference Ernst, Nelson, McClure, Monk, Munson and Eshel2005) results suggested a dissociative pattern of neural activation in the ventral striatum and amygdala across age groups, providing support to the idea that risk taking in adolescence may be more likely characterized by a hyperactive ventral striatum (supporting approach systems) and a hypoactive amygdala (less strongly supporting avoidance mechanisms).
Finally, studies have begun to break down development into more specific age groups to focus on differences across the life span. Galvan et al. (Reference Galvan, Hare, Parra, Penn, Voss and Glover2006) investigated the interplay between the ventral striatum and the OFC in a reward learning task across three groups: children (aged 7–11), adolescents (aged 13–17), and adults (aged 23–29). Although ventral striatum activation increased overall with increasing reward value, adolescents exhibited significantly greater ventral striatum activity than children or adults. However, whereas children demonstrated the strongest OFC activity to high magnitude rewards, adolescent OFC activity was not significantly different from that of adults. The volume of activity, defined here by the number of active voxels, consistently declined with age in both regions, with this effect being more dramatic in OFC, where both children and adolescents showed significantly more diffuse activity than did adults. These results provide support for the theory that risk taking in adolescents is due to increased sensitivity to larger potential gains in neural circuits involved in approach behavior and reward-related processing (e.g., ventral striatum), coupled with less developed regulatory areas (e.g., PFC) that are not yet maximally capable to inhibit subcortical regions.
Reward and risky decision making
The research on reward anticipation and delivery in adolescence suggests an intricate functional interplay between subcortical (e.g., ventral striatum, amygdala) and prefrontal areas (e.g., PFC, OFC) during development that may provide an insight into risky decision making during adolescence. One interesting model of decision making in adolescents that builds upon some of the findings discussed above (Ernst et al., Reference Ernst, Nelson, Jazbec, McClure, Monk and Leibenluft2005; Galvan et al., Reference Galvan, Hare, Parra, Penn, Voss and Glover2006; May et al., Reference May, Delgado, Dahl, Stenger, Ryan and Fiez2004) is the triadic model (Ernst, Pine, & Hardin, Reference Ernst, Pine and Hardin2006), which proposes that motivated behavior is subserved by three neural systems (approach, avoidance, and regulatory) that work in concert. The approach system, which includes the ventral striatum (especially the NAcc), lends itself to reward-related approach behaviors. The avoidance system, in contrast, includes the amygdala, and supports avoiding harmful or threatening situations. Finally, the regulatory system, which includes prefrontal areas, specifically the medial and ventral prefrontal cortices, supports the balance of the approach and avoidance systems as a sort of moderator of the two opposing influences. Ernst, Pine, and Hardin (Reference Ernst, Pine and Hardin2006) hold that decision making in adolescents is skewed by an over influential approach system and a weaker avoidance and/or regulatory system, leading to increased risk-taking behaviors. The model is obviously not without detractors or limitations, such as evidence for the striatum and amygdala to be involved in affective learning irrespective of valence (and thus not specific to approach and avoid), but it does raise interesting ideas and theories to be tested.
Although this topic has only recently become a growing field of interest in the functional neuroimaging literature, initial studies of decision making in developmental populations have attempted to probe the role of corticostriatal circuits in adolescent risk taking. Specifically, these studies were interested in not just the processing of reward anticipation and outcomes, but also how prefrontal areas may or may not intervene and interact with these processes. One such study focused on how children (aged 9–12) and young adults (aged 18–26) differed in their abilities to estimate risk and process feedback while performing a gambling task called the “Cake Task” (van Leijenhorst, Crone, & Bunge, Reference van Leijenhorst, Crone and Bunge2006). In this paradigm, participants had to indicate which flavored wedge of a “cake” the computer was most likely to choose at random. The proportion of flavors in the cake varied across trials, providing the basis for either high- or low-risk conditions. All participants made more errors during high-risk trials, but accuracy (choosing the wedge most likely to lead to a reward) was lower for children. A neural overlap in prefrontal regions was observed across groups, with differences emerging in the level of activity within particular regions of interest. In estimating risk, for instance, the OFC and DLPFC were more active on high-risk trials regardless of group; further, lateral OFC signals in response to negative feedback were stronger in children. Responses to high-risk trials in the mPFC/ACC were stronger in the younger group as well. These results suggest that younger populations exhibit more diffuse and less effective prefrontal recruitment in risky decision making, which could result in poor decisions.
Research has implicated the ACC in cognitive control (Miller & Cohen, Reference Miller and Cohen2001), further highlighting the ACC's participation in both conflict monitoring and representing affective information (Bush et al., Reference Bush, Luu and Posner2000). In line with this, recent developmental research (Bjork, Smith, Danube, & Hommer, Reference Bjork, Smith, Danube and Hommer2007) has reported that medial cingulate regions (Brodmann's Area 24) become more sensitive to subtle risk options with age. Similarly, the selection of moderately risky options tend to decrease with age, coupled with less efficient recruitment of prefrontal regions (OFC, ventrolateral PFC, ACC) in adolescents when choosing risky options compared to adults (Eshel, Nelson, Blair, Pine, & Ernst, Reference Eshel, Nelson, Blair, Pine and Ernst2007). These effects further indicate a role for prefrontal areas in risky decision making and, with age, activity becomes more robust and focal, likely contributing to decreased risky choices and better overall decision making.
The results from the developmental functional neuroimaging literature strongly support the findings from structural work indicating important changes are occurring in corticostriatal circuitry, especially during adolescence. The increased propensity toward risk-taking behaviors typically observed in adolescent behavior is hypothesized to be due in large part to “approach” systems, regions involved in reward-related processing in the brain (e.g., striatum), that are hypersensitive to reward-seeking and thus able to outweigh underdeveloped prefrontal regions that are not yet fully capable to exert control over the approach systems.
Abnormal Reward Processing During Development
The literature on functional development of corticostriatal circuitry coheres with the findings from the structural research. The neural changes that occur during adolescence (increased myelination, synaptic pruning, and decreases in subcortical gray matter structures) are realized functionally by, for example, initial diffuse recruitment of prefrontal, regulatory structures that becomes more coherent and focal with time. This leads to a more effective ability to exert control over behavioral responses. The striatum, which has been a prime focus of recent functional developmental neuroimaging research, exhibits different response patterns in adolescents compared to adults in various reward processing paradigms. This research leads to the hypothesis that hypersensitive striatum responses, in combination with still developing regulatory and/or avoidance capacities, influence risk-seeking behavior in adolescence. The result of normal development into adulthood is a more balanced and efficient relationship between reward, avoidance, and regulatory areas to enable decision making that is not overly biased toward risky behavior.
Although the literature reviewed above details the normal course of structural and functional development of corticostriatal circuitry underlying reward processing, it is also important to consider the implications of dysfunction in this circuitry on behavior. This section of the review will briefly consider research on reward processing in both clinical (e.g., depression) and developmental (ADHD) populations as examples.
A good starting point for this discussion comes from an fMRI study of adolescents described as behaviorally inhibited (Guyer et al., Reference Guyer, Nelson, Perez-Edgar, Hardin, Roberson-Nay and Monk2006), who have been shown to exhibit negative emotion and increased reaction to novelty (Fox, Henderson, Marshall, Nichols, & Ghera, Reference Fox, Henderson, Marshall, Nichols and Ghera2005). This study investigated approach behavior in both behaviorally inhibited and noninhibited adolescents using a paradigm aimed at measuring reward anticipation (Knutson et al., Reference Knutson, Adams, Fong and Hommer2001). However, behaviorally inhibited adolescents were found to demonstrate both increased dorsal (caudate nucleus) and ventral striatum (including NAcc) activity to anticipation of larger incentives, irrespective if such incentive was positive (monetary gain) or negative (monetary loss), than did noninhibited adolescents. The authors took this finding to indicate that behaviorally inhibited adolescents, who could have precursors for anxiety disorders or depression, may show this striatal pattern of activity due to heightened motivation to avoid making errors.
Adolescence is a prime target period for the development of anxiety and depressive disorders. The symptomology of these disorders generally reflects deficits in motivation, and as such, could implicate dysfunction in corticostriatal reward circuitry (Hardin, Schroth, Pine, & Ernst, Reference Hardin, Schroth, Pine and Ernst2007). The changing role or impact of positive emotion has been stressed as an important aspect of depression (Forbes & Dahl, Reference Forbes and Dahl2005). For instance, in a study of reward-related decision making (choosing between safe and risky choices that varied in potential reward magnitude), children and adolescents ages 9 to 17 with major depressive disorders were compared with control participants. Depressed individuals exhibited decreased striatal (caudate) activation and increased inferior OFC activation both while making decisions and while evaluating outcomes (Forbes et al., Reference Forbes, May, Siegle, Ladouceur, Ryan and Carter2006). The decreased activity in corticostriatal circuits could reflect lower levels of motivation in depressed adolescents (Delgado et al., Reference Delgado, Stenger and Fiez2004).
Corticostriatal circuitry also has been implicated in developmental disorders such as ADHD. Early structural neuroimaging work in children and adolescents with ADHD (aged 5–18 years) and controls indicated that over the course of development the right caudate and right prefrontal areas were observed to have reduced volume in ADHD participants compared to controls (Castellanos et al., Reference Castellanos, Giedd, Marsh, Hamburger, Vaituzis and Dickstein1996). The major symptom in ADHD has been described as a deficit in cognitive control and response inhibition (Casey & Durston, Reference Casey and Durston2006). In particular, right corticostriatal circuitry has been implicated as dysfunctional in ADHD such that ADHD children performed a series of response inhibition tasks worse than controls, and this worse performance was seen to correlate with right hemispheric volumetric deficits in ADHD participants (Casey et al., Reference Casey, Castellanos, Giedd, Marsh, Hamburger and Schubert1997). Similar results were found in an investigation of children with ADHD and controls on a go/no-go paradigm (Durston et al., Reference Durston, Tottenham, Thomas, Davidson, Eigsti and Yang2003). Here, control children recruited right caudate nucleus, right ventral PFC, and ACC during no-go trials much more so than did ADHD children, further implicating cognitive control deficits in ADHD.
Although the involvement of corticostriatal circuitry in reward processing and decision making is evident, this has not been considered until recently in populations with ADHD. (Scheres, Milham, Knutson, & Castellanos, Reference Scheres, Milham, Knutson and Castellanos2007) looked at responses to reward anticipation in adolescents (aged 12–17) with and without ADHD. Adolescents with ADHD demonstrated no significant increases in striatal activity to reward trials compared to nonreward trials, whereas control participants exhibited increases in both ventral and dorsal striatum (caudate). Reduced activity in the ADHD group positively correlated with hyperactivity and impulsivity measures. Of interest, ADHD striatum deficits were not observed to rewarding outcomes, suggesting that impairments here were perhaps limited to motivational aspects of approach behavior.
This brief discussion of corticostriatal circuitry in psychopathology and abnormal development is by no means intended to imply that this circuitry is not implicated in other conditions. For example, a recent study highlights the role of the ACC in high-functioning autistic individuals in relation to decision making in social situations (Chiu et al., Reference Chiu, Kayali, Kishida, Tomlin, Klinger and Klinger2008). Rather, it is intended to provide insight into the importance of this circuit in normal functioning and the extent to which dysfunction here can have dire consequences. More research in this realm is necessary to further probe deficits and potential treatment options for abnormal reward processing that may contribute to the maladies of some of these psychiatric conditions.
Conclusions
The interplay between frontal corticobasal ganglia networks serves an integral role in everyday human behavior and specifically in reward-related processing. This circuitry undergoes important structural and functional modifications during development, particularly during adolescence, which in combination with other neural and bodily changes may explain the increased risk taking and poor decision making observed during this time. Although progress in research in general reward processing in adults continues, new technologies in structural and functional neuroimaging allow investigations in this area to further extend to differences and similarities across development. The rapid advancement of the field and technology brings forth new and exciting questions regarding decision making. More refined investigations will probe social interaction and goal-directed behavior while considering contributions of specific components of corticostriatal circuits across development.
One such question involves parcellation of striatum function across development. Much of the research reviewed here regarding underlying neural structures subserving reward processing in adolescence suggests that the adolescent ventral striatum is overresponsive to the expectation of potential rewards compared to adults. This might lead to increased risk-taking behavior. Although this hypothesis is consistent with a general role for the ventral striatum in risky decision making (e.g., Delgado, Frank, et al., 2005; Kuhnen & Knutson, 2005) there are some inconsistencies with respect to other regions important for reward-related processing. A discrepancy between functional and structural developmental research, for instance, lies in the fact that much of the structural work reviewed mentions specific decreases in caudate nucleus volume as development progresses toward adulthood. The caudate nucleus and its cortical connections are important for mediating processes that aid goal-directed behavior (e.g., response inhibition) and affective learning, particularly when a contingency between reward and behavior exists. Yet, the function of the caudate nucleus across development has not received as much attention as the ventral striatum in studies of reward and decision making. Considering the caudate's structural changes over time, involvement in reward processing during adulthood and implication in disorders such as ADHD, it would be beneficial to understand the maturation and adaptation of caudate function across development. Such research would address additional questions regarding adolescent motivation to pursue rewards (even when they are maladaptive) and the formation of habits throughout development.
Other questions involve extensions into the burgeoning field of neuroeconomics (Glimcher & Rustichini, Reference Glimcher and Rustichini2004), which has emerged to build from initial work on reward-related processing to investigate more complex and applied behavior. A current topic of interest, for example, is intertemporal choices, which refers to how people value rewards of differing magnitudes at different time points (e.g., an immediate gain of $5 vs. a gain of $10 in 1 week). Findings from this research implicate corticostriatal reward circuitry in evaluating such choices (Kable & Glimcher, Reference Kable and Glimcher2007; McClure, Ericson, Laibson, Loewenstein, & Cohen, Reference McClure, Ericson, Laibson, Loewenstein and Cohen2007; McClure, Laibson, et al., Reference McClure, Li, Tomlin, Cypert, Montague and Montague2004), with applications to different types of rewards and decisions. Understanding how these processes vary across development may open a window into adolescent decision making given their propensity for risky decisions and diminished capacity to inhibit responses that may not be in their best interest. This research could elucidate how the adolescent valuation system operates and how impulsivity may result.
Research has also begun to investigate the influence of social factors on decision making and corticostriatal circuitry (e.g., how moral information can bias decision-making behavior, and how people respond to unfairness in a social exchange). Reyna and Farley (Reference Reyna and Farley2006) discuss research indicating that adolescents tend to engage in more risky behavior and make poorer decisions when with peers than when alone, such as being more likely to drive through a yellow light at an intersection (Gardner & Steinberg, Reference Gardner and Steinberg2005). Combining these ideas could help to further probe the influence of social factors in adolescent decision making, such as understanding the contributions of trust and status seeking within a social network in risky decision making, or even how a desire to conform to the majority affects day-to-day choices. A vast array of questions remain regarding the formation of fairness and moral beliefs that impact decision making and its neural basis across development (de Quervain et al., Reference de Quervain, Fischbacher, Treyer, Schellhammer, Schnyder and Buck2004; Delgado, Frank, et al., 2005; Greene, Nystrom, Engell, Darley, & Cohen, Reference Greene, Nystrom, Engell, Darley and Cohen2004; Sanfey, Rilling, Aronson, Nystrom, & Cohen, Reference Sanfey, Rilling, Aronson, Nystrom and Cohen2003). Responses to these types of situations invoke a sense of moral reasoning, which may rely on differential bases as we progress from childhood to adulthood (Killen, Reference Killen2007). Reasoning about issues such as unfairness and exclusion and associated neural correlates during adolescence would seem to be of prime interest, given the neural changes that occur combined with the fostering of social relationships and influence of social peer groups during this time.
Seeking out rewards is a motivating force for everyday behavior. The neural circuitry underlying reward processing is a complex connection of cortical, executive, regulatory areas, and subcortical, more primitive and emotion related areas. As both cortical and subcortical regions continue to develop into young adulthood, it is no surprise that during that development, interesting behavioral and functional neural changes are observed. Changes in adolescent reward processing, and observed increases in adolescent risk taking manifest neurally via differences in activity of corticostriatal reward circuitry compared to adults, indicating an increased sensitivity to rewards and less developed regulatory functions; dysfunction in this circuitry often underlies different clinical and developmental conditions.
