Introduction
Positive reinforcers serve several basic functions. They promote selected behaviors, speed actions, maintain stimulus-response associations and, in humans, induce subjective feelings of pleasure and other positive emotions (Thut et al., Reference Thut, Schultz, Roelcke, Nienhusmeier, Missimer, Maguire and Leenders1997). Given the importance of reinforcers in promoting adaptive behavior, identifying the brain substrate underlying reinforced movement has been a central task of behavioral neuroscience.
Advances in functional brain imaging have made possible the visualization of the distributed brain systems underlying reward processing in humans. Findings from functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) studies indicate that the key anatomical nodes in the reward system are midbrain dopaminergic neurons, the ventral striatum (which includes the nucleus accumbens), the ventral pallidum, the orbitofrontal cortex, and the anterior cingulate (Haber & Knutson, Reference Haber and Knutson2010; Volkow et al., Reference Volkow, Fowler, Gatley, Logan, Wang, Ding and Dewey1996; Wallis & Kennerley, Reference Wallis and Kennerley2011). These brain areas appear to be involved in the appetitive or anticipatory aspects of reward (Rademacher et al., Reference Rademacher, Krach, Kohls, Irmak, Gründer and Spreckelmeyer2010). The ventral striatum has also been shown to respond to the delivery of rewards in both human (Daniel & Pollmann, Reference Daniel and Pollmann2014; Delgado, Nystrom, Fissell, Noll, & Fiez, Reference Delgado, Nystrom, Fissell, Noll and Fiez2000) and nonhuman primates (Apicella, Ljungberg, Scarnati, & Schultz, Reference Apicella, Ljungberg, Scarnati and Schultz1991). The anterior caudate, which receives projections from the prefrontal cortex and midbrain dopaminergic neurons, may also be engaged as part of its role in goal-directed behavior (Balleine & O’Doherty, Reference Balleine and O’Doherty2010; Yin & Knowlton, Reference Yin and Knowlton2006). Of the brain regions mediating reward, the ventral striatum has been theorized to contribute a central role to reward processing due to massive projections of midbrain dopaminergic neurons into this region (Daniel & Pollmann, Reference Daniel and Pollmann2014). An important current theory holds that dopaminergic neurons code differences between anticipated and actual rewards (Daniel & Pollmann, Reference Daniel and Pollmann2014; Schultz, Reference Schultz2007). This reward error prediction theory is supported by animal studies showing high bursts of neuronal activity when rewards are unpredictable or when stimuli predict the probabilistic occurrence of rewards, but not when reward occurs 100% of the time (Daniel & Pollmann, Reference Daniel and Pollmann2014; Schultz, Reference Schultz2002). In addition to contributing to the basic neuroscience of reward, imaging studies of reward have stimulated the development of neurobiological theories of psychological phenomena that engage reward associated brain systems – including accounts of addiction, depression, schizophrenia, eating disorders, and financial decision making (Chau, Roth, & Green, Reference Chau, Roth and Green2004; Neary & Batterham, Reference Neary and Batterham2010; Volkow et al., Reference Volkow, Fowler, Gatley, Logan, Wang, Ding and Dewey1996; Wu, Sacchet, & Knutson, Reference Wu, Sacchet and Knutson2012).
Functional MRI studies in humans have temporally separated the component processes involved in reward expectation and reward processing. A common finding is that the expectation of reward preferentially activates the ventral striatum, including the nucleus accumbens, whereas reward outcome preferentially activates the ventromedial prefrontal cortex (Knutson, Fong, Adams, Varner, & Hommer, Reference Knutson, Fong, Adams, Varner and Hommer2001; O’Doherty, Deichmann, Critchley, & Dolan, Reference O’Doherty, Deichmann, Critchley and Dolan2002). Other human studies of reward focus upon its response to decision-making outcomes (Delgado et al., Reference Delgado, Nystrom, Fissell, Noll and Fiez2000), reporting activation in more dorsal regions of the striatum. A recent meta-analysis of 142 neuroimaging studies of reward valence processing confirmed contributions of the orbitofrontal cortex and anterior cingulate in reward processing and identified a sub-region of the ventral striatum, the nucleus accumbens, as a brain area broadly involved in different stages of reward processing (Liu, Hairston, Schrier, & Fan, Reference Liu, Hairston, Schrier and Fan2011).
Investigators using functional brain imaging to study the neural substrates of reward have often interpreted their findings by relating them to single cell recording studies in animals (Haber & Knutson, Reference Haber and Knutson2010; Rademacher et al., Reference Rademacher, Krach, Kohls, Irmak, Gründer and Spreckelmeyer2010), but differences in task structure across species may limit interpretation. For example, nonhuman primate studies of reward outcome typically involve events that predict feedback, such as a tone indicating the animal should approach a target to receive a reward, but they may also include events that predict no feedback, such as a tone indicating the animal should return to the starting position in preparation for the next trial (Gdowski, Miller, Bastianen, Nenonene, & Houk, Reference Gdowski, Miller, Bastianen, Nenonene and Houk2007).
Perhaps a more important difference is that in many human fMRI studies, the response is a brief, ballistic button press (Elliott, Friston, & Dolan, Reference Elliott, Friston and Dolan2000; Ernst et al., Reference Ernst, Nelson, McClure, Monk, Munson, Eshel and Pine2004; Knutson, Westdorp, Kaiser, & Hommer, Reference Knutson, Westdorp, Kaiser and Hommer2000; Tanaka et al., Reference Tanaka, Doya, Okada, Ueda, Okamoto and Yamawaki2004), whereas in animal studies the response is often a capture movement that engages both proximal and distal musculature (Tremblay, Hollerman, & Schultz, Reference Tremblay, Hollerman and Schultz1998), requiring greater movement control and accuracy. These differences in response requirements between human and animal reward processing studies might explain why neurons in the dorsal striatum (comprised of the caudate nucleus and putamen) are often found to be responsive during movements elicited by reward cues in animal studies (Hollerman, Tremblay, & Schultz, Reference Hollerman, Tremblay and Schultz1998; Kawagoe, Takikawa, & Hikosaka, Reference Kawagoe, Takikawa and Hikosaka1998), whereas the ventral striatum is often activated in human fMRI studies of reward cuing (Delgado, Miller, Inati, & Phelps, Reference Delgado, Miller, Inati and Phelps2005; Galvan et al., Reference Galvan, Hare, Davidson, Spicer, Glover and Casey2005; Jensen et al., Reference Jensen, Smith, Willeit, Crawley, Mikulis, Vitcu and Kapur2007; Knutson et al., Reference Knutson, Fong, Adams, Varner and Hommer2001). Differences in response requirements might also explain why some studies of reward-cued movement in non-human primates have found neurons in the internal globus pallidus, outside of the ventral pallidum, that discharge during reward-cued capture-movements (Gdowski et al., Reference Gdowski, Miller, Bastianen, Nenonene and Houk2007; Gdowski, Miller, Parrish, Nenonene, & Houk, Reference Gdowski, Miller, Parrish, Nenonene and Houk2001). Single cell recording data support the view that the basal ganglia are involved in setting movement parameters, thus contributing to movement control (DeLong et al., Reference DeLong, Georgopoulos, Crutcher, Mitchell, Richardson and Alexander1984; Evarts & Wise, Reference Evarts and Wise1984). When movements are complex and unfold over time, continual updating of amplitude, force, direction, and velocity parameters is required to control the movement (DeLong, Reference DeLong1973; Kornhuber, Reference Kornhuber1971). This provides a setting where reward-cue modulation of the basal ganglia activity may occur more broadly than the ventral striatum and pallidum’s response to reward cues involving ballistic movements.
Some support does exist for the thesis that reward-cued movements may activate different components of the basal ganglia when paradigms require greater movement control than that needed for ballistic movements. For example, this hypothesis is supported by a human fMRI study involving action and movement control in the oculomotor system (Harsay et al., Reference Harsay, Cohen, Oosterhof, Forstmann, Mars and Ridderinkhof2011). The investigators found that although both the dorsal and ventral striatum activated in anticipation of future rewards, the dorsal striatum was directly involved in the modulation of oculomotor behavior through motivational processes (Harsay et al., Reference Harsay, Cohen, Oosterhof, Forstmann, Mars and Ridderinkhof2011). What the profile of basal ganglia activation in the skeletomuscular system might be for reward-cued movements executed in a motor control paradigm remains an open question.
In this study, fMRI was used during a movement paradigm requiring action control to investigate the neural substrate of reward-cued movement. We adapted a paradigm, originally designed for nonhuman primates (Gdowski et al., Reference Gdowski, Miller, Parrish, Nenonene and Houk2001), for human use during fMRI. Based on findings from primate studies, we hypothesize that reward related skeletomotor movements will modulate the BOLD signal compared with non-reward movements (Balleine, Delgado, & Hikosaka, Reference Balleine, Delgado and Hikosaka2007; Gdowski et al., Reference Gdowski, Miller, Parrish, Nenonene and Houk2001; Turner & Anderson, Reference Turner and Anderson2005). Specifically, we predict that the caudate nucleus, pallidum, and putamen will exhibit greater context dependent modulation for visual cues predicting potential financial gains relative to cues associated with no outcome. Furthermore, striatal regions will also exhibit a greater response to financial gains relative to neutral outcomes. To determine whether reward expectancy and reward outcome selectively alter neural activity in the basal ganglia, the BOLD response in motor and sensory areas (supplementary motor area, precuneus, and the precentral gyrus) involved in visually cued movement will also be studied (Benton & Tranel, Reference Benton and Tranel1993; Brown et al., Reference Brown, Caligiuri, Meloy, Eberson, Kindermann, Frank and Lohr2004).
Methods
Subjects
Twenty right-handed participants (8 male, 12 female) aged 27.7±8.53 years (range, 18–42 years) with 16.08±3 years of education (range, 12–23.5 years) gave written informed consent in accordance with the University of California at San Diego Institutional Review Board. Exclusion criteria included: past history of dependence on alcohol, stimulants, opiates, or hallucinogens; serious medical conditions; color blindness; psychiatric illness (e.g., anxiety, depression); and conditions contraindicative to magnetic resonance imaging. Participants received $50 for their participation and were told they may earn up to an additional $25 based upon their performance. All participants, regardless of performance, received the additional $25 in compensation.
Experimental Design
Participants performed a reward expectancy task with probabilistic reward. Targets were presented on a projection monitor controlled by a Dell Inspiron computer running Presentation (Neurobehavioral Systems, Inc., Albany, CA, www.neurobs.com), which also recorded choices and response times. Responses were made with a fiberoptic joystick (Current Designs, Philadelphia, PA, www.curdes.com) to manipulate a red square cursor. Three square targets with white sides on a black background were presented. Participants initially aligned the cursor to the center target. Visual cues were randomly ordered through the use of an m-sequence (Buracas & Boynton, Reference Buracas and Boynton2002): Feedback Cues, occurring 50% of the time and in which one of the targets was illuminated in white, indicating that either a (1) positive or (2) neutral outcome would be presented following the movement (Figure 1a); (3) Nonfeedback Cues, occurring 25% of the time and in which one of the targets was illuminated in gray, indicating that no outcome stimulus would occur on this trial after a movement was performed (Figure 1b); and (4) Fixation Cues, occurring 25% of the time, where the participant remained fixated at the current location. For Feedback and Nonfeedback Cues, fixation occurred for 500 ms before one of the two remaining squares was randomly selected as the target and illuminated. After a variable length cue period (750–1250 ms), the illuminated target was extinguished, and the participant had 1000 ms to acquire the target with the cursor; the participant then held position inside the target frame for a variable period (750–1250 ms). For a Feedback Cue, the target was re-illuminated in either green (positive outcome) or yellow (neutral outcome) following the hold period; both positive and neutral outcomes were equally likely. Positive outcomes were worth 20 cents, and there was no financial gain for the neutral outcome. The current cursor location then served as the fixation position for the next trial. For a Nonfeedback Cue, no outcome stimulus was provided, and participants held position within the target frame until the next trial period. Each trial lasted approximately 4 sec, with jittered hold and outcome presentation periods (Figure 1c), both to ensure that activation patterns did not become phase-locked with the slice acquisition rate and to minimize anticipatory movements. Each functional run contained 63 trials and lasted 221–284 s. The eight functional runs were separated by a 30-s rest period.
Examples of the two behavioral trial types. (a) For the Feedback Cue, the white stimulus predicting future feedback is presented; upon extinguishing, the participant uses the fiberoptic joystick to move to the previously lit target’s position and holds during the delay period. Feedback is provided with either a green filled target, indicating the participant earned 20 cents for the trial, or a yellow target, indicating the participant did not earn any money for this trial. (b) For the Nonfeedback Cue, the gray stimulus indicating no feedback for this trial is presented; upon extinguishing, the participant uses the joystick to move to the target’s location. (c) The timeline for a typical trial.
MRI
Functional MRI was performed with gradient-recalled echoplanar imaging [repetition time (TR)=2000 ms; echo time (TE)=30 ms; flip angle=90°; 64×64 matrix; 32 4 mm (no gap) contiguous oblique slices aligned to anterior commissure-posterior commissure with an in-plane resolution of 3.75 mm×3.75 mm; 150 volumes with a General Electric Signa HDx 3 Tesla scanner (Kwong et al., Reference Kwong, Belliveau, Chesler, Goldberg, Weisskoff, Poncelet and Turner1992; Ogawa et al., Reference Ogawa, Tank, Menon, Ellermann, Kim, Merkle and Ugurbil1992). The first four volumes of each run were discarded so as to discount T1 saturation. High resolution T1-weighted SPGR anatomical images (TR=8 ms; TE=min full; 256×192 matrix; 124 1.2 mm contiguous sagittal slices) were obtained for subsequent spatial normalization.
Behavioral Analysis
The joystick was sampled at 60 Hz and filtered offline with a Butterworth lowpass filter at 10 Hz to remove jitter. Response time was defined as the interval of time from when the target was extinguished to the onset of joystick movement. Movement duration time was defined as the interval of time from the onset of joystick movement to the point in time at which maximum displacement occurred. To calculate these parameters, movement onset and offset were determined with a three-step algorithm (Teasdale, Bard, Fleury, Young, & Proteau, Reference Teasdale, Bard, Fleury, Young and Proteau1993). To calculate movement onset, the algorithm traced backward from the event with the peak tangential velocity to the event that was 10% of its peak value, t 2, and traced back further to event t 1, which was 10% of the velocity value of t 2. Movement onset was then defined as the average value (plus standard deviation) between events t 1 and t 2; averaging minimizes the contribution of noise and is more accurate than an absolute criterion (Teasdale et al., Reference Teasdale, Bard, Fleury, Young and Proteau1993). Movement offset was calculated in a similar manner, but with the average calculated from events t 1 and t 2 located on the downward slope of the tangential velocity profile (i.e., tracing forward from the peak tangential velocity event).
MRI Statistical Analysis
Imaging data were analyzed using Statistical Parametric Mapping (SPM5; Wellcome Department of Cognitive Neurology, London UK) (Friston et al., Reference Friston, Holmes, Worsley, Poline, Frith and Frackowiak1995) in Matlab 7.5.0.338 (R2007b). Data were motion corrected to the first functional scan, coregistered to the individual’s high-resolution MRI using mutual information, spatially normalized to the Montreal Neurologic Institute (MNI) template with a 12-parameter affine transformation followed by nonlinear warping (Ashburner & Friston, Reference Ashburner and Friston1999), resampled to 2 mm isotropic voxels, and smoothed with a 6 mm isotropic Gaussian kernel. Two participants failed to complete all eight runs due to technical problems; therefore only their completed runs were included in the analyses. Each participant’s data were analyzed using the general linear model with an event-related design. The onset times of cue and outcome stimuli were convolved with the canonical hemodynamic response with time and dispersion derivatives. Contrasts were constructed at the subject level for comparing the cue stimuli (Feedback Cues vs. Nonfeedback Cues), and for comparing outcome stimuli (Positive Outcomes vs. Neutral Outcomes).
A region of interest (ROI) analysis was used on structures associated with reward and goal-oriented movements. Anatomical representations of the precentral gyrus, supplementary motor area, precuneus, pallidum, putamen, and caudate nucleus were obtained from the AAL atlas (Tzourio-Mazoyer et al., Reference Tzourio-Mazoyer, Landeau, Papathanassiou, Crivello, Etard, Delcroix and Joliot2002), a toolbox available for SPM. Due to issues with signal dropout in ventral regions, we did not pursue an ROI analysis within the ventral striatum (including the nucleus accumbens). Statistical maps for each ROI were thresholded at p<.05 using the false discovery rate (FDR, Benjamini & Hochberg, Reference Benjamini and Hochberg1995) for multiple comparisons using a small volume correction (SVC). To account for testing multiple ROIs (6 per hemisphere), we applied a Bonferroni-corrected threshold of p<.004. To test for both condition and temporal (run) effects, the MarsBar toolbox (http://marsbar.sourceforge.net) (Brett, Anton, Valabregue, & Poline, Reference Brett, Anton, Valabregue and Poline2002) was used to extract the average percent signal change within our reward-related ROIs for each of the four conditions (Feedback Cue, Nonfeedback Cue, Positive Outcome, Neutral Outcome), and the somatosensory regions for trial type (Feedback Cue, Nonfeedback Cue). These extractions were 5-mm spheres centered upon the peak activations found within our ROIs as determined by SVC. The radius was chosen so as to ensure the extraction was smaller than the smallest ROI, and so that there would be the same number of observations within each extraction. Hypotheses related to these ROIs were tested by a two (condition) by eight (run) mixed effects analysis of variance (ANOVA) with subject as a random effect. The condition effect either contrasted Feedback Cues with Nonfeedback Cues or Positive Outcomes with Neutral Outcomes. We used Mauchly’s W to test for violations of sphericity in the run effect and in the run by condition interaction. When Mauchly’s test indicated a significant deviation from sphericity, we used the Huynh-Feldt adjustment to the degrees of freedom. Finally, a voxelwise group analysis was used to look for other regions that showed significant changes in our contrasts of interest (p FDR<.05).
Results
Response Behavior
Motor response with the joystick was broken down into two components: (1) the time to initiate the response, and (2) the movement duration. Overall, participants tended to initiate and respond more slowly for the Nonfeedback Cue than for the Feedback Cue. A two (Cue Type: Feedback, Nonfeedback) by eight (run) mixed-effects ANOVA with subject as a random effect reported an interaction between measures for movement response time, F(7,133)=2.56; p=.02; ηp 2=.12. There was also a significance for trial type, F(1,19)=6.15; p=.02; ηp 2=.24 (Figure 2a). While movement duration failed to show an interaction between trial type and run, it was also significant for trial type, F(1,19)=10.09; p=.005; ηp 2=.35 (Figure 2b). Run effects were not found to be significant for either the movement response time or duration. All participants successfully captured all targets.
(a) Response time in relation to Cue stimuli (Feedback Cue, Nonfeedback Cue) across eight runs. Overall, response initiation was slower for the Nonfeedback Cues relative to the Feedback Cues. (b) Movement duration in relation to Cue stimuli (Feedback Cue, Nonfeedback Cue) across eight runs. Participants exhibited longer movement durations for the Nonfeedback Cues relative to the Feedback Cues. Error bars indicate standard errors.
Hypothesized Effect of Cue Type
Subcortical regions of interest
Using SVC, we compared activation differences between Feedback Cues and Nonfeedback Cues within the pallidum, putamen, and caudate nucleus. Most ROIs reported significant clusters (FDR-corrected) for Feedback Cues relative to Nonfeedback Cues (Table 1; Figure 3). Several of these regions also reported significant clusters for Nonfeedback Cues relative to Feedback Cues; these latter results included the posterior zones of the pallidum and putamen. We also extracted and averaged the percent signal change within each ROI for the Feedback Cues and Nonfeedback Cues for each run. A two (Cue Type: Feedback, Nonfeedback) by eight (run) ANOVA did not show an interaction within subcortical ROIs. However, a difference of cue type was found for the bilateral medial caudate nucleus [Left: F(1,19)=33.46; p<.001; ηp 2=.64; Right: F(1,19)=105.50; p<.001; ηp 2=.85], the bilateral rostroventral putamen [Left: F(1,19)=11.14; p=.003; ηp 2=.38; Right: F(1,19)=37.83; p<.001; ηp 2=.67], the bilateral posterior putamen [Left: F(1,19)=20.98; p<.001; ηp 2=.52; Right: F(1,19)=26.44; p<.001; ηp 2=.58], the bilateral pallidum [Left: F(1,19)=29.59; p<.001; ηp 2=.61; Right: F(1,19)=56.97; p<.001; ηp 2=.75], and the bilateral posterior pallidum [Left: F(1,19)=12.26; p=.002; ηp 2=.39; Right: F(1,19)=8.51; p=.009; ηp 2=.31].
Differences in response to Feedback and Nonfeedback Cues. (a) Upper panel: The contrast of Feedback Cues > Nonfeedback Cues revealed effects within several a priori predicted regions, including the medial caudate. Lower panel: The contrast of Nonfeedback Cues > Feedback Cues revealed differences within the posterior putamen and posterior pallidum. (b) Average percent signal change extracted from a priori selected regions of interest for Feedback Cues and Nonfeedback Cues. Anterior subregions (medial caudate) tended to be more responsive to Feedback Cues, whereas posterior subregions (putamen) were more responsive to Nonfeedback Cues. Error bars indicate standard errors.
Significant clusters of BOLD response within regions of interest in relation to the cue stimuli
Note. Brodmann’s areas for appropriate regions are in parentheses. Z-score refers to the activation peak within the cluster. Coordinates are given in MNI space.
*Significant at Bonferroni-corrected threshold of FWE p<.007 (with SVC).
FDR=false discovery rate.
Several regions also demonstrated run effects. Although the main effect of run fell just short of being significant in the left medial caudate nucleus [F(7,133)=1.88; p=.08; η2=.09], several non-linear trends were significant. The data suggested that early runs showed larger effects than later runs. The right medial caudate nucleus, bilateral rostroventral putamen, and bilateral pallidum also showed a pattern of significant non-linear effects reflecting larger effects for early runs.
Cortical regions of interest
Using SVC, the right precentral gyrus and bilateral supplementary motor area both exhibited a greater response for Feedback Cues relative to Nonfeedback Cues (Table 1). Several cortical regions also demonstrated significant clusters for Nonfeedback Cues relative to Feedback Cues; these included the bilateral precuneus and more posterior zones of the bilateral supplementary motor area and precentral gyrus. A two (Cue Type: Feedback, Nonfeedback) by eight (run) ANOVA detected a difference of cue type within the left precuneus [F(1,19)=15.28; p=.001; ηp 2=.45] and the right precuneus [F(1,19)=6.09; p=.023; ηp 2=.24]. The right precentral gyrus also indicated a cue type effect [F(1,19)=8.73; p=.008; ηp 2=.32].
Hypothesized Effect of Feedback
Subcortical regions of interest
SVC was used to compare positive and neutral outcomes within the subcortical ROIs (Table 2, Figure 4). All ROIs exhibited a significant response to Positive Outcomes relative to Neutral Outcomes; no regions were more responsive to Neutral Outcomes relative to Positive Outcomes. The average percent signal change within each ROI was extracted and subjected to a two (Outcome: Positive, Neutral) by eight (run) ANOVA. None of the ROIs demonstrated a run by outcome interaction. Several regions were significant for type of outcome, including the bilateral medial caudate nucleus [Left: F(1,19)=56.75; p<.001; ηp 2=.75; Right: F(1,19)=77.62; p<.001; ηp 2=.80], the bilateral rostroventral putamen [Left: F(1,19)=63.98; p<.001; ηp 2=.77; Right: F(1,19)=59.19; p<.001; ηp 2=.76], and the bilateral pallidum [Left: F(1,19)=40.81; p<.001; ηp 2=.68; Right: F(1,19)=42.26; p<.001; ηp 2=.69].
Differences in response to Positive Outcomes and Neutral Outcomes. (a) The contrast of Positive Outcomes > Neutral Outcomes revealed effects within several a priori predicted regions, including medial caudate. (b) Average percent signal change extracted from a priori selected regions of interest for Positive Outcomes and Neutral Outcomes. Overall, regions of interest were more responsive for Positive Outcomes than for Neutral Outcomes. Error bars indicate standard errors.
Significant clusters of BOLD response within regions of interest in relation to the outcome stimuli
Note. Brodmann’s areas for appropriate regions are in parentheses. Z-Score refers to the activation peak within the cluster. Coordinates are given in MNI space.
*Significant at Bonferroni-corrected threshold of FWE p<.007 (with SVC).
FDR=false discovery rate.
Several regions demonstrated a significant linear increase in the BOLD response over runs: bilateral medial caudate nucleus [Left: Flin(1,19)=10.32; p=.005; ηp 2=.35; Right: Flin(1,19)=14.84; p=.001; ηp 2=.44]; bilateral rostroventral putamen [Left: Flin(1,19)=28.57; p<.001; ηp 2=.60; Right: Flin(1,19)=19.69; p<.001; ηp 2=.51], and the bilateral pallidum [Left: Flin(1,19)=24.62; p<.001; ηp 2=.56; Right: Flin(1,19)=23.44; p<.001; ηp 2=.55]. For these ROIs the advantage of later trials over the first trial leveled off, adding a quadratic trend to the linear effect.
Cortical regions of interest
None of the cortical ROIs demonstrated a differential response to Positive Outcomes relative to Neutral Outcomes.
Exploratory Voxelwise Analysis
Effects of cue type
A secondary voxelwise analysis at p<.01 (FDR-corrected) and a minimum of 10 contiguous voxels per cluster reported additional regions active for Feedback Cues relative to Nonfeedback Cues (Supplementary Table 1). These additional regions included the right inferior parietal lobule, the right middle frontal gyrus, the bilateral anterior insula, the right middle occipital gyrus, and a cluster that included the right inferior frontal gyri. In comparison, additional regions that were significantly activated for Nonfeedback Cues relative to Feedback Cues included the inferior parietal lobule bilaterally, the left middle frontal gyrus, and the right superior frontal gyrus (Supplementary Table 2).
Effects of outcome
A secondary voxelwise analysis at p<.01 (FDR-corrected) and a minimum of 10 contiguous voxels per cluster reported additional regions active for Positive Outcomes relative to Neutral Outcomes, including the bilateral inferior parietal lobule and bilateral superior frontal gyrus (Supplementary Table 3). No regions survived significance for the Neutral Outcomes relative to Positive Outcomes contrast.
Comparison of Trial Type with Behavioral Response
Finally, multiple regression was used to determine whether the contrast difference comparing Feedback Cues to Nonfeedback Cues was related to the individual’s mean response duration between the two cue types. SVC within the ROIs failed to find a relationship between the difference in trial types and mean response duration (p<.05, FDR-corrected). Similarly, a voxelwise analysis revealed no regions surviving the p<.05 (FDR-corrected) threshold.
Discussion
Our study yielded two main results. First, as hypothesized, portions of the dorsal striatum and pallidum exhibited greater context dependent modulation for visual cues predicting potential financial gains compared to cues associated with no outcome. These results extend the finding of dorsal striatal activation during anticipation of future reward reported in the oculomotor system (Harsay et al., Reference Harsay, Cohen, Oosterhof, Forstmann, Mars and Ridderinkhof2011) to the skeletomotor system. Second, both striatal and pallidal regions displayed a greater response to financial gains compared to neutral outcomes. Our finding of context dependent modulation of basal ganglia response converges with the pallidal results reported by Gdowski and colleagues (2007, 2001) in monkey and with the role of the dorsal striatum in reward and decision-making described in the model by Balleine and colleagues (2007). Overall, these finding support the notion of a broader effect of context dependent modulation within subregions of the basal ganglia.
Of interest, the direction of the reward expectancy modulation varied – some subregions within each basal ganglia nucleus displayed increases in the BOLD response, whereas other areas displayed decreases. The directional difference in modulation can be explained by variations in the neuronal response to cues predicting reward or by differences in metabolic-blood flow coupling (Buxton, Reference Buxton2009; Davis, Kwong, Weisskoff, & Rosen, Reference Davis, Kwong, Weisskoff and Rosen1998). Primate data indicate that both explanations are plausible. Single cell recordings of the activity of individual neurons in the external globus pallidus of monkeys during a probabilistic visuomotor task show that some neurons increase their activity to predicted reward, whereas others decrease their activity (Arkadir, Morris, Vaadia, & Bergman, Reference Arkadir, Morris, Vaadia and Bergman2004). This differential response to reward cues is compatible with the view that dopamine release has a brief excitatory effect on some neurons and a slow inhibitory effect on others (Shepherd, Reference Shepherd1994, p. 472). Although the BOLD signal appears to be more tightly regulated by alterations of local field potentials rather than by neural spikes, sustained neural firing of a pool of neurons can alter nearby local field potentials (Logothetis, Pauls, Augath, Trinath, & Oeltermann, Reference Logothetis, Pauls, Augath, Trinath and Oeltermann2001; Mishra et al., Reference Mishra, Ellens, Schridde, Motelow, Purcaro, DeSalvo and Blumenfeld2011).
Differential metabolic-blood flow coupling associated with neural activity in different neural pools within the basal ganglia is an alternative to the purely neural account of the differential BOLD response. A study of the WAG/Rij rat found regions within the caudate-putamen where cerebral blood flow decreased even though multiunit neural activity increased during either whisker stimulation or spike-wave discharges (Mishra et al., Reference Mishra, Ellens, Schridde, Motelow, Purcaro, DeSalvo and Blumenfeld2011). Cerebral blood volume (CBV) within the caudate-putamen is modulated in part by dopamine, with D1/D5 receptors mediating increases in CBV and D2/D3 receptors mediating decreases (Choi, Chen, Hamel, & Jenkins, Reference Choi, Chen, Hamel and Jenkins2006). Based on data showing reduced decreases of CBV to pain stimulation in the striatum of rodents following administration of a D2 antagonist (Shih et al., Reference Shih, Chen, Shyu, Lin, Chiang, Jaw and Chang2009), Mishra and colleagues (2011) hypothesize that dopamine release following neural activation causes vasoconstriction in the striatum that in turn reduces the BOLD signal. These differential vascular effects in basal ganglia subregions could confound the expression of the BOLD signal, perhaps leading to different directions of the effect, even if the neural effects were in the same direction. If true, a calibrated BOLD study would be necessary to identify the true directions of the effects (Ances et al., Reference Ances, Leontiev, Perthen, Liang, Lansing and Buxton2008).
All locations within the striatum and pallidum showing a significant BOLD response to outcome type revealed a significant response to positive reward outcome. While the animal literature has demonstrated striatal neurons that fire to either reward prediction or delivery (Hollerman et al., Reference Hollerman, Tremblay and Schultz1998; Lau & Glimcher, Reference Lau and Glimcher2007), few human neuroimaging studies have reported activation to both anticipatory cues and reward outcome within a single experiment in the absence of learning or decision-making. Human studies have generally either demonstrated responses to rewarding outcomes (Delgado, Locke, Stenger, & Fiez, Reference Delgado, Locke, Stenger and Fiez2003; Delgado et al., Reference Delgado, Nystrom, Fissell, Noll and Fiez2000) or only to the anticipatory phase if one is present (Knutson & Greer, Reference Knutson and Greer2008). However, the anticipatory element in these kinds of tasks predicts a specific outcome (e.g., receipt of a reward), whereas our task’s anticipatory cue predicted only that an outcome (either positive or neutral) would be presented. In that regard, our task is similar to that of Breiter, Aharon, Kahneman, Dale, and Shizgal (Reference Breiter, Aharon, Kahneman, Dale and Shizgal2001); in their paradigm, participants were presented with one of three different anticipatory cues, each of which was associated with three different probabilistic outcomes. They reported that several of their regions of interest increased activation both to the cues associated with the greatest potential gain as well as the most rewarding outcome for each cue. Thus, our findings support the notion that the striatum can encode both anticipation and outcome during the same paradigm when sufficient uncertainty is present.
We also saw differences in regional specificity to cue type, with anterior striatal regions more responsive to feedback cues, and posterior regions more responsive to no feedback cues. While the anterior striatum has been associated with reward anticipation in humans (Tricomi, Delgado, & Fiez, Reference Tricomi, Delgado and Fiez2004) as well as nonhuman primates (Lauwereyns, Watanabe, Coe, & Hikosaka, Reference Lauwereyns, Watanabe, Coe and Hikosaka2002), the role of the posterior striatum in outcome anticipation is less certain. Prediction error studies in humans have suggested that posterior regions are more linked to punishment prediction error signals (Mattfeld, Gluck, & Stark, Reference Mattfeld, Gluck and Stark2011; Seymour, Daw, Dayan, Singer, & Dolan, Reference Seymour, Daw, Dayan, Singer and Dolan2007), although a meta-analysis suggests this relationship may follow a U-shaped profile (Bartra, McGuire, & Kable, Reference Bartra, McGuire and Kable2013), with an emphasis on cues predicting non-neutral outcomes. The no feedback cue may have been interpreted as predicting a negative outcome, as it led to no potential for monetary gain. Such an interpretation is supported by studies demonstrating that the context of alternative options can influence the subjective value of a stimulus (Breiter et al., Reference Breiter, Aharon, Kahneman, Dale and Shizgal2001; Bunzeck, Dayan, Dolan, & Duzel, Reference Bunzeck, Dayan, Dolan and Duzel2010; Nieuwenhuis et al., Reference Nieuwenhuis, Heslenfeld, Alting von Geusau, Mars, Holroyd and Yeung2005).
There was no differential brain response within motor and visual areas when positive feedback was compared with no feedback. In comparison, supplementary motor area, precentral gyrus, and precuneus all exhibited a modulated BOLD response to cued outcome. These results indicate that corticostriatal pathways that have been postulated to be involved only in the acquisition of cued actions and habits might also be involved in probabilistic decision-making, at least for reward modulated motor control tasks (Balleine et al., Reference Balleine, Delgado and Hikosaka2007). The reward cue modulation of sensorimotor areas indicates that the neural effects of reward expectancy extend well beyond the basal ganglia. Our exploratory voxelwise analyses suggested that the impact of reward expectancy on neural activity includes occipital and parietal cortex, as well as limbic cortex (insula) and multiple regions within the lateral frontal cortex (see supplementary material). Overall our results support theories about the brain-substrate of reward expectancy and reward outcome that emphasize the contributions of multiple brain systems, including emotional, attentional, motor and memory systems, in accounting for the neural effects of reward (e.g., Delgado & Dickerson, Reference Delgado and Dickerson2012).
These results must be viewed in light of the study’s limitations. The participants we studied were generally young and college-educated, and our results should only be generalized to similar samples. Evidence suggests that older adults may have reduced reward expectancy, but not outcome, response in the striatum (Samanez-Larkin, Worthy, Mata, McClure, & Knutson, Reference Samanez-Larkin, Worthy, Mata, McClure and Knutson2014). Education may also have had an influence; some studies suggest performance differences in decision making, such as with the Iowa Gambling Task (Davis et al., Reference Davis, Fox, Patte, Curtis, Strimas, Reid and McCool2008; Evans, Kemish, & Turnbull, Reference Evans, Kemish and Turnbull2004), and the influence of feedback during decision making processes may be modulated by IQ, which is in turn correlated with education (Brand, Laier, Pawlikowski, & Markowitsch, Reference Brand, Laier, Pawlikowski and Markowitsch2009). Thus, our reward expectancy findings in particular are limited to young, well-educated adults. A second limitation of the current approach is the use of secondary versus primary reinforcers. In humans, both primary (e.g., juice) and secondary (e.g., money) reinforcers activate similar neural substrates (Kim, Shimojo, & O’Doherty, Reference Kim, Shimojo and O’Doherty2011; McClure, Ericson, Laibson, Loewenstein, & Cohen, Reference McClure, Ericson, Laibson, Loewenstein and Cohen2007; McClure, Laibson, Loewenstein, & Cohen, Reference McClure, Laibson, Loewenstein and Cohen2004), thereby supporting the use of financial rewards. Third, due to signal dropout we did not acquire adequate data within the ventral striatum (including the nucleus accumbens) and orbitofrontal cortex. Future imaging studies using approaches to minimize susceptibility distortion (Deichmann, Gottfried, Hutton, & Turner, Reference Deichmann, Gottfried, Hutton and Turner2003) should help minimize this problem. Fourth, although we were interested in whether complex movements in humans would demonstrate a similar dorsal striatal response to future rewards as seen in the animal literature, we did not perform a direct comparison of simple versus complex movements. Simple motor responses are common in the literature, thus we wanted to focus on more complex motor responses. Because there were eight functional runs, there was insufficient time for also including a simple response task. Moreover, we were concerned about the potential for negative transference of the simple motor task if it were in the same session as the complex version. Finally, we did not model the movement or hold periods to avoid overfitting, but note that we did see response differences due to cue type, which might reflect preparatory or motor response activity. Future studies examining response complexity during reward processing are needed to better determine whether more complex responses are region-specific.
These findings demonstrate the importance of context in human reward processing neuroimaging paradigms. Furthermore, robust modulation of dorsal striatal and pallidal response to reward expectancy and reward outcome was seen in a probabilistic reward task involving larger scale arm movements as those seen in non-human primate studies of probabilistic reward (Gdowski et al., Reference Gdowski, Miller, Bastianen, Nenonene and Houk2007, Reference Gdowski, Miller, Parrish, Nenonene and Houk2001), supporting the importance of response complexity in the interpretation of neuronal processing across species.
Acknowledgments
This research was supported by NIH grant K01DA15499. The authors have no conflicts of interest to disclose.
Supplementary Material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1355617715000491.
