Introduction
Psychiatric disorders are the leading cause of disability world-wide with major depressive disorder (MDD) the commonest cause (Whiteford et al., Reference Whiteford, Degenhardt, Rehm, Baxter, Ferrari, Erskine and Vos2013). Severe and enduring mental illness is associated with a reduction in lifespan of 5–15 years (Chang et al., Reference Chang, Hayes, Perera, Broadbent, Fernandes, Lee and Stewart2011) and suicide is a leading cause of death in young adults (WHO, 2018). However, understanding of illness mechanisms remains rudimentary, there are no biomarkers in clinical use, clinical outcomes are difficult to predict for individual patients and its widely recognised clinical practice in psychiatry has not progressed significantly in the past 50 years (Stephan et al., Reference Stephan, Bach, Fletcher, Flint, Frank, Friston and Breakspear2016Reference Stephan, Bach, Fletcher, Flint, Frank, Friston and Breakspeara, Reference Stephan, Binder, Breakspear, Dayan, Johnstone, Meyer-Lindenberg and Friston2016Reference Stephan, Binder, Breakspear, Dayan, Johnstone, Meyer-Lindenberg and Fristonb). Better understanding of illness mechanisms is crucial for progress.
Dolan has argued that emotional experience is closely linked to valuation (Dolan, Reference Dolan2002). Normal mood can be viewed as a bias to experience positive or negative emotions and abnormally biased subjective reward valuation (anhedonia) and cognitions are core characteristics of MDD (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders and Steele2011; Kumar et al., Reference Kumar, Waiter, Ahearn, Milders, Reid and Steele2008). The origin and persistence of core symptoms of MDD, such as anhedonia, helplessness, rumination and cognitive biases can be explained as arising from biased internal processing; i.e. a biased evaluation of internal states and biased cognitions (Huys & Renz, Reference Huys and Renz2017; Huys, Daw, & Dayan, Reference Huys, Daw and Dayan2015). Such a decision-theoretic approach allows quantitative coupling of valuation and action which is a central aspect of emotion (Dolan, Reference Dolan2002). A behavioural meta-analysis found evidence for reduced primary reward value sensitivity in depression (Huys, Pizzagalli, Bogdan, & Dayan, Reference Huys, Pizzagalli, Bogdan and Dayan2013) and other recent reviews have argued for blunted reward valuation in anxiety and depression (Bishop & Gagne, Reference Bishop and Gagne2018; Rizvi, Pizzagalli, Sproule, & Kennedy, Reference Rizvi, Pizzagalli, Sproule and Kennedy2016) modulated by stress vulnerability (Pizzagalli, Reference Pizzagalli2014). This conceptualisation of MDD is consistent with the National Institute of Mental Health (NIMH), Research Domain Criteria (RDoC, Cuthbert & Insel, Reference Cuthbert and Insel2013) framework, implying a blunted positive valence system, increased sensitivity of the negative valence system and cognitive biases in line with both (Johnston et al., Reference Johnston, Tolomeo, Gradin, Christmas, Matthews and Steele2015).
Model-based functional magnetic resonance imaging (fMRI) can be used to determine brain region encoding of signals derived from a computational model such as estimated value or reward prediction error (RPE) (O'Doherty, Hampton, & Kim, Reference O'Doherty, Hampton and Kim2007). Meta-analyses have highlighted the importance of the striatum and ventromedial prefrontal cortex (vmPFC) as regions encoding value (Bartra, McGuire, & Kable, Reference Bartra, McGuire and Kable2013; Chase, Kumar, Eickhoff, & Dombrovski, Reference Chase, Kumar, Eickhoff and Dombrovski2015). Using model-based fMRI with an instrumental task, we reported blunted encoding of expected reward value in chronically medicated patients with treatment-resistant MDD and schizophrenia (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders and Steele2011); however, the effect of medication on these results was unclear. A recent meta-analysis of fMRI and electroencephalography studies found converging evidence for blunted striatal activation and feedback-related negativity responses to reward in depression which may precede the first episode of illness (Keren et al., Reference Keren, O'Callaghan, Vidal-Ribas, Buzzell, Brotman, Leibenluft and Stringaris2018). Very recently, we reported behavioural evidence for impairments in both the learning and decision-making phases of a novel Pavlovian conditioning task using computational modelling (Rupprechter, Stankevicius, Huys, Steele, & Series, Reference Rupprechter, Stankevicius, Huys, Steele and Series2018). Here we extend that behavioural analysis to identify the neural substrates of these abnormalities.
Although a number of studies have reported RPE abnormalities (e.g. most recently, Kumar et al., Reference Kumar, Goer, Murray, Dillon, Beltzer, Cohen and Pizzagalli2018), to our knowledge only a few have tested for expected reward value encoding abnormalities using fMRI with a computational model in MDD patients: we reported blunted reward value encoding (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders and Steele2011) and reduced reward value signals have been reported in elderly depressed patients with a history of suicide attempts (Dombrovski, Szanto, Clark, Reynolds, & Siegle, Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013). In addition, Greenberg et al. reported that healthy subjects but not unipolar unmedicated depressed patients showed the expected theoretical inverse relationship between prediction error and reward expectancy, mediated by anhedonia (Greenberg et al., Reference Greenberg, Chase, Almeida, Stiffler, Zevallos, Aslam and Phillips2015) with similar observations in medicated depressed patients with MDD or bipolar disorder (Chase et al., Reference Chase, Nusslock, Almeida, Forbes, LaBarbara and Phillips2013). Notably though, Greenberg et al. did not find evidence for blunted reward value or RPE signals in unmedicated unipolar depression (Greenberg et al., Reference Greenberg, Chase, Almeida, Stiffler, Zevallos, Aslam and Phillips2015).
Here we tested the following four hypotheses: (a) is it possible to replicate previous findings of blunted striatal reward response signals in MDD (Keren et al., Reference Keren, O'Callaghan, Vidal-Ribas, Buzzell, Brotman, Leibenluft and Stringaris2018), (b) do unmedicated subjects with MDD exhibit abnormal brain encoding of learned Pavlovian reward values during decision making, (c) are there correlations between aberrant brain encoding and illness severity and (d) is there evidence for abnormal event-related connectivity in MDD for brain regions identified as exhibiting abnormal encoding of reward values.
Methods and materials
Participants
The study was approved by the East of Scotland Research Ethics Committee (REC reference 13/ES/0043) and written informed consent obtained from all subjects. Thirty-nine subjects comprising 19 satisfying DSM-IV criteria for MDD not receiving antidepressant medication and 20 healthy controls matched on age, sex and IQ (NART; Nelson & Wilson, Reference Nelson and Wilson1991) were recruited. Diagnosis was made according to MINI Plus v5.0 structured diagnostic criteria (Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller and Dunbar1998). Demographics and illness severity (Beck Depression Inventory, BDI; Beck, Steer, Ball, & Ranieri, Reference Beck, Steer, Ball and Ranieri1996) scores are summarised in Table 1 with more details in online Supplementary Materials. Exclusion criteria were claustrophobia, serious physical illness, pre-existing cerebrovascular or other neurological disease, previous history of significant head injury and receipt of medication likely to affect brain function. Subjects were recruited using the University of Dundee advertisement system HERMES and compensated for participation (£20) with up to £10 extra depending on task performance. One MDD subject and four controls were excluded due to problems with fMRI data acquisition, so data from 18 MDD subjects and 16 controls were analysed. Power estimation in fMRI is recognised as difficult because of the complexity of the analyses and not possible in this instance as no previous similar data existed to allow such an estimate. We did however know on the basis of previous work that the behavioural data, acquired in the same experimental session, showed a significant abnormality (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018).
Clinical characteristics of subjects
BDI, Beck Depression Inventory; NART, National Adult Reading Test.
Data are displayed as n or mean ± standard deviation. For more details see online Supplementary materials.
Paradigm
The task was adapted from our earlier work (Stankevicius, Huys, Kalra, & Series, Reference Stankevicius, Huys, Kalra and Series2014) and described in detail in online Supplementary Materials. Subjects passively observed a series of different fractals; each fractal was always followed by either a reward symbol (£) indicating ‘value’ or a blank screen indicating ‘no value’. Each fractal was observed on four occasions. Participants had to form an internal estimate of the value (reward probability) associated with each fractal (i.e. number of observed rewards divided by total number of observations). The fractal then appeared at a later time in a single decision trial where subjects were asked to choose the higher reward probability, which required comparison of their internally estimated value for the fractal with a displayed numeric value. Participants made a choice by pressing one of two available buttons (‘choose fractal’ and ‘choose explicit probability’). Either option could have a value 10, 20 or 30% higher than the other or be of equal value. Either option could have a value 10, 20 or 30% higher than the other or be of equal value. This means a total of 240 fractals (60 × 4) were observed with 60 decisions being made. The sequence of observations and decisions were interleaved in a pseudo-random order and identical for all subjects. The study was divided into four sessions of 15 min each, between which there were periods where participants could briefly rest. Each session was split into three blocks and during each block participants made five decisions after having observed 5 × 4 fractals. Participants did not receive feedback during the task but were told their performance scores would be converted into money they would receive at the end of the experiment. The task is summarised in Fig. 1.
Pavlovian learning paradigm. Participants passively observed different fractals followed by reward or no reward. From these observations they estimated the probability of reward for each fractal then choose the higher of their estimated fractal value or an explicitly presented value.
Computational modelling of behaviour
To measure individuals' performance, we plotted their psychometric response curves as the percentage of times a fractal option was chosen as a function of the difference between the probabilities associated with each option with curves fitted with a sigmoid function (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018). The slopes of the sigmoid curves were significantly steeper for controls compared to MDD (p = 0.025) and detailed computational analyses indicated that MDD was associated with impaired value learning. Details on these behavioural analyses are summarised in the online Supplementary Materials and have been published elsewhere (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018).
Briefly, to reveal which decision-making components explained the performance difference, three different families of models were compared, reflecting distinct hypotheses about how participants make decisions. All models assumed participants internally estimated a value for each observed fractal then compared this estimate to the explicitly presented value when making a decision. For model fitting, parameters were estimated using maximum a posteriori estimates incorporating an empirical prior estimated from behavioural data initialised using maximum likelihood estimates. Thereafter, expectation–maximisation was used to iteratively improve the value estimates and the model that best fitted the behavioural data, taking into account model complexity, was identified using the integrated Bayesian information criterion (Huys et al., Reference Huys, Pizzagalli, Bogdan and Dayan2013; Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018). Here we focus on the best model identified from that work (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018) as this was used for model-based fMRI analyses.
The model that best described observed behaviour was termed ‘Leaky’ and included a retrospective discounting factor or memory loss parameter (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018). Internal value estimates were assumed to be updated after observing fractal i and associated reward r occurring at time t as
$$V_i^{t + 1} = A \times V_i^t + r_i^t \comma \;$$where A is a memory parameter (range 0–1) and smaller A reflected increased forgetting or retrospective discounting, r was unity if a £ reward symbol was observed and zero otherwise. The probability of choosing fractal i was calculated using a softmax function
$$p\,\lpar {\rm choose}\;{\rm fractal}\;i\rpar \,{\rm}= \,\sigma \,\lpar \beta \,\times \,\lpar f\,\lpar V_i\rpar \,-\,\phi _i\rpar \rpar \,= \,\displaystyle{1 \over {1\,+ \,\exp \,\lpar \!-\!\beta \times \lpar f\lpar V_i\rpar -\phi _i\rpar \rpar }}\comma \;$$incorporating estimated value (V) and explicitly presented value (ϕ) where f(x) = x/4 is a transformation of the internal value estimate compared to the explicitly displayed reward probability of the alternative choice. The inverse temperature β determined the ability of participants to use internal value estimations to make decisions. Smaller values of β indicated a more variable use of internal values.
Image acquisition and pre-processing
Functional whole brain images were acquired using a 3T Siemens Magnetom Tim Trio scanner using an echo-planar imaging sequence with the following parameters: repetition time = 2500 ms, echo time = 30 ms, flip angle = 90°, field of view = 224 mm, matrix = 64 × 64, 37 slices, voxel size 3.5 × 3.5 × 3.5 mm. The first four blood oxygen level-dependent volumes were discarded as standard because of transient effects. Data were pre-processed using Statistical Parametric Mapping 12 (SPM12; https://www.fil.ion.ucl.ac.uk/spm/) with functional images realigned to the first image, unwarped and co-registered to the segmented T1 weighted structural image. An estimated deformation field was used to spatially normalise the images and an 8 mm Gaussian kernel used to smooth the functional images.
Random effects image analyses
Random-effects, event-related designs were used for analyses. Three event times were of particular interest: (a) when participants observed a fractal stimulus and may have retrieved their previously estimated value for that fractal, (b) when participants observed a rewarding Pavlovian association (£ symbol) indicating reward value or alternatively a blank screen in the case of zero value, this being the trial ‘outcome event’ and (c) when participants were prompted to choose between the estimated value of an observed fractal and an explicit probability value this being the ‘decision event’. For first level analyses, events were modelled as truncated delta functions and convolved with the SPM12 canonical haemodynamic response function without time or dispersion derivatives. Vectors representing these events were entered into first level analyses for each subject and six rigid body motion realignment parameters estimated during pre-processing included as covariates of no interest. Activation at these event times was investigated using both model-based and standard fMRI strategies, testing for significant activations across and between groups and for correlations of activity with illness severity scores.
Given strong evidence for blunted striatal responses to rewards in depression, we used the results of an automated meta-analysis of fMRI studies on healthy subjects (‘Neurosynth’, Yarkoni, Poldrack, Nichols, Van Essen, and Wager, Reference Yarkoni, Poldrack, Nichols, Van Essen and Wager2011) with the search term ‘reward’ which identified 922 studies. We then chose voxels with the global maximum z-score in left and right hemisphere located in left (−12, 10, −8) and right (12, 10, −8) nucleus accumbens (NAc). For each participant in our study we extracted median beta values from the reward contrast maps from a 5 mm sphere centred at these co-ordinates, then tested for significant group differences using Welch's t test.
For model-based fMRI, the Leaky model was used to calculate the value of each fractal on each trial. The estimated value was used as a first level analysis parametric modulator at the time when the fractal stimulus was presented. Additionally, the difference between the internally estimated fractal probability value and the displayed explicit probability value was calculated and used as a parametric modulator at the decision time. The value difference was defined as V chosen − V alternative, i.e. the value of the chosen option minus the value of the alternative option. Notably, our model uses the value difference to assign probabilities for choosing each option at the decision time. We therefore expected to observe a value difference encoding signal in regions identified as being active at the decision time.
Event-related functional connectivity between brain regions activated during the task was calculated using the generalised psychophysiological interaction (gPPI) method (McLaren, Ries, Xu, & Johnson, Reference McLaren, Ries, Xu and Johnson2012), which tested the hypothesis that value-based decision making involves a distributed network and MDD is associated with abnormal connectivity in that network. Specifically, we assessed how the ‘decision event’ (the psychological state) modulated activity within brain networks that included our anterior mid-cingulate cortex (aMCC, Tolomeo et al., Reference Tolomeo, Christmas, Jentzsch, Johnston, Sprenglemeyer, Matthews and Steele2016) seed region. For each participant, we calculated the contrast at the first (i.e. subject) level (connectivity at decision time > implicit baseline) and then took these contrasts to a standard second (i.e. group) level analysis using SPM12.
For all calculations, activity was corrected for multiple comparisons using a Monte Carlo method (Slotnick, Moo, Segal, & Hart, Reference Slotnick, Moo, Segal and Hart2003) with simultaneous requirement for a cluster extent threshold of 108 contiguous resampled voxels and a voxel threshold of p < 0.05, resulting in a whole brain corrected cluster threshold of p < 0.01. This threshold was enforced for all contrasts.
Results
There was no significant difference between MDD and control groups in the number of (missed) behavioural responses from subjects during the paradigm: two group t test p = 0.728. Since behavioural responses were matched and subjects were not given feedback during the task, all events were matched between groups.
Striatal reward response
The outcome event time was associated with strong activations in regions including the bilateral striatum (10, 12, −4), (−10, 18, 0), aMCC (−10, 10, 48) and bilateral dorsolateral cortex (−46, 8, 24), (44, 6, 32). Consistent with our first hypothesis using the Region of Interest (ROI) approach, striatal activation to reward symbols were significantly blunted in unmedicated MDD in right NAc (12, 10, −8), t (25.54) = 2.907, p = 0.007 with a trend for left NAc (−12, 10, −8), t(22.80) = 1.953, p = 0.063 (Fig. 2a). Using voxel-based methods not confined to the NAc, we found significantly blunted activation in left (−22, 14, −16) and right striatum (12, 4, −4), (22, 26, 10) (Fig. 2b). This is consistent with our independent studies of chronically medicated patients with treatment-resistant MDD (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders and Steele2011; Johnston et al., Reference Johnston, Tolomeo, Gradin, Christmas, Matthews and Steele2015; Steele, Kumar, & Ebmeier, Reference Steele, Kumar and Ebmeier2007) and other reports from independent groups (e.g. Keren et al., Reference Keren, O'Callaghan, Vidal-Ribas, Buzzell, Brotman, Leibenluft and Stringaris2018).
Reward events. (a) Reward activation in NAc ROIs, (b) decreased reward activation in MDD participants compared to healthy controls (HC) in the striatum. All regions significant at p < 0.01 whole-brain corrected.
Reward value encoding
At the fractal presentation time, the estimated value of the presented fractal was used as a parametric modulator at the first level. Single group second level analyses showed positive encoding of reward value (activation) in controls (Fig. 3a) in areas including hippocampus (−38, −28, 0), (46, −26, −2) and rostral ACC (rACC) (14, 50, −2) and negative encoding (deactivation) of reward value in MDD subjects (Fig. 3b) in hippocampus (−30, −30, −2), (36, −26, −2) and rACC (14, 50, −10). A subsequent two-group comparison revealed significantly larger positive value encoding in controls compared to MDD participants (Fig. 3c, d) in hippocampus (−36, −32, 2), (48, −26, 4) and rACC (14, 50, −8). Within MDD subjects only, there was a significant negative correlation of BDI illness severity with extracted contrast-betas from the rACC (r = −0.59, p = 0.009; Fig. 3e) but not hippocampus (r = −0.02, p = 0.931).
Reward value encoding at fractal presentation time. (a) Positive value encoding within healthy controls. (b) Negative value encoding in depressed participants. (c) Larger value encoding in healthy controls (HC) compared to MDD participants in hippocampus and rostral ACC. All regions significant at p < 0.01 whole-brain corrected. (d) Group comparison of value encoding in hippocampal ROI. (e) Within MDD subjects negative correlation between BDI illness severity and rAC value encoding (r = −0.59, p = 0.009). All regions significant at p < 0.01 whole-brain corrected.
In addition to classical statistical inference it is important to test for individual patient predictive accuracy (Steele & Paulus, Reference Steele and Paulus2019). Logistic regression with leave-one-out cross-validation was used to classify participants as MDD or controls using median beta values of the value encoding contrast at rACC and left hippocampal ROIs. The classifier achieved an individual subject accuracy of 79% (area under the Receiver Operating Characteristic (ROC) curve Area Under Curve (AUC) = 0.86; see online Supplementary Materials).
Decision making
The decision event time was associated with strong activation in regions including the aMCC (−2, 14, 50) and bilateral anterior insula (−28, 22, −2), (32, 26, −6) across both groups (Fig. 4a), a pattern consistent with activation of cognitive control processes as identified in a large meta-analysis (Shackman et al., Reference Shackman, Salomons, Slagter, Fox, Winter and Davidson2011). Bilateral insula, subgenual anterior cingulate cortex (−2, 28, −2) and aMCC (−12, 20, 32) (22, 28, 42) activity was significantly increased in MDD subjects compared to controls (Fig. 4b), with the aMCC region (−6, 26, 36) correlating positively with BDI illness severity scores within the MDD group alone.
Activation during decision making. (a) Activation across all participants (p < 0.05 FWE threshold). (b) Larger activations in MDD compared to controls. (c) Negative value difference encoding signal across participants. (d) Negative absolute value difference encoding signal across participants. (e) Positive absolute value difference encoding signal across participants. (f) Decreased event-related connectivity in depression between dorsal cingulate region and other cingulate regions. All regions significant at p < 0.01 whole-brain corrected.
The difference between the value of the chosen option and the value of the alternative option was used as a parametric modulator at the first level. In the softmax decision rule, the value difference is used together with the beta inverse temperature parameter to calculate choice probabilities. Across participants, we observed a significant negative correlation of value difference encoding in regions including the aMCC region (−14, 16, 48), (12, 24, 28) (Fig. 4c). In addition, a negatively correlated absolute value difference encoding signal was also observed in regions including aMCC (−4, 24, 46), (10, 10, 46) (Fig. 4d) and a positively correlated absolute value difference signal was observed in regions including the rACC (−16, 42, 8), (−4, 50, −14), (24, 38, 4) (Fig. 4e). Mean value difference and mean absolute value difference were weakly correlated across participants (r = 0.36, p = 0.037). We did not identify a significant difference between groups for either value encoding parameter within these dorsal and rostral cingulate regions (see online Supplementary Materials).
Event-related connectivity
The aMCC region from the decision event time activation across groups was used as a seed region for a gPPI analysis, to test whether this region exhibited abnormal event-related connectivity in MDD compared to controls. Significantly weaker connectivity at the decision time between the dACC and posterior, mid and rostral cingulate cortex regions (−12, 42, 4), (8, 50, 8) in MDD was identified as shown in Fig. 4f.
Post-hoc correction for grey matter variation
Because there is evidence for hippocampal volume reductions in recurrent depression (Schmaal et al., Reference Schmaal, Veltman, van Erp, Samann, Frodl, Jahanshad and Hibar2015, Reference Schmaal, Hibar, Samann, Hall, Baune, Jahanshad and Veltman2017) an additional analysis was done (see also online Supplementary Materials) to test for the effect of grey matter variation on fMRI findings. For every participant the estimated forward deformation field was used to normalise the grey matter probability image, thereby obtaining for each resampled voxel an estimate of the probability that a voxel was grey matter. Beta values in the hippocampal and rostral anterior cingulate of the fMRI contrast images were then multiplied by these grey matter probabilities and two group t tests used to test for differences. The results still showed significant fMRI group differences: left hippocampus t(21.36) = 3.313, p = 0.003; right hippocampus t(31.03) = 2.501, p = 0.018; rACC t(31.19) = 2.890, p = 0.007.
Discussion
To our knowledge, this is the first study to test hypotheses about abnormal reward value encoding and event-related connectivity in patients with unmedicated MDD. In our detailed behavioural analyses (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018) we reported impaired behavioural performance in MDD caused by impairments in both value learning and decision phases of our Pavlovian task; MDD subjects also showed lower memory of observed reward and had an impaired ability to use internal value estimations to guide decision making (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018). Here we sought to identify the neural substrates of these behavioural abnormalities.
Consistent with our first hypothesis, we found that the striatal reward activation was blunted as was the reward signal in an independently defined NAc ROI of unmedicated MDD subjects. This is consistent with our previous independent studies on chronically medicated treatment-resistant MDD (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders and Steele2011; Johnston et al., Reference Johnston, Tolomeo, Gradin, Christmas, Matthews and Steele2015; Steele et al., Reference Steele, Kumar and Ebmeier2007) and reports by independent groups (Keren et al., Reference Keren, O'Callaghan, Vidal-Ribas, Buzzell, Brotman, Leibenluft and Stringaris2018; Zhang, Chang, Guo, Zhang, & Wang, Reference Zhang, Chang, Guo, Zhang and Wang2013). Whilst the region is often referred to generically in the literature as the ‘striatum’, which includes the NAc and caudate, the region of significantly blunted reward activation during our Pavlovian task also prominently included the region between the two NAc (Fig. 2b) which is the septum (Mai, Matjtanik, & Paxinos, Reference Mai, Matjtanik and Paxinos2015). This structure is part of the septo-hippocampal system which is strongly implicated in anxiety and in the action of antidepressant and anxiolytic medication (Gray & McNaughton, Reference Gray and McNaughton2000). Notably, using a very different instrumental task to study an independent group of treatment-resistant medicated patients with MDD, we also observed septal reward signal blunting and similarly asymmetric blunting of the NAc (Fig. 3b; Johnston et al., Reference Johnston, Tolomeo, Gradin, Christmas, Matthews and Steele2015). Further study of septal reward response blunting in MDD is indicated.
Consistent with our second hypothesis, we found brain regions with decreased reward value signal encoding in MDD, in particular hippocampus and rACC. We have previously reported decreased reward value encoding in the hippocampus of an independent group of chronically medicated patients with treatment-resistant MDD using an instrumental learning task (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders and Steele2011) and as noted above, there is strong evidence for hippocampal abnormalities in treatment-resistant and recurrent MDD (Johnston et al., Reference Johnston, Tolomeo, Gradin, Christmas, Matthews and Steele2015; Schmaal et al., Reference Schmaal, Veltman, van Erp, Samann, Frodl, Jahanshad and Hibar2015). Here, using a novel Pavlovian reward task with unmedicated MDD subjects, we report positive reward value encoding in the hippocampus of controls and negative reward value encoding of reward value in MDD. Interestingly, a recent Pavlovian study using aversive stimulus learning reported positive encoding of an aversive conditioned stimulus signal in the habenula of controls and negative encoding in MDD (Lawson et al., Reference Lawson, Nord, Seymour, Thomas, Dayan, Pilling and Roiser2017).
Recent meta-analyses and reviews have provided substantial evidence for the involvement of regions in the PFC including the rACC in the encoding of reward value (Bartra et al., Reference Bartra, McGuire and Kable2013; Chase et al., Reference Chase, Kumar, Eickhoff and Dombrovski2015). The ventromedial PFC (vmPFC) is thought to be a key region involved in value-based decision making (Glascher, Hampton, & O'Doherty, Reference Glascher, Hampton and O'Doherty2009; Treadway et al., Reference Treadway, Buckholtz, Cowan, Woodward, Li, Ansari and Zald2012). Notably, Glaescher and colleagues reported that the vmPFC encoded value signals from a computational model in addition to the amygdala-hippocampal complex, although these value signals were related to actions and expected outcomes (Glascher et al., Reference Glascher, Hampton and O'Doherty2009). Reduced expected reward value signals have previously been reported in the vmPFC of suicide attempters (Dombrovski et al., Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013). Importantly and consistent with our third hypothesis, we found a significant negative correlation between illness severity and rACC value encoding within MDD subjects alone. Consequently, there is considerable evidence for reward value encoding in the hippocampus and vmPFC of healthy subjects, and in addition to the present study, evidence for blunted reward value encoding in two independent studies: on MDD (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders and Steele2011) and attempted suicide (Dombrovski et al., Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013). This suggests these two regions are part of the neural substrates of impaired value learning observed in our behavioural analyses (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018).
The aMCC has been highlighted as crucial for decision making in a large meta-analysis of healthy subjects (Shackman et al., Reference Shackman, Salomons, Slagter, Fox, Winter and Davidson2011), and it has been suggested that abnormalities of anterior cingulate reward-linked computational function and connectivity could explain core symptoms in a variety of disorders including MDD (Holroyd & Umemoto, Reference Holroyd and Umemoto2016). Consistent with this, we have reported decision-making abnormalities in treatment-resistant MDD patients receiving aMCC therapeutic lesions (Tolomeo et al., Reference Tolomeo, Christmas, Jentzsch, Johnston, Sprenglemeyer, Matthews and Steele2016) and evidence for electro-convulsive therapy therapeutically altering aMCC connectivity in an independent group of patients with treatment-resistant MDD (Perrin et al., Reference Perrin, Merz, Bennett, Currie, Steele, Reid and Schwarzbauer2012). Also consistent with our second hypothesis, in the present study we found abnormally increased activation in MDD and encoding of a value difference signal in the aMCC region at the decision time, linking our behavioural model (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018) to localised brain function. Consistent with our fourth hypothesis, event-related connectivity analysis at the decision time revealed reduced connectivity between the aMCC and more rostral ACC regions, in MDD compared to controls. An influential theory of aMCC function linking cognitive control, valuation and motivation, proposes that the underlying function of the aMCC is to determine how much control to allocate (Shenhav, Botvinick, & Cohen, Reference Shenhav, Botvinick and Cohen2013). Consistent with our interpretation, the theory posits that the aMCC receives value-representation inputs from regions such as the vmPFC which are used to monitor outcomes and adjust the level of control. There is evidence that abnormal anterior cingulate cortex maturation during adolescence contributes to the development of MDD reflected by inflexible aMCC connectivity (Ho et al., Reference Ho, Sacchet, Connolly, Margulies, Tymofiyeva, Paulus and Yang2017). The present work suggests this could be related to impairment in the communication of value estimates from the rACC to the aMCC where these estimates are used to guide decision making.
A large meta-analysis of subcortical regions found decreased hippocampal volume in recurrent depression (Schmaal et al., Reference Schmaal, Veltman, van Erp, Samann, Frodl, Jahanshad and Hibar2015) and a later meta-analysis reported a range of cortical structural abnormalities including the rACC (Schmaal et al., Reference Schmaal, Hibar, Samann, Hall, Baune, Jahanshad and Veltman2017) although see Shen et al. (Reference Shen, Reus, Cox, Adams, Liewald, Bastin and McIntosh2017). We therefore did additional analyses addressing the possibility of structural differences influencing our results (‘Results’ section and online Supplementary Materials). The value encoding signals remained significantly different between groups and our conclusions are unaltered. Reward and loss have different value functions with overlapping but different neural substrates which are relevant for MDD (Johnston et al., Reference Johnston, Tolomeo, Gradin, Christmas, Matthews and Steele2015) but we could not address this using our current paradigm, although see Lawson et al. (Reference Lawson, Nord, Seymour, Thomas, Dayan, Pilling and Roiser2017). A possible limitation of our analyses is that the voxel threshold p < 0.05 was within the permitted range but not the ideal range. We therefore repeated the analyses using a more stringent voxel threshold p < 0.01 and found the results analogous with the exception of the encoding of negative value difference across subjects which was not significant (see online Supplementary Materials).
Conclusions
A close link between emotional experience and valuation has been proposed (Dolan, Reference Dolan2002). Diverse symptoms of MDD can be explained within a decision-theoretic framework in which abnormal valuation plays a central role (Huys et al., Reference Huys, Daw and Dayan2015; Huys & Renz, Reference Huys and Renz2017). We reported behavioural evidence for abnormal reward value learning and decision making in depression (Rupprechter et al., Reference Rupprechter, Stankevicius, Huys, Steele and Series2018) and here we identified the neural substrates of these abnormalities as being the striatum, septo-hippocampal system and anterior cingulate, with both reward value encoding and event-related connectivity being abnormal. This supports the theory that abnormally biased neural valuation plays a central role in MDD, and suggests there is impaired communication between the neural substrates of valuation and decision making in depression.
To the extent that emotion reflects valuation, abnormal valuation could explain abnormal emotional experience in MDD, reflect a core pathophysiological process and be a target of treatment. Finally, MDD may not be the only common psychiatric illness associated with abnormal neural valuation, as there is also evidence for schizophrenia (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders and Steele2011) and addiction (Redish, Reference Redish2004; Redish, Jensen, & Johnson, Reference Redish, Jensen and Johnson2008), implying different psychiatric disorders may reflect different disorders of neural valuation.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291719003799.
Acknowledgements
We thank all the participants who took part in the study. SR received a Principal's Career Development Scholarship from the University of Edinburgh.
Financial support
The authors acknowledge Doctoral Training Centre, Informatics Forum, University of Edinburgh, which was funded by the EPSRC with contributions from BBSRC and MRC; grant numbers EP/F 500385/1 and BB/F529254/1 (no grant number for MRC as contribution via EPSRC).
Conflict of interest
The authors declare no competing interests.
