Introduction
How an episode of extreme traumatic stress impacts normal brain function to alter the risk of psychopathology is one of the most fundamental questions in mental health research. Despite increasing knowledge about the neuronal adaptation to traumatic stress (Pitman et al. Reference Pitman, Rasmusson, Koenen, Shin, Orr, Gilbertson, Milad and Liberzon2012), we know little regarding how an episode of traumatic stress affects decision-making and the acquisition of optimal choice behaviour. This question becomes even more important in light of altered decision-making being a core feature of several psychiatric disorders associated with traumatic stress (Cella et al. Reference Cella, Dymond and Cooper2010; Sebold et al. Reference Sebold, Deserno, Nebe, Schad, Garbusow, Hagele, Keller, Junger, Kathmann, Smolka, Rapp, Schlagenhauf, Heinz and Huys2014), leading to the idea that traumatic stress may predispose to illness by influencing how we make decisions (Huys et al. Reference Huys, Daw and Dayan2015).
Our decisions are the end product of multiple computationally and neurobiologically distinct mechanisms (Dolan & Dayan, Reference Dolan and Dayan2013). The instrumental system ensures that rewards are harvested and punishments avoided through discrete action choices, controlled by a consideration of the consequences of an action (Dayan et al. Reference Dayan, Niv, Seymour and Daw2006; Dayan & Daw, Reference Dayan and Daw2008). Although humans readily learn to approach reward and avoid punishment, they show greater difficulties learning not to act to obtain a reward and act to avoid a punishment, highlighting a surprising inflexibility in human decision-making (Cavanagh et al. Reference Cavanagh, Eisenberg, Guitart-Masip, Huys and Frank2013; Guitart-Masip et al. Reference Guitart-Masip, Duzel, Dolan and Dayan2014a ). This inflexibility can be understood in a frame of reference whereby stimuli predicting reward are intrinsically associated with behavioural approach, while stimuli predicting punishments are pre-potently coupled to behavioural inaction. These pre-specified response tendencies are referred to as Pavlovian biases, which are known to distort flexible instrumental decision making (Guitart-Masip et al. Reference Guitart-Masip, Duzel, Dolan and Dayan2014a ).
Following extreme traumatic stress, humans and animals show deficits in suppressing or extinguishing Pavlovian fearful associations (Milad et al. Reference Milad, Pitman, Ellis, Gold, Shin, Lasko, Zeidan, Handwerger, Orr and Rauch2009; Jovanovic et al. Reference Jovanovic, Norrholm, Blanding, Davis, Duncan, Bradley and Ressler2010), which can lead to sustained fear and avoidance in response to a Pavlovian cue. The effect of traumatic stress on the instrumental system is less known, though the results of acute or persistent laboratory-induced stress point to a shift from goal-directed towards habitual instrumental behaviours (Dias-Ferreira et al. Reference Dias-Ferreira, Sousa, Melo, Morgado, Mesquita, Cerqueira, Costa and Sousa2009; Schwabe & Wolf, Reference Schwabe and Wolf2009). Another possible mechanism by which traumatic stress might influence instrumental decision making is through its impact on the Pavlovian system. While this effect could potentially be beneficial in situation where both the Pavlovian and the instrumental system promote the same behavioural output, an increased Pavlovian influence on decisions is maladaptive in situations where the behaviour promoted by the two systems are in opposition, causing behavioural inflexibility and a reduced ability to meet current situational demands.
The arbitration between Pavlovian and instrumental systems in controlling decisions and behaviours is thought to be under the control of the prefrontal cortex (Cavanagh et al. Reference Cavanagh, Eisenberg, Guitart-Masip, Huys and Frank2013; Guitart-Masip et al. Reference Guitart-Masip, Duzel, Dolan and Dayan2014a ). The dorsal anterior cingulate cortex (dACC) may be of particular interest here, based on its historical role in governing of cognitive control (Silvetti et al. Reference Silvetti, Alexander, Verguts and Brown2014; Mansouri et al. Reference Mansouri, Egner and Buckley2017), and a recent study showing that individual differences in dACC neurophysiology were associated with corresponding differences in the ability to overcome Pavlovian biases (Cavanagh et al. Reference Cavanagh, Eisenberg, Guitart-Masip, Huys and Frank2013). However, humans differ not only in the neurophysiological features of the dACC, but also exhibit great variability in the neurochemical characteristics, including glutamate levels (Falkenberg et al. Reference Falkenberg, Westerhausen, Specht and Hugdahl2012), of this region. Regional glutamate levels can be measured in-vivo using magnetic resonance spectroscopy (1H-MRS), which allows us to test for an association between individual dACC neurochemistry and the ability to overcome Pavlovian biases of instrumental scenarios. Moreover, mounting evidence suggest that stress, through its impact on glucocorticoids, affects glutamatergic neurotransmission, thereby influencing core aspects of cognitive processing (Popoli et al. Reference Popoli, Yan, McEwen and Sanacora2012). Accordingly, stress may impair flexible instrumental responding through an influence on prefrontal glutamatergic mechanisms.
Here, we combined a behavioural task that dissociates distinct influences of instrumental and Pavlovian mechanisms on action selection (Guitart-Masip et al. Reference Guitart-Masip, Huys, Fuentemilla, Dayan, Duzel and Dolan2012) with 1H-MRS in a target population of trauma survivors, whom we compared with a non-traumatized control group. We tested whether an episode of traumatic stress can have long-term influences on how we make instrumental decisions, and whether the impact of stress on action choice could be attributable to changes in glutamatergic levels.
Materials and methods
Subjects
Twenty-six survivors from the 2011 Norwegian terror attack at Utøya and 30 healthy control subjects between 16 and 25 years were included in the present study after giving written informed consent. The data were collected between 21 and 33 months after the attack. The study was approved by the Regional Committee for Medical and Health Research Ethics South East and complied with the declaration of Helsinki. Subjects received an honorarium of 500 NOK for their participation.
Trauma survivors were recruited by written invitation sent out by the Resource Centre for Violence, Traumatic Stress and Suicide Prevention Region West, Norway. The control sample was subjects matched for age, gender and educational level, which were not involved in the trauma, and were not otherwise related to any of the survivors. Information concerning subjects’ mental status was obtained by the Mini International Neuropsychiatric Interview (MINI, 6.0.0; (Sheehan et al. Reference Sheehan, Janavs, Harnett-Sheehan, Sheehan, Gray, Lecrubier, Weiller, Hergueta, Allgulander, Kadri, Baldwin and Even2009)) administered on the day of examination. See online Supplementary Methods for more details regarding the MINI.
General exclusion criteria were a history of severe somatic illness, head trauma, ongoing substance abuse and MRI-incompatibility. Additional exclusion criteria for the control subjects included history of psychiatric disorders or previous psychological traumas as detected by the MINI. After initial assessments, five subjects were excluded from the control group based on history of psychiatric disorder or recent drug use. Furthermore, one subject was excluded from the trauma survivors due to incidental brain pathology discovered during the MRI session. Finally, two subjects from the control group were excluded due to an overall accuracy of less than 50% in the experimental task. The final samples were thus 25 trauma survivors and 23 controls.
The orthogonalized Go/No-go task
All subjects completed the experimental task before entering the MRI scanner. Subjects performed a modified version of the orthogonalized Go/No-go task (Guitart-Masip et al. Reference Guitart-Masip, Huys, Fuentemilla, Dayan, Duzel and Dolan2012) (Fig. 1a , b ). Each trial consisted of a fractal cue, which after a short delay was followed by an outcome. Subjects had to learn for each fractal whether to press a button or not to obtain a reward or avoid losing money. In total there were four trial types that were indicated by four separate fractal cues; Go to win, Go to avoid punishment, No-go to win and No-go to avoid punishment. See online Supplementary Methods for more task details.
(a) The orthogonalized Go/NoGo task. Subjects had to learn for each image whether to press a button or not to obtain a reward or avoid losing money. (b) The timings of the task. C: Mean accuracy for each experimental condition shown for the trauma survivors and the controls separately. Error bars are ±1 SEM. ITI, Intertrial interval; ISI, Interstimulus interval; NOK, Norwegian Krone; GW, Go to win; GNL, Go to avoid losing; NGW, No-go to win; NGNL, No-go to avoid losing.
Behavioural data analysis
Independent sample t tests were performed to test for overall differences in accuracy or response times between the two groups. Next, we analysed the data in four different ways to reveal if traumatic stress led to a greater Pavlovian bias of instrumental decisions. In the first analysis, the number of correct choices was collapsed across time bins of 10 trials per condition for each participant. We then performed a four factor mixed ANOVA with time, action (go/no-go) and valence (reward/punishment) as within-subject factors, and group as a between-subject factor. Significant interactions were explored using post-hoc t tests corrected for multiple comparisons using Bonferroni correction. A two-tailed p value < 0.05 was used as significance threshold if otherwise not stated.
However, average learning measures often obscures more discrete differences between participants who learned the task and those who did not (Gallistel et al. Reference Gallistel, Fairhurst and Balsam2004). Thus, we also performed non-continuous classification of subjects as learners (=average performance >65%) (Cavanagh & Frank, Reference Cavanagh and Frank2014) and non-learners independently for the conflicting (i.e. No-go to win and Go to avoid punishment) and non-conflicting (i.e. Go to win and No-Go to avoid punishment) conditions, and compared the groups in a χ2 test. Thirdly, we calculated a Pavlovian Performance Bias (PPB) score according to the following formula: [Pavlovian Performance bias = ((Go on Go to win + No-go to win)/Total Go) + (No-go on Go to avoid punishment + No-go to avoid punishment)/Total No-go))/2] (Cavanagh et al. Reference Cavanagh, Eisenberg, Guitart-Masip, Huys and Frank2013), which is a summary measure of how strongly action and valence interact during choice. The PPB score can be separated into two valence-specific components, one representing reward-based invigoration [(Go on Go to win + No-go to win)/Total Go] and the other representing punishment-based suppression [(No-go on Go to avoid punishment + No-go to avoid punishment)/Total No-go] of action.
Computational modelling of the behavioural data
We defined a series of nested models incorporating different instrumental and Pavlovian reinforcement-learning hypotheses so as to capture learning behaviour (Guitart-Masip et al. Reference Guitart-Masip, Huys, Fuentemilla, Dayan, Duzel and Dolan2012). In all models the propensities w(a t , s t ) for action a t (go or no-go) on trial t under condition s t were estimated. The simplest model updated action values Q t (a t , s t ) according to the Rescorla–Wagner equation, and this model was expanded through successive steps to incorporate irreducible choice noise (ξ) and a value-independent static action bias b (Guitart-Masip et al. Reference Guitart-Masip, Huys, Fuentemilla, Dayan, Duzel and Dolan2012). The winning model also contained a Pavlovian parameter π, which inhibited a go tendency when feedback was in terms of punishments (negative V(s t )) and promoted go actions when feedback was in terms of rewards (positive V(s t )). In addition, the model allowed subjects to treat one unit of reward and one unit of punishment differently by letting the parameter ρ take on different values in reward and punishment trials.
$$\eqalign{& W_t (a,\,s) = \left\{ \matrix{Q_t (a,\,s) + b + \pi V_t (s)\quad {\rm if}\,a = {\rm go} \hfill \cr Q_t (a,\,s)\quad \quad \quad \quad \quad \quad \quad \,\,{\rm else} \hfill} \right. \cr & V_t (s_t ) = V_{t - 1} (s_t ) + \varepsilon (\rho r_t - V_{t - 1} (s_t ))} $$
In line with previous publications of these models, we used an expectation-maximization procedure for estimation of the group and the individual subject parameters (Huys et al. Reference Huys, Cools, Golzer, Friedel, Heinz, Dolan and Dayan2011; Guitart-Masip et al. Reference Guitart-Masip, Huys, Fuentemilla, Dayan, Duzel and Dolan2012). The model fitting procedures were confirmed on surrogate data from an established decision process. Model comparisons utilized the integrated Bayesian Information Criterion (iBIC). Different from the BIC, which gives an estimate of the penalized likelihood of the data given a set of parameters at the subject-level, iBIC gives the penalized group-level likelihoods from the distribution of the group level hyperparameters. Low iBIC scores indicate a good model fit of the data, and the difference in iBIC values is indicative of the evidence. See online Supplementary Methods for a more detailed description of the models.
Magnetic resonance spectroscopy (1H-MRS) acquisition and analysis
1H-MRS spectra were obtained from the dACC cortex using a single voxel point resolved spectroscopy (PRESS) sequence acquired with a GE Signa HDx 3T scanner. The region was chosen on a priori grounds due to its involvement in resolving Pavlovian-instrumental conflicts (Cavanagh et al. Reference Cavanagh, Eisenberg, Guitart-Masip, Huys and Frank2013) and cognitive control (Silvetti et al. Reference Silvetti, Alexander, Verguts and Brown2014). Based on the importance of ventromedial prefrontal cortex (vmPFC) in regulating Pavlovian behaviours (Etkin et al. Reference Etkin, Egner and Kalisch2011), we also obtained 1H-MRS spectra from the vmPFC for comparison purposes.
We used the resting-state Glx (Glx = glutamine + glutamate) level relative to Creatine from the LCModel (Provencher, Reference Provencher1993) output. We did not obtain dACC 1H-MRS spectra from one of the controls, and neither dACC 1H-MRS or vmPFC 1H-MRS spectra from one of the trauma survivors. Furthermore, vmPFC 1H-spectra data from one trauma survivor and two controls had to be excluded due to poor data quality. A detailed description of 1H-MRS data acquisition and analyses is provided in the online Supplementary Methods.
Results
Trauma survivors and the controls were well matched on age, gender and years of education (see Table 1). In total 14 survivors reported the presence of symptoms meeting the criteria for at least one of the disorders assessed in the MINI interview (see Table 1). Among the 14, six subjects had two or more ongoing disorders. The trauma survivors had significantly increased post-traumatic stress disorder (PTSD) symptom scores compared with a mean score of zero (t 24 = 6.23, p < 0.001, mean symptom score ± s.d. = 5.15 ± 4.14), suggesting an impact on mental health even 2 years after the attack. None of the trauma survivors reported a prior diagnosis of a psychiatric disorder preceding the terrorist attack. Except from one trauma survivor who occasionally used a low-dose benzodiazepine for insomnia, none of the survivors were prescribed any medications.
Characteristics of the subjects
s.d., standard deviation.
a The χ2 test was used for sex and psychopathology comparisons across the two groups; two-sample t test was used for age and years of education comparisons.
Performance
Overall performance was reduced in the trauma survivor group (t
(46) = −1.99, p = 0.05, Cohen's d = 0.57) though mean response times did not differ from controls (t
(46) = 0.81, p = 0.42, Cohen's d = 0.23). A four factor mixed ANOVA revealed a main effect of time (F
(2,46) = 70.49,
$\eta _p^2 $
= 0.61, p < 0.001), a main effect of action (F
(2,46) = 62.11,
$\eta _p^2 $
= 0.58, p < 0.001), an action by valence interaction (F
(2,46) = 54.11,
$\eta _p^2 $
= 0.54, p < 0.001) and an action by time interaction (F
(2,46) = 17.01,
$\eta _p^2 $
= 0.27, p < 0.001). There was no main effect of valence (F
(2,46) = 0.52, p = 0.48). Importantly, there was a significant three-way interaction between action × valence × group (F
(2,46) = 3.61,
$\eta _p^2 $
= 0.07, p = 0.03, one-way), indicating that traumatic stress interfered with how Pavlovian and instrumental systems interact.
The results on action and valence are in line with previous reports using a similar orthogonalized Go/No-go task (Guitart-Masip et al. Reference Guitart-Masip, Huys, Fuentemilla, Dayan, Duzel and Dolan2012). Specifically, subjects learned equally well from rewards and punishments, but performed better in a condition requiring a go compared with a no-go response (t (47) = 7.93, p < 0.001, Cohen's d = 1.14), and they learned go trials faster than no-go trials (as indicated by better performance in the go conditions compared with no-go conditions in the first 10 trials; t (47) = 7.52, p < 0.001, Cohen's d = 1.08). Furthermore, subjects were better in learning go to win (compared with go to avoid punishment) (t (47) = 7.16, p < 0.001, Cohen's d = 1.03) and no-go to avoid punishment (compared with no-go to win) (t (47) = 3.98, p < 0.001, Cohen's d = 0.57), supporting the idea that learning is facilitated when Pavlovian and instrumental valuation systems promoted the same behavioural response.
The action × valence interaction differed across the groups. Planned comparisons demonstrated that trauma survivors performed significantly worse in the two conditions where Pavlovian and instrumental actions conflicted (i.e. go to avoid punishment and no-go to win) (t (46) = −2.15, p = 0.04 Cohen's d = 0.62) compared with the controls. Importantly, there was no difference between the groups in the conditions where Pavlovian biases were aligned with instrumental learning (i.e. go to win and no-go to avoid punishment) (t (46) = −0.98, p = 0.34, Cohen's d = 0.28). These findings suggest a greater Pavlovian bias during decision-making in the trauma survivors. To visualize differences between groups and conditions, the mean accuracy for each condition divided by group is plotted in Fig. 1c .
To determine whether the percentage of participants learning the task successfully differed depending on group, we classified subjects as learners and non-learners for the conflicting v. non-conflicting conditions separately. In line with our hypothesis, fewer participants in the trauma survivors group performed successfully when Pavlovian and instrumental systems were in opposition (Learners in trauma survivors group: 14, Learners in control group: 19, χ2 = 3.95, p < 0.05) compared with conditions where they promoted the same behavioural response (Learners in trauma survivors group: 25, Learners in control group: 23, p = 1). To bolster this conclusion, we next compared the PPB scores. In line with our a priori predictions, trauma survivors showed greater PPB scores (t (46) = 2.13, p = 0.04, Cohen's d = 0.62), indicating greater expression of Pavlovian behaviours when it was inappropriate to do so, compared with control subjects. This difference was significant for both punishment-based suppression (t (46) = 2.12, p = 0.04, Cohen's d = 0.62) and for reward-based invigoration (t (46) = 2.36, p = 0.02, Cohen's d = 0.68).
Computational modelling
Figure 2a shows the results of fitting the computational models to the overall data. In line with previous studies (Guitart-Masip et al. Reference Guitart-Masip, Duzel, Dolan and Dayan2014a ; de Berker et al. Reference de Berker, Tirole, Rutledge, Cross, Dolan and Bestmann2016), the most parsimonious model had both two separate reinforcement sensitivity parameters and a Pavlovian parameter (ΔiBIC = 25.5 with the next-best model). Not only did this model capture the data collapsed across the two groups, but it also generated data that captured each group individually. Comparing the Pavlovian parameter between the groups (Fig. 2b ), showed a significant greater bias in the trauma survivors compared with controls (t (46) = 2.49, p = 0.02, Cohen's d = 0.73), although the difference was relatively small (Δ mean Pavlovian Bias = 0.04). However, the result was robust to exclusion of three trauma survivors with values more than two standard deviations (s.d.) from the group mean (t (43) = 2.62, p = 0.01, Cohen's d = 0.78). To visualize differences between groups and conditions, we plotted the average behaviour in each of the four conditions independently for the two groups (Fig. 2d–k ).
Model fits from 25 trauma survivors and 23 controls. (a) Model evidence (iBIC). The smaller the number, the better the model trades off complexity and fitting the data. The most parsimonious model contained both a Pavlovian component and separate sensitivities to rewards and losses. (b) Pavlovian bias parameters π from the most parsimonious model for controls and trauma survivors. Group mean differences are robust to exclusion of trauma survivors in red who are >2 standard deviations from the group mean. (c) Scatter plot of the association between the individual Pavlovian bias parameter and days since trauma. Subjects with a Pavlovian bias parameter >2 standard deviations from the group mean have been excluded. (d–k) Detailed learning curves for all four conditions are separated by group. The background shows each choice for each subject (go in white, no-go in grey). The black lines represent time-varying probability, across subjects, of making a go response. Note that panel d–g represents controls, while h–k represents trauma survivors. The coloured lines show the same time-varying probabilities, but evaluated on choices generated from the different models (colours as in panel a).
If an increased Pavlovian bias in the trauma group is driven by their traumatic experiences, we might expect this bias to be related to time elapsed since the traumatic event. To test this, we calculated for each subject the number of days between times of testing and the traumatic event. While this effect did not reach significance in the full sample, there was a significant negative association between the Pavlovian bias and time since trauma (r = −0.49, p = 0.02, Fig. 2c ) after excluding the three subjects with outlying Pavlovian bias scores. Thus, subjects tested closest to the traumatic event had the highest bias. We also investigated if PTSD symptom load was associated with the Pavlovian influence on choice. However, there was no significant association between PTSD symptom scores and the Pavlovian bias in the trauma survivors (r = 0.11, p = 0.61). Furthermore, the group effect was not driven by the two most common axis 1 disorders in the trauma survivors: Excluding subjects fulfilling the criteria of PTSD did not affect the results (t (39) = 2.06, p < 0.05, Cohen's d = 0.63), and the group difference in Pavlovian bias was robust to exclusion of subjects with panic disorder (t (38) = 2.50, p = 0.02, Cohen's d = 0.76).
1H-MRS
Figure 3a shows the positioning of the dACC 1H-MRS voxel. Analysis of dACC 1H-MRS data revealed a significant reduction in Glx (t (44) =−2.54, p = 0.02, Cohen's d = 0.76) in trauma survivors v. controls (Fig. 3b ). The dACC Glx levels in the trauma survivors were not related to PTSD symptom load (r = 0.11, p = 0.62) or time since trauma (r = 0.31, p = 0.15). There was no difference in Glx levels in vmPFC between the groups (t (42) = −1.48, p = 0.15, Cohen's d = 0.44). Mean Glx values for the vmPFC and the dACC divided by group are presented in online Supplementary Table S1. If the ability to overcome the inherent Pavlovian bias depends on glutamatergic signalling in dACC, then subjects with low dACC Glx scores, potentially reflecting diminished excitatory neurotransmission (Yang et al. Reference Yang, Quan, Peng, Zhong, Tan and Gong2015), should have greater Pavlovian influence on behaviour. In line with our predictions, decreasing levels of dACC Glx predicted a greater Pavlovian bias across subjects (r = −0.27, p = 0.04, one-way). The association was robust to exclusion of the three trauma survivors with Pavlovian bias values more than two s.d. from the group mean (r = −0.33, p = 0.03, Fig. 3c ).
1H-MRS data. (a) Positioning of the 1H-MRS voxel in dorsal anterior cingulate cortex (dACC). (b) Scatterplot of Glx/Cr in the dACC for the controls (n = 22) and the trauma survivors (n = 24). (c) Scatter plot of the association between the individual Pavlovian bias parameter and Glx/Cr. Subjects with a Pavlovian bias parameter >2 standard deviations from the group mean have been excluded.
To explore if the association between dACC Glx and a Pavlovian bias differed depending on group, we performed independent correlations for each group and compared the correlation coefficients using the Fisher's r-to-z test. The analysis revealed no significant differences between the groups (z = −0.03, p = 0.98). We also tested for a group difference using a two-way between-group ANOVA with participant group (trauma survivors v. controls) and dichotomized dACC Glx scores (high (n = 23) and low (n = 23) based on the population median = 2.24) as independent variables. The analysis revealed no significant interaction between group and dACC Glx (F (2,42) = 0.30, p = 0.59), bolstering the conclusion of a lack of group difference in the association between prefrontal Glx and the Pavlovian bias. Traumatic stress thus appears to alter glutamatergic levels in a medial prefrontal region. This is accompanied by a reduced ability to overcome Pavlovian influences, pointing to a potential neurochemical basis of the altered learning of action choices in the trauma survivors.
Discussion
Our data indicate that experiencing an episode of traumatic stress exerts long-term influences on the balance between decision-making systems. Young survivors of traumatic stress showed greater difficulties in an instrumental learning task, which was explained by a greater Pavlovian bias of instrumental decisions compared with a matched control group. Through the use of MR Spectroscopy, we also show that having experienced traumatic stress reduces the levels of Glx in dACC, and that across groups; subjects with the lowest levels of dACC Glx expressed the highest Pavlovian bias. The association between medial prefrontal glutamatergic levels and the Pavlovian bias of instrumental learning brings novel insight into the neurochemical basis of decision-making, and suggest a mechanism by which traumatic stress can influence motivated instrumental behaviours.
In line with previous reports, subjects were better at learning to emit a behavioural response in anticipation of reward, and withhold a response in the anticipation of punishment (Guitart-Masip et al. Reference Guitart-Masip, Huys, Fuentemilla, Dayan, Duzel and Dolan2012). Interestingly, this apparent inflexibility in instrumental decision-making was increased following an episode of extreme traumatic stress. Differences in reward or punishment sensitivity alone could not explain these findings. If experiencing traumatic stress mainly affected reward or punishment sensitivity, then both rewarding (or punishment) conditions should be equally affected, rather than the observed pattern of decision-making impairments in one rewarding (i.e. No-go to win) and one punishment (i.e. Go to avoid punishment) condition alone. Moreover, performance in the Go to win and No-go to avoid punishment were indistinguishable between the two groups, precluding a general performance deficit in the trauma survivors. Instead the findings support a greater dependency upon Pavlovian biases following traumatic stress, in accordance with observations that stress-mediates a general shift from computationally demanding flexible systems towards more automatic forms of control (Schwabe & Wolf, Reference Schwabe and Wolf2013).
Stress is typically associated with high metabolic demands and uncertainty (Koolhaas et al. Reference Koolhaas, Bartolomucci, Buwalda, de Boer, Flugge, Korte, Meerlo, Murison, Olivier, Palanza, Richter-Levin, Sgoifo, Steimer, Stiedl, van Dijk, Wohr and Fuchs2011). Switching behavioural control from flexible ‘cognitive’ to a more rigid Pavlovian system might save cognitive resources needed to deal with the stressor during the period of stress. However, a greater reliance on the Pavlovian system is maladaptive in the long run, as it promotes less behavioural flexibility, and may render a subject susceptible to maladaptive and potentially harmful behaviours (Huys et al. Reference Huys, Cools, Golzer, Friedel, Heinz, Dolan and Dayan2011; Schwabe & Wolf, Reference Schwabe and Wolf2013). Although previous studies have reported that other types of behavioural flexibility, including those identified in working memory and set-shifting tasks, are affected by uncontrollable stress (Arnsten, Reference Arnsten2015; Gamo et al. Reference Gamo, Lur, Higley, Wang, Paspalas, Vijayraghavan, Yang, Ramos, Peng, Kata, Boven, Lin, Roman, Lee and Arnsten2015), they have not addressed how this type of stress influence motivated decisions per se, a prerequisite to understand how traumatic stress contributes to various forms of psychopathology.
As in other areas of stress research, the effect of stress upon decision-making is highly dependent on the duration, intensity and controllability of the stressor (Hollon et al. Reference Hollon, Burgeno and Phillips2015). A recent study found that acute laboratory induced stress impaired learning to act, but did not render subjects more susceptible to Pavlovian influences in general (de Berker et al. Reference de Berker, Tirole, Rutledge, Cross, Dolan and Bestmann2016). The apparent discrepancy can be related to both quantitative and qualitative differences between the stressors (i.e. an ecologic valid traumatic experience v. a laboratory stress test). A vast literature has acknowledged that acute transient stressors v. more severe, persistent, ones affect both the structure and the functions of the prefrontal cortex differently, with opposing consequences for behaviour (Arnsten, Reference Arnsten2015; McEwen et al. Reference McEwen, Bowles, Gray, Hill, Hunter, Karatsoreos and Nasca2015). Accordingly, studies of trauma exposed individuals may be a unique opportunity to test theoretical models of decision-making in an ecological valid sample.
A fundamental, albeit complex, question is whether excitatory or inhibitory neurotransmitter levels are important for overcoming Pavlovian-instrumental conflicts when this is necessary for optimal decisions. In the present study, an individual Pavlovian bias parameter covaried with dACC Glx, suggesting that glutamatergic mechanisms in the prefrontal cortex may be of central importance for controlling Pavlovian influences. Interestingly, increasing the level of the glutamate transporter, GLT-1, which increases extracellular clearance of glutamate, impairs the activity of widespread neural circuits, and this is evident as a reduction in frontal and parietal theta power in EEG (Bellesi et al. Reference Bellesi, Vyazovskiy, Tononi, Cirelli and Conti2012). Frontal theta is presumed to be generated in mid-cingulate/dorsal anterior cingulate cortex (Van Veen & Carter, Reference Van Veen and Carter2002) and its oscillatory power is indicative of an ability to overcome a Pavlovian bias in the orthogonalized Go/No-go task (Cavanagh et al. Reference Cavanagh, Eisenberg, Guitart-Masip, Huys and Frank2013). Our finding that increasing levels of dACC Glx is associated with greater ability to overcome a Pavlovian bias across the groups adds to a literature on a previous association between prefrontal cortex activity and performance in the orthogonalized Go/No-go task (Guitart-Masip et al. Reference Guitart-Masip, Huys, Fuentemilla, Dayan, Duzel and Dolan2012; Cavanagh et al. Reference Cavanagh, Eisenberg, Guitart-Masip, Huys and Frank2013). Moreover, glutamatergic mechanisms may act to drive neuronal activity in prefrontal networks that ensure adequate suppression of Pavlovian influences when these conflict with optimal instrumental responses.
The notion that repeated stress disrupts prefrontal glutamatergic neurotransmission (Yuen et al. Reference Yuen, Wei, Liu, Zhong, Li and Yan2012), is supported by our finding of a significant reduction in dACC Glx levels in trauma survivors compared with controls. In contrast to the rapid increase in glutamate following acute stress, prolonged or extreme stress has been associated with decreased transmission efficiency and reduced glutamate levels, with detrimental effects on prefrontal-dependent cognitive processes (Yuan & Hou, Reference Yuan and Hou2015). Although work has been done to uncover the impact of such stress-mediated changes in glutamate on a number of cognitive tasks (Graybeal et al. Reference Graybeal, Kiselycznyk and Holmes2012), they do not address how altered glutamate following stress affects motivation and motivated behaviours per se (Hollon et al. Reference Hollon, Burgeno and Phillips2015). Based on the present findings, we could speculate that the reduction in dACC Glx made subjects less able to regulate Pavlovian influences on instrumental choices, predisposing to dysfunctional behaviours when the systems prompt different behavioural outputs.
In this study, we report data from a unique group of young traumatized individuals. However, there are important limitations we need to acknowledge, especially related to the trauma group. First, given the cross-sectional design of this study, causal interferences cannot be made. Moreover, the trauma group was of modest size and relatively heterogeneous, with a subset of the sample reporting mood- and anxiety disorders following the traumatic exposure. Such heterogeneity is difficult to avoid in these types of studies given the known variability in response to stressors, as well as potential biases when recruiting from the target population. Although we endeavoured to control for the two most common axis 1 disorders in our analyses, the findings await replication in a larger sample of young trauma survivors ideally without existing psychopathologies. In addition, future studies should attempt to obtain family history of psychopathology as well as a history of traumatic life events, to ensure groups are matched on all other relevant measures apart from the trauma of primary interest. However, because terror attacks generally strike randomly, it is unlikely that the trauma survivors differed from the general population with respect to risk for psychopathology or previous traumatic experiences (North et al. Reference North, Nixon, Shariat, Mallonee, McMillen, Spitznagel and Smith1999; North & Pfefferbaum, Reference North and Pfefferbaum2013). Furthermore, the finding of an association between time since trauma and the Pavlovian bias suggests that the behavioural effect was indeed directly related to the traumatic experience. Finally, it is important to note that the association between dACC Glx and the Pavlovian bias was modest. Additionally, we only provide indirect evidence for a relationship between stress-induced change in glutamatergic levels and increased Pavlovian control. It is also the case that other neuromodulators, such as dopamine, are implicated in controlling a balance between instrumental and Pavlovian mechanisms (Guitart-Masip et al. Reference Guitart-Masip, Economides, Huys, Frank, Chowdhury, Duzel, Dayan and Dolan2014b ). Accordingly, we consider future studies might usefully focus on a wider set of neurotransmitters and neuromodulators to uncover the neurochemical underpinnings of enhanced Pavlovian biases following traumatic stressful experiences.
Our findings support that traumatic stress influences the interaction of Pavlovian and instrumental mechanisms during learning of action choices. Specifically, in the aftermath of a traumatic stress experience, instrumental decisions are more susceptible to Pavlovian control, in line with a rich animal literature supporting an attenuation of prefrontal control and a concomitant strengthening of amygdala-dependent circuits (Arnsten, Reference Arnsten2015; McEwen et al. Reference McEwen, Bowles, Gray, Hill, Hunter, Karatsoreos and Nasca2015). Moreover, the present findings suggest that prefrontal glutamatergic mechanisms are important for overcoming this Pavlovian bias, such that disruption in glutamatergic signalling secondary to severe stress can render subjects more prone to Pavlovian influences on instrumental action selection. Stress may precipitate and influence a number of mental illnesses, thus a deeper understanding of how it impacts on cognition and behaviour represents an important step towards bridging the gap between stress exposure and onset of illness.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S003329171700174X
Acknowledgements
This work was supported by the Western Norway Regional Health Authority (O.T.O. #911780 and #911813); the Wellcome Trust (R.J.D. #098362/Z/12/Z) and the Swiss National Science Foundation (Q.J.M.H. #320030L_153449/1). The authors would like to thank Eva Øksnes, Roger Barndon, Turid I. Randa, Christel Jansen, Trond M. Øvreaas, Lena Matre, Silje Haukenes Stavestrand, Maja Holmeng and Kristine Lorentzen for assisting with data collection.
Declaration of Interest
Kenneth Hugdahl, Alex Craven and Lars Ersland have shares in the NordicNeuroLab Inc. which produces functional MRI equipment. They do not declare any conflict of interest. All other authors declare no competing financial interests.
