Introduction
Internet gaming disorder (IGD) is a serious mental health issue worldwide (Petry et al., Reference Petry, Rehbein, Gentile, Lemmens, Rumpf, Mößle and Borges2014). It was included in section three of the Diagnostic and Statistical Manual of Mental Disorder, 5th edition (DSM-5) as a disorder that deserves more research (APA, 2013). It is regarded as a type of behavioural addiction (i.e. syndromes analogous to substance addiction but with a behavioural focus, such as gambling disorder) (Potenza, Reference Potenza2015), characterised by impaired control over gaming, increasing priority given to gaming over other activities, and continuation or escalation of gaming despite the occurrence of negative consequences (https://www.who.int/features/qa/gaming-disorder/en/).
Accumulated evidence suggests that IGD shows typical features of addiction. For example, it shares similar symptoms with drug addictions and gambling disorder, such as preoccupation, tolerance, and withdrawal symptoms (Petry et al., Reference Petry, Rehbein, Gentile, Lemmens, Rumpf, Mößle and Borges2014). Besides, IGD is highly comorbid with depression and anxiety disorders (Gentile, Reference Gentile2009; Van Rooij et al., Reference Van Rooij, Kuss, Griffiths, Shorter, Schoenmakers and Van De Mheen2014), similarly to drug addictions (Conner, Pinquart, & Duberstein, Reference Conner, Pinquart and Duberstein2008). Yet, many pathophysiological aspects of IGD are still elusive.
There is considerable evidence for specific abnormalities of the dopaminergic system in addiction (Volkow, Koob, & McLellan, Reference Volkow, Koob and McLellan2016). According to the incentive sensitisation theory, one of the most cited theory of addiction, the excessive exposure to drugs ‘hijack’ the brain reward system towards drug-related cues due to acquired incentive salience (Berridge & Robinson, Reference Berridge and Robinson2016). As a result, the brain reward systems of addicted individuals become highly sensitive to addiction-related cues, but meanwhile, less sensitive to cues that are related to daily reward (Luijten, Schellekens, Kühn, Machielse, & Sescousse, Reference Luijten, Schellekens, Kühn, Machielse and Sescousse2017; Volkow et al., Reference Volkow, Koob and McLellan2016). The increased sensitivity towards addiction-related cues has been consistently reported in IGD using the cue-reactivity paradigm (Starcke, Antons, Trotzke, & Brand, Reference Starcke, Antons, Trotzke and Brand2018; Zheng et al., Reference Zheng, Hu, Wang, Wang, Du and Dong2019). For example, while viewing game-related stimuli, greater activities have been found in the nucleus accumbens (NAc), orbitofrontal cortex (OFC), dorsolateral prefrontal cortex (DLPFC), dorsal anterior cingulate cortex (dACC), ventral medial prefrontal cortex, and insula in IGD individuals relative to controls (Han et al., Reference Han, Bolo, Daniels, Arenella, Lyoo and Renshaw2011a; Han, Hwang, & Renshaw, Reference Han, Hwang and Renshaw2011b; Han, Kim, Lee, Min, & Renshaw, Reference Han, Kim, Lee, Min and Renshaw2010; Ko et al., Reference Ko, Liu, Hsiao, Yen, Yang, Lin and Chen2009, Reference Ko, Liu, Yen, Chen, Yen and Chen2013; Ma et al., Reference Ma, Worhunsky, Xu, Yip, Zhou, Zhang and Yao2019; Zhang et al., Reference Zhang, Lin, Zhou, Xu, Du and Dong2016a). Moreover, cue-induced activity in these brain regions is positively correlated with self-reported gaming urge in IGD individuals (Han et al., Reference Han, Kim, Lee, Min and Renshaw2010; Ko et al., Reference Ko, Liu, Hsiao, Yen, Yang, Lin and Chen2009, Reference Ko, Liu, Yen, Chen, Yen and Chen2013). On the other hand, blunted sensitivity to daily rewards have been observed in gambling disorder and various drug addictions as well (Fauth-Bühler, Mann, & Potenza, Reference Fauth-Bühler, Mann and Potenza2017; Linnet, Reference Linnet2014; Luijten et al., Reference Luijten, Schellekens, Kühn, Machielse and Sescousse2017). For example, individuals with gambling disorder showed decreased ventral striatum activity to winning (v. losing) monetary rewards (Reuter et al., Reference Reuter, Raedler, Rose, Hand, Gläscher and Büchel2005). Indeed, altered reward processing brought on by functional and structural changes in the mesocorticolimbic reward system was considered as a hallmark of gambling disorder in a recent review (Fauth-Bühler et al., Reference Fauth-Bühler, Mann and Potenza2017). Finally, a recent electrophysiological study reported decreased feedback-related negativity amplitudes following reward gains in adolescents with IGD relative to controls, indicating a blunted sensitivity to daily rewards (Li et al., Reference Li, Wang, Yang, Dai, Zheng, Sun and Liu2020).
Blunted sensitivity to daily rewards could potentially impair one's reinforcement learning through diminishing the role of daily rewards as a reinforcer of action selection (Pizzagalli et al., Reference Pizzagalli, Evins, Schetter, Frank, Pajtas, Santesso and Culhane2008), thereby making it difficult for addicted individuals to derive pleasure from everyday life, e.g. to cultivate a hobby or maintain a relationship (Allison, Von Wahlde, Shockley, & Gabbard, Reference Allison, Von Wahlde, Shockley and Gabbard2006). The impaired reinforcement learning of daily rewards hence represents a crucial aspect of the pathology of addiction that bears both practical and theoretical importance (Volkow et al., Reference Volkow, Koob and McLellan2016). The blunted sensitivity to daily rewards may contribute to reinforcement learning impairment in addictions via the reward prediction error (RPE) signals of the dopaminergic neurons (García-García, Zeighami, & Dagher, Reference García-García, Zeighami and Dagher2017; Keiflin & Janak, Reference Keiflin and Janak2015). RPE signal reflects a discrepancy between the predicted and the actual outcome, which serves as a teaching signal of reinforcement learning (Schultz, Reference Schultz1998). Functional magnetic resonance imaging (fMRI) studies have found enhanced striatal activity in response to an unexpected reward (positive RPE, +RPE) and decreased activity in these regions when an expected reward is omitted (negative RPE, −RPE) (Morris et al., Reference Morris, Vercammen, Lenroot, Moore, Langton, Short and Weickert2012). Despite the rich literature on disrupted reinforcement learning in drug addictions and gambling disorder (Deserno et al., Reference Deserno, Beck, Huys, Lorenz, Buchert, Buchholz and Heinze2015; Park et al., Reference Park, Kahnt, Beck, Cohen, Dolan, Wrase and Heinz2010; Parvaz et al., Reference Parvaz, Konova, Proudfit, Dunning, Malaker, Moeller and Goldstein2015; Rose et al., Reference Rose, Ross, Salmeron, Lee, Shakleya, Huestis and Stein2012, Reference Rose, Salmeron, Ross, Waltz, Schweitzer, McClure and Stein2014; Tanabe et al., Reference Tanabe, Reynolds, Krmpotich, Claus, Thompson, Du and Banich2013; Tau et al., Reference Tau, Marsh, Wang, Torres-Sanchez, Graniello, Hao and Kangarlu2014; Wiehler & Peters, Reference Wiehler and Peters2020), however, only a handful of previous studies have examined RPE signals towards daily rewards in drug addictions. Blunted RPE signals in the striatum and frontal areas have been documented in polysubstance dependence (Tanabe et al., Reference Tanabe, Reynolds, Krmpotich, Claus, Thompson, Du and Banich2013), nicotine addiction (Rose et al., Reference Rose, Ross, Salmeron, Lee, Shakleya, Huestis and Stein2012), and cocaine users (Parvaz et al., Reference Parvaz, Konova, Proudfit, Dunning, Malaker, Moeller and Goldstein2015; Rose et al., Reference Rose, Salmeron, Ross, Waltz, Schweitzer, McClure and Stein2014). Two studies reported intact RPE signals in abstinent alcohol-dependent individuals (Deserno et al., Reference Deserno, Beck, Huys, Lorenz, Buchert, Buchholz and Heinze2015; Park et al., Reference Park, Kahnt, Beck, Cohen, Dolan, Wrase and Heinz2010), but ventral striatal RPEs were correlated inversely with craving in patients (Deserno et al., Reference Deserno, Beck, Huys, Lorenz, Buchert, Buchholz and Heinze2015). Overall, these findings highlighted that blunted RPE signals could be a characteristic feature of individuals with drug addictions. To our knowledge, no previous study has examined whether RPE signals are blunted in behavioural addictions.
The study aimed to assess the putative alterations of the RPE signal in IGD using task-based fMRI. We hypothesised that IGD would exhibit blunted RPE signals in regions of the brain reward system when compared with controls.
Materials and methods
Participants
Ninety-seven young male adults were recruited for this study. Ten participants were excluded due to quality control (one had >20% missed trials, three had an accuracy of <70% in the task, and six had excessive head motion during scanning, see below for details). Data of the remaining 87 participants, including 45 individuals with IGD and 42 healthy controls (HCs) were analysed (Table 1). The diagnosis of IGD was made based on the DSM-5 criteria by qualified psychiatrists (KZL and XML). An individual was deemed as IGD if he met at least five of the nine diagnosis criteria proposed in DSM-5. Participants were excluded if they reported lifetime substance dependence (including nicotine), current substance abuse, mood disorders, anxiety disorders, schizophrenia spectrum disorders, and neurological disorders. All subjects were Han Chinese and right-handed. Participants provided written consent and received monetary compensation depending on their task performance. The study was approved by the Institutional Review Board of the Southwest Medical University and was conducted in accordance with the latest revision of the Declaration of Helsinki.
Sample demographic information
IGD, internet gaming disorder; HC, healthy controls; d, Cohen's d value; IGDS, Internet Gaming Disorder Scale; DGH-LY, daily gaming hour in the past year; DGH-EL, daily gaming hours for the entire life (since gaming started); BIS-11, Barratt Impulsiveness Scale, 11th version; IAT, internet addiction test; BAI, Beck Anxiety Inventory; BDI, Beck Depression Inventory; RSPM, Raven's Standard Progressive Matrices; Accuracy-12, decision accuracy of the first 12 trials; Accuracy-all, decision accuracy of all trials, except for the first 12 trials.
***p < 0.001.
Clinical measures
To assess the severity of IGD, participants completed the 9-item dichotomous Internet Gaming Disorder Scale (IGDS) (Lei et al., Reference Lei, Liu, Zeng, Liang, Huang, Gong and Chen2020) and the Internet Addiction Test (IAT) (Young, Reference Young1998). Additional information such as age of start gaming, daily gaming hours in the past year, daily gaming hours since they started playing games, and craving for gaming right before fMRI scan (via a visual analogue scale from 0 = not at all to 10 = extremely) were collected. Impulsivity score was acquired using the Barratt Impulsiveness Scale, 11th version (BIS-11) (Patton, Stanford, & Barratt, Reference Patton, Stanford and Barratt1995). Self-esteem scale (Rosenberg, Reference Rosenberg1986), general self-efficacy scale (Bandura, Reference Bandura1977), Beck Anxiety Inventory (BAI) (Beck, Epstein, Brown, & Steer, Reference Beck, Epstein, Brown and Steer1988a), and the Beck Depression Inventory (BDI) (Beck, Steer, & Carbin, Reference Beck, Steer and Carbin1988b) scores were collected. Finally, the general cognitive ability of participants was assessed using Raven's Standard Progressive Matrices (Raven, Reference Raven2000).
Reward-related prediction-error task
We adopted a reward-related prediction-error task (Morris et al., Reference Morris, Vercammen, Lenroot, Moore, Langton, Short and Weickert2012) (Fig. 1). The paradigm first establishes an association between a ‘trump card’ (one of four distinct cards) and the reward stimulus through training. During the training, participants were instructed to decide whether the card-pair is rewarding or not, by pressing one of two buttons. Since a card-pair with a trump card was always linked to a reward during training, the judgement could be made based on whether the card-pair had a trump card. To establish associations between the trump card and the reward, each participant was trained until there were six consecutive correct responses prior to the scanning session.
Examples of trial types in the reward-related prediction-error task. Four different card stimuli were used (diamonds, triangles, squares, and circles) in which one card was ‘trump card’ (diamonds in this example). Each trial starts with a crosshair fixation (1000–5000 ms), then a card-pair was presented for 2000 ms. Participants were asked to respond by pressing a button on the presentation of the card-pair. Finally, after a 2000–4000 ms inter-stimuli-interval the feedback stimulus was presented for 2000 ms. The feedback was of two-fold, reward stimulus was either presented (an image of ten ¥100 RMB notes) or omitted (an image of grey noise), and predictions were either correct or incorrect (Chinese characters underneath the reward stimulus, ‘正确’ means correct, ‘错误’ means incorrect). ER, expected reward; EO, expected reward omission; UO, unexpected reward omission; UR, unexpected reward.
The scanning session was identical to the training, with the only difference being that the scan session contains a small proportion of unexpected trials, in which the trump card was not linked to a reward as was trained to induce prediction errors. By manipulating the combinations of the cue (with trump card v. without trump card) and outcome (reward v. reward omission), four conditions of reward were introduced: Expected Reward (ER), Expected reward Omission (EO), Unexpected Reward (UR), and Unexpected reward Omission (UO). A total of 120 trials were presented in two runs of 60 trials during the scan session, of which 96 trials were expected trials (ER = 48 trials, EO = 48 trials) and 24 trials were unexpected trials (UR = 12 trials and UO = 12 trials).
The first 12 trials in the scanning session were consistent with training to reinforce learned associations. The rest of the trials were randomly arranged. The identity of the trump card was counterbalanced between diamonds, squares, circles, and triangles. Participants were required to respond within 2 s on the presentation of the card-pair. Otherwise, the trial would be labelled as a missing trial and be excluded in the analysis. One IGD participant failed to respond in more than 20% of the trials and was excluded. Of note, every prediction which was incongruent with the outcome was included as an unexpected event (UR or UO). Thus, the obtained number of trials in each condition differed for each participant according to their response history.
fMRI acquisition
fMRI data were collected using a Phillips Achieva 3T scanner with a 16 channels head coil using echo-planar images sequence: slice thickness = 3 mm, gap = 1 mm, 38 axial slices in interleave order, repetition time = 2000 ms, echo time = 30 ms, flip angle = 90°, matrix = 64 × 64, field of view = 240 mm, inplane resolution = 3.75 mm × 3.75 mm. A T1-weighted high-resolution anatomical scan was obtained for each participant for registration and screening: repetition time = 7.6 ms, echo time = 3.8 ms, field of view = 256 mm, matrix = 256 × 256, inplane resolution = 1 mm × 1 mm, slice thickness = 1 mm with no gap, 170 sagittal slices.
Data analysis
Clinical and behavioural data
Comparisons of demographic and clinical data were conducted using two-sample t tests. The decision accuracy in the first 12 trials was calculated to confirm the effectiveness of training. The accuracy of all trials excluding the first 12 trials was also calculated for each participant. Participants with an accuracy of <70% in either of these two accuracy measurements (two IGD and one HC) were excluded.
fMRI data
fMRI data were processed using SPM12 (http://www.fil.ion.ucl.ac.uk/spm). Pre-processing steps included the following: discard the first five volumes to achieve magnet-steady images; slice timing; realignment; co-registration of T1-weighted image; segmentation of T1-weighted images; spatially normalisation to the Montreal Neurological Institute (MNI) space using the mapping parameters derived from segmentation of T1-weighted images and interpolated to 3 mm × 3 mm × 3 mm voxels; and spatial smooth using an 8 mm Gaussian kernel. Participants showing excessive head movement (translational movement >3 mm or rotation >3°) during fMRI scanning (three IGD and three HC) were excluded.
In the first-level analysis, six regressors were defined. Four regressors of interest were time locked to the onset of the four types of feedback stimuli (i.e. ER, UR, EO, and UO). Two regressors of no interest were corresponding to the onset of the card-pair stimuli representing the response of the participants in anticipation of whether the card-pair would result in a reward (i.e. reward and non-reward). The six head movement parameters were included as covariates. The resultant design matrix was convolved with a canonical haemodynamic response function in the context of the general linear model. The time-series was high-pass filtered to 1/128 Hz and modelled for temporal autocorrelation across scans with an AR(1) process. Three paired t tests were specified in the first-level analysis. The contrast of UR > ER was tested to identify regions activated related to +RPE. The contrast of EO > UO was tested to identify regions activated related to −RPE. We also tested the reward effect (i.e. reward v. reward omission) using the contrast of ER + UR − EO − UO.
In the second-level analysis, RPE signals were first identified in the HC and the IGD groups separately using one-sample t tests and then compared between groups using two-sample t tests. The case-control difference in reward effect was identified using two-sample t tests. Age was included as a covariate. A grey matter mask, derived from SPM prior template (TPM.nii) by setting the threshold at >0.2, was used to restrict the analysis within the grey matter area. The results were evaluated at a threshold of cluster-level family-wise error (FWE) corrected p < 0.05 using small volume correction (SVC) on images with a threshold of uncorrected voxel-level p < 0.001 and cluster size ⩾20 voxels. As we were interested in RPE signal alterations in the dopaminergic brain reward system, we identified the group-level difference restricted to the masked areas of this circuit. Based on previous literature, the brain reward system was composed of the ventral tegmental area (VTA), substantia nigra (SN), NAc, dACC, anterior insula, OFC, caudate, putamen, DLPFC, medial prefrontal cortex, amygdala, thalamus, and hippocampus (Haber & Knutson, Reference Haber and Knutson2010; Makris et al., Reference Makris, Oscar-Berman, Jaffin, Hodge, Kennedy, Caviness and Harris2008; Volkow, Wise, & Baler, Reference Volkow, Wise and Baler2017). The mask of DLPFC was confined to the combined atlas of Brodmann areas BA46 and BA9 (Cieslik et al., Reference Cieslik, Zilles, Caspers, Roski, Kellermann, Jakobs and Eickhoff2013) whereas the mask of the medial prefrontal cortex was confined to the combined atlas of BA10 and BA32 (Haber & Knutson, Reference Haber and Knutson2010). The remaining regions were derived directly from the Automated Anatomical Labelling atlas 3 (Rolls, Huang, Lin, Feng, & Joliot, Reference Rolls, Huang, Lin, Feng and Joliot2020). Each cluster was tested separately in SVC. For completeness, we also reported exploratory whole-brain results with a threshold set at the uncorrected voxel level p < 0.001 and cluster size ⩾20 voxels in Table 2. Original t statistical maps from all comparisons, both thresholded and unthresholded, can be found in neurovault.org (https://neurovault.org/collections/8988/).
Results of whole-brain case-control comparisons and PPI analysis
Threshold at voxel-level p < 0.001 uncorrected and cluster size ⩾20 voxels. +RPE, positive reward prediction error; −RPE, negative reward prediction error; Coordinates, Montreal Neurological Institute coordinates of x, y, z; SVC, FWE small volume corrected p value; L, left; R, right; OFC, orbitofrontal cortex; DLPFC, dorsolateral prefrontal cortex; dACC, dorsal anterior cingulate cortex.
Each cluster that showed significant RPE signal alterations in IGD in the whole-brain case-control comparisons (see Table 2) was defined as a region of interest (ROI) for further ROI-wise analysis. Repeated measures analysis of variance (ANOVA) with extracted beta values of ROIs were carried out, with group (IGD v. HC) as the between-subject variable and conditions (‘UR v. ER’ or ‘UO v. EO’) as the within-subject variable. Correlation analyses were performed to assess the clinical significance of the regional RPE alterations. For each ROI and each participant, the +RPE signals were computed as BetaUR − BetaER, and −RPE signals were computed as BetaEO − BetaUO. Correlations between RPE signals of ROIs and clinical measures (including IGDS, IAT, BIS-11, age of start gaming, daily gaming hours in the past year, daily gaming hour for the entire life, and craving forgaming) were performed separately in the IGD group and the HC group using Spearman's rho.
Psychophysiological interaction (PPI) analysis
To explore the potential changes of task modulated functional connectivity related to RPE signal alterations, a PPI analysis was performed using the gPPI toolbox (McLaren, Ries, Xu, & Johnson, Reference McLaren, Ries, Xu and Johnson2012). The volume of interest (VOI) was defined based on the results of case-control comparisons (see Table 2) as spheres with 6 mm radius centred around the peak voxel of each cluster. These VOIs were selected based on the case-control comparisons of the current study, instead of referring to previous study findings or anatomical masks, so we could explore the connectivities that related to the voxels where IGD showed disrupted RPE signalling. PPI analysis was carried out in each subject for each VOI, with the interested contrast (UR v. ER) as a psychological variable. PPI connectivity alterations in IGD, relative to controls, were identified using two-sample t tests. The results were based on a threshold set at SVC corrected p < 0.05 on images with a threshold of uncorrected voxel-level p < 0.001 and cluster size ⩾20 voxels.
Results
Clinical and behavioural data
The IGD group had similar age, education, general cognitive ability, self-esteem, self-efficacy, age of start gaming, and anxiety level in comparison with the HC group (Table 1). IGD participants had more severe gaming problems than HCs, IGD had significantly higher scores on IGDS and IAT, higher craving level, longer daily gaming hours in the past year, and longer daily gaming hours since gaming started (p < 0.001). Additionally, IGD participants were marginally more depressed than HCs (p = 0.072). In the scan session, both groups had similarly high accuracy in the first 12 trials (96% v. 96%) and the rest of the trials (98% v. 98%), suggesting that participants learned the correct association between the trump card and reward during training, and applied the same association throughout the scanning session.
RPE maps in IGD and HC
The +RPE (UR > ER) maps in the HC group showed activations in the SN/VTA, bilateral frontal operculum/anterior insula, DLPFC, dACC/medial prefrontal cortex, caudate, and other regions (online Supplementary Table S1 and Fig. S1). In addition, negative activations (ER > UR) are shown in online Supplementary Table S1 and Fig. S1. The −RPE (EO < UO) maps of HCs revealed activities in most of the same regions (online Supplementary Table S2 and Fig. S1). The RPE maps of IGD were similar to those in HCs, except for the peak heights and size of clusters were smaller (online Supplementary Tables S3, S4 and Fig. S1). These results were consistent with the previous meta-analysis of RPE maps in healthy subjects (Garrison, Erdeniz, & Done, Reference Garrison, Erdeniz and Done2013).
Case-control comparisons
When compared with HCs, participants with IGD showed blunted +RPE signal in the right caudate, left OFC, and right DLPFC (Table 2, Fig. 2a). The case-control comparisons of −RPE maps and reward effect revealed no significant differences.
Regions showing blunted +RPE signals in IGD. (a) Compare with HCs, participants with IGD showed blunted +RPE signals in the right caudate, left OFC, and right DLPFC. The threshold set at voxel-level p < 0.001, cluster-level small volume corrected p < 0.05 (areas circled by green lines indicate anatomical masks used in SVC). Colour bar represents the voxel-level t values. (b) The mean beta values of each ROI were extracted and illustrated as bar charts. Error bars represent standard error. p values represent the statistical significances in pairwise comparisons. (c) correlations between +RPE signals in the left OFC and IGDS scores in the IGD (left panel) and HC group (right panel). L, left; R, right; OFC, orbitofrontal cortex; DLPFC, dorsolateral prefrontal cortex; IGDS, internet gaming disorder scale; n.s., non-significant.
The right caudate, left OFC, and right DLPFC were defined as ROIs for further ROI-wise analyses. Repeated measures ANOVA with extracted beta values of ROIs confirmed significant group (HC and IGD) by conditions (ER and UR) interaction in all three ROIs (p < 0.001, online Supplementary Table S5). Pairwise comparisons indicated that while the HC group showed significantly higher activity under the UR condition than under ER condition in ROIs (p < 0.001), IGD group showed no (in the right caudate and left OFC) or much less (in the right DLPFC) of such difference between conditions (Fig. 2b, online Supplementary Table S5).
The correlational analysis revealed a positive correlation between the +RPE signals of left OFC and IGDS scores in the IGD group (ρ = 0.50, p < 0.001 Bonferroni corrected) but not in the HC group (ρ = 0.24, p = 0.121 uncorrected) (Figure 2c). Correlations between the +RPE signals of right DLPFC and right caudate and clinical measures were not significant in both groups (online Supplementary Table S6).
PPI connectivity
The PPI analyses revealed a pattern of hyperconnectivity among regions of the brain reward system in IGD (Table 2). Compared with HCs, the IGD group showed higher connectivity in the right putamen, right DLPFC, and right dACC to the right DLPFC VOI, and higher connectivity in the left DLPFC to the right caudate VOI (Fig. 3). Single group analysis suggests a pattern of hyperconnectivity mainly derived from increased connectivities between the right DLPFC VOI to the other regions of the brain reward system, including bilateral caudate, right dACC, and right putamen, in IGD (online Supplementary Table S7).
PPI results. For each VOI (green spheres on the left side), participants with IGD showed hyperconnectivity to the VOI among regions of the brain reward system. Threshold at voxel-level p < 0.001, cluster-level small volume corrected p < 0.05 (areas circled by green lines indicate anatomical masks used in SVC). Colour bar represents the voxel-level t values. L, left; R, right; DLPFC, dorsolateral prefrontal cortex; dACC, dorsal anterior cingulate cortex.
To explore how these increased PPI connectivities contributed to the RPE signal alterations in IGD, we conducted an additional linear regression analysis in the IGD group. The regression models were set up for each ROI separately, with the extracted regional connectivity strengths as the independent variables and the +RPE signals of ROIs as the dependent variable. Linear regression showed that the connection between the right DLPFC and right dACC could significantly predict the variation of +RPE signals in left OFC (β = 0.52, p < 0.001, adjusted r 2 = 0.26).
Discussion
Using a reward-related prediction-error task, we investigated neural mechanisms underlying reward learning in HCs and IGD individuals. Three central findings emerged. First, relative to HCs, IGD participants were characterised by blunted +RPE signals in the right caudate, left OFC, and right DLPFC. Second, the PPI analysis revealed a pattern of hyperconnectivity between the right caudate, right putamen, bilateral DLPFC, and dACC in the IGD group. Finally, linear regression analysis showed that the connection between the right DLPFC and right dACC significantly predicts +RPE signal variations in the left OFC. Collectively, these findings highlight neural evidence of disrupted reinforcement learning in IGD.
The current study revealed blunted +RPE signals towards daily rewards in participants with IGD. Blunted +RPE signals in IGD have been found in multiple regions of the brain reward system, including the right caudate, left OFC, and right DLPFC.
The caudate is part of the dorsal striatum, which is involved in reward-related processing and the development of addiction (Wise, Reference Wise2009). RPE signals have been reported to encode dorsal and ventral striatum in two meta-analyses using instrumental and Pavlovian reinforcement learning tasks (D'Astolfo & Rief, Reference D’Astolfo and Rief2017; Garrison et al., Reference Garrison, Erdeniz and Done2013). In line with our results, reduced RPE signals in the caudate were reported in nicotine dependence (Rose et al., Reference Rose, Ross, Salmeron, Lee, Shakleya, Huestis and Stein2012). It has been proposed that the caudate is involved in evaluating post-decision outcomes, through interpretive monitoring of ongoing cognitive processing and error monitoring (Packard & Knowlton, Reference Packard and Knowlton2002; Yanike & Ferrera, Reference Yanike and Ferrera2014). Nagai et al. (Reference Nagai, Kikuchi, Lerchner, Inoue, Ji, Eldridge and Hori2016) demonstrated that silencing bilateral caudate could produce a significant and reproducible loss of sensitivity to the value of reward in monkeys. The decreased +RPE signals in caudate thus could reflect a generally blunted sensitivity to value changes to daily reward in IGD.
Reduced RPE signal in the OFC finding was consistent with a previous study in individuals with polysubstance dependence (Tanabe et al., Reference Tanabe, Reynolds, Krmpotich, Claus, Thompson, Du and Banich2013). The OFC is a neural substrate linked with positive hedonic feelings associated with receipt of rewards and encoding of the reward value (O'Doherty, Reference O'Doherty2004). The reduced RPE signals in the OFC and caudate suggested that neural representation of a change in reward value upon receipt of unexpected reward was impaired in IGD.
Blunted +RPE signals in DLPFC of participants with IGD was in line with previous findings suggesting that the DLPFC activates when expectations are violated, and also the activity correlated with prediction-error magnitude (Corlett et al., Reference Corlett, Aitken, Dickinson, Shanks, Honey, Honey and Fletcher2004; Fletcher et al., Reference Fletcher, Anderson, Shanks, Honey, Carpenter, Donovan and Bullmore2001; Tobler, O'Doherty, Dolan, & Schultz, Reference Tobler, O'Doherty, Dolan and Schultz2007). DLPFC has been linked with salience attribution, impulsivity, and motivation (Goldstein & Volkow, Reference Goldstein and Volkow2011). Dysfunction of these regions may manifest in abnormalities in motivational salience, such as craving (Steinberg et al., Reference Steinberg, Keiflin, Boivin, Witten, Deisseroth and Janak2013). Indeed, increased activations in DLPFC induced by game cues were correlated with the urge to play games in young male gamers (Ko et al., Reference Ko, Liu, Hsiao, Yen, Yang, Lin and Chen2009). Hayashi, Ko, Strafella, and Dagher (Reference Hayashi, Ko, Strafella and Dagher2013) found that the DLPFC attenuates the subjective craving-related signal in the medial OFC, and blunted DLPFC reduces craving-related signals in the ACC and ventral striatum. Thus, the blunted +RPE signal in DLPFC could reflect a deficit in salience attribution for daily reward in IGD.
Taking together, the finding of blunted RPE signals in IGD was in line with previous findings of blunted fronto-striatal RPE signalling in polysubstance dependence (Tanabe et al., Reference Tanabe, Reynolds, Krmpotich, Claus, Thompson, Du and Banich2013), smokers (Rose et al., Reference Rose, Ross, Salmeron, Lee, Shakleya, Huestis and Stein2012) and cocaine users (Parvaz et al., Reference Parvaz, Konova, Proudfit, Dunning, Malaker, Moeller and Goldstein2015; Rose et al., Reference Rose, Salmeron, Ross, Waltz, Schweitzer, McClure and Stein2014). Also, our findings were consistent with previous studies reporting disrupted reinforcement learning in drug addictions and gambling disorder (Deserno et al., Reference Deserno, Beck, Huys, Lorenz, Buchert, Buchholz and Heinze2015; Park et al., Reference Park, Kahnt, Beck, Cohen, Dolan, Wrase and Heinz2010; Parvaz et al., Reference Parvaz, Konova, Proudfit, Dunning, Malaker, Moeller and Goldstein2015; Rose et al., Reference Rose, Ross, Salmeron, Lee, Shakleya, Huestis and Stein2012, Reference Rose, Salmeron, Ross, Waltz, Schweitzer, McClure and Stein2014; Tanabe et al., Reference Tanabe, Reynolds, Krmpotich, Claus, Thompson, Du and Banich2013; Tau et al., Reference Tau, Marsh, Wang, Torres-Sanchez, Graniello, Hao and Kangarlu2014; Wiehler & Peters, Reference Wiehler and Peters2020). This suggests that IGD, similarly to other types of drug and behavioural addictions, shows disrupted reinforcement learning towards daily rewards. Notably, blunted fronto-striatal RPE signals have also been reported in patients with depression, and were associated with anhedonia (Kumar et al., Reference Kumar, Goer, Murray, Dillon, Beltzer, Cohen and Pizzagalli2018; Rothkirch, Tonn, Sterzer, Köhler, & Rothkirch, Reference Rothkirch, Tonn, Sterzer, Köhler and Rothkirch2017). This might suggest an underlying comorbidity between IGD and depression (Van Rooij et al., Reference Van Rooij, Kuss, Griffiths, Shorter, Schoenmakers and Van De Mheen2014). However, in the current study, the activation elicited by receipt of rewards was intact in IGD, which was not consistent with a recent meta-analysis reporting opposing abnormalities in the reward system of depression: decreased in the striatum and enhanced in the OFC (Ng, Alloy, & Smith, Reference Ng, Alloy and Smith2019), indicating that future studies of these two disorders are needed.
There are currently not enough research findings to differentiate RPE signals specific in IGD in comparison with other types of addictions or other psychiatric disorders. However, studies on polysubstance dependence (Tanabe et al., Reference Tanabe, Reynolds, Krmpotich, Claus, Thompson, Du and Banich2013) and nicotine addiction (Rose et al., Reference Rose, Ross, Salmeron, Lee, Shakleya, Huestis and Stein2012) have reported blunted RPE signals in the ventral striatum, which is not shown in IGD. These results indicated that the involvement of dorsal and ventral striatum might differ in IGD in comparison with drug addictions.
The positive correlation between the +RPE signals of left OFC and IGDS scores in the IGD group was unexpected, given that +RPE signals were decreased in left OFC and no sign of increased +RPE (or −RPE) signals have been found across the whole brain in IGD. These results may suggest an addiction-induced shift from voluntary goal-directed control to habitual control over reinforcement learning in IGD individuals (Everitt & Robbins, Reference Everitt and Robbins2005; Lüscher, Robbins, & Everitt, Reference Lüscher, Robbins and Everitt2020), given the role of OFC in goal-directed control of drug-seeking behaviours (Parkes et al., Reference Parkes, Ravassard, Cerpa, Wolff, Ferreira and Coutureau2018). Further studies using paradigms that can differentiate RPE signals provoked by these two control systems should reveal more insights into this issue (e.g. Daw, Gershman, Seymour, Dayan, & Dolan, Reference Daw, Gershman, Seymour, Dayan and Dolan2011).
The PPI analyses revealed a pattern of hyperconnectivity in brain-related reward system regions including the right caudate, right putamen, bilateral DLPFC, and dACC in participants with IGD. Given that the PPI connectivity is very similar to what has been observed in resting state (Cole, Bassett, Power, Braver, & Petersen, Reference Cole, Bassett, Power, Braver and Petersen2014; Di, Huang, & Biswal, Reference Di, Huang and Biswal2017), these results were partly in line with a recent study reporting hyperconnectivity between the anterior insula, ACC, putamen, angular gyrus, and precuneus, among individuals with IGD (Zhang et al., Reference Zhang, Yao, Li, Zang, Shen, Liu and Fang2016b), and hyperconnectivity among midbrain-striatal dopaminergic circuits in cannabis dependence (Manza, Tomasi, & Volkow, Reference Manza, Tomasi and Volkow2018). Notably, the PPI connectivity maps of the right DLPFC in the current study are not consistent with a recent meta-analysis, where DLPFC reliably connected to the posterior cingulate cortex and amygdala (Smith, Gseir, Speer, & Delgado, Reference Smith, Gseir, Speer and Delgado2016). This inconsistency may be attributable to differences in the location of VOIs and task paradigms.
More importantly, the linear regression analysis showed that the DLPFC-dACC connection contributed to +RPE signal variations in the left OFC. The DLPFC-dACC connection was previously shown to be involved in the effortful control of distress and conflict (Puetz et al., Reference Puetz, Kohn, Dahmen, Zvyagintsev, Schüppen, Schultz and Konrad2014). It has been proposed that the dorsal part of the prefrontal cortex (including DLPFC and dACC) is involved in inhibitory control of addictive behaviours via suppressing responses triggered by drug-cues (Goldstein & Volkow, Reference Goldstein and Volkow2011). This result thus suggested that the impaired RPE signalling in the left OFC might associated with disrupted top-down control of gaming.
Limitations of this study should be noted. First, the study recruited only males with IGD and thus results may not generalise to female populations. Second, it should be noted that the paradigm used in the current study adopted a qualitative definition of RPE, where the expected value of options was fixed during the test and no learning-related trial-to-trial fluctuations were expected. Notably, this is distinct from standard reward learning tasks (e.g. Daw et al., Reference Daw, Gershman, Seymour, Dayan and Dolan2011) or paradigms recommended in the Research Domain Criteria (Carcone & Ruocco, Reference Carcone and Ruocco2017). Further studies with quantitative modelling of these variables should provide more details on RPE alterations in IGD. Third, this study focused on the RPE signal alterations in the brain reward system by using SVC, leaving the effects in other brain regions (in our case the decreased +RPE signals in right precuneus, see Table 2) unattended. We have carried out additional ROI-wise analysis with extracted beta values of the right precuneus cluster, which revealed similar patterns as the other three ROIs and no significant brain-symptom correlations (data not shown). Finally, RPE signals were measured cross-sectionally in the current study, making it difficult to make causal inferences regarding changes in the brain reward system and addiction. Longitudinal studies are needed to understand the changes of RPE signals that linked with IGD.
In conclusion, we found that IGD is characterised by blunted RPE signals in response to daily rewards, as well as hyperconnectivity among regions of the brain reward system. These findings highlight important reinforcement learning deficits in IGD and the underlying pathophysiology.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S003329172000402X
Financial support
This research was supported by the National Science Foundation of China (81701322); Humanity and Social Science Youth Foundation of Ministry of Education of China (17YJC190009 and 17YJC190001); National undergraduate innovation and entrepreneurship training programme (201910632014); Nuclear Medicine and Molecular Imaging Key Laboratory of Sichuan Province (HYX18013); and Luzhou Science and Technology Bureau & Southwest Medical University (2018LZXNYD-ZK03 and 2019LZXNYDJ39). The funding agencies did not contribute to the experimental design or conclusions, and the views presented in the manuscript are those of the authors and may not reflect those of the funding agencies.
Conflict of interest
ST has received unrestricted educational grants from Indivior, Lundbeck Foundation, and Merck Serono.
