INTRODUCTION
Measures of decision-making (DM) have been commonly used with various populations. However, few studies have reported detailed psychometric properties of these tasks, including how they might relate to each other, particularly in adolescent samples. Some studies have found overlap in the constructs measured by these tasks (Buelow & Blaine, Reference Buelow and Blaine2015; Monterosso et al., Reference Monterosso, Ehrman, Napier, O’Brien and Childress2001). Others suggest that between-task correlations may vary with population characteristics, or that different DM tasks may capture different aspects of this complex construct (Brown et al., Reference Brown, Hack, Gold, Carpenter, Fischer, Prentice and Waltz2015; Buelow & Blaine, Reference Buelow and Blaine2015). Neuroimaging studies support this conclusion, as the performance of different DM tasks activates different brain networks (Labudda et al., Reference Labudda, Woermann, Mertens, Pohlmann-Eden, Markowitsch and Brand2008; Li et al., Reference Li, Lu, D’Argembeau, Ng and Bechara2010; Xue et al., Reference Xue, Lu, Levin, Weller, Li and Bechara2009). For example, DM during the Game of Dice Task (GDT) has elicited activation of the dorsolateral prefrontal cortex, anterior cingulate cortex, and inferior parietal lobule (Labudda et al., Reference Labudda, Woermann, Mertens, Pohlmann-Eden, Markowitsch and Brand2008). DM during the Iowa Gambling Task (IGT), in contrast, has been associated with increased activation in dorsolateral prefrontal cortex, insula, anterior and posterior cingulate cortex, orbitofrontal and ventromedial prefrontal cortex, and ventral striatum (Li et al., Reference Li, Lu, D’Argembeau, Ng and Bechara2010). An fMRI study employing the Cups Task has found that all risky choices elicited activation in the dorsomedial prefrontal cortex, whereas the gain and loss domains led to differential activation in the ventromedial prefrontal cortex and nucleus accumbens (Xue et al., Reference Xue, Lu, Levin, Weller, Li and Bechara2009). Thus, although research suggests overlap in the frontal areas engaged during these tasks, there is also variability in the involvement of other regions.
Studies have shown that a variety of participant characteristics or behaviors can impact DM performance. For instance, sex differences in DM are well documented (Shulman et al., Reference Shulman, Harden, Chein and Steinberg2015; van den Bos et al., Reference van den Bos, Homberg and de Visser2013). Participant age can also influence DM because the neural circuits underlying executive functions follow a pattern of protracted development during adolescence, causing younger participants to make riskier choices (Casey et al., Reference Casey, Jones and Hare2008). Additionally, cognitive factors such as intelligence have been shown (albeit inconsistently) to moderate DM task performance (Brand et al., Reference Brand, Laier, Pawlikowski and Markowitsch2009; Toplak et al., Reference Toplak, Sorge, Benoit, West and Stanovich2010). Finally, both substance abuse (Bechara & Damasio, Reference Bechara and Damasio2002; Bechara et al., Reference Bechara, Dolan and Hindes2002; Brand et al., Reference Brand, Roth-Bauer, Driessen and Markowitsch2008; Brevers et al., Reference Brevers, Bechara, Cleeremans, Kornreich, Verbanck and Noël2014) and use of specific substances such as cannabis (Churchwell et al., Reference Churchwell, Lopez-Larson and Yurgelun-Todd2010; Gonzalez et al., Reference Gonzalez, Schuster, Mermelstein and Diviak2015) have been linked to poorer DM. It is therefore important to take relevant characteristics into account when assessing DM in a given population.
Because these tasks are increasingly used in both research and clinical settings, further work is needed to determine whether measures of DM assess the same underlying construct, even when measured among participants with different characteristics. The current study aimed (1) to determine whether three commonly used DM tasks – the Cups Task, GDT, and IGT – load onto the same factor by creating a latent construct of DM; and (2) to examine whether this latent construct and its individual components are commensurate across a set of theoretically relevant covariates using moderated nonlinear factor analysis (MNLFA). MNLFA allows for a simultaneous assessment of measurement invariance and differential item functioning (DIF) across variables reflecting individual differences. Because these three DM tasks involve maximizing monetary gains and minimizing losses by making choices at varying levels of risk and reward, and neuroimaging evidence suggests that similar networks are activated during performance, we hypothesized that the three DM tasks would load onto a single factor, and that the structure of this factor would not be influenced by covariates (e.g., sex, age, IQ, and cannabis use) in our adolescent sample.
METHODS
Participants
Participants were 401 (54.1% male) adolescents aged 14–17 (M = 15.40, SD = .72), recruited through Miami-Dade County middle and public schools, flyers posted throughout the community, and word-of-mouth. Participants were primarily of Hispanic/Latino ethnicity (89.8%). Self-reported race was White (76.8%), Black/African American (7.7%), mixed race (12.0%), and other (3.5%).
The sample consisted of participants from a longitudinal study examining associations between DM, memory, and trajectories of cannabis use (R01 DA031176, P.I.: Gonzalez). Eligibility for the parent study was ascertained through telephonic screening. Inclusion criteria were developed to recruit a sample consisting predominantly of adolescents at risk of escalation in cannabis use, that is, adolescents who reported early experimentation with substances without reaching problematic levels of use at the time of screening, and with little to no exposure to substances other than alcohol, cigarettes, or cannabis. The majority of the sample (90%) reported some use, even if minimal, of alcohol, cigarettes, or other drugs at the time of screening. Additional information on participant selection can be found in prior publications with this cohort ( Duperrouzel et al., Reference Duperrouzel, Hawes, Lopez-Quintero, Pacheco-Colón, Coxe, Hayes and Gonzalez2019; Hawes et al., Reference Hawes, Trucco, Duperrouzel, Coxe and Gonzalez2018; Lopez-Quintero et al., Reference Lopez-Quintero, Granja, Hawes, Duperrouzel, Pacheco-Colón and Gonzalez2018; Ross et al., Reference Ross, Graziano, Pacheco-Colón, Coxe and Gonzalez2016). Participants were also between the ages of 14 and 17 at baseline, and able to read and write in English. We excluded participants who reported developmental disorders, birth complications, neurological conditions, or a history of diagnosed or significant mood or thought disorders (excluding ADHD), and those who reported frequent or recent use of drugs other than alcohol, nicotine, or cannabis, or whose answers at the time of screening suggested the presence of an alcohol or cannabis use disorder. Participants underwent oral fluid toxicology screening to test for recent drug use. We excluded seven participants who tested positive for any drug, five with a past dependence on substances other than cannabis, and 15 participants who reported using a drug (besides alcohol, nicotine, cannabis, and hallucinogens) within 14 days of the assessment. Participant and substance use characteristics (displayed in Table 1) and analyses were based on the remaining 372 participants.
Participant demographics at baseline assessment.
M = mean, SD = standard deviation, Md = median, IQR = interquartile range; WRAT = Wide Range Achievement Test.
Procedures
All procedures were approved by the Florida International University IRB. We obtained parental consent and participant assent for all participants. The parent study involves five assessment waves conducted at 6-month intervals over a 2-year period, each of which involves a detailed assessment protocol. Participants received monetary compensation for their time in the study, earning $75 for completing the baseline assessment. The main analyses of the current study focus on data acquired during baseline assessment. Data from the 1-year follow-up assessment were used for a subset of analyses.
Measures
Decision-making
We assessed DM through three computerized tasks: the Cups Task, GDT, and IGT.
Cups Task
The Cups Task was designed to assess DM in children and adolescents (Levin et al., Reference Levin, Hart, Weller and Harshman2007). This task measures DM under conditions of specified risk in both gain and loss domains (Levin & Hart, Reference Levin and Hart2003). Performance on this task has been linked to performance in other DM tasks, such as the IGT (Weller et al., Reference Weller, Levin and Bechara2010). Participants are shown a visual display of 2, 3, or 5 cups on both sides of the screen, and are asked to choose a cup from either side for a total of 54 trials. Choices from one side always yield a reward (e.g., definite gain of one-quarter) or loss (e.g., definite loss of one-quarter), whereas choices from the other side provide the chance for a greater reward (e.g., chance to gain multiple quarters) or loss (e.g., chance to lose multiple quarters). The latter were considered risky choices. We used the total number of risky choices in the gain domain, and the total number of risky choices in the loss domain as our indices of DM for this task.
Game of Dice Task
The GDT was designed to assess the influence of executive functioning (e.g., performance monitoring, strategizing) on DM under uncertain or risky conditions, in which participants are explicitly provided with rules and probabilities for gains and losses (Brand et al., Reference Brand, Fujiwara, Borsutzky, Kalbe, Kessler and Markowitsch2005). This task has been successfully used with adolescents (Drechsler et al., Reference Drechsler, Rizzo and Steinhausen2008; Ross et al., Reference Ross, Graziano, Pacheco-Colón, Coxe and Gonzalez2016). Performance on this task has also been correlated to performance on the IGT (Brand et al., Reference Brand, Recknor, Grabenhorst and Bechara2007). Participants are instructed to win as much money as possible within 18 throws of a die. Before each trial, participants choose a single number, or a combination of two, three, or four numbers. Each choice is associated with specific gains and losses depending on the probability of the occurrence of the participant’s choice. In other words, choices with more numbers have a higher probability of occurring, but are associated with a lesser reward, than choices with fewer numbers, which have a lower probability of occurring. We used the total number of risky choices; that is, the number of times the participant chose the options with the lowest probability of occurring (i.e., one or two numbers), as the index of DM for this task.
Iowa Gambling Task
The IGT assesses DM under conditions of ambiguous risk. This task was developed to capture DM impairments seen in patients with damage to the ventromedial prefrontal cortex (Bechara et al., Reference Bechara, Damasio, Damasio and Anderson1994). Although originally designed for use with adults, the IGT has been successfully used with adolescents (Hooper et al., Reference Hooper, Luciana, Conklin and Yarger2004; Ross et al., Reference Ross, Graziano, Pacheco-Colón, Coxe and Gonzalez2016). In this task, participants are shown a visual display of four decks of cards, and are told that the goal is to win as much money as possible. They are instructed that every time they choose a card they will win money, but sometimes also lose money, and that some decks are worse than others. More choices from good decks lead to a positive total score, whereas more choices from bad decks yield a negative total score at the end of 100 trials. As per the IGT Professional Manual, we used the reverse-scored IGT net total, that is, choices from good decks (decks C and D) minus choices from bad decks (decks A and B), as the index of DM for this task (Bechara, Reference Bechara2007).
Substance use
The Drug Use History Questionnaire is a detailed semi-structured interview used to assess the frequency and amount of use of 15 different drug classes over a participant’s lifetime (Rippeth et al., Reference Rippeth, Heaton, Carey, Marcotte, Moore, Gonzalez, Wolfson and Grant2004). We used frequency (i.e., number of days) of cannabis use in the past 30 days as a covariate in our analyses.
Estimated IQ
We used the Word Reading subtest of the Wide Range Achievement Test–4th Edition (Wilkinson & Robertson, Reference Wilkinson and Robertson2006) to estimate participants’ IQs, which we used as a covariates in our analyses.
Statistical Analyses
Covariate selection
We selected a set of covariates based on theoretical relevance. Following the rationale outlined in the Introduction, we included age, sex, estimated IQ, and recent cannabis use as covariates in our analyses.
Correlations
To better characterize correlations between DM tasks among adolescents, we examined bivariate correlations between all indices of DM used in the current study.
Latent DM construct
All analyses were conducted using Mplus 8 (Muthén & Muthén, Reference Muthén and Muthén2012). We conducted a confirmatory factor analysis (CFA), where four DM indices derived from three DM tasks were combined into a single-factor model. Specifically, we used the reverse-scored IGT net total, the number of risky choices in the GDT, and the total number of risky choices in the gain and loss domains from the Cups Task. Higher scores in this construct reflect poorer DM. This model used maximum likelihood estimation with standard errors and a chi-square statistic that are robust to non-normality (MLR). Model fit was assessed using the comparative fit index (CFI), root mean square error of approximation (RMSEA), and sample size-adjusted Bayesian Information Criterion (BIC). To handle missing data, we used full information maximum likelihood estimates.
Moderated nonlinear factor analysis
After establishing unidimensionality of the DM construct, we used MNLFA (Bauer, Reference Bauer2017) to examine the potential effects of study covariates on the parameters of the latent DM model. Specifically, we tested for differences in the DM construct mean and variance, as well as differential item functioning (DIF) of item intercepts and factor loadings as a function of participants’ age, sex, estimated IQ, and past 30-day cannabis use frequency. DIF was examined on an item-by-item basis, accounting for covariate effects on the DM construct mean and variance. We then established a final MNLFA model by retaining all significant covariate effects on the DM factor (mean and variance) and items (thresholds and loadings). We used parameter estimates from this final model to produce maximum a posteriori (MAP) scores on the DM construct, which account for differences in the DM factor mean and variance and item DIF resulting from participants’ sex, age, IQ, and/or cannabis use. Unlike traditional summed score approaches, MAP scores provide information about individual differences by providing participant-specific factor scores for the DM construct (Curran et al., Reference Curran, McGinley, Bauer, Hussong, Burns, Chassin, Sher and Zucker2014)
Construct stability
To determine whether the DM construct was stable over time, we re-created our final MNLFA model using data collected at the 1-year follow-up assessment of the parent study. We calculated the correlation between participants’ MAP scores on the DM factor at baseline with their MAP scores on the DM factor at the 1-year follow-up assessment.
RESULTS
Table 2 shows bivariate Pearson’s correlations between all indices of DM used in the current study among adolescents. The number of risky choices in the GDT was significantly correlated with risky choices in the Cups Task for both the risk and loss domains. Additionally, the reverse-scored IGT net total was significantly correlated with the number of risky choices in the loss domain in the Cups Task. Correlations between GDT and IGT indices were not significant.
Bivariate Pearson’s correlations between DM indices from the Cups Task, GDT, and IGT (N = 372).
DM = decision-making; GDT = Game of Dice Task; IGT = Iowa Gambling Task
* Significance at p < .005.
The unidimensional CFA model of DM demonstrated good fit (CFI = .989; RMSEA = .038, BIC = 10383.72). All indices loaded significantly onto the DM factor. Table 3 shows detailed MNLFA results. Participant sex significantly influenced mean scores of the DM construct (p = .007), suggesting that female cannabis users demonstrated higher risk-taking compared with males. There was some evidence of DIF across DM indices. Specifically, participant IQ significantly moderated the IGT index intercept (p = .027), whereas participant age had a marginally significant moderating influence on the GDT index factor loading (p = .048). Although these effects are marginally significant and would attenuate to nonsignificance upon correcting for multiple comparisons, we chose to retain these effects in the final MNLFA model to more conservatively correct for any potential DIF.
Moderated nonlinear factor analysis (MNLFA) results and covariate effects.
Values represent unstandardized factor loadings and standard errors. Bolded effects were retained in the final MNLFA model.
a The value of the parameter was fixed (not estimated) to set the scale of the latent variables.
* Significance at p<.05.
** Significance at p < .01 level.
Furthermore, participants’ MAP scores on the DM factor at baseline (N = 372) were moderately and significantly correlated with their MAP scores on the DM factor at the 1-year follow-up assessment (N = 356), r = .35, p < .001. This provides evidence of moderate stability of the DM factor across the follow-up window.
DISCUSSION
Research suggests that different commonly employed DM tasks may measure different aspects of DM (Buelow & Blaine, Reference Buelow and Blaine2015), which raises questions as to whether these instruments measure the same underlying factor. The current study addressed this issue by examining whether different DM tasks load onto a single latent factor. Despite low correlations between indices from the IGT and those of the GDT and Cups Task, our results indicated that four indices derived from the Cups Task, GDT, and IGT loaded significantly onto a single DM factor in our sample composed predominantly of adolescent cannabis users. This suggests that, although these tasks may assess different aspects of DM (e.g., DM under ambiguous vs. specified conditions of risk) that may provide nuanced information, they nonetheless sufficiently tap into a single underlying construct of DM that is common across tasks. Collectively, these measures assess the ability to make optimal choices that maximize rewards in the presence of risk. This finding is consistent with results from neuroimaging studies, which showed cross-task overlap in activation in frontal regions during DM, such as the prefrontal cortex, while also showing differences in activation in other regions (e.g., insula, posterior cingulate cortex) (Labudda et al., Reference Labudda, Woermann, Mertens, Pohlmann-Eden, Markowitsch and Brand2008; Li et al., Reference Li, Lu, D’Argembeau, Ng and Bechara2010; Xue et al., Reference Xue, Lu, Levin, Weller, Li and Bechara2009).
Furthermore, we used MNLFA to determine whether our DM factor was commensurate across participant age, sex, general cognitive ability, and recent cannabis use (Bauer, Reference Bauer2017). These analyses indicated that, on average, girls made riskier decisions compared with boys in our sample of adolescents at risk of cannabis escalation. Although previous work suggests that men make riskier choices compared with women, it is important to consider that the current study assessed DM during adolescence, a period characterized by neuromaturational changes that result in increased risk-taking and sensation-seeking (Casey et al., Reference Casey, Jones and Hare2008). Further, females reach peak levels of sensation-seeking at an earlier age (14–17 years) than males (16–17 years), which may explain why girls in our sample engaged in more risk-taking (Shulman et al., Reference Shulman, Harden, Chein and Steinberg2015). We also found that participant characteristics such as age and IQ caused DIF in our model. We chose to retain these effects in the final MNLFA model, ensuring that our DM factor was commensurate across participant characteristics. In other words, accounting for these effects in our final model ensured that individual differences in DM factor score represent true differences in DM performance, rather than differences in other characteristics that may not be of interest.
One previous study used exploratory and confirmatory factor analyses to examine whether three DM tasks (IGT, Columbia Card Task, and Balloon Analogue Risk Task) loaded onto a single factor among college students (Buelow & Blaine, Reference Buelow and Blaine2015). Although this study found overlap between some of the task indices, a three-factor model provided the best fit, suggesting that all of these tasks assess unique components of DM. Our findings, in contrast, suggest that a single-factor model provides good fit. Discrepancies in our findings may be explained by several factors. First, it is possible that our tasks were more similar to each other than those employed by Buelow and Blaine (Reference Buelow and Blaine2015). For instance, neuroimaging studies suggested that, in addition to activation in frontal areas such as the anterior cingulate and dorsolateral prefrontal cortex, DM during the Balloon Analog Risk Task was associated with robust activation in mesolimbic regions, such as the midbrain and ventral and dorsal striatum, contrasting with other tasks used in the current study (Labudda et al., Reference Labudda, Woermann, Mertens, Pohlmann-Eden, Markowitsch and Brand2008; Li et al., Reference Li, Lu, D’Argembeau, Ng and Bechara2010; Rao et al., Reference Rao, Korczykowski, Pluta, Hoang and Detre2008; Xue et al., Reference Xue, Lu, Levin, Weller, Li and Bechara2009). Second, there are several methodological differences that may have influenced our findings, including the specific task indices used as well as the type of analysis. For instance, our latent factor approach employed confirmatory factor analysis to test our hypothesis that the indices loaded onto a single factor. Thus, the number of factors was decided a priori, whereas this is not the case with exploratory factor analysis. Buelow and Blaine (Reference Buelow and Blaine2015) used a combination of exploratory and confirmatory factor analyses, which may have contributed to differences in our findings. Third, the participants in our sample were significantly younger. Although speculative, it is possible that different tasks may load onto a single factor at this developmental stage due to the protracted pattern of development observed in brain areas underlying executive functions such as DM (Casey et al., Reference Casey, Jones and Hare2008), and that this single factor may later differentiate into multiple factors as the brain continues to mature.
Finally, scores on our latent construct at baseline were significantly correlated with scores on this construct at the 1-year follow-up assessment, suggesting that our construct was relatively stable over time. This effect was moderate in size and was consistent with previous estimates for individual DM tasks (Buelow & Barnhart, Reference Buelow and Barnhart2018).
These results should be interpreted in light of several limitations. First, our sample consisted of adolescents primarily of Hispanic/Latino descent who were cannabis users, which may limit the generalizability of our findings to other populations. The limited age range in our sample may have also limited our ability to detect significant age-related DIF. Future studies should aim to replicate these results in a more representative sample of healthy adolescents. In addition, although some of the DM tasks we employed yielded several informative and fine-grained indices of DM, the current study only used one or two indices per task, as this was better suited to the purpose of our analyses. Considering ongoing debates about which indices best capture DM in each task (e.g., IGT), future studies should examine whether different indices for these tasks influence loadings onto the DM factor, resulting in improved model fit. Nonetheless, we created a latent factor of DM that accounts for DIF caused by different participant characteristics.
Our results provide further support that various DM measures assessed a single underlying construct of DM – the ability to make optimal choices that maximize rewards in the presence of risk – when applied to a sample of adolescent cannabis users. Future studies of DM should apply similar approaches to examine whether their DM tasks tap into a single underlying construct in their sample and whether that construct is commensurate across participants with different characteristics. Such methods will ensure that our measures of DM are comparable for all participants, ultimately enhancing the generalizability of findings.
ACKNOWLEDGMENTS
This work was supported by grants R01 DA031176 and U01 DA041156 (P.I.: Gonzalez) from the National Institute of Drug Abuse, as well as the Presidential Fellowship at Florida International University and McKnight Doctoral Fellowship from the Florida Education Fund (Recipient: Pacheco-Colón). The authors declare no conflicts of interest.
