INTRODUCTION
Drug use is a worldwide problem with a high socioeconomic cost (UNODC, 2016). The negative consequences of drug use not only impacts the user but affects their families and social circles as well (Taylor et al., Reference Taylor, Mackay, Murphy, McIntosh, McIntosh, Anderson and Welch2012). Among the most significant consequences of repeated drug use are deficits in executive functioning (Fernández-Serrano, Pérez-García, & Verdejo-García, Reference Fernández-Serrano, Pérez-García and Verdejo-García2011). Executive deficits are particularly important because they negatively interfere with activities of daily living (ADL), and may lead to job difficulties, accidents and financial problems (Diamond, Reference Diamond2011). An optimal approach to evaluating executive deficits requires a comprehensive assessment that is able to define different executive profiles (Gonzalez, Bechara, & Martin, Reference Gonzalez, Bechara and Martin2007), and assess the repercussions of repeated drug use on day-to-day functioning.
Higher order executive functions, such as planning, have shown to be sound predictors of academic, professional, and social outcomes (Baars, Nije Bijvank, Tonnaer, & Jolles, Reference Baars, Nije Bijvank, Tonnaer and Jolles2015; Gonzalez et al., Reference Gonzalez, Mills, Genee, Li, Piquette, Rosen and Gibb2014). Furthermore, they have the strongest relationship with ADL (Frisch, Förstl, Legler, Schöpe, & Goebel, Reference Frisch, Förstl, Legler, Schöpe and Goebel2012; Krabbendam, de Vugt, Derix, & Jolles, Reference Krabbendam, de Vugt, Derix and Jolles1999). However, most of the classic laboratory tests of planning, such as the Tower of London or the Stockings of Cambridge, have low ecological validity, as they were originally designed by experimental neuropsychologists to interrogate cognitive constructs and processes, not to assess functional outcomes, one of the major interests for clinical neuropsychologists (Burgess et al., Reference Burgess, Alderman, Forbes, Costello, Coates, Dawson and Channon2006). Despite this limitation, classic laboratory tests are also used in clinical assessment, although in many cases the participant’s performance does not reflect their behavior in real-world situations (Shallice & Burgess, Reference Shallice and Burgess1993). More recent neuropsychological research has shown that the use of tests that mimic everyday activities (e.g., the Behavioural Assessment of the Dysexecutive Syndrome battery or the Revised Strategy Application Test) enable better assessment of daily life problems associated with executive dysfunction in drug users (Verdejo-García & Pérez-García, Reference Verdejo-García and Pérez-García2007; Verdejo-García, Reference Verdejo-García2017).
Based on the tests’ contents and complexity, high order executive tests could be classified in three groups. The first group, often referred to as planning tasks, consists of planning tasks with simple goals and specific rules. These tasks are administered in highly structured artificial contexts and provide explicit instructions about the sequence of steps that lead to goal achievement. Examples are the tower tasks (e.g., of London, of Hanoi) (Shallice, Reference Shallice1982), Zoo Map Test (Wilson et al., Reference Wilson, Alderman, Burgess, Emslie and Evans1996), or the Key Search Test (Wilson et al., Reference Wilson, Alderman, Burgess, Emslie and Evans1996). The tasks included in this group have been the focus of most of the research on planning processes.
The second group, traditionally known as multitasking tests, consists of planning tasks with simple goals and complex rules, in which the examinee must maintain and monitor a high amount of online information in the form of rules. In these tasks, compliance to the rules is the central and most important aspect, even more so than the actual performance of the tasks themselves. The contexts in which they are administered are, like the first group, artificial. The Six Elements Test (Wilson et al., Reference Wilson, Alderman, Burgess, Emslie and Evans1996), the Revised Strategy Application Test (Levine, Dawson, Boutet, Schwartz, & Stuss, Reference Levine, Dawson, Boutet, Schwartz and Stuss2000), and the Greenwich Multitask Test (Burgess, Veitch, de Lacy Costello, & Shallice, Reference Burgess, Veitch, de Lacy Costello and Shallice2000) are examples of this type of test.
The third group, namely complex planning tasks, consists of planning tasks with multiple goals and rules and whose contexts, whether natural or virtual, are unstructured. Their distinctive features include greater temporal duration, sequencing targets and the absence of immediate feedback. Notable examples of complex planning tasks include the Multiple Errands Test (MET) (Shallice & Burgess, Reference Shallice and Burgess1991), the Executive Secretariat Task, the Hotel Task (Lamberts, Evans, & Spikman, Reference Lamberts, Evans and Spikman2010), and the Jansari-Agnew-Akesson-Murphy Task (Jansari, Agnew, Akesson, & Murphy, Reference Jansari, Agnew, Akesson and Murphy2004). In the context of addiction, there is specific interest in the study of these tests, as they replicate scenarios that resemble the daily life of drug users. Therefore, they may have superior discriminability and ecological/predictive validity (e.g., in relation to treatment outcomes) relative to traditional tests (Verdejo-García, Reference Verdejo-García2017).
Planning is a construct that has been scarcely studied using tasks with multiple goals and rules in substance users, with the exception of one study performed on individuals with alcohol dependence using a virtual reality task (Laloyaux et al., Reference Laloyaux, Michel, Mourad, Bertrand, Domken, Van der Linden and Larøi2012). It is important to further examine planning processes in people with substance dependence, as there are meaningful associations between planning deficits and addiction treatment retention (Verdejo-García et al., Reference Verdejo-García, Betanzos-Espinosa, Lozano, Vergara-Moragues, González-Saiz, Fernández-Calderón and Pérez-García2012) or the ability to maintain employment after discharge (Moriyama et al., Reference Moriyama, Mimura, Kato, Yoshino, Hara, Kashima and Watanabe2002). However, there have been no studies in people with substance dependence using a high ecological test, such as the MET. Even though the MET was published in the 1990s and is the pioneer test among the complex planning tasks (Shallice & Burgess, Reference Shallice and Burgess1991), it has not been widely used until recently (Alderman, Burgess, Knight, & Henman, Reference Alderman, Burgess, Knight and Henman2003).
The task raises multiple goals and is administered in a real-life context. The duration of the test is not pre-determined, and is often extensive due to the fact that the participant’s performance determines the beginning and ending time of the test. The examiner’s role is to explain the instructions, ensure understanding of the test, and observe and record the participant’s behavior without providing any feedback or interacting during the test. Several studies using the MET have been conducted on patients with acquired brain injury (Cuberos-Urbano et al., Reference Cuberos-Urbano, Caracuel, Vilar-López, Valls-Serrano, Bateman and Verdejo-García2013), stroke (Manes, Villamil, Ameriso Rock, & Torralva, Reference Manes, Villamil, Ameriso, Roca and Torralva2009; Morrison et al., Reference Morrison, Giles, Ryan, Baum, Dromerick, Polatajko and Edwards2013), schizophrenia (Bulzacka et al., Reference Bulzacka, Delourme, Hutin, Burban, Méary, Lajnef and Schürhoff2016), bipolar disorder (Caletti et al., Reference Caletti, Paoli, Fiorentini, Cigliobianco, Zugno, Serati and Altamura2013), and multiple sclerosis (Roca et al., Reference Roca, Torralva, Meli, Fiol, Calcagno, Carpintiero and Correale2008). The findings of these studies have shown good inter-rater reliability, internal consistency (Knight, Alderman, & Burgess, Reference Knight, Alderman and Burgess2002), and predictive validity, that is, better ability to estimate daily functioning and social autonomy compared to traditional tasks (Bulzacka et al., Reference Bulzacka, Delourme, Hutin, Burban, Méary, Lajnef and Schürhoff2016; Cuberos-Urbano et al., 2016). The MET is well-suited to further characterize planning processes in drug users because it is a test related to general, important daily activities that are meaningfully related to daily functioning and recovery capital (Knight et al., Reference Knight, Alderman and Burgess2002).
This study is the first to assess executive functions in drug users by administering the MET in a real-life setting. Several versions of the MET have been adapted to specific contexts such as hospital settings (Knight et al., Reference Knight, Alderman and Burgess2002) or a shopping center (Alderman et al., Reference Alderman, Burgess, Knight and Henman2003). Despite being adaptations of the same version of the MET, they have very different objectives and, therefore, may yield different results depending on the appropriateness of the specific context in which they operate. In this case, we tested a new version of the MET adapted to one of the most typical treatment settings for drug users – a therapeutic community (TC) (European Drug Report, 2014). Therefore, the specific aims of this study are (i) to apply the new version of the MET, namely, the Multiple Errands Test – Contextualized Version (MET-CV), to people with substance dependence (PWSD) following addiction treatment in three TCs; (ii) to obtain evidence of testing context reliability (equivalence of scores in different contexts for similar populations) and convergent validity (i.e., correlations with other planning and executive function tests); (iii) to analyze differences in performance of the new version of MET and other traditional planning and executive function tests among PWSD and a control group. We anticipated that MET-CV scores would show evidence of testing context reliability (equality of scores in different contexts for similar populations) and convergent validity, and that MET-CV and traditional neuropsychological tasks could detect differences in cognitive performance between PWSD and healthy controls.
METHODS
Procedures
Participants were assessed between September 2012 and December 2013. The assessment sessions lasted up to 4 hr including breaks. On the day of testing, the participants had not taken medications with a significant impact on cognition, such as benzodiazepines. The order of administration of the tests was: diagnostic interviews, Letter Number Sequencing (Wechsler, Reference Wechsler1997), Zoo Map Test (Wilson et al., Reference Wilson, Alderman, Burgess, Emslie and Evans1996), Stocking of Cambridge (SOC) [Cambridge Neuropsychological Test Automated Battery (CANTAB, 2004)], Information Sampling Test (IST) (CANTAB, 2004), Revised Strategy Application Test (RSAT) (Levine et al., Reference Levine, Dawson, Boutet, Schwartz and Stuss2000), and MET-CV.
The tasks were administered in the three TCs where the sample was recruited. In the case of controls, the assessment was administered in the Research Center, Mind, Brain and Behavior (CIMCYC) of the University of Granada. Participants did not receive a reward for participating in this study. The Ethics Committee of the University of Granada approved the study, and all participants provided informed consent.
Participants
Sixty participants with alcohol, cocaine, and heroin dependence, 12 of whom were women, enrolled in residential treatment and 30 healthy comparison participants, 10 of whom were women, aged between 18 and 52 years, participated in this study. PWSD were recruited while receiving treatment in three residential, long-stay (6–12 months) public TCs: Centro de Rehabilitación Cortijo Buenos Aires de Granada, Comunidad Terapéutica Proyecto Hombre de Huetor-Santillán, and Comunidad Terapéutica Proyecto Hombre de Algarrobo. All PWSD were abstinent, as indicated by urine toxicological tests for alcohol, cannabis, cocaine, opiates, benzodiazepines, and amphetamines conducted before assessments. Participants for the healthy comparison group were recruited through advertisements in different newspapers and also via email through mailing lists. There were no differences between the groups in gender, age, educational level, and general cognitive ability indicated by the vocabulary subscale of the Kaufman Brief Intelligence Test (Kaufman & Kaufman, 1990).
Table 1 displays the sociodemographic characteristics of both groups. With respect to the three PWSD groups, there were no differences in age (TC1, M=37.32, SD=8.51; TC2, M=36.5, SD=7.51; TC3. M=33.11, SD=6.09; F=1.683, p=.195), educational level in years (TC1, M=10, SD=2.97, TC2, M=10.10, SD=2.31, TC3, M=10.94, SD=2.98; F=0.665, p=.518) or general cognitive ability (TC1, M=44.2, SD=0.83: TC2, M=44.37, SD=0.74; TC3, M=43.80, SD=1.30; F=0.571, p=.577). When considering gender, there were differences between TC1 and TC2 (χ2=4.863; p=.027) and TC3 (χ2=12.222; p<.001), due to the fact that the TC1 is a residential addiction center only for men.
Sociodemographic and consumption patterns in PWSD and HC
Note. HC=healthy controls.
The inclusion criteria for substance users in the study were: (i) meeting DSM-IV-TR criteria for substance dependence including alcohol, cocaine and/or heroin; (ii) having a minimum abstinence interval of 15 days, as determined by urine toxicological tests completed in TCs; (iii) absence of any current psychiatric comorbidity on Axis I and Axis II as indicated with the SCID and the International Personality Disorders Examination (IPDE; Loranger et al., Reference Loranger, Sartorius, Anfreoli, Berger, Buchheim and Channabasa-vanna1994; Spanish version by López-Ibor, Reference López-Ibor1996); (iv) absence of a history of head injury and neurological, infectious, systemic, or any other disease affecting the central nervous system; (v) not taking prescription drugs that affect the central nervous system (e.g., benzodiazepines, antipsychotics, etc.).
In relation to the DSM-IV-TR criteria for substance dependence, 17 participants met the criteria for alcohol dependence, 29 for cocaine, and 14 for heroin. The majority of the participants met criteria for more than one substance. In such cases, the main diagnosis, as noted above, was determined based on their self-reported drug of choice.
The number of participants who met criteria for more than one substance was as follows: among people with alcohol dependence, 8 met criteria for cannabis dependence (9 were “pure” users). Among people with cocaine dependence, 14 met criteria for alcohol dependence, 5 for cannabis, and 9 for both alcohol and cannabis (1 “pure” user). Among people with heroin dependence, 2 met criteria for cocaine dependence, 3 for cocaine and alcohol, 3 for cocaine and cannabis, and 6 for cocaine, alcohol and cannabis (no “pure” users).
The inclusion criteria for the comparison group were the same as for the clinical group, except the first criterion, i.e., meeting DSM-IV-TR criteria for substance dependence (with the exception of nicotine dependence). Other substance dependence diagnoses were considered exclusion criteria for this group.
Instruments
The interview for research on addictive behavior
This is a semi-structured interview to assess the lifetime severity of drug use (Verdejo-García et al., Reference Verdejo-García, López-Torrecillas, Aguilar de Arcos and Pérez-García2005). This interview collects detailed information about the use of each drug from the onset of substance use until the present moment. The average amount consumed per day (number of alcoholic drinks, grams of cocaine, etc.), the frequency of use (daily, four to five times a week, weekends, occasional use in a month, etc.), and the number of years consuming are recorded. With these data, a lifetime composite severity index is obtained for each substance (alcohol, cocaine, heroin, and cannabis)=[Quantity used per day×Frequency of use per month×Duration in years]. A global composite lifetime severity index was calculated as the sum of Z-scores of each drug used. The outcome measures are a composite severity index, the age of initiation of each substance, the frequency of use of each substance (units/month), the duration of consumption of each drug, and the duration of abstinence from all drugs.
Traditional Neuropsychological Instruments
Three traditional planning tasks were used in this study. Furthermore, cognitive domains related to planning abilities according to Miyake’s model of executive functions (Miyake et al., Reference Miyake, Friedman, Emerson, Witzki, Howerter and Wager2000), such as working memory, were also assessed.
Letter-Number Sequencing (Wechsler Adult Intelligence Scale, WAIS-III)
This is a working memory test. The examiner reads a sequence of numbers and letters, with a frequency of one second per letter/number. The participant must recall the sequence and list the numbers in ascending order and the letters alphabetically. The outcome variable is the number of correct responses.
Information Sampling Test (IST)
This is a computerized CANTAB test. This test assesses the reflection-impulsivity ability, defined as the ability to evaluate the available contextual information before making a decision. During this test, participants are presented with a square chart composed of 25 gray boxes on a screen. When the box is touched, one of the two possible colors presented in the bottom of the screen are revealed in two colored boxes. The participant must tap as many boxes as desired, and when the subject is certain of which color he or she thinks predominate, choses a color believed to be predominately hidden under the square chart. There are two conditions. In the “fixed” condition, the participant wins 100 points each time if he or she is right and loses 100 points if he or she fails. In the “decreasing” condition, the participant could win a maximum of 250 points each time, but the amount of points decreases by 10 points each time a box is opened. If the decision is not correct, the participant loses 100 points. The goal is to win the highest amount of points possible. The outcome variables are the number of sampling errors (the number of trials where the participant fails but in the moment of his or her decision, the color selected was the predominant), probability of success at the moment of the decision, the number of boxes opened, and the number of trials where the participant selects the hidden color correctly in the majority of the squares.
Stocking of Cambridge (SOC)
The SOC is a computerized version of the traditional Tower of London task (Shallice, Reference Shallice1982), which assesses spatial planning abilities. Participants view two screens in which there are three different color balls and three cavities that have a different ball-holding capacity (three, two, and one ball, respectively). The aim is to reproduce the ending position of the top screen in the bottom screen with the least number of movements possible. The task consists of problems of varying difficulty (two, three, four, and five moves to be solved). Participants must respect two rules that the program does not allow to be violated: a ball cannot be drawn from the cavity if there is another ball on top, and the balls can only be placed in the lower available cavity. The main outcome variable is the sum of displacements in four and five movement problems.
Zoo Map Test (Behavioral Assessment of the Dysexecutive Syndrome)
This test assesses the ability to formulate and implement a plan. In this task, participants must draw a route to visit the six indicated places on a paper map of a zoo following three simple rules. These rules determine the performance, so that there are only four correct routes to solve the task correctly. In this study, only the first part of the test was used, in which the problem has low structure and is considered as high demand. The outcome variable is the total score on the first part of the test.
Revised Strategy Application Test (RSAT)
This test assesses multitasking and strategy application abilities. The RSAT consists of six stacks of 10 pages each. Each page has twelve items that could be big or small, and long, medium, or short. Two stacks have drawn figures where the participant must trace them, while two of the other stacks are composed of sentences that must be copied. In the last two stacks, the participant must numerate a variety of group figures. The main goal is to gain the maximum amount of points possible, knowing that each small item completed earns points. Furthermore, the participant must remember and learn a list of rules that need to be followed. The test ends when the participant has completed 50 items (except for the items of the first page of each stack). The most effective strategy is to complete the small items (which give points) because they are brief and faster to solve. At the end of the task, the participant is asked about the strategy implemented. The outcome variables are the number of small items, the percentage of the sample that recognizes the efficient strategy, and the self-regulation index (percentage of small items divided by total items performed).
Multiple Errands Test – Contextualized Version (MET-CV)
This test is an adaptation of the hospital version of the MET developed by Knight et al. (Knight et al., Reference Knight, Alderman and Burgess2002). The original task was designed to measure everyday planning abilities in the context of a hospital setting. Participants are instructed to perform 12 tasks (e.g., buying a list of items, collecting information about the hospital setting, keeping an appointment) following a specific set of rules. The MET-CV was designed to adapt to the characteristics of our population and the new assessment context (i.e., TCs). The specific characteristics of the MET-CV are as follows: (1) all subtasks are characterized by one main goal (e.g., preparing a meal) that is clear and relevant to everyday life; (2) the difficulty of the test has been slightly increased due to the better neurocognitive state of PWSD compared to people with traumatic brain injury [i.e., the location of some key objects were not directly pointed, the shopping list included general items (e.g., sodas) but did not specify the product/amount that had to be purchased, thereby letting participants decide, the phone call and the information to provide in the announcement required additional collection of information, and there were more distracters in the map]; (3) in the absence of certain elements of the original version, such as a shop, we designed a supermarket brochure, and the shopping task consisted of making a shopping list; (4) sending a letter was replaced with hanging a sign to announce the location of a meal celebration; (5) the task of finding out opening times for different services was replaced with taking note of the temperature using a thermometer; (6) the laundry timetable, which is well known by the users of the TC, was replaced by the task of seeking a newspaper and recording the weather forecast for the following day; (7) writing down the price of a product from the supermarket was replaced with writing down the address of the shop (provided in the supermarket brochure) where the items would be purchased; (8) the objects to count were modified according to the availability of outside assessment centers; and (9) given that the use of money is prohibited in TCs, money was replaced by a hypothetical budget provided to participants.
The test consists of performing 11 tasks that are grouped into three goals. The first goal was to complete the following tasks: picking up a bag that contains the necessary material to carry out the task.; preparing a shopping list with three products; following specific instructions from a supermarket catalog (represents three tasks); making a reservation for a venue to host a celebration by telephone, and create and put up a poster with the information about the event.
The second goal was to obtain information about the surroundings through four tasks: studying a thermometer and taking note of the current temperature; making a weather prediction using a newspaper; finding and taking note of the supermarket address; counting the number of specific items in the area (benches, lampposts, etc.).
The third goal consisted of meeting in a specific place 20 min before the start of the task.
Test Administration
Participants were informed that they would conduct some activities related to preparing a meal for the center’s users. The examiner explained the instructions and rules for the task. All questions were resolved and the participant was asked to explain what they should do to ensure that they understood the instructions. The participant was informed that the examiner would observe them at a safe distance and would not interact with them unless the task required it. Finally, all the necessary material was provided (pen and folder with instructions and a map). During test performance, the examiner took note of the implementation of the tasks, any instances of breaking the rules or any other aspect of interest, without interacting with the participant. The task was considered complete when the participant indicated they had finished.
When performing the test, the participants’ behavior was registered and classified following the original procedures given by Shallice and Burgess (Reference Shallice and Burgess1991). Error measures are defined in Table 2. One point is assigned for each error. Subsequently, a weighted score is calculated to achieve a sensitive scoring method based on the frequency of error in the control group according to Shallice and Evans (Reference Shallice and Evans1978) and Alderman et al. (Reference Alderman, Burgess, Knight and Henman2003). This process allows for discrimination between common errors and unique errors associated with PWSD. In any case, weighted errors are assigned to both groups. The errors made exclusively in the experimental group are assigned three points, and the errors made in less than 6.66% of the control group are allotted two points. The errors made in more than 6.66% of the control group are not weighted, and instead maintain the same value (one point error). In summary, the higher the number of points, the poorer the performance is.
Description of the dependent variables of the MET-CV
Other variables measured included: familiarity (the subjective rating of the subject’s knowledge of the TC environment), total performance time (time between the start and the end of the task), initial planning time (time between the completion of the instructions and the start of the task), and number of times that the participant consults signals or the map. The outcome variables are: total number of errors (task failures, total number of rules broken, total number of inefficiencies, and total number of misinterpretations).
Statistical Analysis
All analyses were conducted using SPSS version 19. We first explored the data to detect outliers (indicated with the SPSS Explore command as three standard deviations) and missing data points. After removing outliers and accounting for missing data, the sample size was as follows: IST=59, RSAT=59, MET-CV=58, SOC=58, Zoo Map Test=60, and Letter Number Sequencing=60. Analyses of covariance were conducted to determine testing context reliability of test scores for participants from different TCs. We found a gender difference between the groups; thus, this variable was included in the model as covariate. The scores of the MET-CV and the traditional neuropsychological tasks variables were analyzed as dependent variables and TCs as independent variables. Pearson correlation analyses were conducted to examine the convergent validity of the evidence in relation to other traditional planning and executive function measures.
Furthermore, we performed correlations between MET-CV and traditional neuropsychological tasks with the duration of abstinence and the severity of consumption index. For the group comparison, parametric and non-parametric tests were carried out. Duration of abstinence was included as a covariate in the analysis for Zoo Map Test, as abstinence was significantly correlated with performance in this task (see the Results section). Scores were represented in means for parametric analysis and median for non-parametric analysis. Following the recommendations of Blakesley et al. (Reference Blakesley, Mazumdar, Dew, Houck, Tang, Reynolds and Butters2009), Bonferroni adjustments were made to prevent Type I error in the comparison of multiple hypotheses (23 variables were analyzed, adjusting the level of significance determined at 0.05 / 23=0.0021). Cohen’s d values were calculated for each of the intergroup contrasts to index effect sizes. We used Pearson correlation analyses to examine the association between patterns of drug use and performance measures.
A hierarchical regression was conducted to determine the impact of the scores of traditional neuropsychological planning tasks and the MET-CV (independent variables) on severity of consumption (dependent variable). Independent variables were entered into the regression model in sequential blocks: the first block included the scores of traditional neuropsychological planning tasks (SOC, Zoo Map Test, brief items/total items of RSAT); the second block included the scores of the MET-CV (task failures weighted, inefficiencies weighted, interpretation failures weighted, and instances of rule breaking weighted). We did not find correlations between demographic variables and consumption severity; hence, we did not include any covariate in the model. The number of predictors (n=7) is aligned with statistical recommendations for sample size in regression analyses (Hair et al., 2000). For each new block, we estimated the R 2 of the prediction change associated with that block, and its statistical significance to determine their unique contribution.
RESULTS
Testing Context Reliability of MET-CV Scores between Participants from Different TCs
No significant differences were found among the three groups in the main indexes of the MET-CV (Table 3). Nor were any significant differences found in other secondary variables, with the exception of the total performance time, in which we found differences between two TCs (1 and 3; p=.006). With respect to the other traditional neuropsychological tasks, there were no significant differences in the major variables of neuropsychological tasks (Table 3).
Analyses of covariance comparisons between participants from the three different TCs on the MET-CV and traditional neuropsychological tests
Note. DC=decreasing condition; FC=fixed condition.
Evidence of convergent validity: correlations between MET-CV, traditional neuropsychological tasks, and drug use variables
The correlations between the scores of the MET-CV and those of traditional neuropsychological tasks are shown in Table 4. Error variables of the MET-CV correlated with discrete executive function tests (i.e., working memory and planning tests). With regard to drug use variables, rule breaks correlated with abstinence (r=−0.306; p<.001) and the severity of consumption index (r=0.453, p<.001), and interpretation failures (r=0.269; p<.001). The only correlation between traditional neuropsychological tests and drug use measures was the one between the Zoo Map Test scores and the duration of abstinence (r=0.309; p=.016).
Pearson’s correlations between the MET-CV and traditional neuropsychological tasks in polysubstance users
Note. DC=decreasing condition; FC=Fixed condition. Drug use measures were obtained by the Interview for Research on Addictive Behavior.
*p<.05.
*p<.001.
MET-CV Differences between PWSD and Controls
Between-group comparisons on the scores of the complex planning task can be found in Table 5. Task failures and inefficiencies showed significant differences between groups, with PWSD showing significantly more errors than the comparison group (Cohen’s d=0.771 for task failures and Cohen’s d=0.879 for inefficiencies).
Descriptive scores, group comparisons, and effect sizes between PWSD and HC on the MET-CV and traditional neuropsychological tests
Note. HC=healthy controls; FC=fixed condition; DC=decreasing condition; ANCOVA=analysis of covariance.
There were also differences between the groups in the following secondary variables: initial planning time, number of times that the signals and map were consulted, and familiarity with the center. Participants in the experimental group showed less thinking time and fewer checks of signals on the map than the comparison group.
Traditional neuropsychological task differences between PWSD and controls
Performances of both groups are presented in Table 5. The PWSD performed significantly worse than the control group on the Letter Number Sequencing working memory task, with a large effect size (Cohen’s d>0.8). The IST showed differences between groups for the main variables of the fixed condition with a large effect size (Cohen’s d>0.8), but no significant differences in the decreasing condition. Regarding the simple goal planning tasks, SOC (problems 4 and 5 movements) and the Zoo Map Test, there were no significant differences between groups. Finally, in the RSATs, the number of brief items was significantly lower among PWSD. However, there were no significant differences for the index of self-regulation (brief items/total items) and the percentage of people who recognized the strategy.
Multiple hierarchical regression model
Results are displayed in Table 6. Traditional neuropsychological planning tasks were not a significant predictor of lifetime severity of drug use. Prediction significantly improved after inclusion of MET-CV variables (R 2 change=0.247; p<.001; R 2 adjusted=0.278; p<.001). The full model explained 27.8% of the variance. Interpretation failures weighted and rule breaks weighted were significant individual predictors.
Multiple hierarchical regression model, contribution of traditional neuropsychological planning tasks and MET-CV to explain consumption severity
Note. Traditional neuropsychological planning tasks: SOC, Zoo Map Test, and brief items/ total items of RSAT; MET-CV (task failures weighted, inefficiencies weighted, interpretation failures weighted, and rule breaks weighted).
DISCUSSION
The aims of this study were to examine the feasibility of a contextualized version of the MET (MET-CV) among people with substance dependence (PSWD) enrolled in TCs, and to analyze the testing context reliability of MET-CV scores and the convergent validity in terms of association with classic neuropsychological measures of higher order executive functions. We also sought to analyze performance differences between PWSD and a non–drug-using comparison group.
Our first goal was to test the applicability of the MET-CV in different natural environments. We found that the MET-CV can be effectively applied in the context of TCs, which is one of the main therapeutic settings for addiction treatment, and is sensitive to detecting executive function deficits in PWSD. Our second goal consisted of testing the context reliability and convergent validity of the MET-CV. The findings showed adequate testing context reliability across three different TCs, and moderate convergent validity in relation to traditional neuropsychological tests of planning and related skills. Complex planning tasks, such as the MET, have been criticized for being difficult to apply in clinical settings (Rand, Basha-Abu Rukan, Weiss, & Katz, Reference Rand, Basha-Abu Rukan, Weiss and Katz2009). This study confirms that it is possible to adapt this type of test to these contexts, using everyday tasks and universal elements. There were only differences in the secondary variable of total performance time, which is likely due to the different size of the TCs’ facilities. These findings indicate that the results of the MET-CV are reliable.
The scores of the MET-CV correlated with performance indices of traditional neuropsychological tasks, providing evidence of convergent validity. The MET-CV tasks failures correlated with traditional working memory test and rule breaks correlated with classic single goal planning tasks (SOC and Zoo Map Test). These results confirm that the MET-CV is related to planning abilities and working memory, constructs closely related involved in high order cognitive functions (Frisch et al., Reference Frisch, Förstl, Legler, Schöpe and Goebel2012; Köstering, Stahl, Leonhart, Weiller, & Kaller, Reference Köstering, Stahl, Leonhart, Weiller and Kaller2014). The lack of correlation with inefficiencies may be explained by the fact that traditional neuropsychological tasks tend to assess performance in a dichotomous manner (error/hit), while the MET-CV can encode inefficiencies as less optimal behavior to achieve a goal. Altogether, the modest correlations suggest that planning tasks with multiple goals/rules may be tapping into different aspects of executive functioning relative to traditional tasks. This notion resonates with the general premise that real-life problems (as modeled in the MET-CV) are complex and unstructured, whereas laboratory tasks are simple and highly structured. Indeed, it has been proposed that ecological tasks like the MET would mobilize specific cognitive resources, such as divergent thinking (Goel, Reference Goel2010).
Drug use variables (severity of consumption index and abstinence) also correlated with performance on the MET-CV in the expected manner (i.e., higher use was associated with poorer performance). Similarly, a longer duration of abstinence correlated with fewer broken rules. Previous studies indicate that prolonged periods of abstinence are associated with better executive performance (Schulte et al., Reference Schulte, Cousijn, den Uyl, Goudriaan, van den Brink, Veltman and Wiers2014), but the results vary, depending on the type of task used (Fernández-Serrano, Pérez-García, et al., Reference Fernández-Serrano, Pérez-García and Verdejo-García2011). This study suggests that the MET-CV is sensitive to fluctuations in performance associated with the abstinence period.
Finally, we sought to compare between-group results (PWSDs vs. healthy comparison participants) in traditional neuropsychological tasks and the MET-CV. The results indicated that PWSD had significant deficits in both types of tasks. Overall, they made a higher number of errors compared to the control group, but differences were only significant for task failures and inefficiencies. The differences in number of rules broken and misinterpretations were not significant. The absence of differences in broken rules is striking. However, some of these rules were context dependent, and the greater familiarity of the PWSD group with their setting, compared to the control group, may have reduced the discriminative power of this variable. As for the misinterpretation, the low number of errors in both groups suggests that this type of error is not common in this population. It is noted that, in other studies conducted in individuals with brain damage, these types of errors are also rare (Alderman et al., Reference Alderman, Burgess, Knight and Henman2003; Knight et al., Reference Knight, Alderman and Burgess2002).
The initial planning time of the MET-CV was also different between the two groups. This finding is relevant because lengthier initial planning time has been associated with better performance in the traditional single goal planning tasks (Kaller, Unterrainer, Rahm, & Halsband, Reference Kaller, Unterrainer, Rahm and Halsband2004). Similarly, the higher number of checks for signs and maps in the control group indicate that PWSD benefit less from goal-related available information. These results are consistent with those found in the variable number of boxes opened in the IST task, where PWSD receive less information about the context, represented by the number of boxes opened.
Finally, the results from traditional neuropsychological tasks showed that PWSD have deficits in higher order executive functions, consistent with previous studies (van Holst & Schilt, Reference van Holst and Schilt2011; Verdejo-García & Pérez-García, Reference Verdejo-García and Pérez-García2007; Zorko, Marusic, Cebasek-Travnik, & Bucik, Reference Zorko, Marusic, Cebasek-Travnik and Bucik2004). The deficits found in working memory are consistent with previous research (Lawton-Craddock, Nixon & Tivis, Reference Lawton-Craddock, Nixon and Tivis2003; Pitel et al., Reference Pitel, Beaunieux, Witkowski, Vabret, de la Sayette, Viader and Eustache2008). The reflection-impulsivity task, IST, showed different results depending on the condition being evaluated. Both groups showed similar performance in the decreasing condition, while significant differences were found in the fixed condition in the probability of success, sampling errors, and boxes opened. The discrepancy between conditions may be linked to their different levels of risk: in the decreasing condition, considered high risk, PWSD would have more thoughtful decision-making influenced by the perception of risk present (Lawrence, Luty, Bogdan, Sahakian, & Clark, Reference Lawrence, Luty, Bogdan, Sahakian and Clark2009).
In contrast, in the fixed condition, the absence of risk facilitates impulsivity. In the simple goal planning tasks, namely the SOC and the Zoo Map Test, there were no significant differences, although PWSD showed poorer performance than controls. Previous studies with these tests have shown mixed results (Brand, Roth-Bauer, Driessen, & Markowitsch, Reference Brand, Roth-Bauer, Driessen and Markowitsch2008; Fishbein et al., Reference Fishbein, Krupitsky, Flannery, Langevin, Bobashev, Verbitskaya and Tsoy2007; Flannery et al., Reference Flannery, Fishbein, Krupitsky, Langevin, Verbitskaya, Bland and Zvartau2007; Ornstein et al., Reference Ornstein, Iddon, Baldacchino, Sahakian, London, Everitt and Robbins2000). Similarly, the RSAT did not differentiate between groups, consistently with previous studies (Fernández-Serrano et al., 2012; Moriyama et al., Reference Moriyama, Mimura, Kato, Yoshino, Hara, Kashima and Watanabe2002).
Multiple hierarchical regression models indicated that traditional neuropsychological planning tasks did not significantly contribute to explaining severity of consumption. In contrast, MET-CV variables did. These findings have implications for clinical practice, as ecologically valid planning tasks with multiple goals and rules may enable better understanding of clinical severity among the populations with substance use disorders.
Finally, the following limitations in this study should be mentioned: the sample size was relatively limited; the cross-sectional design does not allow us to draw conclusions regarding causation effects; blind assessment ratings were not feasible; generalization of findings was limited according to the absence of psychiatric comorbidity in the sample; divergent validity of the scores has not been established; it was not possible to analyze the influence of motor skills in MET-CV performance; and the use of a crystallized measure of intelligence (vocabulary) to assess possible changes in intelligence secondary to drug use. Moreover, although we defined PWSD according to their main diagnosis, most PWSD participants had more than one substance use disorder, and they had a history of polysubstance use. Future studies should address these limitations and incorporate fluid measures of intelligence that are more sensitive to change.
In conclusion, this study shows that it is feasible to apply the MET-CV in the TC context, the results showed good testing context reliability and a moderate convergent validity. Both traditional neuropsychological tasks and new the naturalistic assessment task, namely, the MET-CV, allowed us to differentiate between PWSD and non–drug-using controls. In addition, the complex nature of the MET-CV can differentiate better than simple goal planning tasks (SOC and Zoo Map Test) and RSATs. Our findings reinforce the need to consider the dynamics of real environments in neuropsychological assessment to obtain information on executive functioning.
ACKNOWLEDGMENTS
C. Valls-Serrano is funded by FPU predoctoral research grant from “Plan propio, University of Granada”. A. Verdejo, X. Nöel, A. Caracuel report no conflict of interest.
