Introduction
There is increasing interest in community-based, universal risk assessment to identify youth who either have a substance-use disorder or who will develop one in the future. Universal risk assessment, followed by appropriate intervention, could potentially reduce the population burden of disease associated with substance-use disorders. There are many risk-assessment tools that screen adolescents for current or future substance-use disorder (Clark et al. Reference Clark, Chung and Martin2006; Vanyukov et al. Reference Vanyukov, Kirisci, Moss, Tarter, Reynolds, Maher, Kirillova, Ridenour and Clark2009; Chung et al. Reference Chung, Smith, Donovan, Windle, Faden, Chen and Martin2012; Kirisci et al. Reference Kirisci, Reynolds, Carver and Tarter2013; Levy et al. Reference Levy, Weiss, Sherritt, Ziemnik, Spalding, Van Hook and Shrier2014). None, to our knowledge, incorporate the key features of the most successful universal risk assessment tools to date, such as the Framingham risk score for cardiovascular disease (Wilson et al. Reference Wilson, D'Agostino, Levy, Belanger, Silbershatz and Kannel1998; D'Agostino et al. Reference D'Agostino, Vasan, Pencina, Wolf, Cobain, Massaro and Kannel2008). The Framingham risk score was developed using population-based samples, estimates total risk briefly and inexpensively by incorporating a relatively small number of well-established risk factors, and aggregates risk factors using a simple algorithm.
In the present study, we developed a risk score to identify adolescents in the general population who are at risk for persistent substance dependence. To maximize translation to practice in community settings, we incorporated the key features of the most successful universal risk assessment tools. That is, we used data from a population-representative longitudinal study; we selected a relatively small number of risk factors that have been shown in longitudinal studies to consistently and robustly predict substance dependence; and we aggregated these risks into a risk score using a simple algorithm, namely the sum of dichotomous risks. This summation approach draws on the large body of research showing that number of childhood risk factors predicts poorer mental and physical health in adulthood (i.e. cumulative risk) (Rutter, Reference Rutter1981; Sameroff et al. Reference Sameroff, Seifer, Barocas, Zax and Greenspan1987; Felitti et al. Reference Felitti, Anda, Nordenberg, Williamson, Spitz, Edwards, Koss and Marks1998; Evans et al. Reference Evans, Li and Whipple2013).
We evaluated the accuracy with which the cumulative risk score in adolescence predicted risk for substance dependence through young adulthood to early midlife. However, rather than predicting an adolescent's risk for lifetime substance dependence, we predicted risk for severe, persistent substance dependence. Our rationale was that epidemiological studies show that the prevalence of lifetime substance dependence is quite high, and most people with substance dependence remit on their own without treatment (Heyman, Reference Heyman2013; Meier et al. Reference Meier, Caspi, Houts, Slutske, Harrington, Jackson, Belsky, Poulton and Moffitt2013; Grant et al. Reference Grant, Goldstein, Saha, Chou, Jung, Zhang, Pickering, Ruan, Smith and Huang2015). Therefore, to avoid over-treating adolescents who would benefit from brief, harm-reduction interventions, and under-treating adolescents who might require more intensive intervention, we developed a population-based risk score that distinguishes those with the poorest long-term prognosis from those with a relatively good prognosis. In addition, rather than predicting risk for dependence on specific substances (e.g. alcohol v. cannabis), we collapsed across substances in defining persistent forms of substance dependence. Our reasoning was that practitioners conducting universal screening (e.g. primary care physicians) want to assess risk for severe dependence on any substance, as opposed to risk for particular types of substance dependence. We further reasoned that the development of substance-specific risk assessment tools would result in the proliferation of risk assessments, thereby reducing implementation in practice.
Method
Participants
Participants are members of the Dunedin Multidisciplinary Health and Development Study, a longitudinal investigation of health and behavior in a complete birth cohort (Poulton et al. Reference Poulton, Moffitt and Silva2015). Study members (N = 1037, 91% of eligible births, 52% male) were all individuals born between April 1972 and March 1973 in Dunedin, New Zealand, who were eligible for the longitudinal study based on residence in the province at age 3 and who participated in the first follow-up assessment at age 3. The cohort represents the full range of socioeconomic status (SES) in the general population of New Zealand's South Island and is primarily white (Moffitt et al. Reference Moffitt, Caspi, Rutter and Silva2001). On adult health, the cohort matches the NZ National Health & Nutrition Survey (e.g. body mass index, smoking, general practitioner visits) (Poulton et al. Reference Poulton, Hancox, Milne, Baxter, Scott and Wilson2006). Assessments occurred at birth and at ages 3, 5, 7, 9, 11, 13, 15, 18, 21, 26, 32, and, most recently, 38 years, when 95% of the living 1007 study members took part. At each assessment phase, study members were brought to the Dunedin Research Unit for a full day of interviews and examinations.
Ethics statement
The study protocol was approved by the institutional ethical review boards of the participating universities. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The Otago Ethics Committee approved each phase of the study. Informed consent was obtained from all study members.
Persistent substance dependence
Past-year substance dependence diagnoses were made using the Diagnostic Interview Schedule (DIS) following Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria (Robins et al. Reference Robins, Helzer, Croughan and Ratcliff1981, Reference Robins, Cottler, Bucholz and Compton1995; APA, 1987, 1994). We assessed alcohol, tobacco, and cannabis dependence at ages 21, 26, 32, and 38 and hard-drug dependence (e.g. heroin, cocaine) at ages 26, 32, and 38. DSM-III-R criteria were used at age 21 and DSM-IV criteria were used at ages 26, 32, and 38. We have previously compared prevalence rates of alcohol and cannabis dependence in the Dunedin Study with other representative studies of same-age respondents (Moffitt et al. Reference Moffitt, Caspi, Taylor, Kokaua, Milne, Polanczyk and Poulton2010). The past-year prevalence of alcohol dependence was similar in the Dunedin Study, the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC), and the National Comorbidity Survey (NCS; Moffitt et al. Reference Moffitt, Caspi, Taylor, Kokaua, Milne, Polanczyk and Poulton2010). The past-year prevalence of cannabis dependence was slightly higher in the Dunedin Study than in representative U.S. surveys, but was similar to another longitudinal, population-representative survey of New Zealanders (Moffitt et al. Reference Moffitt, Caspi, Taylor, Kokaua, Milne, Polanczyk and Poulton2010). The past-year prevalence of tobacco dependence in the Dunedin Study (averaged across ages 18–38, 17%) was similar to NESARC participants aged 18+ (13%) (Grant et al. Reference Grant, Hasin, Chou, Stinson and Dawson2004). (NCS did not assess past-year tobacco dependence.) Direct comparison of rates of hard-drug dependence across studies is difficult due to differences between studies in the drugs included in this category. However, in general, rates of hard-drug dependence appear to be slightly higher in the Dunedin Study. To summarize, the prevalence of alcohol and tobacco dependence in the Dunedin Study is similar to other representative U.S. studies, but the prevalence of cannabis and hard-drug dependence is slightly higher in Dunedin. One potential explanation for this is that the prevalence of cannabis and hard-drug dependence is, indeed, higher in New Zealand. Another potential explanation for this pattern of findings is that Dunedin Study participants, interviewed repeatedly over the course of their lives, have learned to trust the study's confidentiality guarantee, and are, therefore, more forthcoming about their illicit drug use.
We classified study members as persistently substance dependent if they were diagnosed with one or more of alcohol, tobacco, cannabis, or hard-drug dependence at 3+ assessment ages (ages 21, 26, 32, 38). For example, a study member could be considered persistently dependent if they were diagnosed with alcohol dependence at 3+ assessment ages (homotypic continuity). A study member could also be considered persistently dependent if they were diagnosed with tobacco dependence at age 21, followed by cannabis dependence at age 26 and hard-drug dependence at age 32 (heterotypic continuity). Of those classified as persistently dependent, 73% both diagnosed persistently for a single substance and across different substances. We chose a threshold of 3+ diagnoses to ensure that we were capturing individuals with severe, chronic dependence throughout adulthood.
We collapsed across substances in defining persistent dependence because practitioners want to predict risk for severe dependence on any substance, rather than dependence on a particular substance. This decision to collapse across substances is bolstered by evidence that (a) different substance-use disorders tend to co-occur (Table 1) (Krueger et al. Reference Krueger, Hicks, Patrick, Carlson, Iacono and McGue2002; Kendler et al. Reference Kendler, Jacobson, Prescott and Neale2003; McGue et al. Reference McGue, Iacono and Krueger2006), (b) a common liability underlies all substance-use disorders (Krueger et al. Reference Krueger, Hicks, Patrick, Carlson, Iacono and McGue2002; Kendler et al. Reference Kendler, Jacobson, Prescott and Neale2003; McGue et al. Reference McGue, Iacono and Krueger2006) and, (c) our results were similar across specific substances (Supplementary Tables S1–S4).
Prevalence of persistent (3+ diagnoses between ages 21 and 38) and lifetime (at least one diagnosis between ages 21 and 38) substance dependence by study member category
This table shows the prevalence of the disorders listed in the rows given the group listed in the columns. For example, of all adults in the cohort (column 1, N = 961), 4.17% had persistent alcohol dependence (row 1). As another example, of those with persistent alcohol dependence (column 2, N = 40), 47.50% had persistent tobacco dependence (row 2).
a Even after excluding tobacco dependence from the criteria for persistent substance dependence, 38.55% of the group with persistent substance dependence also met criteria for persistent tobacco dependence (last column, second row). This is because many of those who were persistently dependent on alcohol, cannabis, or hard drugs were also persistently dependent on tobacco, and 75.90% (last column, sixth row) had been dependent on tobacco at some point in their life between ages 21 and 38.
To be classified as persistently substance dependent, study members had to have been assessed for dependence at three of four assessment occasions. Ninety-three percent of the original 1037-member cohort were classified (910 had diagnostic data for four assessment occasions and 961 had data for three). Of those not classified, nearly half (n = 37) had either died or left the study before age 18 or had severe developmental disabilities that prevented their being interviewed with the DIS.
Risk factors
The nine childhood and adolescent risk factors are described in Table 2: SES, family history of substance dependence, conduct disorder, depression, early exposure to substances, frequent alcohol use, frequent tobacco use, frequent cannabis use, and male sex. We selected these particular risk factors because they (i) have been shown in longitudinal studies to consistently and robustly predict adult substance dependence, (ii) represent pre-specified domains of obvious interest (sociodemographic characteristics, mental health, and substance use), and (iii) have fairly natural cut-offs (Grant & Dawson, Reference Grant and Dawson1998; Chassin et al. Reference Chassin, Flora and King2004; Fergusson et al. Reference Fergusson, Horwood and Ridder2007; Pardini et al. Reference Pardini, White and Stouthamer-Loeber2007; Brook et al. Reference Brook, Lee, Brown, Finch and Brook2011; Hussong et al. Reference Hussong, Jones, Stein, Baucom and Boeding2011; Stone et al. Reference Stone, Becker, Huber and Catalano2012; Kendler et al. Reference Kendler, Myers, Damaj and Chen2013).
Childhood and adolescent risk factors
SES, Socioeconomic status.
a Male sex was also included as a risk factor in the cumulative risk index.
b Measures showed reliability >0.70.
c See Supplementary Table S6 for a sample brief screen form that could be used in community settings.
Statistical analysis
We summed the nine dichotomous childhood and adolescent risk factors to produce a single cumulative risk index that allowed us to classify individuals as persistently substance dependent based on their number of risks. We evaluated predictive accuracy using the traditional performance measures: area-under-the-curve, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Choosing a higher threshold for number of risks results in lower sensitivity (more false negatives) but higher specificity (fewer false positives). We plotted sensitivity against 1 – specificity for every value of the index, yielding a receiver-operating characteristic (ROC) curve. The area under the curve (AUC) provides a measure of predictive accuracy that reflects the probability of correctly classifying a randomly selected pair of individuals in which one has persistent substance dependence and the other does not. The AUC can take on any value between 0.50 (indicating chance prediction) and 1.00 (indicating perfect prediction). AUC values of 0.54, 0.64, and 0.71 correspond to Cohen's d values of 0.20, 0.50, and 0.80, and reflect small, medium, and large effects, respectively (Rice & Harris, Reference Rice and Harris2005).
Results
The prevalence of persistent adult substance dependence to age 38 in this population-representative cohort was 19% (n = 183). Table 3 shows that each childhood and adolescent risk factor significantly predicted persistent substance dependence in adulthood, and frequent tobacco use in adolescence was the best predictor (Table 3, left panel; AUC = 0.74). Adolescent tobacco use remained a top predictor even when tobacco dependence was excluded from the outcome (Table 3, right panel; AUC = 0.69), which was unsurprising given the high rates of co-morbidity between tobacco dependence and dependence on other substances (Table 1).
Relative risk of persistent substance dependence in adulthood given each childhood and adolescent risk factor
RR, Relative risk; CI, confidence interval; AUC, area under the curve.
The risk factors in this table were modestly correlated (r's ranged in absolute magnitude from 0.00 to 0.33). Note that AUC is a better indicator of classification accuracy than relative risk (Pepe et al. Reference Pepe, Janes, Longton, Leisenring and Newcomb2004).
The cumulative risk index (mean = 1.78, s.d. = 1.50) predicted persistent substance dependence in adulthood with considerable accuracy. The ROC analysis revealed an AUC of 0.80, a large effect, meaning that we had an 80% probability of correctly predicting, from a randomly selected pair of adolescents, which adolescent would have persistent substance dependence in adulthood. Results were similar when tobacco dependence was excluded from the outcome (Table 3, right panel; AUC = 0.81) and when predicting persistent dependence on each substance individually (Supplementary Tables S1–S4).
The prevalence of persistent substance dependence increased markedly as a function of number of childhood and adolescent risks (Fig. 1a , striped bars): 3% of adolescents with 0 risk factors, 27% of adolescents with 3 risk factors, and 74% of adolescents with 6+ risk factors had persistent substance dependence as adults.
(a) Percentage of the population-representative cohort who had 1+, 2+, or 3+ dependence diagnoses as a function of number of childhood and adolescent risks. Percentages (shown above each bar) can be used to calculate an adolescent's relative risk for persistent adult substance dependence. For example, adolescents with 2 risks were seven times more likely to diagnose with dependence 3+ times between ages 21 and 38 than their peers with 0 risks (21/3 = 7). (b) Mean number of substance dependence diagnoses between ages 21 and 38 as a function of number of childhood and adolescent risks. Note. Number of diagnoses ranged from 0 (was not diagnosed with dependence at age 21, 26, 32, or 38) to 4 (was diagnosed with dependence at age 21, 26, 32, and 38). Mean number of dependence diagnoses is shown above each bar. For example, the mean number of substance dependence diagnoses between ages 21 and 38 for people with 2 risks was 1.29.
Table 4 shows sensitivity, specificity, PPV, NPV, and overall classification accuracy of the cumulative risk index as a function of number of risks. Overall accuracy was greatest at a cut-off on the cumulative risk index of 4+ to 5+ risks.
Sensitivity, specificity, positive predictive value, negative predictive value, and overall prediction accuracy of the cumulative risk index as a function of number of risk factors (n = 961)
PPV, Positive predictive value: the % of those predicted to develop persistent substance dependence (3+ diagnoses of substance dependence between ages 21 and 38) who actually developed persistent substance dependence. NPV, negative predictive value: the % of those predicted not to develop persistent substance dependence who actually did not develop persistent substance dependence. Overall accuracy = the proportion of the sample that was correctly classified. We included study members with missing predictor data in our analysis of the cumulative risk index. Ninety-five percent of the 961 study members had data for 7+ of the nine predictors. When we restricted the analysis to participants with no missing predictor data, results were unchanged.
Moderation by sex
Male sex was included as a risk factor in the cumulative risk index, but to test the possibility that the cumulative risk index is a more accurate predictor of persistent substance dependence for one sex, we examined sex as a moderator. First, we tested whether there were sex differences in the associations between each risk factor in the cumulative risk index and persistent substance dependence. There was no evidence that sex moderated any of these associations. Next, we removed male sex as a risk factor in the cumulative risk index and recomputed the AUC for the cumulative risk index separately by sex. The cumulative risk index (without sex as a risk factor) predicted persistent substance dependence similarly well for girls (AUC = 0.81) and boys (AUC = 0.78). Moreover, the cut-off score that maximized overall classification accuracy was 4+ to 5+ risks for girls and 3+ risks for boys, v. 4+ to 5+ risks for both girls and boys when male sex was included as a risk factor in the cumulative risk index. Findings suggest that the cumulative risk index predicts persistent substance dependence similarly for girls and boys. A practical advantage of including male sex as a risk factor in the cumulative risk index is that it equates the cut-off score for girls and boys.
Sensitivity analyses
We tested whether prediction could be improved by adding another risk factor to the cumulative risk index – either low childhood self-control or childhood maltreatment. These risk factors were selected because they have been shown to predict risk for substance dependence and because they were available in the dataset. Both low childhood self-control and childhood maltreatment were associated with increased risk of persistent substance dependence, and they predicted persistent substance dependence with similar accuracy to other predictors (Supplementary Table S5; AUCs = 0.55 and 0.57, respectively). Adding these risk factors to the cumulative risk index did not improve accuracy. When either risk factor was added, the cumulative risk index predicted persistent substance dependence with an AUC of 0.79.
Next we tested the effects of dropping a risk factor from the cumulative risk index. Table 5 shows that the AUC did not drop substantially with the exclusion of any single risk factor, with exception of frequent tobacco use in adolescence.
Accuracy of the cumulative risk index in predicting persistent substance dependence after systematically removing one risk factor from the index
AUC, Area under the curve.
Finally, we tested the robustness of the cumulative risk index by reducing the threshold for ‘persistent’ substance dependence from 3+ to 2+ diagnoses across assessment ages. The cumulative risk index was about as accurate when predicting 2+ substance dependence diagnoses from ages 21 to 38 (AUC = 0.77) as when predicting 3+ diagnoses (AUC = 0.80). The cumulative risk index was also fairly accurate when predicting 1+ diagnoses from ages 21 to 38 (AUC = 0.76).
The cut-off score on the cumulative risk index that maximized overall classification accuracy dropped from 4–5+ risks to 3–4+ risks to 2+ risks for predicting 3+, 2+, and 1+ dependence diagnoses, respectively, from ages 21 to 38. This is not surprising given the dose-response association between number of childhood and adolescent risk factors and persistence of substance dependence. For example, the mean number of substance dependence diagnoses from ages 21 to 38 increased in a fairly linear fashion as a function of number of childhood and adolescent risk factors (Fig. 1b ). Fig. 1a shows that cohort members with a greater number of risk factors were at higher risk of dependence, regardless of how we defined the dependence outcome (1+, 2+, or 3+ diagnoses). The vast majority of individuals with 0 risks never developed dependence (79%), whereas this was true of only 6% of those with 4+ risks.
Comparison with other risk screens
Both NIDA's Quick Screen and NIAAA's Alcohol Screening Guide for Children and Adolescents rely exclusively (NIDA) or heavily (NIAAA) on assessing frequency of drug use and/or alcohol use. Table 3 and Supplementary Tables S1–S4 show that by expanding risk assessment beyond frequency of drug and alcohol use, prediction was improved. For example, frequent cannabis use in adolescence predicted persistent cannabis dependence in adulthood with an AUC of 0.67, whereas the cumulative risk index predicted persistent cannabis dependence with an AUC of 0.83 (Supplementary Table S3). The cumulative risk index was more accurate in predicting persistent dependence on each substance than frequency of alcohol or drug use alone, with one exception. Frequent tobacco use in adolescence predicted persistent tobacco dependence as well as the cumulative risk index (Supplementary Table S2).
Brief screen adaption
We adapted our risk measures for a brief screen to see whether the cumulative risk index could be used in community settings by practitioners with limited time to conduct the detailed risk assessments in our research study. Table 2 shows how we adapted each measure, and Supplementary Table S6 shows the brief screen. The brief screen (i.e. a cumulative risk index based on the sum of the eight adapted risks) performed nearly as well the cumulative risk index based on our more detailed risk assessments (AUC = 0.79 v. 0.80, respectively). Like the cumulative risk index based on our more detailed risk assessments, the brief screen was more accurate in predicting persistent dependence on each substance than frequency of alcohol or drug use alone with one exception. Frequent tobacco use in adolescence predicted persistent tobacco dependence as well as the brief screen. We report additional information on the accuracy of this brief-screen adapted risk index in Supplementary Table S7 and Supplementary Fig. S1.
Discussion
This report advances knowledge by suggesting answers to three recently posed questions about screening adolescents for risk of substance-use disorders (Subramaniam & Volkow, Reference Subramaniam and Volkow2014). The first question is: How can we combine multiple risk factors to estimate an adolescent's risk? Current screening tools rely heavily on assessing adolescent substance use, yet other risk factors, such as psychiatric disorder, also predict risk (Subramaniam & Volkow, Reference Subramaniam and Volkow2014). In this report, we showed that summing a small set of dichotomous risks into a single cumulative risk index is a clinically useful way to integrate risk factors and accurately predict persistent substance dependence. Many studies have shown that cumulative childhood risk is associated with adult mental and physical health problems (Rutter, Reference Rutter1981; Sameroff et al. Reference Sameroff, Seifer, Barocas, Zax and Greenspan1987; Felitti et al. Reference Felitti, Anda, Nordenberg, Williamson, Spitz, Edwards, Koss and Marks1998; Evans et al. Reference Evans, Li and Whipple2013). A recent study even showed that cumulative risk distinguishes those with persistent alcohol problems from those with time-limited alcohol problems (Copeland et al. Reference Copeland, Angold, Shanahan, Dreyfuss, Dlamini and Costello2012). The current report extends this work to suggest how cumulative risk could be used as an actuarial risk assessment tool in community settings to accurately predict persistent substance dependence in the general population.
The second question we address is: Who is at risk? This report provides initial population-representative estimates of risk that can be used to gauge an individual adolescent's likelihood of having persistent substance dependence in adulthood. For example, 3% of adolescents with 0 risks, 27% of adolescents with 3 risks, and 74% of adolescents with 6+ risks developed persistent substance dependence in adulthood. In the future, practitioners may use risk estimates such as these (aggregated across more population-representative studies like ours) to make actuarial judgments and referrals to treatment. One may even envisage members of the general population calculating their risk for persistent substance dependence on their own, just as they now can calculate their risk for heart attack (http://cvdrisk.nhlbi.nih.gov/calculator.asp).
The third question we address is: How should we decide the appropriate level of intervention for an at-risk adolescent? By triaging adolescents into different levels of intervention based on their risk, we might avoid under- or over-treatment and reduce the costs of treatment. The cumulative risk index lends itself to triaging, because we documented a dose-response association between number of childhood and adolescent risk factors and persistence of substance dependence. That is, individuals with more risks had more persistent substance dependence. This dose-response association suggests that adolescents with more risks may require more intensive intervention, whereas individuals with fewer risks may benefit from brief interventions. Although more work will be needed to determine the cut-off scores on the cumulative risk index that decide level of care, we illustrate one possible way that the cumulative risk score could be used.
Adolescents with 4+ risks might be candidates for intensive intervention. A cut-off score of 4+ risks maximized overall accuracy in predicting persistent dependence in adulthood. Moreover, specificity at 4+ risks was high (93%), ensuring that costly interventions go to the small portion of the population (13.6%) that really need it (Table 4). Overtreatment of adolescents with 4+ risks is unlikely, as 94% were diagnosed with dependence at least once between ages 21 and 38 and 82% were diagnosed at least twice. These adolescents may benefit from broad-based interventions (i.e. interventions that target a variety of risk factors, not just substance use) because of the number and variety of their accumulated risks. Interventions need not focus exclusively on the small set of predictors studied here. These predictors were selected for the explicit purpose of efficiently identifying those at highest risk. Intervention design should be guided by the literature on effective treatments that target mutable risks for a variety of problem behaviors.
Brief interventions might be offered to adolescents with 2 and 3 risk factors. These adolescents were also at increased risk for persistent substance dependence, but compared with adolescents with 4+ risks, their struggles with dependence were more likely to be time-limited (Supplementary Fig. S2). The screening itself could serve as the basis for a brief intervention, as the cumulative risk index provides a powerful means of communicating to adolescents their overall risk in clear and understandable terms. Adolescents with 2 risks were seven times more likely than their peers with 0 risks to struggle with persistent dependence through early midlife. Adolescents with 3 risks were nine times more likely than their peers with 0 risks to have persistent dependence. For some adolescents, information about their risk status might motivate behavior change towards a lower risk category and prevent initiation or escalation of substance use. For adolescents who have 2 or 3 risk factors, and at least one of those risk factors includes frequent substance use, feedback about risk status might be combined with brief motivational interviewing and/or harm-reduction strategies focused on substance use. For adolescents with symptoms of depression or conduct problems, brief interventions targeting these symptoms might be appropriate (e.g. behavioral activation for depression or brief behavioral parenting strategies for disruptive behavior).
Our results may well prove useful in designing and evaluating intensive prevention/early intervention programs in the future. Researchers planning these programs need to know approximately what proportion of their treatment group would develop persistent dependence. This information could be used to calculate the ‘number needed to treat’ to prevent one case – a measure that is increasingly used to gauge the effectiveness of an intervention. For example, if adolescents with 4+ risks were the target of intervention, 60% would otherwise develop persistent substance dependence in adulthood (Table 4, PPV). Given a perfectly effective intervention, the ‘number needed to treat’ to prevent one case would be 1.67 (1/.60 = 1.67) (Cook & Sackett, Reference Cook and Sackett1995). As the proportion of the treatment sample who would otherwise develop persistent dependence decreases, the number needed to treat increases. Thus, prevention/early intervention studies that cast too wide a net in defining their treatment group have, from the start, limited their treatment's effectiveness.
The results of this study should be interpreted in the context of its limitations. First, although we selected some of the best predictors of substance dependence, prediction might be marginally improved by adding childhood and adolescent risks. Our sensitivity analyses showed that, with exception of adolescent frequent tobacco use, adding or subtracting a predictor did not make a difference in accuracy of the cumulative index. Moreover, each additional predictor yields diminishing returns while lengthening assessment (Ware, Reference Ware2006). Substituting different predictors from the ones we examined here could lead to improvements in prediction, but multiple datasets will be needed to fully explore this possibility to avoid overfitting the model to this cohort. Substantial improvement in prediction may be unlikely however, because prediction is already quite good. A more promising strategy for improving upon prediction might be to include more predictors in models that could reveal combinations of risk factors that predict an especially high risk of persistent substance dependence (e.g. decision tree, cluster analysis, or neural network models).
A second limitation is that our findings are based on a single New Zealand cohort and require replication in independent samples. We view this report as proof of concept for using a cumulative risk index as an actuarial tool. The next steps include testing and refining the cumulative risk index in contemporary cohorts and in special populations, such as Native American adolescents, as well as testing the accuracy of the cumulative risk index in predicting persistent substance-use disorder, as defined by DSM-5.
In summary, we developed a universal screen for persistent substance dependence that (1) is based on population-representative data, (2) estimates total risk briefly and inexpensively by incorporating a relatively small number of well-established risk factors, and (3) aggregates risk factors using a simple algorithm. Although findings are preliminary, they suggest that we can predict with considerable accuracy which adolescents in the general population will struggle with persistent substance dependence in adulthood. The cumulative risk index may be simpler to use, less costly, and more accurate (AUC = 0.80) than more comprehensive screens, and these features are important considerations in universal screening. For example, an extensive profile of neuropsychosocial risk, including measures of brain and cognitive function, predicted adolescent binge drinking with an AUC of 0.75 (Whelan et al. Reference Whelan, Watts, Orr, Althoff, Artiges, Banaschewski, Barker, Bokde, Buchel, Carvalho, Conrod, Flor, Fauth-Buhler, Frouin, Gallinat, Gan, Gowland, Heinz, Ittermann, Lawrence, Mann, Martinot, Nees, Ortiz, Paillere-Martinot, Paus, Pausova, Rietschel, Robbins, Smolka, Strohle, Schumann and Garavan2014). We also showed that the cumulative risk index compared favorably to current risk assessment approaches, which narrow in on substance use as the primary risk indicator. Moreover, an adapted version of the cumulative risk model for use in community settings (a version that simply summed readily obtained adolescent risks) yielded an AUC of 0.79. Additional research is needed to validate the cumulative risk index, evaluate its practical utility, and address potential ethical issues that may be raised by screening adolescents for persistent substance dependence (Carter & Hall, Reference Carter and Hall2011; Hall et al. Reference Hall, Carter, Yücel, Clausen and Levy2015). The results presented here represent a first step toward establishing population-representative estimates of risk for persistent substance dependence that may be useful in research and practice.
Supplementary material
For supplementary material accompanying this paper visit http://dx.doi.org/10.1017/S0033291715002482.
Acknowledgements
We thank the Dunedin Study members, their families, Unit research staff, and Study founder Phil Silva. The Dunedin Multidisciplinary Health and Development Research Unit is supported by the New Zealand Health Research Council and New Zealand Ministry of Business, Innovation, and Employment (MBIE). This research received support from US-National Institute on Aging (AG032282, AG048895); UK Medical Research Council (MR/K00381X); and ESRC (ES/M010309/1). Additional support was provided by the Jacobs Foundation. Wayne Hall was funded by a National Health and Medical Research Council Australia Fellowship. Daniel W. Belsky was supported by the National Institute on Aging (T32-AG000029).
Declaration of Interest
None.
