Introduction
Globally, depression and anxiety are leading causes of disability among young people (Whiteford et al., Reference Whiteford, Degenhardt, Rehm, Baxter, Ferrari, Erskine and Vos2013). These disorders typically first emerge during adolescence, a period of marked psychological, biological and social change (Andrews, Ahmed, & Blakemore, Reference Andrews, Ahmed and Blakemore2021; Kessler et al., Reference Kessler, Angermeyer, Anthony, De Graaf, Demyttenaere, Gasquet and Üstün2007; Merikangas et al., Reference Merikangas, He, Burstein, Swanson, Avenevoli, Cui and Swendsen2010). During adolescence, one's mental health status is predictive of future lifetime mental health, and the subsequent social and economic costs associated with depression and anxiety are large (Hu, Reference Hu2006; Patton et al., Reference Patton, Coffey, Romaniuk, Mackinnon, Carlin, Degenhardt and Moran2014; Whiteford et al., Reference Whiteford, Degenhardt, Rehm, Baxter, Ferrari, Erskine and Vos2013). To ameliorate these effects, scalable, and effective, preventive interventions are necessary.
Schools are among some of the key places with which to deliver preventive interventions, and if administered early enough may have the potential to prevent the emergence of mental health problems by fostering resilience and coping strategies among young people. However, current evidence suggests the effects of school-based universal mental health interventions are small and most effective at targeting anxiety, rather than depressive, symptoms, leaving significant room for improvement (Caldwell et al., Reference Caldwell, Davies, Hetrick, Palmer, Caro, López-López and Welton2019). Despite these small effects, there remains considerable potential for preventive universal school-based mental health interventions to improve the well-being of young people, and this necessitates the development, implementation and evaluation of novel programmes (Tanner-Smith & Grant, Reference Tanner-Smith and Grant2018; Werner-Seidler, Perry, Calear, Newby, & Christensen, Reference Werner-Seidler, Perry, Calear, Newby and Christensen2017). Indeed, universal school-based interventions may be favoured over targeted interventions as they are cost effective and less likely to confer social risks, such as bullying or stigma, that may be more commonly experienced by young people enrolled in targeted interventions (Fazel & Kohrt, Reference Fazel and Kohrt2019).
One factor that has been associated with adolescent mental health outcomes is a person's position within their respective social network. For example, studies have indicated that adolescents who are less connected to others in their school classes report higher depressive symptoms, relative to more well-connected peers (Okamoto et al., Reference Okamoto, Johnson, Leventhal, Milam, Pentz, Schwartz and Valente2011; Ueno, Reference Ueno2005). In addition, one study which used advanced remote sensing to track the real-world movements of a cohort of adolescents at a summer camp found that individuals who were more socially isolated and spent less time interacting with others, relative to their peers, reported higher depressive symptoms (Elmer & Stadtfeld, Reference Elmer and Stadtfeld2020). In another study of over 10 000 high school students, pupils belonging to less cohesive school-classes, characterised by fewer reciprocal friendships, were at increased risk of suicide ideation and suicide attempts (Wyman et al., Reference Wyman, Pickering, Pisani, Rulison, Schmeelk-Cone, Hartley and Valente2019). Indeed, forming and maintaining stable peer relationships and a strong sense of belonging is proposed to be critical for mental health during adolescence (Tomova, Andrews, & Blakemore, Reference Tomova, Andrews and Blakemore2021). It is important to note these findings are based on correlational evidence; relationships between social network position and mental health problems are likely to be reciprocal and temporally dynamic. Whilst these findings highlight an important association between a person's position within their school social network and their mental health, it is unknown whether individuals who are more or less central (i.e. more of less connected to their peers) are equally responsive to school-based mental health interventions. This is an important question to answer as arguably the less connected young people may have more to gain from these interventions.
In the current study we assessed the efficacy of a universal eHealth preventative school-based mental health intervention, relative to a passive control, which formed part of a large multicentre, cluster-randomised controlled trial in Australian secondary schools: The Climate Schools-combined (CSC) trial. The CSC study aimed to compare the effectiveness of a combined substance use and mental health eHealth intervention with (1) a mental health-only intervention, (2) a substance use only intervention and (3) a passive control (Teesson et al., Reference Teesson, Newton, Slade, Chapman, Allsop, Hides and Andrews2014). Previous studies reporting the outcomes of this trial have focused on the effectiveness of the combined intervention and the substance use intervention. The main trial findings indicated that, compared with controls, the combined intervention reduced odds of drinking and reduced increases in anxiety symptoms, but no effect on depressive symptoms was observed. Furthermore, individuals randomised to the combined intervention showed improvements in mental health outcomes at 6 months post intervention, relative to the mental health-only intervention (Teesson et al., Reference Teesson, Newton, Slade, Chapman, Birrell, Mewton and Andrews2020). As outlined in the published protocol for the main study, this trial was also powered to investigate differences between each of the study arms. As such, secondary analysis of the trial data has shown that, those who received the substance use intervention showed a slower increase in their odds of drinking and heavy episodic drinking when compared with controls at 12 months (Newton et al., Reference Newton, Chapman, Slade, Birrell, Healy, Mather and Teesson2020). However, comparisons between the mental health-only intervention and the control group have not yet been examined and form the basis of the present study.
We preregistered two hypotheses (https://osf.io/68f47). First, that individuals randomised to receive the Climate Schools mental health intervention will report improved mental health outcomes across time, relative to those individuals randomised to the control group. Second, that baseline individual social network characteristics will moderate the effectiveness of the school-based mental health intervention.
Method
Trial design and randomisation
The present study analyses data from a multicentre, cluster-randomised controlled trial which was conducted in Australian secondary schools, across three states (New South Wales, Western Australia and Queensland). Eighty-eight participating schools were randomly assigned to one of the four study conditions: Climate Schools-combined, Climate Schools-substance use, Climate Schools-mental health or passive control. The current study analyses data from the schools that either received the Climate Schools-mental health intervention or were allocated to the passive control. Schools were assigned to the four interventions in a 1:1:1:1 ratio, using blocked randomisation, with blocks of four using the Stata ralloc command.
Participants and sample size
The sample size calculations for the full study aimed to achieve 80% power to detect a standardised between-group mean difference of 0.15 (p = 0.05) in outcomes between each of the study arms at the end of the trial with up to seven measurement occasions. An effect size of 0.15 is comparable to previous trials of digital mental health interventions for depression and anxiety in youth (Garrido et al., Reference Garrido, Millington, Cheers, Boydell, Schubert, Meade and Nguyen2019).
The only exclusion criterion to participation was the lack of parental consent. All data were collected on site, in the schools. Following randomisation, 17 schools withdrew (the main reasons include lack of time and school resources to run the study), leaving a final cohort of 71 schools, of which 18 were allocated to the Climate Schools-mental health condition and 19 to the control condition. Further details can be found in the study protocol paper (Teesson et al., Reference Teesson, Newton, Slade, Chapman, Allsop, Hides and Andrews2014). For modelling purposes, two schools in the Climate Schools-mental health condition had two cohorts assessed and are therefore treated as two separate groups in subsequent analyses, bringing the total number of school groups to 39 (19 control; 20 mental health). Number of participants who answered at least one of our mental health outcomes were at baseline (control: 1490; intervention: 1710), immediately following the intervention (control: 1267; intervention: 1272), at 6 months (control: 1285; intervention: 1195), at 12 months (control: 1154; intervention: 1094) and at 18 months (control: 1049; intervention: 1016).
All year 8 pupils from participating schools were invited to participate. Only those pupils who consented, and whose parents also consented to their participation, were eligible for inclusion in the trial. The mean age of participants in the control group at baseline was 13.5 and, in the Climate Schools-mental health condition it was 13.6 years. At 18 months follow-up the mean age in the control group was 15.8 years, and in the mental health group it was 15.5 years.
Intervention
Schools that were randomised to Climate Schools-mental health delivered the universal eHealth mental health course to their year 9 (year 10 in Queensland) pupils during health education classes. This course is based on cognitive-behavioural principles and incorporates skill acquisition, psychoeducation, management of psychological symptoms, cognitive symptoms, behaviour and additional skills that are specific to anxiety and depression. The course comprised of six 40-min lessons aimed at reducing anxiety and depression (see online Supplementary Table S1 for more details). This intervention was delivered alongside each school's standard mental health education. Each lesson comprised of a 20-min online cartoon component that was completed individually by students, followed by a 20-min activity delivered by the teacher, which reinforced the information in the cartoons and allowed for interactive communication. The cartoon storyboards were co-developed with young people and evaluated by teachers in order to be appropriately engaging. Teachers were provided with both an online and hard-copy manual containing the activities, implementation guidelines, links to the education syllabus and teacher and student summaries for each lesson. Teachers and students were provided with confidential login details to access the study website. Any additional homework assignments or reinforcement activities were introduced at the discretion of the teacher, in addition to the six teacher-led sessions. Parents were informed of the skills being taught in the consent process. Students were unable to progress to the content of the next online lesson until the previous lesson had been completed. Data on the fidelity of the mental intervention are reported in the online Supplementary material as is additional information on teaching as usual for the control group.
Schools that were randomised to the passive control condition delivered their usual health education classes over the year, including lessons on alcohol, other drugs and mental health. In Australia, this education is a mandatory part of the secondary-school health curriculum and all control schools reported delivering such lessons during this trial (for more details see Teesson et al., Reference Teesson, Newton, Slade, Chapman, Birrell, Mewton and Andrews2020).
For the purposes of the current study, baseline data were collected in term 3, 2014, prior to the administration of the mental health intervention. Data were subsequently collected in term 1, 2015 (immediately following the mental health intervention), in term 3, 2015 (6 month follow-up), in term 1, 2016 (12 month follow-up) and in term 3, 2016 (18 month follow-up) to establish the immediate and longer-term outcomes. For more information on the timing of data collection see the study protocol (Teesson et al., Reference Teesson, Newton, Slade, Chapman, Allsop, Hides and Andrews2014).
Trial registration and ethics
The research protocol, which details informed consent procedures and sample size calculations, was approved by the University of Sydney, University of New South Wales, Curtin University, Queensland University of Technology Human Research Ethics Committees, the Western Australian and Brisbane Catholic Education Offices, and the New South Wales, Queensland and Western Australian Department of Education and Training (Teesson et al., Reference Teesson, Newton, Slade, Chapman, Allsop, Hides and Andrews2014).
The trial is registered with the Australian and New Zealand Clinical Trials registry (ACTRN12613000723785).
Measures
Each of these outcome measures and social network measures were pre-specified in our pre-registration for the current study (https://osf.io/68f47). Each of the outcome measures included were also included in the study protocol and (Teesson et al., Reference Teesson, Newton, Slade, Chapman, Allsop, Hides and Andrews2014) and the trial registration, for the main study (see the above section).
Outcomes
Our primary outcome measure was internalising symptoms, which were measured using the Strengths and Difficulties Questionnaire (SDQ). This is a composite score combining the emotional problems and peer problems subscale of the SDQ and is scored on a scale of 0–20 and is a preferable measure to the individual subscales in community samples (Goodman, Lamping, & Ploubidis, Reference Goodman, Lamping and Ploubidis2010). Our secondary outcomes of depression and anxiety were assessed using the Patient Health Questionnaire-8 (PHQ-8) and the Generalised Anxiety Disorder scale (GAD-7), respectively. Scores on the PHQ-8 range from 0 to 24 and scores on the GAD range from 0 to 21 and have been shown to be reliable measures of depression and anxiety among adolescents (Johnson, Harris, Spitzer, & Williams, Reference Johnson, Harris, Spitzer and Williams2002; Spitzer, Kroenke, Williams, & Löwe, Reference Spitzer, Kroenke, Williams and Löwe2006). In addition to a continuum assessment of depressive and anxiety symptoms, we also created two binary outcome variables for probable depression and anxiety, determined by individuals scoring above 10 on the PHQ-8 and GAD-7, respectively (Levis, Benedetti, & Thombs, Reference Levis, Benedetti and Thombs2019; Spitzer et al., Reference Spitzer, Kroenke, Williams and Löwe2006). We additionally measured psychological distress with the six item K6 (Kessler et al., Reference Kessler, Barker, Colpe, Epstein, Gfroerer, Hiripi and Zaslavsky2003) and also mental health knowledge which was assessed using a 13 item multiple choice questionnaire (for more details, and the full questionnaire, see the online Supplementary material). All outcome measures were assessed at each time point, except for the SDQ which was not asked directly following the intervention at the post-intervention follow-up, but first asked at 6 months post intervention. This was because a reduced survey was asked at this time point to minimise respondent burden in the context of assessments across the full trial.
Social network measures
Not all schools consented to participate in the social network component of the trial. Therefore, a reduced subset of participants from participating schools completed a social network survey at baseline (N = 1100). The social network survey asked students to nominate up to three people with whom they spent most of their free time with from their year group who also provided consent to be included in the CSC trial. As per ethical approvals, students could only nominate other students in their year group who also consented to participate in the CSC trial. Schools participating in this data collection were from NSW and WA and required computerised survey collection. From this we computed two measures of social connectedness: in-degree centrality (the number of friendship nominations an individual received from their peers) and betweenness centrality (the extent to which an individual connects two otherwise unconnected individuals). For more detail on the network construction see the online Supplementary material.
Statistical analyses
Analyses were conducted using R version 4.0.2 (R Core Team, 2021) and followed a pre-registered analysis plan. Outcomes were analysed using multilevel (generalised) linear mixed-effects models to account for the clustered structure of the data, with clustering occurring both within schools and within individuals over time. Linear mixed-effects models were used for continuous outcomes. We also computed binary scores for probable depression and anxiety, for which the analysis procedure and results are presented in online Supplementary Tables S6–S8. All analyses used an intention-to-treat approach, including all relevant measurements for all students. Time was treated as a categorical variable representing number of years/months since baseline. The fixed effects parameters included group (i.e. intervention condition, Climate Schools-mental health v. control) and time as categorical predictors, as well as the group × time interaction. The control group and baseline time point were treated as the reference. Random intercepts were included at the individual and school levels, as well as random slopes at the individual level to allow for different individual trajectories over time. Random slopes for individuals were only included if they improved model fit over the random intercept model. Where models failed to converge or produce reliable estimates we reverted to the simpler model. Intervention effects were assessed on the basis of the group × time interaction coefficients, representing the change from baseline to each follow-up time point for the intervention group compared with the control group.
Effect sizes, calculated as standardised mean differences (Cohen's d) for continuous outcomes and odds ratios for binary outcomes, on the basis of group × time coefficients, were also computed. Sensitivity analyses controlling for covariates considered to be confounders including gender, baseline depression (for anxiety outcomes) and baseline anxiety (for depression outcomes) were also conducted.
Subsequent analyses were conducted to examine the moderating effect of each social network characteristic (in-degree centrality and between centrality) on our internalising problems, depression and anxiety outcome variables. We compared the fit of a model including each social network variable (taken from the baseline testing session), to a model without. If the model including the social network variable provided a better fit to the data, we explored this model further. A moderating effect of the social network characteristics was assessed on the basis of three-way group × time × social network interaction coefficients. For continuous outcomes model fit was established by using standard Akaike's information criteria (AIC). The model with the lowest AIC was selected and confirmed with a likelihood ratio test (Lewis, Butler, & Gilbert, Reference Lewis, Butler and Gilbert2011).
Results
Descriptive statistics for each outcome assessing internalising problems, depression and anxiety, across time and by condition are reported in Table 1 (see online Supplementary Table S2 for binary outcomes). For each of our continuous outcome measures we report the results of a linear mixed effects model predicting our outcome measure with main effects of time and condition, a time by condition interaction and nested random intercepts for participant and school. Across each outcome variable a model including an additional random slope for participant did not converge.
Mean and standard deviations (s.d.) for continuous outcomes across time and by condition
Hypothesis one
Primary outcome: internalising problems
Omnibus tests revealed a statistically significant time by condition interaction (F = (3, 6728.8) = 7.15, p < 0.001). This interaction was explained by a statistically significant condition by time interaction for internalising scores at 6 [B = 0.67, 95% confidence interval (CI) 0.37–0.97, p < 0.001, d = 0.11] and 12 months (B = 0.38, 95% CI 0.07–0.99, p = 0.016, d = 0.06) such that individuals in the Climate Schools-mental health condition reported higher internalising scores relative to the control at the 6- and 12-month follow-up (see Table 2 and Fig. 1). However, there were no significant difference at the 18-month follow-up. There was a statistically significant main effect of time (F = (3, 6728.8) = 22.15, p < 0.001), which was explained by a statistically significant effect for internalising scores at 18 months (B = 0.55, 95% CI 0.33–0.78, p < 0.001, d = 0.12) (see Table 2). There was no statistically significant main effect of condition (F = (1, 39.0) = 1.17, p = 0.29). All effects remained statistically significant when controlling for gender (see online Supplementary Table S3 for the results of these sensitivity analyses).
Model-predicted means for each outcome variable. Error bars represent standard errors. Panel A depicts depressive symptom scores (PHQ), panel B depicts anxiety symptom scores (GAD). There was no significant time by condition interactions for depression (F = (4, 9241.0) = 0.83, p = 0.51) or anxiety (F = (4, 9234.3) = 0.84, p = 0.50). Panel C depicts internalising symptom scores (SDQ). There was a significant time by condition interaction (F = (3, 6728.8) = 7.15, p < 0.001). This interaction was explained by a statistically significant condition by time interaction for internalising scores at 6 (B = 0.67, 95% CI 0.37–0.97, p < 0.001, d = 0.11) and 12 months (B = 0.38, 95% CI 0.07–0.99, p = 0.016, d = 0.06) such that individuals in the Climate Schools-mental health condition reported higher internalising scores relative to the control at the 6- and 12-month follow-up, however the observed differences were small and did not persist by 18-month follow.
Model estimates for internalising problems
σ 2 represents the residual variance, unexplained by the predictors in the model. τ 00 represents the between subject variance. The interclass correlation coefficient (ICC) is the proportion of the variance explained by the grouping structure in the population.
Secondary outcomes
Depression. Omnibus tests revealed no statistically significant time by condition interaction (F = (4, 9241.0) = 0.83, p = 0.51) or main effect of condition (F = (1, 35.8) = 0.27, p = 0.87). There was a statistically significant main effect of time (F = (4, 9241.0) = 8.43, p < 0.001). This main effect of time was explained by a statistically significant effect for depressive scores post-intervention (B = 0.36, 95% CI 0.04–0.68, p = 0.027, d = 0.05) and at18 months (B = 0.65, 95% CI 0.31–0.99, p < 0.001, d = 0.08) such that across all participants we observed significant increases in depressive scores immediately post, and at 18-month post, intervention (see Table 3 and Fig. 1). All effects remained statistically significant when controlling for gender and baseline anxiety (see online Supplementary Table S4 for the results of these sensitivity analyses).
Model estimates for depressive and anxiety symptoms
σ 2 represents the residual variance, unexplained by the predictors in the model. τ 00 represents the between subject variance.
Anxiety. Omnibus tests revealed no statistically significant time by condition interaction (F = (4, 9234.3) = 0.84, p = 0.50) or main effect of condition (F = (1, 34.3) = 0.36, p = 0.55). There was a statistically significant main effect of time (F = (4, 9234.3) = 9.81, p < 0.001). This main effect of time was explained by a statistically significant effect for anxiety scores at 18 months (B = 0.64, 95% CI 0.34–0.94, p ≤ 0.001, d = 0.09) such that across all participants we observed significant increases in anxiety scores at 18-month post intervention (see Table 3 and Fig. 1). All effects remained statistically significant when controlling for gender and baseline depression (see online Supplementary Table S5 for the results of these sensitivity analyses).
Hypothesis two
Participants from 10 of the 39 school groups (N = 1100) included in hypothesis one provided social network data. Four of these schools were randomised to receive the Climate Schools-mental health intervention and six were randomised to the control group. Before exploring the moderating effect of the social network measures, we re-ran the analyses from hypothesis one on the subset of participants who had complete social network data. The results from this subsample of schools and participants show a similar pattern of results as the full sample. For a full breakdown of these additional models see online Supplementary Tables S9–S11.
Moderating effect of social network position
Across all our outcome measures a model including a three-way interaction between time, condition and indegree, or betweenness, centrality did not provide a better fit to the data, relative to a simpler model. See Table 4 for a comparison of model fits for our continuous measures and online Supplementary Table S12 for our binary measures. We ran several exploratory correlation analyses between our continuous baseline outcome and social network measures. Across each outcome we observed no significant relationship with our social network measures (see online Supplementary Fig. S1). Despite some observable floor effects in these plots, mixed effects models are robust to violations of distributional assumptions (Schielzeth et al., Reference Schielzeth, Dingemanse, Nakagawa, Westneat, Allegue, Teplitsky and Araya-Ajoy2020).
AIC scores for models including an interaction with social network measures compared to a model without, for each continuous outcome variable
To make appropriate comparisons, models predicting continuous outcomes were estimated using maximum likelihood (ML), rather than restricted maximum likelihood (REML) and were compared based on AIC. In each case the model with the lowest AIC was statistically confirmed as the best fitting model with a likelihood ratio test (p > 0.05).
Additional analyses
In addition to our pre-registered analyses, we also assessed intervention effects on psychological distress (assessed with the K6) and mental health knowledge. There was no statistically significant time by condition interaction effect for psychological distress (F = (4, 9254.7) = 2.12, p = 0.08). However, there was a statistically significant time by condition interaction effect for mental health knowledge (F = (4, 8461.3) = 0.18.23, p < 0.001), such that individuals in the Climate Schools-mental health condition reported higher mental health knowledge at each time point following the intervention, relative to the control group (see online Supplementary material for the full results; model estimates are presented in online Supplementary Tables S13 and S14).
Discussion
We assessed the effectiveness of a universal eHealth preventive school-based mental health intervention, relative to a control (school health education as usual), which formed part of a large cluster-randomised controlled trial in Australian secondary schools (Teesson et al., Reference Teesson, Newton, Slade, Chapman, Allsop, Hides and Andrews2014). Compared to health education as usual, we found evidence that the standalone mental health intervention improved knowledge of mental health, however there was no evidence that the intervention improved other mental health outcomes. Across all participants we observed small, but significant, increases in internalising problems, depression and anxiety at 18 months post intervention, relative to baseline. Student's social connectedness did not influence intervention outcomes.
We found that compared to baseline, internalising, depressive and anxiety symptoms were higher at 18 months post intervention when participants were 15–16 years of age. This effect was true for students in both the Climate Schools-mental health condition and the control group. These findings are consistent with population-based statistics from Australia that between the ages of 11–15 and 16–17 incidences of depression and emotional problems are on an increasing trajectory (Lawrence et al., Reference Lawrence, Johnson, Hafekost, de Haan, Sawyer, Ainley and Zubrick2015).
Relative to baseline, and compared with controls, we found that students in the Climate Schools-mental health condition had higher internalising scores at 6- and 12-months post intervention. These effects were not maintained at 18-months post intervention and were not observed on any of the other four outcome measures of depressive or anxiety symptoms and probable depression or probable anxiety. In addition, we did not assess internalising problems immediately following the intervention, which limits our interpretation of this finding. These results do, however, warrant further evaluation of the intervention components that may have contributed to this result. For example, despite finding that mental health knowledge increased among students in the intervention group, relative to the control group, anecdotal evidence from teacher feedback indicated that student engagement with the cartoon component reduced across the intervention. However, this finding should also be considered within the context of wide individual variability seen in our outcome measures and the small effect sizes associated with these differences. Indeed, despite increases in mental health knowledge, a key curriculum outcome in place in Australian secondary schools, our findings indicate that the standalone mental health intervention did not improve upon the standard curriculum in terms of reducing anxiety or depression. Furthermore, despite the alternative explanations presented above, it should not be fully discounted that the intervention is causally related to the observed short-term increases in internalising problems, noting that the size of the increases were small and did not differ from the control group by the 18-month follow-up.
Our results sit within a growing body of work indicating the limited efficacy of universal interventions for adolescents in reducing anxiety and depression. For example, two systematic reviews focusing on the effectiveness of cognitive behavioural therapy (CBT)-informed resilience interventions for young people found no benefits for depression or anxiety symptoms post intervention (Bastounis, Callaghan, Banerjee, & Michail, Reference Bastounis, Callaghan, Banerjee and Michail2016; Dray et al., Reference Dray, Bowman, Campbell, Freund, Wolfenden, Hodder and Wiggers2017; Werner-Seidler et al., Reference Werner-Seidler, Spanos, Calear, Perry, Torok, O'Dea and Newby2021). Indeed, another recent systematic review and network meta-analysis of school-based interventions to prevent depression and anxiety found weak evidence that universal CBT-based interventions reduce anxiety in both primary and secondary school settings. Furthermore, outcomes for both universal and targeted interventions for depression appeared to be poor across both primary and secondary school settings (Caldwell et al., Reference Caldwell, Davies, Hetrick, Palmer, Caro, López-López and Welton2019). Given that our intervention approach was universal and founded on cognitive behavioural principles, our results are consistent with these findings.
Whilst the current intervention is universal in nature, other prevention programmes that have been designed to target individuals at risk for anxiety and depression have shown some promise. Relative to universal prevention programmes, the literature on targeted interventions for depression and anxiety is smaller. However, it does point towards stronger effects for these other programmes, with targeted approaches showing more positive outcomes in the short term for depressive symptoms (Caldwell et al., Reference Caldwell, Davies, Hetrick, Palmer, Caro, López-López and Welton2019; Werner-Seidler et al., Reference Werner-Seidler, Perry, Calear, Newby and Christensen2017). This is in line with a recent commentary suggesting that preventive psychiatry should move towards prevention programmes that identify and intervene with selected high-risk individuals, rather than programmes that have been designed with a population wide focus, such as the current one (Cuijpers, Smit, & Furukawa, Reference Cuijpers, Smit and Furukawa2021). However, given the evidence that increased bullying and stigma are associated with inclusion in targeted school-based interventions, further work is needed to establish best practices for these interventions that limit these risks for those included (Fazel & Kohrt, Reference Fazel and Kohrt2019).
It is also possible that we did not observe any positive effects of our intervention as it was administered too late to have an impact on trajectories of mental ill-health. At baseline, students were aged 13–14, which is around the same time that lifetime trajectories of mental ill-health are already established (Patton et al., Reference Patton, Coffey, Romaniuk, Mackinnon, Carlin, Degenhardt and Moran2014). For example, in Australia, 1% of individuals aged 4–11 have reported experiencing depression, compared with 4–6% of 12–17 years old's reporting depression (Lawrence et al., Reference Lawrence, Johnson, Hafekost, de Haan, Sawyer, Ainley and Zubrick2015). Preventive interventions may, therefore, be most successful if delivered before this sharp rise in depressive episodes. However, previously published findings from the wider Climate Schools-combined trial did find benefits of a combined mental health and substance use intervention on mental health symptoms, relative to the standalone mental health intervention evaluated here, at 6 months post intervention (Teesson et al., Reference Teesson, Newton, Slade, Chapman, Birrell, Mewton and Andrews2020). These effects were not present immediately after the intervention was delivered, but this finding suggests that a combined approach, whereby multiple risk factors are targeted may hold more long-term promise in reducing mental health problems among adolescents. In addition, given the observed mental health benefits of the combined approach, future analyses of the CSC trial should evaluate the potential transfer effects of the standalone substance use intervention on mental health outcomes, relative to the control and standalone mental health group. It is possible that we found no effect of the mental health intervention here, as the potential mental health benefits observed in the combined approach are an indirect benefit from the substance use intervention content (Newton et al., Reference Newton, Chapman, Slade, Birrell, Healy, Mather and Teesson2020; Teesson et al., Reference Teesson, Newton, Slade, Chapman, Birrell, Mewton and Andrews2020).
Future preventative intervention trials may also benefit from involving young people themselves in the delivery of the intervention. This peer-led approach has shown promising results for a number of behavioural interventions including victimisation and substance use problems (e.g. smoking, alcohol use and drug use) (MacArthur, Harrison, Caldwell, Hickman, & Campbell, Reference MacArthur, Harrison, Caldwell, Hickman and Campbell2016; Paluck, Shepherd, & Aronow, Reference Paluck, Shepherd and Aronow2016). However, some peer-led interventions have resulted in iatrogenic effects, especially in the context of substance use and should therefore be carefully developed, taking into account local contextual factors (Valente et al., Reference Valente, Ritt-Olson, Stacy, Unger, Okamoto and Sussman2007). Despite our intervention's focus being on mental health, rather than changing a specific behaviour, some aspect of peer involvement in the delivery of the intervention may increase fidelity through allowing young people themselves to be agents of their own change (Yeager, Dahl, & Dweck, Reference Yeager, Dahl and Dweck2018). There are already promising developments in this area: one trial – ‘The Mind Your Mate Study’ – is a web-based peer support prevention strategy that facilitates discussion around mental health and is currently being trialled in Australian secondary schools (Birrell, Furneaux-Bate, Chapman, & Newton, Reference Birrell, Furneaux-Bate, Chapman and Newton2021).
We further found that a model including students social network position did not explain any additional variance in any of our outcome variables across time. Exploratory correlation analyses also found no significant cross-sectional association between baseline social network position and mental health symptoms. This result counters many of the previous findings which show that individuals who are less well connected have higher depressive symptoms (Okamoto et al., Reference Okamoto, Johnson, Leventhal, Milam, Pentz, Schwartz and Valente2011; Ueno, Reference Ueno2005). However, these results should be interpreted within the context of a number of limitations. Firstly, students were unable to nominate individuals from different year groups to their own and were restricted to choosing just three peers. Secondly, for ethical reasons students were limited to choosing only other students who also consented to participate in the CSC trial, and we did not control for the number of friendships each student may have outside of their school environment. Whilst we had high rates of participation (85.5%), it is still possible that some participants were well connected to other students, but this was not captured by the current study design. Finally, given that only a sub-sample of schools and students provided social network data, we may not have had sufficient power to detect an effect. That said, these measures offer a proxy for each young person's social connectedness which has benefits over and above commonly used, subjective, self-report measures. In the future, researchers developing school-based preventative interventions should consider including comprehensive social network measures which would facilitate further insight into the role that social connectedness plays in intervention efficacy. These studies should also ascertain, and control for, the degree to which participants report having strong social connections beyond those they are able to nominate.
Conclusion
We found evidence that our standalone mental health intervention improved knowledge of mental health, however there was no evidence that the intervention improved other mental health outcomes, relative to a control. Across all participants we observed small, but significant, increases in internalising problems, depression and anxiety at 18 months post intervention, relative to baseline. Student's social connectedness did not influence intervention outcomes. These findings sit within an existing body of evidence that CBT-informed universal preventive mental health interventions for adolescent mental health show limited efficacy. We suggest future studies should move towards earlier intervention, to explore ways to reduce stigma within targeted intervention approaches and to incorporate comprehensive measures of each student's position within their school-wide social network.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291722002033.
Acknowledgements
The authors acknowledge the schools, students, teachers, health professionals and research assistants who were involved in this research. The research team also acknowledges the assistance of the New South Wales Department of Education and Communities (SERAP 2013054), the Western Australian Department of Education (D13/0475948), the Western Australian Catholic Education Office, the Brisbane Catholic Education Office and Queensland Department of Education and Training for access to their schools to conduct this research.
Financial support
This research was funded by the National Health and Medical Research Council (NHMRC) project grant (APP1047291) and a Centre of Research Excellence Grant in Prevention and Early Intervention in Mental Illness and Substance Use (PREMISE, APP1134909). The NHMRC did not influence the design of the study, the collection, analysis and interpretation of data or the writing of the manuscript.
Conflict of interest
MT and NN are two of the developers of the Climate Schools programmes and directors of Climate Schools Pty Lts, a social enterprise established in 2015 to distribute the Climate Schools programmes and maximise social well-being. All other authors declare no conflicts of interest.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
