Introduction
School retention and educational achievement are predictive of good health status (Freudenberg and Ruglis, Reference Freudenberg and Ruglis2007). Young people who leave school under the minimum school leaving age have high rates of a broad range of health and social problems such as depression, substance abuse, criminal offending and reduced earning potential (Ou, Reference Ou2008). Partly to address these issues, Alternative Education and other alternative schooling programmes (such as youth wrap-around and transition programmes) for young people who are excluded or alienated from mainstream education have been established. Students in these programmes have high mental health needs. For example, those in Alternative Education (AE) in New Zealand have approximately twice the rates of depression and more than three times the rates of suicide attempts as their peers in mainstream high schools (Clark et al., Reference Clark, Smith, Raphael, Jackson, Fleming and Denny2010; Denny, Clark and Watson, Reference Denny, Clark and Watson2004).
Evidence-based treatments for adolescent depression are available, with Cognitive Behavioural Therapy (CBT) being the treatment of choice for young people with mild to moderate depression (National Institute for Health and Clinical Excellence, 2005). Computerized CBT (cCBT) has been shown to be effective and acceptable for adults with depressive disorders (Andrews, Cuijpers, Craske, McEvoy and Titov, Reference Andrews, Cuijpers, Craske, McEvoy and Titov2010) and holds promise in terms of potentially increasing the availability of CBT for teenagers. There have been positive or promising results for cCBT for depression among young people utilizing General Practice care (Van Voorhees, Ellis, Stuart, Fogel and Ford, Reference Van Voorhees, Ellis, Stuart, Fogel and Ford2005; Van Voorhees et al., Reference Van Voorhees, Fogel, Reinecke, Gladstone, Stuart and Gollan2009), utilizing mental health services (Abeles et al., Reference Abeles, Verduyn, Robinson, Smith, Yule and Proudfoot2009; Stallard, Richardson, Velleman and Attwood, Reference Stallard, Richardson, Velleman and Attwood2011) and for students attending high schools (O'Kearney, Gibson, Christensen and Griffiths, Reference O'Kearney, Gibson, Christensen and Griffiths2006; O'Kearney, Kang, Christensen and Griffiths, Reference O'Kearney, Kang, Christensen and Griffiths2009). However, there is a lack of evidence regarding the use of cCBT for adolescents alienated or excluded from mainstream high schools. While cCBT may be just as effective for this group as for other teens, they are different from many research samples in terms of socio-economic status (mainly being from higher deprivation communities), gender (mainly male), age (mainly 13–16 years) and ethnicity (typically including high numbers of indigenous and ethnic minority young people). In addition, young people in alternative schooling have high rates of exposure to family stressors, low rates of literacy, low rates of help seeking for mental health needs and a lack of engagement with mainstream health services (Albert, MacKay, Stewart, Saewyc and the McCreary Centre Society, Reference Albert, MacKay, Stewart, Saewyc and Centre Society2007; Clark, et al., Reference Clark, Smith, Raphael, Jackson, Fleming and Denny2010), which might mean different approaches are required.
We developed a cCBT programme called SPARX. SPARX has been tested in comparison to treatment as usual for young people in mainstream high schools, youth health services and traditional primary health care (Merry et al., unpublished observations; for info re the trial see www.sparx.org.nz). In this smaller companion study we aimed to test the efficacy of SPARX for students in alternative schooling programmes for adolescents excluded, or at risk of being excluded, from mainstream education at and under the New Zealand minimum school leaving age of 16 years. Specifically, we aimed to investigate whether SPARX reduced symptoms of depression, anxiety and hopelessness and improved quality of life and locus of control scores, compared to those wait listed for SPARX. The study used pragmatic design features to provide an indication of potential efficacy of SPARX in alternative schooling environments.
Method
Trial design
The study employed an immediate vs. delayed intervention randomized controlled trial. The intervention was the 7-module SPARX cCBT programme, to be completed at a rate of 1–2 modules per week, from week 0 to week 5 for the immediate treatment group and weeks 5–10 for the delayed treatment group. Assessments were carried out by TF at baseline and at 5 and 10 weeks.
Participants
Participants were recruited from:
- Three Alternative Education (AE) schools. In New Zealand these are educational programmes for teenagers aged 13½- 15½ who have a history of school exclusions, behaviour problems or long term truancy and are considered to be at risk of major behavioural difficulties in a usual high school environment (O'Brien, Thesing and Herbert, Reference O'Brien, Thesing and Herbert2001).
- An educational programme for students aged 12 to16 years who have been temporarily excluded and are considered at risk of permanent exclusion from school.
- A transition programme for young people who have been in AE and are now aged 15½ -16½ and are not ready for work, school or tertiary training.
Recruitment plans were developed in conjunction with staff at each site to reflect how such an intervention would be likely to be implemented in that site. In 4 out of the 5 study sites all students were invited to participate, irrespective of the presence of depressive symptoms (opt out). In one site all students were informed about the study and were able to ask to participate or have a tutor suggest they participate (opt in). Students were excluded if: they had severe depression, high suicide risk or other mental health issues that may have meant that they were not safe on the computer programme; they had a disability, or insufficient proficiency in English that may have resulted in them not being able to use the programme or not being able to comprehend the functioning scales; or they were not intending to remain enrolled in the participating schooling programme for at least 10 weeks. Students with no or minimal symptoms of possible depression (raw CDRS score under 30, indicating a depressive disorder is unlikely) were allowed to participate and were randomized; however, for the purposes of this analysis of efficacy their data were excluded.
Recruitment took place from July 2009 to June 2010. Written consent was obtained from participants; where they were aged under 16 consent was also obtained from a parent or guardian. Ethics approval was given by the Northern Y Regional Ethics Committee, reference number NTY/09/04/036.
Intervention
The SPARX programme consisted of 7 modules or levels, each of approximately 30 minutes duration. The content was developed by clinical and academic experts at the University of Auckland in partnership with a computer games company, and with advice from young people and from Maori, Pacific and Asian cultural advisors. Content was based on CBT and included psycho-education, relaxation skills, problem solving, activity scheduling, challenging and replacing negative thinking and social skills. The programme includes direct instructional content as well as narrative and experiential learning components. Voice over, written text and music were used. Each module involved meeting with the “guide” who spoke in first person about dealing with depression, presented mood and safety checks, offered further help beyond the SPARX programme, and outlined the purpose of the following level. Next the user entered the “game world”, where they inhabited a personalized character and helped to restore the balance in the game world by using skills from a “shield against depression”. An overarching narrative, metaphor and visual and verbal memory aids and a range of game elements (from helping characters to solve problems to shooting negative thoughts) were used. At the end of each level the user returned to the guide who again communicated with them in first person, reflected on the learning and how that might be applied in real life and set homework challenges. Images from the programme can be viewed at www.sparx.org.nz.
Students completed SPARX during class time under minimal supervision from their educational service provider. One site had a suite of computers and students completed SPARX in two groups. The remainder had one or two computers so students completed SPARX individually. Each site was visited or telephoned weekly by TF (a PhD candidate with experience working as a clinician in adolescent health and mental health services) to address any safety concerns or problems that may have arisen or to support students in the use of the programme. In two sites the tutors had indicated that they were too busy to organize the intervention and so students were prompted to do a module of SPARX if they had not already done so during weekly visits.
Measures and outcomes
Demographic data were collected at baseline. Where students reported more than one ethnicity they were categorized using the NZ Census ethnicity prioritization method (Ministry of Health, 2004). The primary outcome measure was the Children's Depression Rating Scale, Revised (CDRS-R), (Poznanski and Mokros, Reference Poznanski and Mokros1996). This is an observer-rated scale with good sensitivity to change (Brooks and Kutcher, Reference Brooks and Kutcher2001), and well established reliability and concurrent validity (Myers and Winters, Reference Myers and Winters2002).
The following self-report scales were used as secondary measures: Reynolds Adolescent Depression Scale (RADS-2) (Reynolds, Reference Reynolds2002); Paediatric Quality of Life Enjoyment and Satisfaction Questionnaire (PQ-LES-Q) (Endicott, Nee, Yang and Wohlberg, Reference Endicott, Nee, Yang and Wohlberg2006); Spence Anxiety Scale (Spence, Barrett and Turner, Reference Spence, Barrett and Turner2003); Kazdin Hopelessness Scale (HPLS) (Kazdin, French, Unis, Esveldt-Dawson and Sherick, Reference Kazdin, French, Unis, Esveldt-Dawson and Sherick1983) and the Children's Nowicki-Strickland Internal-External Control Scale short (20 item) form (CNSIE) (Nowicki and Duke, Reference Nowicki and Duke1983). Students completed these themselves or, if they preferred, had them read aloud by the researcher.
Remission was defined as a reduction in raw score on the CDRS-R to below 30. Clinically significant change was defined as remission or a reduction of at least 30% in symptoms on the CDRS-R. An adverse event was defined as an episode of self-harm or an increase in depressive symptoms of 5 or more points in CDRS-R raw score or an adverse change in category on the CDRS (for example, from “possible” to “likely” depression).
Randomization and blinding
Randomization was carried out in a 1:1 ratio using a computer generated randomization sequence. Allocation was stratified by study site and arranged in permuted blocks. Allocation concealment was ensured by allocating each participant a unique study number in sequence as they met the researcher (TF). This occurred prior to eligibility being assessed as consultation had identified that young people wanted to test the programme regardless of whether they had difficulties, and did not want their level of distress to be identifiable to their peers. Thus the same process was used for all students; however analysis was of those who had symptoms of possible depression at baseline. A sealed envelope for each study number containing treatment allocation had been prepared in advance by an independent research assistant. Following baseline assessment the young person opened this envelope with the researcher, and access to immediate or delayed treatment was arranged.
It was not possible to blind participants to their treatment allocation. The researcher was also unblinded after the baseline assessment. However, 19% of CDRS-R interviews were audio-recorded and were scored by a research assistant blinded to allocation; these scores were compared to the assessor's scores by an independent statistician. No systematic inter-rater differences were found (means 25.7 and 26.4 with a co-efficient of between-rater variation (CV) of 7.9%).
Statistical analysis
This exploratory study was powered (80% power) to detect a large effect size (d> = 1.0), with a sample size of 15/group allowing up to 50% loss of participants at follow-up. Although all participants ultimately received SPARX, comparisons of changes from baseline to week 5 between the randomized groups were planned as the main analyses of efficacy. The comparison of 5-week and 10-week data for those who received SPARX first provided an estimate of the maintenance of changes. A further analysis tested whether changes associated with SPARX were the same for those who received it first compared with those who waited.
As a small study, the main analyses utilized all participants with pre and post SPARX assessments who had not broken treatment allocation. An intention-to-treat analysis was also undertaken, although the sample size was not adequate for this to be conducted as a main analysis.
Statistical analyses were carried out using SPSS software. For primary and secondary outcome measures the changes from baseline to 5 weeks were compared between SPARX and wait groups using ANCOVA, with the baseline level as the covariate. Differences between groups at 5 weeks in remission and in clinically significant reductions in symptoms were tested using Fishers Exact Test. A series of paired t-tests compared posttreatment to follow-up changes for those randomized to immediate SPARX. An Analysis of Variance compared the magnitude of change (pre-post treatment) between groups. A two-sided p-value of smaller than or equal to .05 was taken to indicate statistical significance.
Out of a total of 12,104 assessment items 88 were missing. Where up to 5 of the 14–30 items on any questionnaire were missing, the total scores were calculated from the available data and weighted to compensate for the missing items. In the two situations where more than 5 items were missing from a single scale, that scale was considered missing at that time point. As there is considerable correlation amongst the outcome measures, no correction for multiple comparisons was made.
The trial was registered with the Australian New Zealand Clinical Trials Registry, registration number ACTRN1261000074099. The statistical analysis was planned and documented prior to data analysis.
Results
Participant flow, completion rates
The participant flow is shown in Figure 1. There were between 1 and 11 students recruited from each of the 5 sites using block randomization for each site. In the 4 “opt out” sites 51 young people were approached, 3 declined. Of those excluded, 12 had no or minimal symptoms of depression; all 12, however, wished to participate as planned but their data were excluded from this analysis. In the one “opt in” site no students asked to be in the study; tutors invited two to participate, one of whom agreed. These processes resulted in uneven group sizes (20:12).
Flow chart of participants in the trial
Baseline data were available for all 32 participants, 5-week assessment data for 31 (97%) and 10-week assessment data for 27 (84%) of participants. Twenty-six (81%) of participants completed 4 or more sessions of SPARX, with 22 (69%) completing all 7 levels. Completion rates were higher in Group 1 - SPARX first (19/20 completed four or more levels and 15/20 completing all levels) than in Group 2 - wait first (7/12 completed four or more levels and 7/12 completed all levels). Most participants who did not complete reported running out of time and said that they did wish to finish. In Group 2, one participant broke randomization and did several levels of SPARX while allocated to wait; this young person completed the 5-week assessment but then left the schooling programme and was lost to follow-up. The primary analyses were carried out with all 30 (93.7%) participants who had baseline and 5-week data and had not broken treatment allocation.
Baseline data
The mean age of participants was 14.9 years (range 13–16, SD = .79), 18 (56%) were male, 11 (34%) were Maori, 12 (38%) were Pacific island, 8 (25%) were New Zealand European and 1 was of other ethnicity. All participants had a CDRS-R score of over the 70th percentile of depressive symptoms as detailed in the CDRS-R manual (Poznanski and Mokros, Reference Poznanski and Mokros1996). Although treatment groups were different sizes they were comparable in terms of severity of baseline symptoms (Table 1).
Primary and secondary outcomes at baseline and mean changes at 5 weeks*
* Data are for all participants who completed at least one level of SPARX in the group that they were randomised and completed 5 week assessment
1Remission is defined as decrease in CDRS-R raw score to under 30
2Clinically significant change is defined as decrease in CDRS-R raw score to under 30 or a decrease of 30% or more in CDRS-R raw score.
Outcomes
There were significantly greater reductions in CDRS and RADS scores from baseline to week 5 for the SPARX group compared with those who waited; however, there were no significant differences in the changes in the remaining self-report measures (Table 1). Effect sizes were 1.61 for CDRS-R and 0.77 for RADS-2. Those in the SPARX group were significantly more likely to be in remission or to have had a clinically significant reduction in symptoms than those in the wait group (Table 1).
Effect of treatment for wait group
The group allocated to wait first also improved when they were able to do SPARX. The magnitude of improvement was not statistically different from the group that did SPARX initially. Pre-post SPARX (week 5 to week 10) mean changes (with 95% confidence intervals) for those allocated to wait first were: CDRS-R: –13.2 (–10.1 to –16.2); RADS-2: –7.3 (–0.5 to –14.1); PQ-LES-Q: 3.7 (–0.1 to 7.6); HPLS: –1.1 (–2.9 to .6); Spence: 0.7 (–7.4 to 8.8); CNSIE: –2.1 (–0.2 to –4.1).
Maintenance of effect
Maintenance of effect was tested for participants who were randomized to SPARX immediately. There were no significant changes in outcomes from posttreatment (5 weeks) to follow-up (10 weeks) (Table 2).
Maintenance of effects: mean scores on primary and secondary outcomes immediately postintervention (5 weeks from baseline) and at follow-up (10 weeks from baseline) for participants who received SPARX immediately
Adverse events
There were a total of six adverse events. Four young people had an increase in depressive symptoms. Each of these young people was in the waitlist, pre-SPARX condition. There were two incidents of self-harm reported; each was reviewed as per the study protocol and was found to be unrelated to the intervention. Both events were in the group allocated to SPARX first, with one event occurring during treatment and the other post SPARX, during the follow-up period.
Intention to treat analyses
Primary and secondary outcome analyses were re-run using Intention to Treat (ITT) analyses. Baseline data for the missing case in the SPARX group were brought forward. Week 5 data for the participant who broke allocation and received the intervention while allocated to wait were included in the wait group data. This resulted in CDRS-R pre-post mean changes of –13.9 (95% CI 10.0–17.9) for the SPARX group and –1.9 (95% CI –7.0–+3.3) for the waitlist; F 14.5, p = .001. The RADS pre-post mean change for SPARX was –4.5 (95% CI-9.0-+.1) and for waitlist was +2.3 (95% CI –3.6–+8.1); F value 3.4, p = .075. Remission rates using ITT analyses were 15/20 (75%) of the SPARX group compared to 5/12 (41.7%) of those in the wait group (Fishers Exact Test p = .130). Clinically significant change was 17/20 (85%) for the SPARX group and 5/12 (42%) in the wait group (Fishers Exact Test p = .018).
Discussion
The results of this small study suggest that the SPARX cCBT programme is engaging and may be effective in reducing depressive symptoms in adolescents excluded from mainstream education. There were high rates of completion of the cCBT programme. The cCBT group demonstrated improvements in depression scores over the waitlist control as measured by the CDRS-R, RADS-2 and remission rates. The effects were maintained at 10-week follow-up. However, there were no significant effects on measures of hopelessness, locus of control, anxiety, or quality of life.
In terms of the feasibility of cCBT, as well as in terms of impact on depressive symptoms, the results are promising. The alternative schooling sites had few computers and limited supervision was available. Additionally, our finding that it was practical to offer the intervention to whole classes is important, given that the majority of students had symptoms of possible depression and that embarrassment about being seen to be depressed and low rates of help-seeking had been reported as major barriers to treatment (Fleming, Dixon and Merry, in press). Young people excluded from mainstream schooling often have multiple challenges and many had previously tried other interventions; so it was pleasing that this relatively short programme was associated with a reduction in depressive symptoms.
A recent systematic review (Richardson, Stallard and Velleman, Reference Richardson, Stallard and Velleman2010) identified one study of an online chat group for adolescents (Gerrits, van der Zanden, Visscher and Conijn, Reference Gerrits, van der Zanden, Visscher and Conijn2007) and five other studies of cCBT for depression among adolescents. These included universal trials (of all students in a year group) of “MoodGym” in a boys and in a girls high school (O'Kearney et al., Reference O'Kearney, Gibson, Christensen and Griffiths2006, Reference O'Kearney, Kang, Christensen and Griffiths2009); and studies among young people utilizing General Practice (Van Voorhees et al. Reference Van Voorhees, Ellis, Stuart, Fogel and Ford2005, Reference Van Voorhees, Fogel, Reinecke, Gladstone, Stuart and Gollan2009) or mental health services (Abeles et al., Reference Abeles, Verduyn, Robinson, Smith, Yule and Proudfoot2009). Since that review, an additional trial of cCBT (“Think, Feel, Do”) with young people utilizing CAMS services (Stallard et al., Reference Stallard, Richardson, Velleman and Attwood2011) has been published. No previous research regarding the use of cCBT with adolescents who were neither in mainstream schooling nor getting help from health services was found. Of the existing studies, only those testing “MoodGym” (O'Kearney et al., Reference O'Kearney, Gibson, Christensen and Griffiths2006, Reference O'Kearney, Kang, Christensen and Griffiths2009) and “Think, Feel, Do” (Stallard et al., Reference Stallard, Richardson, Velleman and Attwood2011) were randomized controlled trials comparing cCBT to non cCBT conditions.
The research protocol for the current study was finalized after consultation with alternative school students and providers. This led to a shorter follow-up period than we would have preferred so as to fit in with the preferences of the schooling programmes and to minimize loss to follow-up. However, participants were recruited more quickly than expected. This contrasts with many computerized therapy trials (Waller and Gilbody, Reference Waller and Gilbody2009). It is notable that in study sites where all students were invited to participate, levels of recruitment and completion were high. In contrast, the one study site which relied on students help seeking or being suggested by a staff member resulted in only one participant.
The unequal sample size of the two groups was unfortunate but does not represent any bias in the group allocation or in the drop-outs. The baseline symptoms in the two treatment groups were comparable and fewer than expected students dropped out of treatment so the sample size remained adequate for the intended main analyses.
Once students had begun SPARX, most completed it. Again, this is in contrast to many studies of cCBT (Waller and Gilbody, Reference Waller and Gilbody2009) but it is consistent with the “Think, Feel, Do” trial. Both “Think, Feel, Do” and SPARX are specifically designed for use with young people and include graphic interfaces rather than being text based. Further, in both of these trials participants were randomized to cCBT immediately or after a short delay. It may be that these cCBT interventions were particularly engaging or that this study design was appealing to young people. The prompting by tutors or the researcher and the fact that the intervention was offered during class time may have also been factors in our high completion rates.
Non completion was higher in the delayed treatment group. Most participants reported that this was because they ran out of time, usually because of frequently being absent from the course. It is possible that the delay might have had an impact on motivation. This could be tested in future research.
The finding that cCBT appeared to reduce depressive symptoms is consistent with positive findings regarding the impact of cCBT on depression among adults (Andrews, et al., Reference Andrews, Cuijpers, Craske, McEvoy and Titov2010) and with positive or promising findings from previous studies in young people (Richardson et al., Reference Richardson, Stallard and Velleman2010).
We did not detect an effect for anxiety, locus of control, hopelessness or quality of life. Abeles et al. (Reference Abeles, Verduyn, Robinson, Smith, Yule and Proudfoot2009) and Van Voorhees et al. (Reference Van Voorhees, Fogel, Reinecke, Gladstone, Stuart and Gollan2009) reported reductions in anxiety of cCBT programmes for adolescent depression. However, others have not reported anxiety findings (O'Kearney et al., Reference O'Kearney, Gibson, Christensen and Griffiths2006, Reference O'Kearney, Kang, Christensen and Griffiths2009; Van Voorhees et al., Reference Van Voorhees, Fogel, Reinecke, Gladstone, Stuart and Gollan2009) or have had non-significant findings (Stallard et al., Reference Stallard, Richardson, Velleman and Attwood2011). Although SPARX includes relaxation techniques, baseline anxiety scores in this group were under the cut-off associated with children at high risk for anxiety (Spence, Reference Spence1997); hence we might have encountered a floor effect (Everitt, Reference Everitt2002). The other negative findings may indicate that SPARX had no impact on quality of life, hopelessness or locus of control in this group. This should be tested further in future research. Encouragingly, therapeutic gains were maintained at 10-week follow-up. The longer term impacts of cCBT within this group should also be investigated further.
Limitations
This study has a number of limitations. These include its small size and short follow-up period. As a small study with limited funds and working in often poorly resourced sites, the introduction to the programme, recruitment and assessments were completed by one researcher. Care was taken to ensure adequate allocation concealment, which was done centrally by computer; however the researcher was not blinded when conducting postintervention and follow-up assessments. Although blind review of audio recorded interviews did not suggest any bias, and all other measures were self-report scales, this remains an important limitation.
Outcome measures used in this study have not been validated for use with this specific group. This may be significant (Stewart and Nápoles-Springer, Reference Stewart and Nápoles-Springer2003). For example, questionnaire items for the CNSIE scale are aimed at a 5th grade level of literacy (Nowicki and Strickland, Reference Nowicki and Strickland1973), which may be a higher level of literacy than that of some AE students.
Thus the current findings should be viewed as preliminary. Nevertheless, given the small number of trials regarding cCBT for depression in young people, and the lack of research for young people excluded from mainstream education, the findings are useful.
Conclusion
Findings from this small pragmatic study suggest that the SPARX cCBT programme is engaging and appears to be a promising intervention for depressive symptoms among young people in alternative schooling programmes for those excluded or alienated from mainstream education. These findings with this vulnerable group suggest that cCBT may hold promise for young people who are reluctant to engage in traditional health services and who have poor health outcomes.
Acknowledgements and Conflict of interests
The New Zealand Ministry of Health provided funding for the development of SPARX. Theresa Fleming is supported by a Top Achievers Doctoral scholarship from the New Zealand Tertiary Education Commission. Theresa Fleming and Sally Merry contributed to the development of the SPARX programme and the intellectual property of SPARX is owned by Uniservices at the University of Auckland.
Comments
No Comments have been published for this article.