Introduction
Adolescence is a critical period for mental health. Approximately half of all mental disorders develop before the age of 15 years, and 75% emerge by 18 years (Kim-Cohen et al., Reference Kim-Cohen, Caspi, Moffitt, Harrington, Milne and Poulton2003). Emotional disorders are especially common in adolescence, when increased social challenges interact with immature systems for emotion regulation, posing heightened risks for both anxiety and depression (Ahmed et al., Reference Ahmed, Bittencourt-Hewitt and Sebastian2015; Steinberg, Reference Steinberg2005). These syndromes, which are often comorbid, cause marked distress and impairment for approximately 4% of adolescents in the community, as well as accounting for most referrals to specialist child and adolescent mental health services (CAMHS) in the UK (Green et al., Reference Green, McGinnity, Meltzer, Ford and Goodman2005; Wolpert et al., Reference Wolpert, Jacob, Napoleone, Whale, Calderon and Edbrooke-Childs2017).
Extensive evidence supports the use of psychological interventions to reduce the burden of youth mental disorders (Weisz et al., Reference Weisz, Kuppens, Ng, Eckshtain, Ugueto, Vaughn-Coaxum and Fordwood2017), yet economic pressures limit the availability of evidence-based therapies and specialist mental health care more generally (Abdinasir and Pona, Reference Abdinasir and Pona2015). A recent review by the Children’s Commissioner for England (2016) identified that only one in 250 young people were referred to CAMHS in 2015. Of these referrals, 28% were rejected outright, primarily because symptom presentations did not reach high thresholds for entry. For those young people who were accepted, large disparities were found in waiting times across geographical regions, ranging from 14 to 200 days. Notably, these figures do not incorporate ‘hidden waiting times’ for the intervening period from initial assessment to treatment (Frith, Reference Frith2016). Such delays can have devastating impacts on young people’s quality of life and prospects (House of Commons Health Committee, 2014). For older adolescents, access is even more problematic due to poor transitions from youth to adult services, resulting in young people losing access to support during the period when they are most vulnerable (Memarzia et al., Reference Memarzia, St Clair, Owens, Goodyer and Dunn2015; Pona et al., Reference Pona, Royston, Bracey and Gibbs2015).
Even when young people are seen in CAMHS, there is often limited provision of cognitive behavioural therapy (CBT) (Edbrooke-Childs et al., Reference Edbrooke-Childs, Calderon, Wolpert and Fonagy2015; Stallard et al., Reference Stallard, Udwin, Goddard and Hibbert2007), which is widely recommended in evidence-based practice guidelines for youth anxiety and depression (Higa-McMillan et al., Reference Higa-McMillan, Francis, Rith-Najarian and Chorpita2016; Hopkins et al., Reference Hopkins, Crosland, Elliott and Bewley2015; Weersing et al., Reference Weersing, Jeffreys, Do, Schwartz and Bolano2017). The unwillingness (or inability) of many practitioners to implement evidence-based CBT protocols has been reported in multiple surveys (Hagermoser Sanetti et al., Reference Hagermoser Sanetti, Collier-Meek, Fallon and Maltzman2016). This is often related to a perceived lack of fit between empirically tested (usually disorder-specific) structured treatments and the vagaries of routine practice (Southam-Gerow et al., Reference Southam-Gerow, Rodríguez, Chorpita and Daleiden2012). ‘Transdiagnostic’ protocols have been heralded as a way of addressing this implementation gap, offering a more parsimonious and flexible approach to dealing with real-world challenges of comorbidity and case complexity (Bearman and Weisz, Reference Bearman and Weisz2015). Transdiagnostic interventions for anxiety and depression have been designed to target common mechanisms of emotion regulation implicated in both syndromes (Newby et al., Reference Newby, McKinnon, Kuyken, Gilbody and Dalgleish2015). Comorbidities can be targeted simultaneously in a single treatment protocol, therefore improving treatment efficiency, and in theory, improving access to psychological therapy (Chorpita et al., Reference Chorpita, Taylor, Francis, Moffitt and Austin2004).
Although direct head-to-head comparisons are lacking, between-study comparisons show that transdiagnostic CBT for adult emotional disorders is at least as effective as disorder-specific variants (Newby et al., Reference Newby, McKinnon, Kuyken, Gilbody and Dalgleish2015). Equivalent evidence is now emerging from transdiagnostic trials with young people (Chu et al., Reference Chu, Temkin and Toffey2016). However, these studies have generally involved downward adaptations of emotion-focused transdiagnostic models developed with adults, which may give insufficient attention to adolescents’ preferred outcomes, delivery formats, and key social contexts (Sclare and Michelson, Reference Sclare, Michelson, Menzies, Kyrios and Kazantzis2016). Future developments require the active participation of young people in the conception, design and formative evaluation of new intervention models.
‘DISCOVER’ CBT workshops
The DISCOVER programme was born out of the need for more accessible and age-appropriate psychological support for distressed older adolescents. The delivery format and specific content of DISCOVER were co-created with a Teenage Advisory Group (TAG) to ensure that the social, emotional and relational needs of older adolescents were comprehensively addressed. The programme employs a one-day, group workshop format with individualized telephone follow-up, which has evolved from an established ‘well-being workshop’ template (Brown et al., Reference Brown, Cochrane and Hancox2000). This corresponds to young people’s preferences for more practical, interactive and less time-intensive modes of delivery (Persson et al., Reference Persson, Hagquist and Michelson2017; Plaistow et al., Reference Plaistow, Masson, Koch, Wilson, Stark, Jones and Lennox2014).
DISCOVER follows a structured manual that covers numerous problem- and emotion-focused coping skills. These elements have been commonly applied in other evidence-based CBT interventions (Chorpita and Daleiden, Reference Chorpita and Daleiden2009), and were subjected to further verification by the TAG, in order to improve fit with adolescents’ perceptions of ecological relevance (Ng et al., Reference Ng, Eckshtain and Weisz2016). The content is organized in a standardized sequence, which differs from certain other transdiagnostic programmes that deliver discrete modules according to individual participant needs (e.g. MATCH; Chorpita et al., Reference Chorpita, Weisz, Daleiden, Schoenwald, Palinkas, Miranda and Gibbons2013). Instead, workshop facilitators encourage participants to consider the suitability of different coping strategies from a curriculum of transdiagnostic self-regulatory skills. Further personalization is enabled through the provision of follow-up telephone calls, which focus on progress towards individual participant goals.
The first iteration of DISCOVER was implemented in a variety of inner-London community venues as a stress management intervention for self-referred 16- to 18-year-olds, and tested in an uncontrolled pre–post cohort study (Sclare et al., Reference Sclare, Michelson, Malpass, Coster and Brown2015). It was subsequently evaluated in a feasibility cluster randomized controlled trial (RCT) in secondary schools (Brown et al., Reference Brown, Blackshaw, Stahl, Fennelly, McKeague, Sclare and Michelson2019). The promising findings from these studies have raised the prospect of applying DISCOVER in other settings. Notably, the focus on common stressors and associated emotional symptoms is consistent with transdiagnostic approaches previously used in clinical settings (Craske, Reference Craske2012). We also identified a novel opportunity to evaluate DISCOVER for young people who had been placed on waiting lists after being referred to CAMHS with emotional difficulties. This aligned with key local and national service priorities to increase mental health service access and reduce waiting times (Department of Health and NHS England, 2015).
Aims
The present study aimed to investigate the feasibility, acceptability and preliminary outcomes of the DISCOVER intervention when applied to a clinical population of 15- to 18-year-olds with anxiety and/or depression, recruited from CAMHS waiting lists. The targeted age range was slightly wider relative to earlier evaluations of DISCOVER, based on an intention to broaden access within specialized services. A second aim was to examine the feasibility of undertaking a RCT of DISCOVER in CAMHS.
Objectives
The specific objectives were as follows:
(1) To assess feasibility of intervention delivery, considering:
(a) Attendance rate at DISCOVER workshops;
(b) Participation rate in telephone follow-up calls.
(2) To assess acceptability of intervention and associated trial procedures, considering:
(a) Quantitative measures of service satisfaction;
(b) Qualitative feedback.
(3) To assess feasibility of research procedures, considering:
(a) Number of eligible cases identified through CAMHS waiting lists;
(b) Consent rate;
(c) Randomization rate;
(d) Data collection rates at baseline and 8-week follow-up.
(4) To obtain clinical outcome data to inform the design of a full-scale trial, considering:
(a) Intervention effect estimates and confidence intervals, as indications of likely ranges for outcomes;
(b) Outcome variance estimates necessary for sample size calculations.
Method
Design
We mounted a feasibility RCT with two parallel arms: an intervention arm (DISCOVER) and a control arm (usual care). Due to the study’s feasibility objectives, an allocation ratio was not pre-specified, but was determined by minimum group size requirements for DISCOVER workshops. Outcomes were assessed at baseline and 8-week follow-up.
Ethics
Ethical approval was obtained for the main study and a protocol modification (see below) by the London-Harrow NHS Research Ethics Committee (reference: 16/LO/0231). The trial was registered on clinicaltrials.gov (identifier: NCT02752945).
Participants
Eligibility criteria
Participant eligibility criteria were: (i) aged 15–18 years; (ii) fluent in English; (iii) currently on a CAMHS waiting list for specialist assessment/treatment, following a referral for anxiety and/or depression; (iv) willing and able to attend a DISCOVER workshop; and (v) position on the waiting list indicated that the young person would be unlikely to receive a CAMHS appointment within 8 weeks of completing a workshop. To determine criterion (iii), it was required that referral letters should indicate a need for assessment and/or treatment based on primary symptoms of anxiety and/or depression (although a confirmed diagnosis was not stipulated).
Young people were excluded if they were: (i) presenting with an acute risk of harm to themselves or others; (ii) presenting with severe learning difficulties; and/or (iii) unable to provide consent to participate. For exclusion criterion (iii), this included parental consent for participants aged 15 years.
Settings
The study was carried out in two outpatient CAMHS clinics in adjacent inner-London boroughs. Compared with the general UK population, both boroughs are characterized by high levels of social disadvantage and a high proportion of black and minority ethnic residents (Stewart et al., Reference Stewart, Soremekun, Perera, Broadbent, Callard, Denis and Lovestone2009). The clinics provide a specialist multi-disciplinary service for adolescents with a variety of mental health needs.
Sample size
A formal sample size calculation was not appropriate for our feasibility design. A recruitment target of n = 30 was set, based on recommendations for obtaining outcome variance estimates for trial sample size calculations (Browne, Reference Browne1995).
Interventions
DISCOVER
Structure
Previous iterations of DISCOVER (see Michelson et al., Reference Michelson, Sclare, Stahl, Morant, Bonin and Brown2016) stipulated a minimum of four participants in each workshop and a maximum of 15. For the current study, the lower limit was increased to six participants. This modification was informed by the workshop facilitators’ initial experience of delivering a small clinic-based group for young people. The larger group size was considered more conducive to open discussion and interaction in a clinical setting. Groups were co-facilitated by two doctoral-level clinical psychologists in accordance with a detailed manual. Workshops lasted for one day (6.5 hours) and were delivered in a CAMHS clinic on a weekday.
Content
The programme’s content was rooted in cognitive behavioural theory of emotional difficulties (Beck, Reference Beck2011). CBT techniques (see Box 1) were introduced and practised through group discussion, role play, individual written tasks and hand-outs. Video vignettes of teenage actors depicting common adolescent challenges were used to normalize experiences, stimulate discussion and enhance learning. Members received a workbook containing home-practice exercises and a summary of the workshop content.
DISCOVER workshop topics
1. Introductions and icebreakers
2. ‘About stress’ (psychoeducation)
3. ‘The stress cycle’ (basic CBT model – thoughts, feelings, physical sensations and behaviour)
4. ‘Thoughts, different perspectives and thinking styles’ (negative thinking patterns or biases)
5. ‘How to change your thinking and feel better’ (distraction, thought challenging, mindfulness)
6. ‘Behavioural changes’ (graded exposure, problem-solving, time management)
7. ‘Mind and body connections’ (sleep hygiene, relaxation)
8. ‘Tackling my problems’ (goal setting and maintaining motivation)
Sclare et al. (Reference Sclare, Michelson, Malpass, Coster and Brown2015)
Participants were invited to set personal goals at the end of the workshop. These were discussed approximately one week later in a 20–30 minute ‘telephone goal review’ with one of the group facilitators, aimed at monitoring and supporting progress. Participants were offered up to three additional reviews within the 8-week post-workshop period.
Usual care
Participants in both arms received usual care from CAMHS while on the waiting list. This consisted of a clinic letter sent to the home of each participant, detailing local support services and emergency contacts in case of risk concerns. Participants were free to access any other treatment or professional support available to them outside CAMHS, including medication.
Measures
Feasibility
Please see study objectives for an outline of feasibility indicators.
Acceptability
At the end of the workshop, participants completed the 8-item Client Satisfaction Questionnaire (CSQ-8; Larsen et al., Reference Larsen, Attkisson, Hargreaves and Nguyen1979). Items are summed to produce an overall treatment satisfaction score. We used an established categorization system (Smith et al., Reference Smith, Roche, O’Loughlin, Brennan, Madigan, Lyne and O’Donoghue2014) to denote different aggregate satisfaction scores: poor (score of 8–13); fair (14–19); good (20–25); and excellent (26–32). Three open-ended questions were appended to the CSQ-8, exploring perceived helpfulness, suggestions for improvements and any other comments.
Participants in the intervention group were also invited to complete semi-structured exit interviews with a researcher (lead author) immediately after the follow-up outcome assessment. These 20–30 minute interviews followed a semi-structured topic guide, with questions and prompts that explored (i) views of the recruitment and assessment process; (ii) intervention content and structure; and (iii) perceived impact of the programme on well-being and service use. Interviews were audio-recorded and transcribed verbatim.
Clinical outcomes
Outcomes were assessed using three validated self-report measures at baseline (before randomization) and 8-week follow-up. There were no changes to outcome measures after the trial commenced. The selected measures have been recommended by the UK’s Child Outcomes Research Consortium (CORC; www.corc.uk.net) for routine use in CAMHS.
Primary outcome
The primary outcome was depression severity, measured by the 33-item Mood and Feelings Questionnaire (MFQ) (Costello and Angold, Reference Costello and Angold1988). This measure was particularly sensitive to change in an earlier evaluation of DISCOVER (Sclare et al., Reference Sclare, Michelson, Malpass, Coster and Brown2015). The clinical cut-off is ≥27, with higher scores indicating greater severity.
Secondary outcomes
Anxiety severity was assessed using the total anxiety score of the 47-item Revised Child Anxiety and Depression Scale (RCADS) (Chorpita et al., Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000). Raw scores were converted into standardized T-scores, with a clinical cut-off of ≥70. Higher scores represent greater symptom severity.
The 14-item Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) (Clarke et al., Reference Clarke, Friede, Putz, Ashdown, Martin, Blake and Stewart-Brown2011) was used to measure mental well-being. Higher scores represent better mental well-being.
Procedures
Recruitment
Clinicians in collaborating services screened waitlisted cases against study eligibility criteria, by scrutinizing information from routinely available referral materials, with the aid of a structured pro forma. The same clinicians then contacted potential participants by telephone and letter, enclosing a participant information sheet. Interested young people (or their parent/carer for younger adolescents) then opted in to be contacted by a researcher. If agreeable, a meeting was arranged at the clinic to obtain consent (including parallel consent from the parents/carers of 15-year-olds) and to complete baseline assessments.
Randomization and allocation concealment
Following baseline assessment, individual participant names were placed into individual opaque envelopes and sealed. Envelopes were given to an administrator, who generated the randomization sequence by shuffling the envelopes and dividing into two piles according to the specified allocation ratio. Allocation ratios were determined by minimum workshop group size requirements. The two piles were given to the workshop facilitator or an assistant psychologist, who unsealed the envelopes and informed participants of their allocation. Workshop facilitators had no prior knowledge of participants.
Blinding
All follow-up assessments were completed by a researcher (first author) who was blind to group allocation.
Data analysis
Feasibility
Feasibility indicators are described using proportions and 95% confidence intervals.
Acceptability
CSQ-8 data are described using means and 95% confidence intervals. Qualitative data from open-ended questionnaire items and interview transcriptions were subjected to thematic analysis (Braun and Clarke, Reference Braun and Clarke2006). First, data from both sources were amalgamated and reviewed manually by the first author, with general annotations made for potential codes. Second, prominent features of the data were identified and initial codes were created. Third, codes were structured into emergent themes and associated sub-themes. Fourth, themes were inspected by a supervising co-author (D.M.) to certify that data extracts supporting each theme were meaningfully linked and different themes could be clearly distinguished. Fifth, any discrepancies were deliberated and final refinements were made to themes and their definitions.
Data saturation was not formally monitored. However, in selecting an appropriate sample size, guidance from Guest and co-workers (Reference Guest, Bunce and Johnson2006) was followed, recommending that 6–12 interviews are sufficient to gain an understanding of common perceptions and experiences in a relatively homogenous sample.
Clinical outcomes
Given our primary focus on feasibility, the trial was not powered to detect significant differences. However, preliminary effects between groups were explored via one-way analysis of covariance (ANCOVA) for the clinical outcome measures, with baseline scores entered as covariates (Van Breukelen, Reference Van Breukelen2006). Only data from those who completed both baseline and follow-up assessments were used (complete case analysis). Effect sizes were calculated as Cohen’s d, where 0.20 was regarded as small, 0.50 as medium, and 0.80 as large. Due to the small sample size and low power, emphasis was placed on confidence intervals (CI; 95%) of the effect estimates over significance testing, allowing for examination of imprecision around effect sizes.
To provide estimates of the SD necessary for sample size calculations in a full-scale trial, 80 and 95% bootstrap CI estimates for the SD of baseline MFQ, RCADS and WEMWBS scores were calculated. The upper 80% CI is recommended for robust estimates of SDs (Browne, Reference Browne1995).
Results
Recruitment and participant flow
Three rounds of recruitment were completed from two clinics (round 1 in the first clinic, and rounds 2 and 3 in the second clinic) between April and November 2016 (see Fig. 1). Randomization was unsuccessful in the first clinic (round 1) due to insufficient waitlisted cases. This resulted in a protocol amendment to reduce follow-up duration from 12 to 8 weeks, thereby expanding the sampling frame available from existing CAMHS waiting lists.
Participant flow.
Overall, 97 waitlisted patients (round: 1, n = 19; 2, n = 37; 3, n = 41) were eligible to take part. Of these, 28 were enrolled (round: 1, n = 4; 2, n = 12; 3, n = 12). The sample size was too small (n = 4) in the first round to permit randomization; however, a decision was made by the Trial Management Group to deliver the workshop for these adolescents, based on clinical need. Participants in this workshop were removed from outcome analyses, but were included in estimates of feasibility.
In the second round of recruitment, n = 12 participants were randomized in a 2:1 ratio, with eight participants allocated to the intervention arm and four to the control arm.
In the third round of recruitment, n = 12 participants were initially randomized on a 2:1 ratio as above. However, two participants allocated to the intervention arm subsequently expressed uncertainty regarding workshop attendance. As a minimum group size of six was considered paramount to ensure a viable and acceptable group, a further decision was taken by the Trial Management Group to break randomization and allocate an extra participant to the intervention group. This was achieved by randomly selecting a participant from the control group and re-allocating to the intervention. Consequently, the allocation ratio was 3:1 (nine participants in the intervention group and three in the control group) in the third round of recruitment.
Outcomes
Feasibility of research procedures
Recruitment
Ninety-seven 15- to 18-year-olds on clinic waiting lists were identified as having prominent features of anxiety and/or depression, as per eligibility requirements. Thirty-one potential participants (32.0%; 95% CI = 22.9 to 42.2%) could not be contacted via the details provided in the referral, leaving 66 who were contactable (68.0%; 95% CI = 57.8 to 77.1%).
Consent
Of the remaining 66 contactable adolescents, 28 consented to take part (42.4%; 95% CI = 30.3 to 55.2%). Thirty-three adolescents declined; the two main reasons were other timetabled commitments, most commonly school (n = 15); and apprehension about participating in a group (n = 11). The remaining seven declining adolescents did not offer a reason. Five additional adolescents declined clinic contact entirely (i.e. they no longer needed support or were accessing another service) and were therefore discharged from the waiting list.
Baseline assessments
All 28 of the consenting participants completed baseline assessments (see Table 1 for participant characteristics).
Baseline demographics and clinical characteristics
n, number of participants; SD, standard deviation.
Randomization
Four consenting participants could not be randomized from the first round of recruitment due to insufficient sample sizes. Twenty-four participants were randomized in the next two rounds of recruitment (85.7%; 95% CI = 67.3 to 96%): 17 were randomized to the intervention arm and seven to the control arm. Randomization was broken for one participant in the third recruitment cohort, such that a participant was randomly selected from the control group and re-allocated to the intervention (see justification above).
Follow-up assessments
Twenty out of 24 randomized participants (83.3%; 95% CI = 62.6 to 95.3%) completed follow-up assessments. Attempts were made to contact all randomized participants, but four could not be reached.
Feasibility of intervention delivery
Workshop attendance
Fifteen out of 17 participants in the intervention arm attended at least part of a DISCOVER workshop (88.2%; 95% CI = 63.6 to 98.5%). One young person did not attend because of anxiety about participating in a group, and another young person failed to attend because of parental objections to involvement in mental health services. Another participant started, but did not complete the full workshop due to anxiety experienced in the group.
Telephone follow-up
Telephone goal reviews were completed with 11 out of 15 workshop attenders (73.3%; 95% CI = 44.9% to 92.2%). Nine adolescents participated in one call, one received three calls and one received four calls; four could not be reached.
Acceptability
Satisfaction ratings
All 14 participants who completed the full workshop also completed the CSQ-8. The mean treatment satisfaction score was 26.86 (SD = 3.88; 95% CI = 24.82 to 28.89), with 13 out of 14 participants (93%; 95% CI = 66 to 100%) indicating overall service satisfaction in the ‘good’ or ‘excellent’ range.
Qualitative feedback
Eleven participants took part in feedback interviews and 14 participants responded to open-ended questions from the CSQ-8. Participants were satisfied with and posed no suggestions about modifying procedures for initial approach, consent, questionnaire administration, randomization and follow-up process. Data on treatment acceptability were categorized into three over-arching themes reflecting different areas of perceived benefit (see Table 2 for participant quotes underpinning each theme). Suggestions for improvements have been described separately.
Participant quotes underpinning each qualitative theme
Being acknowledged
Participants appreciated the opportunity to access more immediate support and attention from professionals instead of remaining unseen and ‘forgotten’ on a waiting list. The availability of telephone goal reviews also contributed to a sense of being kept in mind and validated as an individual whose needs are worthy of attention.
Valuing the group experience
Group interaction was highly valued for reasons such as finding comfort in shared difficulties, and shared learning of coping strategies. One participant expressed limited gain from the workshop content, but appreciated the group experience. There was a general appreciation of the group structure and dynamics being collaborative and facilitating engagement.
Developing improved ways of coping
Young people valued learning a range of practical coping strategies that they could explore before deciding which was most helpful for them. Participants particularly endorsed methods for re-appraising problem orientation and habitual responses to stressors. Changing perspectives and responses tended to be linked to improvements in relationships and associated reductions in emotional distress.
Although there was consensus that DISCOVER had helped participants become better resourced for coping with stress, there was a felt need for further support to explore individual problems and consolidate learning. There was also concern about being taken off the waiting list, for fear of problems worsening and losing access to specialist care.
Suggestions for improvement
The most commonly suggested improvement was a shortened workshop programme, with several participants preferring a shorter duration to minimize fatigue. Other participants were concerned about losing content, leading to the suggestion of splitting the workshop across separate days. Mixed views were expressed around the value of the video vignettes: some participants found the video characters relatable and helpful for modelling coping strategies, whereas others considered the scenarios depicted to be somewhat unrealistic. There were also reports of discomfort around the use of telephone calls for the goal reviews, with alternative methods of written communication (e.g. email or text messages) preferred.
Clinical outcomes
Intervention effect estimates and confidence intervals
As shown in Table 3, there were no conclusive differences in clinical outcomes between the intervention and control groups. However, trends were observed towards potential intervention effects on anxiety and mental-wellbeing. The effect estimate for depression was imprecise. CIs were wide and the presence of effects on primary and secondary outcomes cannot be ruled out at this stage.
Clinical outcomes: treatment (n = 13) and control (n = 7)
1 Adjusted mean difference is the treatment effect after controlling for baseline differences of outcome. CI, confidence interval; SD, standard deviation; MFQ, Mood and Feelings Questionnaire; RCADS, Revised Children’s Anxiety and Depression Scale; WEMWBS, Warwick-Edinburgh Mental Well-Being Scale.
Outcome variance estimates
Table 4 presents the estimates of SDs for future sample size calculations. The upper 80% CIs are recommended for any future power analyses of a full-scale trial.
Outcome variance estimates for future sample size calculations
CI, confidence interval; SD, standard deviation; MFQ, Mood and Feelings Questionnaire; RCADS, Revised Children’s Anxiety and Depression Scale; WEMWBS, Warwick-Edinburgh Mental Well-Being Scale.
Discussion
The present study aimed to test the preliminary implementation of DISCOVER CBT workshops, for waitlisted depressed and/or anxious adolescents in CAMHS, using a feasibility RCT design. Although we observed specific challenges in participant recruitment and randomization, rates of intervention participation and follow-up were encouraging. Moreover, the intervention was generally well-received by participants based on satisfaction scores and qualitative feedback. In terms of preliminary clinical outcomes, potential trends towards improvements favouring the intervention group were identified on some measures. Potential for impact was corroborated through participant exit interviews.
With respect to trial recruitment challenges, approximately one-third of eligible patients could not be reached using contact information obtained from clinic registers. In real-world clinical settings, referral sources often provide incomplete or inaccurate information (Foot et al., Reference Foot, Naylor and Imison2010; Gandhi et al., Reference Gandhi, Sittig, Franklin, Sussman, Fairchild and Bates2000). This effectively inflates the total sampling frame needed to reach an intended sample size.
Among those adolescents who were successfully contacted, the research consent rate (42%) was considerably lower than the benchmark of >80% reported in other psychological treatment trials with comparable populations (e.g. Chapman et al., Reference Chapman, Loades, O’Reilly, Coyle, Patterson and Salkovskis2016; Goodyer et al., Reference Goodyer, Reynolds, Barrett, Byford, Dubicka, Hill and Fonagy2017). Non-participation primarily related to concerns about the group format and timing of workshops. Pre-group preparation (e.g. clarifying treatment expectations and outlining group rules) has been recommended for alleviating anxieties before embarking on any group-based psychological intervention (Bernard et al., Reference Bernard, Burlingame, Flores, Greene, Joyce, Kobos and Feirman2008). This would be a useful avenue to explore for boosting uptake in further clinical evaluations of DISCOVER. Relevant modifications could include the use of online videos as an engaging and accessible method for socializing potential participants to the workshop model (Campinha-Bacote and Dexter, Reference Campinha-Bacote and Dexter2012), especially considering that the DISCOVER curriculum already makes extensive use of video materials.
With regards to timetabling, there was an inevitable trade-off between the efficiency of a one-day group intervention and scheduling restrictions, such that the group workshops could not be planned around the schedules of individual participants. Moreover, the potential for delivering community-based DISCOVER workshops on non-school days has previously received limited support (Sclare et al., Reference Sclare, Michelson, Malpass, Coster and Brown2015). However, young people in other studies have expressed a desire for greater flexibility from mental health services (Abdinasir, Reference Abdinasir2017); hence there may be value in exploring the delivery of clinic-based DISCOVER workshops on non-school days.
The above-mentioned recruitment barriers meant that randomization was not feasible in the first round of recruitment, and was broken in the third. One option for expanding the sampling frame in a future trial would involve extending eligibility criteria to include young people with impairing but subthreshold symptoms, who might otherwise be denied access to specialist mental health care (Balázs et al., Reference Balázs, Miklósi, Keresztény, Hoven, Carli, Wasserman and Wasserman2013; Children’s Commissioner for England, 2016). This would offer a pragmatic way to boost sample size, while building service capacity to treat young people at an earlier and less severe stage in their mental health presentation. From a service provider perspective, there is also a direction towards scaling-up innovative early interventions for youth mental health, in line with policy initiatives and other drivers of service redesign (Department of Health and NHS England, 2015; McGorry et al., Reference McGorry, Bates and Birchwood2013).
Participation across subsequent phases of the study flow was encouraging. The rate of ‘loss to follow-up’ (17%) was within the range of 15–20% missing data commonly reported in psychological and educational research [Enders (2003), as cited in Dong and Peng, Reference Dong and Peng2013], while the intervention drop-out rate (11.8%) compared favourably with the benchmark of 18.5% identified in a recent meta-analysis of pre-treatment drop-out from CBT in child and adolescent populations (Fernandez et al., Reference Fernandez, Salem, Swift and Ramtahal2015). Participants engaged well with the goal review follow-up calls, with 73.3% of workshop attenders completing at least one call. This is similar to the rate of 78.8% reported in a previous evaluation of DISCOVER (Brown et al., Reference Brown, Blackshaw, Stahl, Fennelly, McKeague, Sclare and Michelson2019). In the qualitative feedback, participants described positive aspects of the calls, such as feeling acknowledged, but also advocated for alternative written communication methods in preference to telephone conversations. This mirrors previous findings that telephone calls may be perceived as less acceptable by adolescents relative to other modes of mental health care delivery (Bradford and Rickwood, Reference Bradford and Rickwood2012).
Both quantitative and qualitative feedback suggested high levels of acceptability for the workshops. All but one participant in the intervention arm rated overall service satisfaction as good or excellent, and the mean satisfaction score (mean = 26.86; SD = 3.88) was similar to CSQ-8 scores in other trials involving more conventional CBT formats, such as 12-session individual CBT for youth with anxiety (mean = 26.0; SD = 4.5; Khanna and Kendall, Reference Khanna and Kendall2010) and depression (mean = 26.75; SD = 4.19; Shirk et al., Reference Shirk, DePrince, Crisostomo and Labus2014). Within the workshop format, interactions with fellow group members and facilitators were highly valued, consistent with other CAMHS research citing collaboration and reduction of power differentials as crucial for meeting the developmental needs of older adolescents (Harper et al., Reference Harper, Dickson and Bramwell2014).
Considering the DISCOVER service model more generally, participants appreciated being able to access more immediate support while on a waiting list in CAMHS. Comments were made by some participants about a need for mental health care after ending their involvement with DISCOVER. This would be consistent with a stepped-care model, where an initial ‘low-intensity’ (less costly and less time-intensive) psychological intervention is followed by another treatment of incremental intensity (Clark, Reference Clark2011).
We also assessed clinical outcomes at baseline and 8-week follow-up. Findings were inconclusive for all outcomes; however, trends were observed towards potential intervention effects on anxiety and well-being, while the effect on depression was imprecise. We reiterate that the study was not designed to provide reliable estimates of effectiveness and caution should be applied in interpreting outcomes derived from small sample sizes (Ioannidis, Reference Ioannidis2005). Nevertheless, the observed pattern of results raises important questions about the transdiagnostic effects of the intervention. Around 70% of the total sample had been referred to CAMHS for depression (with around half of these participants presenting with comorbid anxiety), while the overall rate of anxiety was slightly lower at around 60%. Hence, the observed lack of impact on depression is not obviously explained by low baseline prevalence. Pending replication of this finding, it is possible that treatment elements of more specific relevance to depression may need to be enhanced in any future modifications of the programme. For example, this may call for a greater emphasis on behavioural activation, which is a well-established component of CBT for depression (Weersing et al., Reference Weersing, Rozenman and Gonzalez2009). Measuring depression and anxiety as co-primary outcomes could assist with better understanding the relative effects in a future definitive trial. Lastly, we must note a recent meta-anaysis of 447 trials of psychotherapy for youth internalizing problems, which found strongest post-treatment effects for anxiety (0.61) and weakest effects for depression (0.29) (Weisz et al., Reference Weisz, Kuppens, Ng, Eckshtain, Ugueto, Vaughn-Coaxum and Fordwood2017). Hence, optimizing treatments for depression appears to be a key issue for the field of youth psychotherapy as a whole.
Limitations
The feasibility design and relatively small sample size necessarily limit the conclusions that can be drawn from this trial. Although we observed trends towards certain improved outcomes favouring the intervention, the study was under-powered and the effect size estimates included wide confidence intervals.
Potential allocation bias was also introduced due to randomization being compromised. Additionally, selection bias cannot be ruled out, considering that only around one-quarter of all eligible participants in the study’s sampling frame were successfully contacted and consented. Formal assessment of selection bias could not be undertaken given the relatively sparse clinical and demographic information available from referral letters. Lastly, interviews could not be undertaken with the two participants who attended the workshop but did not complete follow-up assessment. Hence, bias from attrition might have been introduced, as interviews were not conducted with participants who potentially had less favourable experiences of the intervention.
A final limitation pertains to the qualitative analysis. Codes were ordered into thematic categories in consultation with a senior co-author, although the initial codes were derived without independent checks. Having two researchers involved in all aspects of the qualitative analysis might have enhanced the reliability of findings.
Recommendations for future research
The present feasibility trial has demonstrated that a full-scale RCT of DISCOVER is warranted in specialist youth mental health clinics, building on existing studies of DISCOVER in self-referred community samples (Brown et al., Reference Brown, Blackshaw, Stahl, Fennelly, McKeague, Sclare and Michelson2019; Sclare et al., Reference Sclare, Michelson, Malpass, Coster and Brown2015). Our feasibility data point towards short-term improvements in participants’ emotional functioning after participating in the DISCOVER workshop, along with an expressed need for additional therapeutic support to consolidate therapeutic gains. A future trial of DISCOVER would provide more definitive evidence about the potential sensitizing effects of the intervention within a stepped-care framework. This should incorporate protocol refinements to enhance the likely recruitment rate, and compare prospective rates of service use, as well as clinical outcomes for adolescents who received either DISCOVER or usual care. A parallel economic analysis would allow for the assessment of potential cost efficiencies stemming from reductions in waiting times and duration of care.
Clinical implications
Just under half of all young people who were contactable opted in to the study, suggesting significant demand for a brief psychological intervention, that can be accessed promptly by adolescents who would otherwise be waiting for usual care. Issues around service capacity are particularly salient for older adolescents who often ‘fall through the gap’ between adult and child services, failing to access treatment at a crucial developmental stage (Memarzia et al., Reference Memarzia, St Clair, Owens, Goodyer and Dunn2015; Pona et al., Reference Pona, Royston, Bracey and Gibbs2015). The specific innovations of DISCOVER (e.g. age-appropriate content; interactive, video-supported group workshop format) serve as a promising platform for delivering a brief, developmentally attuned, first-line transdiagnostic intervention for clinic-referred adolescents with emotional difficulties. Nevertheless, there is scope for further optimization of this intervention model, such as providing participants with preparatory information to address concerns about the group workshop format; and offering a choice about the preferred mode of contact for 1:1 follow-up sessions, rather than limiting this to telephone calls. The potential for introducing focal treatment elements that are specific to anxiety and depression also warrants consideration.
Conclusion
The current study provides preliminary evidence for the feasibility and acceptability of delivering the DISCOVER one-day CBT workshop intervention for waitlisted, clinic-referred 15- to 18-year-olds with emotional difficulties, while establishing key research parameters needed to design a full-scale trial. Future evidence on clinical effectiveness and service-level outcomes (including associated direct and indirect costs) will help to determine whether DISCOVER can be scaled up to provide accessible, age-appropriate and cost-effective mental health care for the high volume of adolescents presenting to specialist services with anxiety and depression.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1352465819000286
Author ORCID
Christina E. Loucas, 0000-0002-1894-2392
Acknowledgements
We are most grateful to the young people who took part in this study. We would like to thank all staff from Lambeth and Lewisham CAMHS; South London and Maudsley NHS Foundation Trust and King’s College London who supported the project. These include: Dr Tessa Crombie, Dr Sadie Williams, Dr Lauren Breese, Natasha Hutchinson, Delores McPherson, Wendy Kean, Martin Roberts, Joy Osbourne, Anna Garrod, Dr David Turgoose, Vanja Rossberg and Dr Wendy Geraghty. We give particular thanks to Dr Philip Kelly, Lauren Deadman and Nadine Myers for going above and beyond in their support.
Financial support
D.S.’s contribution was part-funded by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
Conflicts of interest
I.S., D.S. and D.M. previously held an NIHR ‘Research for Patient Benefit’ project grant (ref. no. PB-PG-0213-30072) to evaluate DISCOVER workshops in UK secondary schools. I.S. is employed by South London and Maudsley NHS Foundation Trust, where she leads the dissemination and delivery of DISCOVER workshops within the organization and for external commissioning agencies. None of the authors receives any personal royalties, fees or honoraria from their involvement with the DISCOVER programme.
Ethical statement
All authors have abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the American Psychological Association (APA). Ethical approval was obtained by the London-Harrow NHS Research Ethics Committee (reference: 16/LO/0231).
Comments
No Comments have been published for this article.