Introduction
Social recovery therapy
Cognitive behavioural therapy for psychosis (CBTp) is a well-documented and effective intervention for treating the positive symptoms of psychosis (Bighelli et al., Reference Bighelli, Salanti, Huhn, Schneider-Thoma, Krause, Reitmeir, Wallis, Schwermann, Pitschel-Walz, Barbui, Furukawa and Leucht2018; Wykes et al., Reference Wykes, Steel, Everitt and Tarrier2007; Wykes, Reference Wykes2014). However, there are limited studies focusing on interventions for functional outcomes, despite research evidence that such outcomes are poor in this client group (Hodgekins et al., Reference Hodgekins, Birchwood, Christopher, Marshall, Coker, Everard and Fowler2015). Fowler et al. (Reference Fowler, French, Hodgekins, Lower, Turner, Burton, Wilson and Steel2013) propose a specific therapeutic intervention called social recovery therapy (SRT) which incorporates CBT techniques with case management and employment support in order to improve functional outcomes following psychosis. Social recovery can be defined in terms of engagement in activities within occupational and interpersonal domains (Hodgekins et al., Reference Hodgekins, Birchwood, Christopher, Marshall, Coker, Everard and Fowler2015). This may include work, education, valued social activities, and relationships with others. The SRT intervention includes an assessment and a formulation, where the formulation focuses on the individual client’s unique barriers to social recovery, including those within the client’s wider system. The techniques used within the therapy sessions should be formulation driven with a large focus on active behavioural work, where the client and their therapist work together to engage in meaningful activity which is guided by the client’s values and determined by their own goals. Evidence and experiences from behavioural work are used to instil hope and promote positive beliefs about the self (Hodgekins and Fowler, Reference Hodgekins and Fowler2010) as the individual works towards achieving meaningful change in their lives. The therapist also works with the client’s wider system to break down barriers and facilitate change.
SRT was piloted in the Improving Social Recovery in Early Psychosis (ISREP) study and found to be effective in improving hours per week spent engaged in structured activity in individuals with non-affective psychosis both immediately post-intervention (Fowler et al., Reference Fowler, Hodgekins, Painter, Reilly, Crane, Macmillan, Mugford, Croudace and Jones2009) and 12 months later (Fowler et al., Reference Fowler, Hodgekins and French2017b). The SUPEREDEN3 (Sustaining Positive Engagement and Recovery) study was a larger phase II randomized controlled trial, which aimed to test whether the use of SRT was effective in increasing the time spent engaged in structured activity by young people with first episode of psychosis with persistent social disability, compared with those in receipt of usual Early Intervention for Psychosis service provision. This study found an average improvement of 8 hours per week in those who received SRT plus early intervention, compared with early intervention alone (Fowler et al., Reference Fowler, Hodgekins, French, Marshall, Freemantle, McCrone, Everard, Lavis, Jones, Amos, Singh, Sharma and Birchwood2018). Thus, the evidence suggests that SRT may be useful in improving functional outcomes in people with first episode psychosis.
Adherence and competence
In order to effectively interpret the results of therapy trials and to translate the findings of research studies into practice, it is important to understand the extent to which the interventions being tested were delivered. Moreover, in order to examine which therapeutic techniques resulted in positive change for participants, we need to measure the specific components of the therapy intervention and the extent to which they were delivered in the trial (Fowler et al., Reference Fowler, Rollinson and French2011). This has traditionally been explored using adherence scales. Adherence has been defined as ‘the extent to which a therapist used interventions and approaches prescribed by the treatment manual and avoided the use of interventions proscribed by the manual’ (Waltz et al., Reference Waltz, Addis, Koerner and Jacobson1993). This differs from the concept of competence which has been defined as ‘the level of skill shown by the therapist in delivering the treatment … the extent to which the therapists conducting the interventions took the relevant aspects of the therapeutic context into account and responded to these contextual variables appropriately’ (Waltz et al., Reference Waltz, Addis, Koerner and Jacobson1993).
Young and Beck (Reference Young and Beck1980) developed the Cognitive Therapy Rating Scale (CTRS), which was later revised by Blackburn et al. (Reference Blackburn, James, Milne, Baker, Standart, Garland and Reichelt2001), producing the Revised Cognitive Therapy Scale (CTS-R) as a tool for measuring the competence of therapists delivering CBT. Fowler et al. (Reference Fowler, Rollinson and French2011) highlight that whilst the CTRS measures a therapist’s general competency in delivering CBT therapy, it does not capture adherence to CBTp. Startup et al. (Reference Startup, Jackson and Pearce2002) highlight that CBTp has a different content from CBT for other non-psychotic conditions, and consequently reported the development of a new adherence scale, the Cognitive Therapy for Psychosis Adherence Scale (CTPAS).
Rollinson et al. (Reference Rollinson, Smith, Steel, Jolley, Onwumere, Garety, Kuipers and Fowler2008) revised the CTPAS to create an adherence scale that could be used within the Psychological Prevention of Relapse in Psychosis (PRP) trial (Garety et al., Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers2008). The Revised Cognitive Therapy for Psychosis Adherence Scale (R-CTPAS) expanded the original tool from 12 to 21 items. Rollinson et al. (Reference Rollinson, Smith, Steel, Jolley, Onwumere, Garety, Kuipers and Fowler2008) suggest that formally measuring the fidelity of treatment could enhance understanding the mechanisms of change within complex interventions. Indeed, analysis of adherence data from the PRP trial concluded that treatment was only effective if participants received a full dose of CBTp (Dunn et al., Reference Dunn, Fowler, Rollinson, Freeman, Kuipers, Smith, Steel, Onwunere, Jolley, Garety and Bebbington2012), highlighting the importance of routinely assessing treatment adherence in therapy trials in order to understand which techniques work for which patients under which conditions (Fowler et al., Reference Fowler, Rollinson and French2011). In addition to these findings, Spencer et al. (Reference Spencer, McMenamin, Emsley, Turkington, Dunn, Morrison, Brabban, Hutton and Dudley2018) found a dose–response effect in another trial of CBT for psychosis, with more sessions resulting in better outcomes, further highlighting the importance of assessing therapy dose effects.
Similarly, using adherence data from the EDIE-II trial of CBT for people with at-risk mental states, Flach et al. (Reference Flach, French, Dunn, Fowler, Gumley, Birchwood, Stewart and Morrison2015) demonstrated a greater effect of the intervention if both homework and a formulation were part of the therapy. This highlights how it is possible to begin to understand the impact of different components of therapy.
Measuring adherence in social recovery therapy
Whilst the R-CTPAS is a well-validated tool for use in assessing adherence in CBTp, SRT has a different emphasis and thus requires a new adherence checklist. A checklist was developed which included all of the different components of the SRT intervention. This paper aims to describe the SRT adherence checklist and to describe the content of the therapy delivered in the SUPEREDEN3 trial, both in terms of the frequency of different SRT techniques used and the proportion of participants who received a full dose. In line with the aims of SRT, it was hypothesized that behavioural techniques would be frequently used and that cognitive work would mostly focus on fostering a positive sense of self. It was also hypothesized that systemic work (i.e. involving family members and other organizations) would be a key feature of the intervention.
Method
The SUPEREDEN3 trial was a single blind, phase II, randomized controlled trial comparing treatment from the early intervention for psychosis service (EIS) plus SRT, with EIS alone. A detailed description of the SUPEREDEN3 trial is provided in Fowler et al. (Reference Fowler, Hodgekins, French, Marshall, Freemantle, McCrone, Everard, Lavis, Jones, Amos, Singh, Sharma and Birchwood2018).
Participants
Eligible participants were aged 16–35 years; had non-affective psychosis; had been clients of EIS teams in Birmingham, Lancashire, Norfolk and Sussex for 12–30 months; and had low levels of structured activity indicating severe social disability (defined as less than 30 hours a week engaged in structured activity). In total, 154 participants were recruited into the trial across the four sites (Birmingham, Norfolk and Suffolk, Lancashire and Sussex), with 75 randomized to receive SRT plus EIS, and 79 randomized to receive EIS alone. Adherence data are available for 74 of the 75 participants randomized to the SRT plus EIS arm. Further information about participant demographics is provided in Fowler et al. (Reference Fowler, Hodgekins, French, Marshall, Freemantle, McCrone, Everard, Lavis, Jones, Amos, Singh, Sharma and Birchwood2018).
Therapy delivery
The SRT sessions were delivered by 13 therapists across the four sites of Lancashire, Norfolk and Suffolk, Sussex and Birmingham. All therapists were either qualified clinical psychologists or cognitive behavioural therapists and each provided a recording of at least one of their therapy sessions which was rated using the Cognitive Therapy Scale-Revised (CTS-R; James et al., Reference James, Blackburn and Reichelt2001), from which all therapists were deemed to be competent at delivering CBT. Training in the SRT model and intervention was provided including regular therapy training days and teleconferences held for all therapists. Weekly supervision was provided at each site. All therapy tapes were encrypted for data protection purposes.
The Adherence Checklist
The therapy adherence checklist consisted of the 14 key components of the SRT intervention. Table 1 provides a list of the components and their description. The checklist was developed by the authors and corresponded to the key components of the therapy as detailed in the therapy manual (Fowler et al., Reference Fowler, French, Hodgekins, Lower, Turner, Burton, Wilson and Steel2013).
Social recovery therapy adherence components for SUPEREDEN3 study
In addition to the adherence checklist, therapists also completed data about additional between-session therapy contacts. This may include contact with family members or other services (e.g. education and employment providers or voluntary sector agencies). These data were used to provide further information about the systemic elements of the intervention.
Procedure
For each of the therapy sessions delivered the research therapist completed an adherence checklist. All research therapists were encouraged to complete the checklist as soon as possible after their session with the participant. This involved the therapist endorsing (present/absent) which of the key therapy elements they considered had been present in the session, along with providing some notes to demonstrate why they believed this component had been delivered. Where individual therapists did not complete the adherence checklists themselves, these were rated by two other trial therapists using either clinical notes or session notes made by the therapist to determine which components of the SRT intervention had been present. To ensure inter-rater reliability in completing the checklists in this way, two trial therapists initially completed this task and compared ratings to ensure they were concordant.
All of the adherence checklists were reviewed by at least two additional therapists working on the project in order to determine whether an individual participant had received a full dose of SRT. This involved looking at all the sessions of therapy a participant had received and determining whether the essential key components had been present. The therapists rating therapy dose were blind to the clinical outcome of the participant. In order to have received a full dose of therapy, the essential key components were defined as: at least six therapy sessions; the presence of an assessment and a formulation; at least two pieces of behavioural work where the client went out with the therapist (any behavioural work which was conducted as homework tasks or within the clinic room during a therapy session were not included in this). For those participants who received a number of the essential key therapy components, but who did not meet the criteria outlined above for having received a full dose of SRT, a partial dose definition was considered as: at least six sessions; an assessment; a formulation; some behavioural work which was not meeting the definition for a full dose, for example because it was conducted by the participant as a homework task, or it was attempted or planned but not necessarily carried out. If participants received less than six sessions and/or any of the other key therapy components were not endorsed by the therapist then a rating of no dose was given.
Analyses
Descriptive statistics were calculated to determine the proportion of sessions in which a given therapy technique was present and to describe the proportion of participants who received a full, partial or no dose of the intervention.
Results
In total, adherence checklists were completed and collated for all of the therapy sessions which were received by 74 participants who had been randomized to receive the SRT intervention. This totalled 1236 sessions.
The number of SRT sessions received by participants ranged from 0 to 37, with a mean of 16.49 sessions. Of the 75 participants who were randomized to receive SRT sessions, 43 (57.3%) received a full dose, 18 (24%) received a partial dose and 14 (18.7%) received no dose. Agreement between raters was 100%. Seven (9.3%) participants received less than five SRT sessions. One participant received no sessions and therefore no adherence checklists were completed, resulting in adherence data being available for 74 of the 75 participants randomized to SRT.
Using the adherence data available for the 1236 SRT sessions received by participants, it was calculated that the number of techniques used in an individual therapy session ranged from 0 to 12 (mean = 3.75; SD = 2.13). Frequencies for each of the items on the SRT adherence checklist are shown in Table 2. This table also presents frequency data for the SRT components for the three dose groups (full, partial and no dose). On average the no dose group received 3.64 sessions (SD = 2.47), the partial dose group received 15.5 sessions (SD = 5.40) and the full dose group received 21.27 sessions (SD = 5.70).
Descriptive statistics for the SRT adherence checklist
Of the sessions received by participants, 40 (3.2%) were engagement only, and 101 (8.2%) were engagement and/or assessment only.
Additional information about between-session therapy contacts was available for 52 participants (69.3%). Additional contacts with family members, employment/education providers or voluntary sector organizations were present for 44/52 participants (84.6%). Where these contacts were present, the number of contacts recorded ranged from 1 to 44 for a participant, with the length of time recorded for these contacts ranging from 2 to 619 minutes.
Discussion
This study describes an adherence checklist which can be used when delivering SRT in both a research and routine clinical practice context. The adherence data reported in this study describe the content of the therapy delivered in the SUPEREDEN3 study, both in terms of how often specific techniques were used and the proportion of participants who received a full or partial dose of SRT. Whilst the majority of participants received a full dose, there was variation in the dose of therapy delivered across participants. This is not considered to reflect the competence of the therapists working on the trial but rather that it is not always possible for therapists to deliver the full trial therapy.
As hypothesized, behavioural techniques were a prominent feature of the intervention, being present in 624 (50.5%) of the sessions. This is compared with cognitive techniques which were present in 431 sessions (34.9%). This is consistent with the underlying behavioural emphasis within SRT, where it would be expected that sessions involved more behavioural work, and therefore provides an indication of therapists having been adherent to the model. Consistent with the emphasis of SRT on optimism, hopefulness and building a positive sense of self, cognitive work predominantly focused on fostering a positive sense of self.
The adherence data also highlight the systemic focus of the intervention, with 19% of sessions (n = 235) being rated as ‘involving other organizations’. The additional between-session contact data provide additional support, highlighting a high number of contacts with family members, work and education providers and voluntary sector organizations. This demonstrates that for many participants therapists were engaging in a considerable amount of additional systemic work outside of the individual therapy sessions, which is again consistent with the SRT model and indicates adherence. Unfortunately in the current study we were not able to collect the additional contact data for all participants; however, the findings suggest that this would be an important indicator of adherence which should also be recorded and measured.
The adherence data presented individually for each of the three dose groups (no, partial, full) indicate a difference in the delivery of SRT. Those receiving no SRT dose experienced a majority of sessions which focused on engagement, assessment, problem list and goal development and less focus on the ‘active’ therapy techniques which were present in the sessions received by those who received a partial or full dose of SRT. Whilst the partial and full dose groups received similar levels of cognitive techniques, it can be seen that the full dose group received more behavioural and systemic work, which is consistent with the SRT model.
This work has highlighted the need for a specific therapy adherence tool to measure the delivery of SRT. Indeed, behavioural work, cognitive work focusing on building a positive sense of self, and systemic work are not included on adherence scales for CBTp.
Study strengths and limitations
This study utilized the adherence checklists completed by therapists after sessions had been delivered, rather than the rating of a selection of therapy tapes by external raters. This is a strength because it both maximized the amount of data available for analysis, and also ensured that the vast amount of behavioural work completed outside of therapy rooms was captured within the adherence checklist. However, it is also a potential weakness of the study as there was no corroboration of therapist ratings and therapists’ ratings of their own therapy may be subject to bias.
A further strength of this study is that all of the adherence checklists were reviewed by at least two research therapists working on the trial, who were blind to the clinical outcome of participants, and high levels of agreement were present for therapy dose ratings.
Further research
Whilst this analysis of the therapy adherence data has enabled us to answer key questions about both how the therapy looked in practice and to what extent participants received a full, partial or no dose of the trial therapy, there remain a number of further questions to explore. For example, it would be useful to conduct a mediation analysis similar to Dunn et al. (Reference Dunn, Fowler, Rollinson, Freeman, Kuipers, Smith, Steel, Onwunere, Jolley, Garety and Bebbington2012) to explore whether the intervention had an enhanced effect for those receiving a full dose of therapy. It would also be interesting to explore the impact of different presentations on the delivery of the therapy. Although all participants in the SUPEREDEN3 study had poor functioning, the factors underpinning this were variable (e.g. social anxiety, negative symptoms, residual positive symptoms). By analysing the therapy adherence data for different sub-groups it may be possible to explore whether participants with different presentations received SRT with a different focus. Such findings would be useful when thinking about implementation of SRT into routine clinical practice.
In addition, it would be useful to conduct a prospective validation test of the adherence scale by including it in another intervention study. As such, the scale has been included in the PRODIGY RCT (Fowler et al., Reference Fowler, French, Banerjee, Barton, Berry, Byrne, Clarke, Fraser, Gee, Greenwood, Notley, Parker, Shepstone, Wilson, Yung and Hodgekins2017a), a trial of SRT in young people with complex and emerging mental health problems, which will enable the scale to be explored in a larger sample.
Conclusions
This work has highlighted the usefulness of a specific therapy adherence tool to measure the delivery of SRT. Key features include frequent use of behavioural techniques, cognitive work focusing on fostering a positive sense of self, and working with the wider system around the individual. This is a new intervention, requiring a new adherence tool to explore both its delivery and impact. The use of this adherence checklist tool would be considered essential for anyone delivering SRT and looking to ensure adherence to the model.
Acknowledgements
We thank all of the trial therapists who worked on this project: Eleanor Baggott, Mark Bernard, Ruth Clutterbuck, Gillian Dewhurst, Mike Fitzsimmons, Sam Fraser, Charlotte Gregory, Rebecca Ison, Chris Jackson, Christine Lowen, Lulu Preston, Kat Pugh and Ruth Turner. We also thank the participants involved in this study and National Health Service staff in the early intervention services who supported this project.
Ethical statements
Ethics approval was granted by the National Research Ethics Service Committee in the Black Country, West Midlands (reference: 12/WM/0097). The authors have abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the APA.
Conflicts of interests
All authors declare that they have no conflicts of interest with respect to this publication.
Financial support
Funding for the SUPEREDEN3 trial was received from the National Institute for Health Research under the Programme Grants for Applied Research programme (RP-PG-0109-10074). Professor Max Birchwood, Chief Investigator of SUPEREDEN, is part-funded by NIHR CLAHRC West Midlands.
Comments
No Comments have been published for this article.