Introduction
Hands-on training is essential for new therapists to develop their competence in psychological therapies. There is an implicit assumption, propagated by both services and training courses, that trainees ought to be allocated cases that are relatively mild in severity and complexity, so-called “training cases”. On the other hand, it has been argued that clinical psychology trainees are underexposed to work with severe and enduring mental health difficulties and that this should be emphasized more during training (Mueser, Silverstein and Farkas, Reference Mueser, Silverstein and Farkas2013). Moreover, through reduced resources and funding, as well as pragmatic difficulties in identifying perfect “training cases”, trainees may increasingly be allocated severe and complex cases. However it has not been established empirically what the impact of a trainee therapist is on outcome. In the absence of a solid evidence base, negative performance-related beliefs and assumptions may be activated in services and trainee therapists, potentially generating additional anxiety for the trainee, supervisor and, most importantly, the client. As with other areas of clinical practice, clinical decision-making on these issues should be evidence-based. This study sought to provide empirical evidence in this area to help inform the allocation of cases.
One assumption behind the allocation of mild-severity “training cases” may be that the quality and acceptability of care offered by trainee and qualified therapists is more similar for milder severity cases. It may be that such cases allow therapists to better focus on the key components needed to effect change. However, there is a lack of research examining the effect of experience on outcome, particularly when stratifying by different therapy approaches, and the majority have examined this question in qualified therapists only. A review of therapist effectiveness for qualified clinical psychologists highlighted the considerable variance in the relationship between years of practising and client outcome, with effect sizes ranging from small and negative to large and positive (Beutler, Reference Beutler2004). Other studies have examined the performance of qualified therapists on theoretical knowledge and formulation skills. Eells, Lombart, Kendjelic, Turner and Lucas (Reference Eells, Lombart, Kendjelic, Turner and Lucas2005) found that cognitive-behavioural and psychodynamic psychotherapists with “substantial” levels of expertise outperformed those with modest or intermediate levels. In contrast, Witteman and van den Bercken (Reference Witteman and van den Bercken2007) found a non-linear relationship such that novice and experienced psychotherapists both outperformed the intermediates. More recently, Vollmer, Spada, Caspar and Burri (Reference Vollmer, Spada, Caspar and Burri2013) evaluated trainees and behaviour therapists in Germany on both theoretical knowledge and clinical skills, assessed by exam format with vignettes. They found that trainees were superior to experienced therapists in both aspects, and also showed less variability in clinical skills. However, each of these studies utilized theoretical measures of competence and so it is not clear how representative the findings are to actual client outcomes “in the clinic”. In summary, there is considerable variability across studies examining the relationship between experience and therapist effectiveness.
The present study aimed to determine whether outcomes on psychometric measures differ with therapist experience, and whether this interacts with client severity. We addressed these questions across a range of anxiety disorders seen in service across primary and secondary care. Anxiety disorders have the highest prevalence, with estimates of around 20% (Kessler et al., Reference Kessler, Berglund, Demler, Jin, Merikangas and Walters2005). They are associated with high levels of functional impairment and reduced quality of life (Mendlowicz and Stein, Reference Mendlowicz and Stein2000). Cognitive behavioural therapy (CBT) is a first-line treatment (e.g. National Institute for Clinical Excellence, 2004), with some of the largest effect sizes being reported for anxiety disorders (e.g. Hofmann and Smits, Reference Hofmann and Smits2008). We hypothesized that qualified therapists would achieve better outcomes than trainee therapists, and that this would be most apparent for severe presentations. We also used drop-out rates as a proxy for treatment acceptability and hypothesized a higher rate of drop-out for trainee therapists.
Method
Design
We audited the complete therapy outcome data that were collected over the past financial year (April 2012–2013) within the Centre for Anxiety Disorders and Trauma (CADAT) in South London and Maudsley NHS Trust. This is a specialist service offering treatment to primary, secondary and tertiary care levels, forming a part of local primary (Increasing Access to Psychological Therapy; IAPT) and secondary care psychological therapy services. To maximize clinical validity, these data were analysed with an intent-to-treat approach. This also allowed us to assess any effects of therapist group on drop-out.
Participants
Participants were 282 patients treated at primary and secondary care levels within the clinical centre. Therapists were qualified (9 clinical psychologists and 5 CBT therapists) or completing their training for these professions (14 IAPT high intensity and 6 clinical psychology). We assessed experience by means of self-report of the number of cases worked with in their career prior to the annual data that were audited.
Treatment and supervision
All clients received individual outpatient CBT focused on the primary anxiety disorder identified at assessment. Where there is comorbidity, treatments typically focus at least initially on the primary anxiety disorder. The treatments are based on specific cognitive models and treatments using individual formulations to tailor the treatments accordingly. Models and treatments used were typically those developed by the Clark, Salkovskis and Ehlers groups, including Clark (Reference Clark1986) for Panic Disorder, Clark and Wells (Reference Clark and Wells1995) for Social Anxiety Disorder (SAD), Ehlers and Clark (Reference Ehlers and Clark2000) for Post-Traumatic Stress Disorder (PTSD), and Salkovskis (Reference Salkovskis1985) for Obsessive-Compulsive Disorder (OCD).
All therapists received regular supervision. Trainees typically received 60–90 minutes per week for a 3-day per week placement. Full-time substantive therapists received up to 60 minutes of supervision per week. Therapists who worked part-time received supervision pro-rata. Supervision was usually individual with a small proportion of group supervision.
Measures
Primary outcome measures
1) The Generalized Anxiety Questionnaire (GAD-7; Spitzer, Kroenke, Williams and Löwe, Reference Spitzer, Kroenke, Williams and Löwe2006) is a 7-item general measure of anxiety that focuses on physical symptoms and worry with high sensitivity (89%) and specificity (82%) for anxiety disorders, with maximum score of 21 (Spitzer et al., Reference Spitzer, Kroenke, Williams and Löwe2006). We utilized established clinical cut-offs to derive severity bandings for anxiety symptoms: mild (5–9), moderate (10–14) and severe (≥ 15) (Spitzer et al., Reference Spitzer, Kroenke, Williams and Löwe2006). 2) Patient Health Questionnaire (PHQ-9; Kroenke, Spitzer and Williams, Reference Kroenke, Spitzer and Williams2001), a 9-item measure of depressive symptoms (maximum score 27) with 88% specificity and sensitivity for major depression. Cut-offs for severity bandings were: moderate (10–14), moderate-severe (15–19) and severe (≥ 20) (Kroenke et al., Reference Kroenke, Spitzer and Williams2001). Both measures have been demonstrated to have high reliability (Cronbach's α ≥ .89)and validity (Kroenke et al., Reference Kroenke, Spitzer and Williams2001; Löwe et al., Reference Löwe, Decker, Müller, Brähler, Schellberg and Herzog2008).
Secondary outcome measures
1) Disorder-specific measures. The disorder-specific measures were those recommended by Increasing Access to Psychological Therapy (IAPT) Data Handbook (Department of Health, 2011). The Obsessive Compulsive Inventory (OCI) is a 42-item measure with clinical cut-off of 40 (Foa, Kozak, Salkovskis, Coles and Amir, Reference Foa, Kozak, Salkovskis, Coles and Amir1998). The Social Phobia Inventory SPIN is 17-item measure with cut-off of 19 (Connor et al., Reference Connor, Davidson, Churchill, Sherwood, Weisler and Foa2000). The Revised Impact of Events Scale (IES-R) has 22 items and a cut-off score of 30 (Creamer, Bell and Failla, Reference Creamer, Bell and Failla2003). The Agoraphobia-Mobility Inventory (MI) is a 52-item measure with item average cut-off of 2.3 (Chambless, Caputo, Jasin, Gracely and Williams, Reference Chambless, Caputo, Jasin, Gracely and Williams1985). All measures have good internal consistency (Cronbach's α ≥.88) and validity for differentiating clinical from non-clinical populations. 2) Number of therapy sessions. 3) Drop-out frequency, defined as any unscheduled ending of therapy as determined by the treating therapist.
Analyses
We examined baseline characteristics on each of our severity measures by therapist group (trainee, qualified) and then by all therapist categories (CBT therapists and clinical psychologists separately, split by whether qualified or in training). Therapist group and symptom severity bandings were entered as fixed factors into a multivariate analysis of variance (MANOVA) with primary outcomes measures as dependent variables. Severity bandings were categorical and followed the clinical cut-offs for the PHQ-9 (moderate, moderately severe, and severe) and GAD-7 (mild, moderate, and severe). To attempt to control for client complexity, this analysis was repeated with “care cluster” as a covariate. This is a metric used within the local NHS Trust in conjunction with Payment by Results (M. Clark, Reference Clark2011). Patients are rated by their lead clinician primarily in terms of level of care needed using standardized criteria (see Appendix) that include chronicity, difficulties with engagement and challenging behaviour, degree of functional impairment, comorbidity, and level of risk to self or others.
To make comparisons on the disorder-specific measures across different disorders, we determined for each case whether they had achieved reliable and clinically significant change on their disorder-specific measure (Jacobson and Truax, Reference Jacobson and Truax1991). In brief, whereas reliable change indicates that the variance between baseline and posttreatment scores exceeds that accountable by instrument measure error, clinically significant change is defined when the posttreatment score is two standard deviations below the mean baseline score across the sample (Criterion C in Jacobson and Truax, Reference Jacobson and Truax1991). A composite of the two yielded three categories (no reliable change, reliable but not clinically significant change, and reliable and clinically significant change). The rates of these outcomes between groups, as well as baseline and posttreatment caseness, were tested with Chi-squared (χ2) tests. Rates of drop-out were also tested in this way. Finally, we explored whether therapist group interacted with baseline symptom severity in predicting outcome. The interaction term (therapist group x baseline GAD-7 score) was entered as a predictor variable for the composite change measure.
Results
Following assessment, 270 participants attended at least one therapy session. A total of 216 completed treatment (82.3%), with 54 drop-outs.
Baseline sample characteristics
PTSD, OCD, SAD, and panic disorder with agoraphobia (PD-A) were most frequently seen. Of note, trainees were allocated less PTSD and OCD (Table 1). The following anxiety disorders were seen infrequently (<20 total cases): generalized anxiety disorder (7 cases), health anxiety disorder (9 cases), specific phobia (19 cases, including 7 vomiting phobia), hoarding disorder (6 cases) and body dysmorphic disorder (19 cases). To increase statistical power, we collapsed across frequently presenting disorders (≥20 cases; i.e. PTSD, OCD, SAD, PD-A), excluding rarely presenting disorders to reduce heterogeneity. This yielded a final sample of 196 cases, after removing 14 cases missing either baseline or final PHQ and GAD scores. There were no group differences in baseline severity on any measures in this final sample (p ≥ .17; Table 2).
Frequency of cases by disorder and therapist group (trainee, qualified)
Baseline characteristics of sample across disorders
Notes: PHQ = Patient Health Questionnaire; GAD = Generalized Anxiety Disorder Assessment
With regards to experience, qualified therapists reported having seen an average of 88.9 cases (SD = 23.1) across their career, whereas the trainee therapists reported worked with an average of 18.6 cases (SD = 14.9).
Outcome on mood and anxiety measures
MANOVA was carried out with fixed factors of therapist group, baseline severity banding (depression and anxiety separately) and dependent variables % change (depression and anxiety separately) and number of sessions.
Baseline Depression Banding exerted a main effect on % change in depression [F(3,163) = 4.75, p = .003], with post-hoc tests confirming that clients with moderate–severe or severe baseline depression showed significantly more improvement than clients exhibiting mild depression (p ≤ .039). The effect of Baseline Anxiety Banding on % change in anxiety did not reach significance (p = .14), and there was no main effect of therapist group on % change in either depression or anxiety (p ≥ .4). In addition to these main effects, there was a two-way interactive effect between therapist group and Baseline Anxiety Banding on % change in anxiety [F(2,163) = 3.75,p = .026]. Follow-up tests showed that the difference in symptom change between mild and severe cases was greater change in the qualified compared to trainee therapists [F(1,120) = 4.03, p = .047; see Figure 1]. There was no significant difference between groups for the difference in symptom change between moderate and severe cases (p = .13). There were no other main effects or interactions (p ≥ .52).
Percentage change on anxiety (GAD-7) measure from baseline to posttreatment. Whereas outcomes for mild and moderate severity clients follow a similar profile for both qualified and trainee therapists, trainees evidence a statistically significant decline in outcomes with severely anxious clients
Reliable and clinically significant change on disorder-specific measures
There was no difference between the groups in terms of reliable [χ2(1) = .061, p = .81] or clinically significant (p = .33) change, nor the composite measure of both [χ2(2) = .061, p = .45). There was also no difference in rates of caseness on disorder-specific measures, either at baseline [χ2(1) = 0, p = .98] or post-treatment (p = .29). An ordinal regression model in which the therapist x baseline severity interaction term was entered as predictor of the composite change outcome was significant [χ2(1) = 4.67, p = .031; Wald = 4.6, p = .032]. Correlations revealed that baseline GAD-7 score was negatively correlated with composite outcome for trainee [Spearman ρ(68) = −.307, p = .01] but not qualified therapists [p = .71].
Number of therapy sessions and dropout
When considering number of sessions as the outcome measure, there were trends for main effects of baseline GAD severity [F(2,152) = 2.67, p = .073] and therapist group [F(1,152) = 3.31, p = .071]. Post-hoc tests showed that clients initially in either “moderate” (p = .068) or “severe” (p = .064) anxiety categories utilized more sessions that those in the “mild” range, but there was no difference between “moderate” and “severe” clients’ number of sessions (p = .86). Trainees utilized fewer sessions (mean of 10.3, SD = 4.10) than qualified therapists (mean of 12.6, SD = 7.14) [t(195) = 2.06, p = .04]. The effect of therapist group on number of sessions could not be explained by differences in drop-out rates between groups [χ2(1) = 2.13, p = .18]. Furthermore, the main effect of therapist group remained when controlling for severity by covarying baseline depression and anxiety scores (p = .026; and these covariates were both marginally significant predictors of session; p ≤ .06). However, the main effect of therapist group was no longer significant when controlling for care cluster (p = .69). In addition, the “drop-out” category had a relatively high average number of sessions (median of 4 sessions; range of 1–22). This means that “drop-outs” could also have included patients that elected to discontinue therapy after considerable therapeutic gain. There were no main or interactive effects of therapist group or baseline severity banding on caseness at outcome (p ≥ .15).
Discussion
An important finding of this study was that trainee therapists achieved comparable outcomes to qualified therapists. Contrary to our predictions, there were no significant differences in change on broad mood and anxiety measures and no differences in overall rates of reliable and clinically significant change on disorder-specific measures, when examining across the entire sample. This is nonetheless in keeping with the highly equivocal literature on the effect of experience on therapist effectiveness (Beutler, Reference Beutler2004) and with findings that trainee therapists show superior theoretical knowledge and performance on vignettes, compared to qualified therapists (Vollmer et al., Reference Vollmer, Spada, Caspar and Burri2013). In the present study, the recency of acquired theory may have compensated for relative inexperience in interpersonal clinical work. This trade-off between theory and skill may explain the finding that therapists with either low or very high levels of experience (who are highly proficient in recent theoretical knowledge or in interpersonal alliance, respectively) both outperform moderately experienced therapists (Witteman and van den Bercken, Reference Witteman and van den Bercken2007). In addition, it is likely the trainee therapists had more time to prepare for and reflect on cases, as well as more supervision time. Further work will be needed to explore the differential impact of these variables on client outcome.
There was also evidence that trainees utilized fewer sessions and this could not be explained by greater drop-out rates in this group (although this effect was no longer significant when controlling for care cluster). The more fixed limit on number of sessions in this group (imposed by rotating to other training placements) may also play a role, perhaps by reducing therapist drift (Waller, Reference Waller2009). Potential differences between therapist group may also have been further reduced by the relatively standardized cognitive behavioural treatments for anxiety disorders evaluated here, which may have offset the impact of non-specific therapist factors on outcome. Further work will be needed to determine the generalizability of this finding to other disorders, especially those with less standardized treatments.
We found some evidence that outcomes achieved by the two therapists groups diverged for more severely anxious clients. Whereas for qualified therapists improvement (on the GAD-7) increased linearly with baseline severity (most change for severely anxious clients), trainee therapists showed a plateau such that the greatest change was for moderately anxious clients (and least change for severely anxious clients). Outcomes on the disorder-specific measures were also consistent with this picture, with higher levels of baseline anxiety (GAD-7) predicting lower rates of reliable and clinically significant change in trainees, but not qualified therapists. There may be a number of factors (both non-specific and model-specific) contributing to the poorer outcomes achieved by trainees working with relatively severe presentations. Important therapist factors may include the ability to establish and maintain the therapeutic alliance, to effectively contain high levels of distress, and to engender higher levels of therapy. In addition, differences in model-specific skills – such as effecting behavioural and cognitive change and deciding when to change tack with specific techniques – may be more important for clients with relatively pronounced cognitive distortions and biases. An analysis of video recordings of CBT for anxiety showed that therapists frequently switched away from core methods such as exposure and that this was associated with poorer outcomes (Schulte and Eifert, Reference Schulte and Eifert2002).
Whilst the routine clinical setting was a strength in terms of the clinical applicability of our findings, it is important to acknowledge a number of limitations. First, there was non-random allocation of cases and the clients seen by the two therapist groups likely differed in severity and complexity. We attempted to control for this retrospectively by including baseline symptom severity and care cluster as covariates. However, future studies should aim to control or at least measure other important variables such as chronicity, comorbidity, or social adversity. In addition, whereas all cases seen by trainees were seen in primary care settings, qualified therapists also saw cases referred from secondary care services and so may have seen more treatment refractory cases. There is some emerging evidence that anxiety disorder specific therapist competence is associated with better outcomes (Ginzburg et al., Reference Ginzburg, Bohn, Höfling, Weck, Clark and Stangier2012). We were unable to measure therapist competence or adherence to anxiety disorder specific treatments, nor differences between the therapist groups in these, and it may be these constructs that underlie good outcome in our sample. Ultimately, a randomized control trial that avoids many of the confounds observed in routine clinical care may be appropriate to explore the specific question of whether therapist experience impacts on outcome (although other, complementary approaches will be needed to better understand the important mediating factors). Finally, whereas we collapsed across several anxiety disorders in the present study, it will be important to explore whether these differences exist to different degrees in specific disorders. This would also allow inferences to be drawn about outcomes on continuous disorder-specific measures, which may have greater sensitivity to group differences.
In summary, when working with mild and moderately anxious clients trainee therapists achieved outcomes that were comparable to those of qualified therapists. This may be important information both for clients who are anxious about being seen by less experienced therapists, and for trainees who may be anxious about their ability to provide a high standard of care. There was some evidence that qualified therapists obtained superior outcomes when working with severely anxious patients, and so trainees may benefit from greater clinical support for these cases. In this way these findings have implications for service-level decision-making regarding case allocation as well as for supervision of training cases.
Acknowledgements
The authors wish to thank the clients, trainees and therapists at CADAT whose work is analysed here.
This study presents independent research part-funded by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
There is no stated conflict of interest.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Comments
No Comments have been published for this article.