Introduction
Evidence-based practice encourages clinicians to use the best available research evidence, whilst considering the patients’ characteristics and values to inform care (Sackett, Rosenberg, Gray, Haynes and Richardson, Reference Sackett, Rosenberg, Gray, Haynes and Richardson1996). Reviews of the present state of mental healthcare in the UK indicate that there is still a significant gap between the treatment received by patients and the care that research suggests is effective (Shafran et al., Reference Shafran, Clark, Fairburn, Arntz, Barlow and Ehlers2009). For evidence-based care to be received, its benefits need to be understood by clinicians. However, research from the US suggests that few therapists read peer-reviewed journals, preferring instead to use theoretical or practice-oriented materials that do not focus on whether something is effective, but rather how a technique can be practised (Stewart and Chambless, Reference Stewart and Chambless2007).
Surveys show that some therapists do support the use of evidence-based treatments, but have specific arguments against the use of research to inform practice (Beutler, Williams, Wakefield and Entwistle, Reference Beutler, Williams, Wakefield and Entwistle1995; Stewart, Stirman and Chambless, Reference Stewart, Stirman and Chambless2012). For example, it has been argued that randomized controlled trials (RCTs) should not be considered to give better evidence than qualitative studies (Levant, Reference Levant2005; Wachtel, Reference Wachtel2009). The reasons cited for this include the view that evidence-based treatments have been developed and tested with relatively simple, often subclinical, cases and may not work well with the complex, comorbid and severe cases usually seen in clinical care (Nelson, Steele and Mize, Reference Nelson, Steele and Mize2006). This is despite the fact that the most common reason for exclusion from RCTs is low symptom severity, suggesting that patients in trials may have more severe conditions than in routine practice (Stirman, Derubeis, Crits-Christoph and Rothman, Reference Stirman, DeRubeis, Crits-Christoph and Rothman2005). There is also research showing that the presence of comorbidity does not necessarily predict poor outcome in clinical trials (Weisz, Jensen-Doss and Hawley, Reference Weisz, Jensen-Doss and Hawley2006). Surveys in the US have also found that therapists believe that learning evidence-based treatments is not as important as learning a specific treatment well (Stewart et al., Reference Stewart, Stirman and Chambless2012). One of the reasons usually cited for this belief is the view that all therapies are equally effective. This view is commonly referred to as the “Dodo Bird Verdict” (Rosenzweig, Reference Rosenzweig1936). Norcross (Reference Norcross1995) cited 11 arguments to dispel the Dodo Bird Verdict, including that early research into the Dodo Bird Verdict was underpowered and that too few psychotherapy modalities have been robustly evaluated to conclude that they have the same impact. Subsequent evidence that some treatments have been shown to be harmful for some conditions, but not others, also adds weight to the argument against the Dodo Bird Verdict (Lilienfeld, Reference Lilienfeld2007).
Norcross (Reference Norcross1995) also argued against the “exclusivity myth”, which he described as “the pernicious misconception that certain processes and outcomes are the exclusive property of particular therapy systems” (p. 501). The evidence against this is that there is support for common factors between therapies to be important, particularly the therapeutic alliance (Lambert and Barley, Reference Lambert and Barley2001). A recent series of meta-analyses found that the relationship had an important impact on the outcome of psychotherapy (Norcross, Reference Norcross2010), highlighting the need to consider both specific and common factors when making clinical decisions.
Other meta-analyses indicate that the therapeutic alliance might explain a smaller amount of variance in outcome than initial estimates (Horvath, Reference Horvath2005). Crits-Christoph, Gibbons, Hamilton, Ring-Kurtz and Gallop (Reference Crits-Christoph, Gibbons, Hamilton, Ring-Kurtz and Gallop2011) argued that estimates of the therapeutic alliance's impact on outcome may vary because some studies do not consider how the therapeutic alliance and symptomatic change can interact with each other; for example, they found that in later sessions of treatment, symptomatic improvement predicted therapeutic alliance, as well as the other way round. Crits-Christoph and colleagues (Reference Crits-Christoph, Gibbons, Hamilton, Ring-Kurtz and Gallop2011) also found that an averaged measure of the therapeutic alliance across the first half of a course of treatment explained more variance in symptomatic improvement than when a measure was only taken once. This indicates that differences in the size of the impact of the therapeutic alliance may be due to the way it is measured. They conclude that the therapeutic alliance is important, but there is still uncertainty about the true size of the effect that the therapeutic alliance has on treatment outcome. The importance of the therapeutic alliance was cited as a reason for not using controlled research to inform practice in a recent US based interview study. Therapists in this study stated that the therapeutic alliance is more important than the technique being used (Stewart et al., Reference Stewart, Stirman and Chambless2012). This implies that some therapists may believe that there is a trade off between focusing on the alliance and using research to inform practice.
If therapists do not believe that using research to inform practice is valuable, we need to understand what they use instead. Surveys indicate that therapists rely on their experience or their peers’ experience to make clinical decisions rather than research (Morrow-Bradley and Elliott, Reference Morrow-Bradley and Elliott1986; Stewart and Chambless, Reference Stewart and Chambless2007; Stewart et al., Reference Stewart, Stirman and Chambless2012). Westen and Weinberger have argued that clinical experience should be used in psychotherapy as research does not tend to measure all the required variables to make a necessary judgement about which treatment is likely to help patients and that “truth does not reveal itself without interpretation” (Westen and Weinberger, Reference Westen and Weinberger2005, p. 1269). This argument is countered by data demonstrating that clinical experience is subject to biases that research is not (Dawes, Faust and Meehl, Reference Dawes, Faust and Meehl1989) and basing decision-making purely on clinical experience is likely to contribute to “therapist drift” from protocols that may negatively impact therapy and make it difficult to maintain treatment integrity (Stobie, Taylor, Quigley, Ewing and Salkovskis, Reference Stobie, Taylor, Quigley, Ewing and Salkovskis2007; Waller, Reference Waller2009).
Since therapist drift can occur in therapists who do want to use evidence-based treatments, tools can be used to avoid an over reliance on experience and help clinicians implement evidence-based treatments. Treatment manuals and regular outcome measures are two such measures. These have been shown to lead to better outcomes and improve the therapeutic relationship (Hogue et al., 1998; Loeb et al., Reference Loeb, Wilson, Labouvie, Pratt, Hayaki and Walsh2005). Manuals and outcome measures have not been well received by a number of therapists (Addis and Krasnow, Reference Addis and Krasnow2000) although interviews suggest that patients are positive about outcome measures, seeing them as evidence that clinicians were taking their problems seriously through a full and structured assessment (Dowrick et al., Reference Dowrick, Leydon, McBride, Howe, Burgess and Clarke2009).
Work has been undertaken to improve current tools used to disseminate information. If research is presented in a readable and user-friendly format it has been shown to be more likely to be used (Michie and Lester, Reference Michie and Lester2005; Schmidt and Taylor, Reference Schmidt and Taylor2002). An experimental study found that therapists were more likely to be interested in learning a type of treatment when case studies were included with results from trials (Stewart and Chambless, Reference Stewart and Chambless2010).
The current study will investigate the perceived barriers to the translation of research into practice, what sources therapists use to enhance their clinical decision-making, and why these sources are used. A qualitative methodology was used because it allowed an in-depth investigation into therapists’ attitudes to research and could be led by the issues therapists felt to be important.
Method
Procedure
Three hundred and thirty-five participants (45.2%) from a previous questionnaire study (Gyani, Shafran, Myles and Rose, in press) who stated that they would like to take part in further research were randomly selected to be contacted, stratified by theoretical orientation using a random number generator. All therapists were members of professional organizations whose contact details were publicly available on the Internet. Participants were given a consent form and an information sheet. Interviews were conducted and recorded over the telephone. Recruitment was stopped after 40 participants agreed to be interviewed (95 therapists were contacted). Only 33 interviews were conducted as some participants dropped out of the study or an interview could not be arranged.
Materials
A semi-structured interview was designed to assess the barriers to the implementation of evidence-based treatments for mental health. A semi-structured interview was used to ensure that participants were asked the same questions. The questions in the interview schedule were based on the previous research outlined in the introduction. The interview started with a discussion on the sources used to inform clinical practice, before a discussion regarding what factors in addition to, or as opposed to, research were felt to be important determinants of successful treatment outcome. This was followed by a discussion about the translation of research into clinical practice. At the end of the interview all participants were asked whether they wished to add anything, or if they felt that some important issues regarding the use of research in clinical practice were not covered.
Analyses
The interviews were digitally recorded and transcribed by AG and a colleague. Interpretative Phenomenological Analysis (IPA) was used to analyse the transcribed interviews (Smith and Osborn, Reference Smith, Osborn and Smith2008). IPA was used as research or evidence can be considered to mean different things to different therapists. The transcripts were read numerous times, and the themes from the interviews were discerned both within and between therapists. AG analysed the entire dataset, and 10% of the interviews were analysed by ML to ensure reliability. One interview was analysed by AG and ML together to ensure that the same method was used by both raters. Three further interviews were selected at random and rated by ML. ML was involved in the decision of what the final themes were, which were discussed until a final agreement on all the topics was reached. The analysis was undertaken in accordance with the description of IPA given by Smith and Osborn (Reference Smith, Osborn and Smith2008). Neither of the two raters had any particular stance/perspective with regard to interview content but are committed to evidence-based practice.
Results
Thirty-three participants took part in the study. One interview could not be transcribed due to the poor quality of the recording. The composition of the therapists’ theoretical orientation can be seen in Figure 1. Twenty-two of the interviewees were female, and the mean age was 54.4 years old (SD = 10.3). Participants’ mean number of years of clinical experience was 12.4 years (SD = 8.7) and the mean number of hours therapists stated they practised psychotherapy per week was 11.9 (SD = 8.8). Eleven therapists stated that the highest qualification they had received was a Masters degree, 11 stated that it was a postgraduate diploma, four stated that it was a Doctorate, five stated that it was an undergraduate degree, and one stated that it was a post graduate certificate. Twenty-one participants stated that they worked exclusively in private practice, three stated they worked exclusively in the NHS, three stated they worked in both the NHS and in private practice, two stated they worked in both the voluntary sector and in private practice, two stated they worked in both the NHS and in an educational setting, and one participant stated that they worked exclusively in occupational health. The mean length of the interviews was 36 minutes and 33 seconds (SD = 8 minutes and 51 seconds).
The composition of participants’ theoretical orientation. CBT = Cognitive Behavioural Therapy
Two superordinate themes emerged. The first focused on philosophical issues about the nature of evidence, whereas the second focused on the sources of information used by therapists in decision-making and why therapists chose to use them. The former can be broken down into the use of unrepresentative samples, disagreements with evidence hierarchies, and the importance of therapist specific factors, and the latter can be broken down into a preference for clinical experience, theoretical or practical material over empirical material, concerns about the presentation of information, concerns about the rigidity of tools and materials in treatment, and concerns about patient experience.
Philosophical issues relating to the nature of evidence
Therapists’ attitudes varied in terms of the evidence they required for a finding to be implemented. The first quote in Table 1 demonstrates a high threshold for acceptability in research, whereas other therapists preferred to try a treatment and evaluate it on the basis of their experience. Seven therapists had an intuitive preference for a type of treatment, based on an internal gut feeling rather than an external source.
Supporting quotations for the first superordinate theme: philosophical issues relating to the nature of evidence
Use of unrepresentative samples
A number of specific issues in research that can stop findings being applicable to clinical practice were named. One common issue was the view that the samples in RCTs were not representative of the wider population. The issue of diagnoses was mentioned often, although views on the appropriateness of diagnoses varied. In some cases therapists felt that the use of diagnoses was inappropriate and could induce self-stigma. The main arguments against the use of diagnoses were that a mental health diagnosis could falsely indicate a permanent problem to some patients, that a diagnosis is too reductionist, and that a thorough assessment without a diagnosis may be more useful. The main reported benefit of diagnosis was that it could lead to a treatment plan. For example, many clinical guidelines, including the National Institute for Health and Care Excellence (NICE) guidelines are diagnosis based, so if they are to be used it is important for patients to be given a diagnosis.
Disagreement with evidence hierarchies
Many therapists argued that quantitative research does not take a number of aspects of treatments into account, unlike qualitative research. On the other hand, 18 therapists felt that neither qualitative nor quantitative research on their own were adequate to evaluate therapies, as both had biases.
Therapist specific factors and the therapeutic alliance
Therapist specific effects were often mentioned in the interviews. All therapists apart from one stated that the therapeutic alliance was the main determinant of success for treatment. A common assertion made was that different treatments do not have differential efficacy, but rather that different treatments work for different populations, or even for the same patient at different times. Two therapists used the phrase “horses for courses”. Few explanations were given as to whether it was possible to identify which patients would benefit most from different types of treatment.
Sources of information used by therapists in decision making and why therapists chose to use them
In all interviews, much discussion focused on the sources that therapists used in clinical decision-making and why they used them. The main themes from these discussions are described below and Table 2 shows supporting quotations.
Supporting quotations for the second superordinate theme:sources of information used by therapists in decision making and why therapists chose to use them
Preferences for clinical experience, and theoretical or practical information over empirical information
Clinical experience was found to be an important factor for clinical decision-making. The most common reasons for this were that experience was free from the caveats of research and that therapists understood the needs of their patients better than researchers. The shared clinical experience of their peers and supervisors was viewed as an important source of information to make clinical decisions. Discussion with peers also allowed therapists to share knowledge of research.
The majority of therapists (n = 19) stated that they did not focus on whether something was effective but rather on how a technique could be practised, and so preferred work that was more theoretical or practical rather than empirical findings. Theoretical or practical information was felt to be easier to read and understand. Therapists who stated that they did read empirical work said they would prefer reading case studies to the results of RCTs.
The presentation of information
Therapists felt that the presentation of information was very important and that this was a barrier to the dissemination of research. This was not just limited to text sources; workshops and conference presentations were found to be an important source of information and a therapy would be more likely to be implemented if it was well presented. The way a treatment was presented led to the assumption that treatments had been evaluated by research, based on therapists’ knowledge of the presenter.
Rigidity in tools and materials
The prescriptive nature of manuals was felt to be important in limiting their use. Manuals were felt to be less useful if they were too rigid and would therefore affect the therapeutic alliance or therapists’ creativity, but therapists felt manuals might be useful in training. For some therapists, having a rigid and directed outcome evidenced by outcome measures was seen as important, but this view was not shared by all.
Patient experience
The accommodation of patients’ or clients’ choice of treatment was regularly mentioned as an important factor in clinical decision-making. Sixteen therapists questioned whether many patients were informed enough to make a decision on the treatment they receive and stated that this was an area for more work to be undertaken. Participants felt that it would be difficult for patients to choose their treatment, as they would not be able to understand the experience of a certain type of therapy.
Discussion
Two main themes emerged. The first focused on philosophical issues about the nature of evidence, whereas the second focused on the sources used by therapists and why they used them. These will be discussed in turn.
Many therapists argued that RCTs should not be seen as the gold standard of research and that qualitative research was valuable. This argument has been claimed in the literature (Levant, Reference Levant2005; Wachtel, Reference Wachtel2009) and shows a fundamentally different understanding of what constitutes “evidence” compared to the hierarchies of evidence used by NICE and the Cochrane Collaboration (Mulrow, Oxman and Cochrane Collaboration, Reference Mulrow and Oxman1996). This debate is not limited to clinical psychology; Harré and Moghaddam (Reference Harré and Moghaddam2012) argued that psychologists should acknowledge the differences in types of languages or “grammars” used to describe different events. When trying to understand behaviour, psychologists should 1) focus on discourse, interaction and the emergence of meaning, as well as 2) looking for causal explanations between events. They argue that both methodologies serve a purpose, but are dependent on the task at hand and the emergence of meaning through discourse should take priority over the identification of causal relationships in psychology.
Therapists who argued against the use of RCTs frequently stated that the reason for this was because the samples used in RCTs were unrepresentative and were therefore less relevant to their own clinical practice. Weisz and colleagues found in their meta-analysis investigating the comparative effectiveness of evidence-based treatments for children and usual care that a variety of sample characteristics (e.g. demography, severity, comorbidity) resulted in non-significant differences in effect sizes (Weisz et al., Reference Weisz, Jensen-Doss and Hawley2006). It was striking that every participant except for one felt that therapist specific factors were more important than the technique used. Reading of the literature on the therapeutic alliance was reflected in the interviews by some therapists (Lambert and Barley, Reference Lambert and Barley2001; Norcross, Reference Norcross2010). Therapists seemed less aware of the literature that showed that the therapeutic alliance had a fairly modest impact on outcome (Horvath, Reference Horvath2005) and that the therapeutic alliance may interact with symptomatic improvement (Crits-Christoph et al., Reference Crits-Christoph, Gibbons, Hamilton, Ring-Kurtz and Gallop2011). Some therapists indicated that the importance of the therapeutic alliance meant that the treatment technique they used was relatively unimportant, which is not supported by research.
With regards to the sources used by therapists, clinical experience was highly valued by therapists, replicating the studies done in the US (Beutler et al., Reference Beutler, Williams, Wakefield and Entwistle1995; Stewart and Chambless, Reference Stewart and Chambless2007; Stewart et al., Reference Stewart, Stirman and Chambless2012). Participants stated that clinical experience allowed them to understand when the application of research evidence failed. Few therapists acknowledged failures of clinical experience, which has been well documented in experimental studies (Dawes et al., Reference Dawes, Faust and Meehl1989). This could be due to the commonly replicated self-serving attribution bias in which therapists are more likely to attribute their failures to external sources (Schuster, Forsterlung and Weiner, Reference Schuster, Forsterlung and Weiner1989). Some therapists argued that many clients or patients had come to them after other orientations had failed and that their clients’ or patients’ wishes would then influence their treatment plan. Clinical experience could also be shared, as supervision and peer discussion were found to be important influences on therapists’ practice, replicating previous work (Stewart et al., Reference Stewart, Stirman and Chambless2012). The majority of therapists stated that supervision and peer discussion was a method to share clinical experience, rather than discuss research findings, but not in all cases.
The presentation of information was felt to be particularly powerful in influencing whether or not therapists used sources to alter their practice. This has been demonstrated in experimental studies. For example, Schmidt and Taylor (Reference Schmidt and Taylor2002) found that one of the greatest influences on clinicians who were considering using an empirically supported therapy was the change agent's credibility and personal characteristics.
Patient choice was felt to be very important in guiding treatment and is consistent with research. Patient preference has been shown to have positive impacts on patient outcomes (Patel and Bakken, Reference Patel and Bakken2010). Many therapists felt that patients were not informed enough about the options available to make the best choice. Mental health literacy has been shown to be a predictor of treatment seeking, and poor literacy has been argued to hinder public acceptance of evidence-based mental health care (Jorm, Reference Jorm2000). As therapists stated that they would take patients’ choices into account, mental health literacy programmes could be used as a tool to increase the use of evidence-based treatments.
Many therapists stated that they would not use manuals as they were considered too rigid and this could have a detrimental effect on the therapeutic alliance. This view does not fit with the outcome literature. Manuals have been shown to lead to better outcomes and even improve the therapeutic relationship (Hogue et al., Reference Hogue, Henderson, Dauber, Barajas, Fried and Liddle2008; Loeb et al., Reference Loeb, Wilson, Labouvie, Pratt, Hayaki and Walsh2005). Although some manuals are “session by session” (Perlis, Jungquist, Smith and Posner, Reference Perlis, Jungquist, Smith and Posner2005), others are more principle-based and allow a great deal of flexibility in the delivery of the protocol (Fairburn, Reference Fairburn2008). Therapists often felt that manuals were too rigid to be useful to experienced therapists. This fits with the wider literature on the diffusion of innovation which finds that innovations that are more adaptable to a specific context are more likely to be used (Rogers, Reference Rogers1995). Even early manuals, such as Beck and colleagues’ manual for the treatment of depression, specifically warn against using them in a rigid, stereotyped way and recent work suggests that “flexibility within fidelity” will lead to the best outcomes (Beck, Rush, Shaw and Emery, Reference Beck, Rush, Shaw and Emery1979; Kendall, Gosch, Furr and Sood, Reference Kendall, Gosch, Furr and Sood2008). Some therapists may not wish to use manuals on principle. Nevertheless, if researchers, therapists and patients work together to understand the optimal level of flexibility for outcome and usability, patients may benefit.
This study is subject to a few limitations due to its use of semi-structured interviews. These may stop participants from discussing factors that are not included in the interview schedule. Although participants were asked at the end of the study if they wished to add anything to the discussion, a semi-structured interview does give the interviewer a lot of control over the discussion. Open-ended interviews on this subject continue to be warranted.
Second, although we have tried to convey participants’ comments to readers clearly, we may have misinterpreted them. In the same way that Westen and Weinberger stated that “truth does not reveal itself without interpretation” in the context of clinical research, the same is true of this research (Westen and Weinberger, Reference Westen and Weinberger2005, p. 1269). Similarly, it cannot be assumed that this study's findings can be generalized to all therapists. This study has been undertaken to understand the issues experienced by a relatively small number of therapists. Surveys have indicated that CBT therapists are more likely to use research, suggesting that if the sample was limited to CBT therapists the results would differ (Gyani et al., in press; Stewart and Chambless, Reference Stewart and Chambless2007). Conversely, CBT therapists have also been shown to drift from using research, indicating that the results might not differ that much (Stobie et al., Reference Stobie, Taylor, Quigley, Ewing and Salkovskis2007; Waller, Reference Waller2009). Further discourse between researchers and therapists of differing theoretical orientations is required.
In conclusion, this study indicated that in the UK therapists continue to have reservations about the value of empirical data to inform their practice, particularly research from RCTs. Many of these reservations appear to be based on misinformation (e.g. about the “mild” nature of patients in trials) and misunderstanding (e.g. about the rigidity of manuals). Based on these findings, for research to have impact clinicians need to be clear about the relative values of the full range of methodologies. These should be taught at undergraduate level as well as within clinical training, continuing professional development and supervision. Given that this study found that the presentation of information is important, careful consideration should be paid to the way in which the merits of using research and research findings are communicated. In particular, joint dissemination of clinically relevant research by clinicians, researchers, patients and caregivers using both qualitative and quantitative methodologies is likely to be of benefit in helping change therapist views about the value of evidence-based interventions. These discussions will hopefully reduce the gap between science and practice and ultimately benefit those suffering psychological distress.
Acknowledgements
We would like to thank our participants and Susie Formby for help in transcribing half the interviews. Ethical approval was gained through the National Research Ethics Service (ref: 11/SC/0240), and the University of Reading Research Ethics Committee (ref: 11/43) and the School of Psychology Ethics Committee (ref: 2011/16/RS).
Comments
No Comments have been published for this article.