Introduction
Despite the high prevalence of mental illness and associated impairment, many people with a diagnosable psychological disorder receive inadequate treatment or no treatment at all (Judd, Reference Judd2005; Kessler, Chiu, Demler, Merikangas and Walters, Reference Kessler, Chiu, Demler, Merikangas and Walters2005; McManus, Meltzer, Brugha, Bebbington and Jenkins, Reference McManus, Meltzer, Brugha, Bebbington and Jenkins2009). Major depressive disorder is one of the most prevalent among all mental disorders, yet evidence suggests that the minority of people with depression receive adequate treatment (Wang et al., Reference Wang, Lane, Olfson, Pincus, Wells and Kessler2005). This disparity between treatment need and treatment received reflects the well-publicized gap between clinical science and practice (Baker, McFall and Shoham, Reference Baker, McFall and Shoham2008). National initiatives in both the UK and in the US, such as the Increasing Access to Psychological Therapies program (IAPT; Clark et al., Reference Clark, Layard, Smithies, Richards, Suckling and Wright2009) and the Department of Veterans Affairs health care system's (VA) national dissemination of cognitive behaviour therapy (CBT) for depression (Karlin et al., Reference Karlin, Brown, Trockel, Cunning, Zeiss and Taylor2012), reflect the importance of increasing access to evidence-based practices (EBPs).
These large-scale initiatives address the lack of availability of EBPs that is due, in part, to insufficient training of clinicians. Weissman and colleagues (Reference Weissman, Verdeli, Gameroff, Bledsoe, Betts and Mufson2006) estimated that nearly two-thirds of accredited psychology and social work programs in the US do not require formal training or supervision in any EBPs. Beyond initial training in medical or graduate school, there is a reliance on continuing education opportunities (e.g. professional workshops and conferences) and treatment manuals to learn EBPs. Few studies, however, have rigorously tested the efficacy of disseminating EBPs through these methods (see Miller, Yahne, Moyers, Martinez and Pirritano, Reference Miller, Yahne, Moyers, Martinez and Pirritano2004 for a notable exception), and training clinicians with these methods does not typically engender improved client outcomes (Beidas and Kendall, Reference Beidas and Kendall2010). Although more intensive training programs coupled with ongoing supervision, such as those used in the IAPT and VA initiatives (McManus et al., Reference McManus, Meltzer, Brugha, Bebbington and Jenkins2009; Karlin et al., Reference Karlin, Brown, Trockel, Cunning, Zeiss and Taylor2012), have been more successful, increased use of electronic technology within healthcare service settings suggests online training (OLT) as a potential training method (Bennett-Levy and Perry, Reference Bennett-Levy and Perry2009; Cook, Reference Cook2005; Sholomskas and Carroll, Reference Sholomskas and Carroll2006; Weingardt, Reference Weingardt2004; Cucciare, Weingardt and Villafranca, Reference Cucciare, Weingardt and Villafranca2008; Dimeff, Paves, Skutch and Woodcock, Reference Dimeff, Paves, Skutch, Woodcock and Barlow2011).
Compared to traditional training methods, OLT provides a unique learning platform that may be especially well suited for learning EBPs. Such potential advantages include user-friendly designs, easy navigation, media-rich content such as video vignettes and interactive quizzes, self-paced learning, and opportunities for role play in low risk virtual environments (Dimeff et al., Reference Dimeff, Koerner, Woodcock, Beadnell, Brown and Skutch2009; Quinn and Conner, Reference Quinn and Conner2005; Weingardt, Reference Weingardt2004). These attributes are consistent with current learning theories of knowledge acquisition and have the potential to improve learner engagement and satisfaction (Mayer, Reference Mayer2008). Recent studies suggest that interactive, media-rich instructional design can significantly improve knowledge acquisition compared to traditional text or in-person workshop training (Dimeff et al., Reference Dimeff, Koerner, Woodcock, Beadnell, Brown and Skutch2009; Sholomskas and Carroll, Reference Sholomskas and Carroll2006). Similarly, a preliminary evaluation of an online CBT package showed that trainees were satisfied with their learning experience, felt more confident in CBT topics, and increased their CBT knowledge (Westbrook, McManus, Clark and Bennett-Levy, Reference Westbrook, McManus, Clark and Bennett-Levy2012). These findings highlight the potential of OLT for disseminating EBPs to community mental health clinicians.
Among EBPs for depression, interest has increased recently in Behavioural Activation (BA) as a good candidate for broad scale dissemination. BA is a structured, principle-driven psychotherapy that uses a behavioural model of depression and behavioural assessment to guide interventions aimed at increasing pleasure and mastery activities, decreasing maladaptive activities, and solving problems (Dimidjian, Barrera, Martell, Muñoz and Lewinsohn, Reference Dimidjian, Barrera, Muñoz and Lewinsohn2011). BA has demonstrated strong outcomes in clinical trials for both acute depression treatment and prevention of depression relapse and recurrence (Dimidjian et al., Reference Dimidjian, Hollon, Dobson, Schmaling, Kohlenberg and Addis2006; Dobson et al., Reference Dobson, Hollon, Dimidjian, Schmaling, Kohlenberg and Gallop2008; Gortner, Gollan, Dobson and Jacobson, Reference Gortner, Gollan, Dobson and Jacobson1998; Jacobson et al., Reference Jacobson, Dobson, Truax, Addis, Koerner and Gollan1996). These favourable outcomes have also been supported by recent meta-analytic reviews (Cuijpers, van Straten and Warmerdam, Reference Cuijpers, van Straten and Warmerdam2007; Ekers, Richards and Gilbody, Reference Ekers, Richards and Gilbody2008; Mazzucchelli, Kane and Rees, Reference Mazzucchelli, Kane and Rees2009). Moreover, recent work suggests that mental health nurses without prior psychotherapy experience can be effectively trained to use BA to treat adult primary care patients with depression (Ekers, Richards, McMillan, Bland and Gilbody, 2011) and that BA is a cost-effective method for training non-specialists to effectively treat depressed primary care patients (Ekers et al., 2011; Ekers, Dawson and Bailey, Reference Ekers, Richards, McMillan, Bland and Gilbody2012).
Based on basic behavioural principles and a repertoire of concise and straightforward treatment strategies, the simplicity of BA may place relatively less burden on mental health clinicians wishing to learn an EBP for depression as compared to other more complex EBPs for depression. This paper reports the results of two studies representing preliminary steps to examine the dissemination of BA via OLT.
Method
Overview
These preliminary dissemination steps are derived from Rounsaville, Carrol and Onken's stage model of treatment development (2001) in which needs assessment surveys, instrument development, and feasibility demonstration (Stage Ia elements) precede formal pilot testing (Stage Ib); efficacy testing is part of Stage II and typically involves resource-intensive and rigorously conducted randomized controlled trials (RCTs).
Thus, we examine the feasibility of disseminating BA via OLT through conducting: (1) a survey study with practising clinicians to assess interest in learning BA with an OLT format; and (2) a small pilot RCT for preliminary efficacy of teaching BA principles and treatment strategies with a precursor version of BA OLT. The survey was exploratory and we provide descriptive statistics. For the pilot RCT, we hypothesized that compared to clinicians assigned to the control condition, clinicians in the BA OLT condition would evidence greater knowledge of and ability to recall BA terms and concepts, greater self-efficacy using BA with depressed clients, and high satisfaction with the training materials. All procedures were reviewed and approved by the Western Institutional Review Board and all participants provided informed consent.
Study 1: BA and OLT interest survey
Procedures
We conducted a survey study in 2006 with a convenience sample of practising clinicians (N = 540) who were participants at Behavioral Tech, LLC sponsored trainings at 20 different locations in 14 states across the United States. We elected to use a sample of easily recruited participants to establish interest in BA OLT training before extensive testing with more representative samples of clinicians. Behavioral Tech, LLC specializes in training clinicians in EBPs, with an emphasis on training in Dialectical Behaviour Therapy.
Participants completed a questionnaire assessing demographic characteristics, theoretical orientation, and clinical practice patterns. After reading a one-page, objective summary of BA, they answered questions about interest in BA, perceived credibility, and personal reactions to the description of BA. The one-page summary of BA described the behavioural model of depression, commonly used interventions, and the general course of BA from session to session.
Perceived credibility and personal reactions to the description of BA were assessed with the Credibility Scale (CS; Addis and Carpenter, Reference Addis and Carpenter1999) and the Personal Reactions to the Rationale scale (PRR; Addis and Carpenter, Reference Addis and Carpenter1999). The Credibility Scale included seven questions that participants rated on a scale from 1 (not at all) to 7 (extremely), with higher scores indicating a higher rating of credibility of BA. Internal consistency was sufficiently high (Cronbach's alpha = .92) to justify collapsing items into a single mean score used in analyses.
The PRR scale included five questions that participants rated on a 7-point scale from 1 (not at all) to 7 (extremely). Internal consistency was sufficiently high (Cronbach's alpha = .89) to justify collapsing items into a single mean score used in analyses. Higher scores indicated a more positive reaction to the rationale for BA.
Participants
The modal participant was a Caucasian female, approximately 43 years of age, and held a Master's degree working in an outpatient setting from a cognitive/cognitive-behavioural theoretical perspective. See Table 1 for a complete description of participant characteristics.
Study 1 - sample demographics
Results
Based on a one-page description of the BA approach to depression (available upon request from first author), clinicians in this study reported perceiving BA with a high degree of credibility (M = 5.18; SD = 0.99) and having highly positive personal reactions to BA (M = 5.02; SD = 0.97). The majority of these participants (64.8%; n = 335) reported being “very” to “extremely” interested in receiving training in BA for depression, and the majority (78.8%; n = 399) expected BA to be relatively simple to master. Over three-quarters of participants in this sample (77.6%; n = 393) reported that they perceived BA to be “very” or “extremely” relevant to their clinical work with depressed clients and likely to be “very” or “extremely” effective with the depressed clients they treat (70.1%; n = 354). Nearly all reported that BA was compatible with the way they approached the treatment of depression (95.5%; n = 485) and would be helpful in their clinical treatment of depression (95.1%; n = 482).
The majority of these participants (68.02%, n = 319) reported being “very” to “extremely” interested in online training for the treatment of depression and endorsed unique advantages of OLT, including the ability to take the training course at home without travelling (61.3%; n = 314), to schedule the course at a convenient time (72.0%; n = 368), and to complete the training at their own pace (68.7%; n = 352).
Conclusion
These results suggest that the mental health clinicians in this convenience sample have high interest in BA training. These clinicians reported that BA is compatible with their clients’ needs and with their approach to treating depression. An important consideration to dissemination efforts, BA was perceived as relatively easy to learn and OLT was perceived as a convenient method for learning BA. These clinicians found BA to be credible and reacted favourably to the rationale for using BA. Although we recruited a large sample, this was a convenience sample in which the majority of clinicians endorsed a cognitive-behavioural orientation. As Behavioral Tech, LLC workshops are geared toward this subset of clinicians, it will be important for future research to examine interest in BA and OLT among clinicians representing a wide range of theoretical orientations and practice settings. The degree to which these results will generalize to clinicians who identify with a diversity of theoretical orientations remains unknown and warrants future study.
Study 2: pilot RCT of BA OLT
Procedures
In Study 2, we developed a preliminary version of BA OLT and conducted a small RCT study comparing BA OLT to a placebo attention control OLT that was comparable in scope and purpose. The BA OLT took approximately 90–120 minutes to complete and consisted of 81 screens organized into six modules: (1) Introduction to the course; (2) Introduction to BA; (3) BA in a nutshell; (4) Presenting BA treatment rationale; (5) Identifying the targets of intervention; and (6) Developing an action plan: activity and mood monitoring.
The control OLT was entitled “DBT Validation Strategies” and was developed by the same production team that developed BA OLT. This control OLT was intended to teach treatment providers how to validate a client in therapy and was comparable in quality, length, and design elements. Although the control OLT contained no BA-specific content, we were primarily interested in establishing feasibility of using OLT to teach BA to clinicians and accounting for threats to internal validity. Thus, we forewent a formal comparison between different learning methods (e.g. reading a BA text book or attending a BA workshop) and focused solely on determining the extent to which BA OLT helped clinicians learn basic treatment principles and strategies. Participant screening and informed consent procedures were conducted via telephone. Participants completed all assessments and their respective training course online.
Participants
Research assistants contacted mental health agencies, described the study, and invited interested clinicians to contact our research office. Interested clinicians were screened to ensure they did not meet study exclusion criteria, including: (1) exposure to any portion of the previously published treatment manuals for BA (e.g. Martell, Addis and Jacobson, 2001) or (2) attendance at a lecture, workshop, or other intensive training on BA. These exclusionary criteria were used to ensure that the study sample was similar to the target population for the OLT (i.e. clinicians who treat depressed clients with minimal exposure to BA).
Measures
The Demographics Questionnaire is a 23-item questionnaire that assesses participants’ theoretical orientation, basic demographic information, professional degree, experience in treating depressed adults, and the extent of a subject's formal and informal exposure to BA.
The Satisfaction Survey (SS) is a 13-item face valid instrument that examines two primary dimensions of participant satisfaction with the training course: course relevance and usability. Course relevance was measured with eight items on a 5-point Likert scale (Cronbach's alpha ranged from .89 to .94) and usability was measured with five items on a 7-point Likert scale (Cronbach's alpha ranged from .81 to .93). Higher scores indicated more favourable ratings. For all items, we compared the mean score to an a priori cut-off for adequate relevance (3.5 on a 5-point Likert scale) and usability (5.0 on a 7-point Likert scale).
The Self-Efficacy Measure (SEM) is a 28-item self-report measure that assesses self-efficacy and confidence in treating adults with Major Depressive Disorder (MDD). Internal consistency was sufficiently high to justify grouping items into one summary score used for analyses (Cronbach's alpha ranged from .97 to .98, depending on time point).
The BA Knowledge Test (BAKT) is a 13-item multiple-choice test with items assessing knowledge of BA concepts taught in the BA OLT. The 13 test questions were selected from a pool of 34 original questions derived from published BA treatment manuals, and were retained if they were generally answered correctly by highly knowledgeable BA experts, and incorrectly answered by content-naïve individuals.
The BA Recall Test (BART) is a spontaneous recall memory test in which participants were asked to name as many BA concepts and techniques as they could recall from the OLT. The score was the correct number of items identified and participants were not penalized for incorrect responses. Independent coders, blind to participant training condition, used explicit instructions to award one point per acceptable response. Scores ranged from zero to a maximum of 97. Each coder, blind to participant condition and time point, independently rated each participant's recall measure at all assessment time points. Single measure intraclass correlation coefficients ranged from 0.95 to 0.99 indicating high inter-rater reliability.
Analysis overview
Longitudinal outcome measures were examined via Hierarchical Linear Modeling (HLM) using PROC MIXED models in SAS 9.2. HLM allows for the additional clustering of repeated measures within individuals, as well as the specification of the best fitting covariance structure (Raudenbush and Bryk, Reference Raudenbush and Bryk2002). Additionally, HLM allowed the inclusion of all 46 intent-to-train participants (even those without complete data on all measures at the posttraining assessment). Predictors in the analyses were experimental Condition, Time, and the Time by Condition interaction. The best-fitting covariance structures were selected and then specified based on the Akaike's Information Criterion (AIC; Hox, Reference Hox2002). Type III F-tests (omnibus tests to determine if differences existed between conditions) were calculated, and when significant, post hoc pair-wise comparisons of predicted means from regression estimates were used to identify which conditions differed. Outcome measures administered at single time points were analyzed using descriptive statistics or cross-sectional HLM analyses, depending on the measure.
Results
We recruited 49 eligible participants, aged 18 years and older, who were currently employed at community mental health agencies or were students working to obtain a professional degree in a mental health-related field. Using a minimization random assignment procedure that matched participants on educational degree (1 = MD/PhD or doctoral candidate; 2 = MA/MS/RN or current graduate student, not in a doctoral program; 3 = BA/BS level), participants were randomly assigned to BA OLT (n = 33) or the control OLT (n = 16). Of the original 49 who provided their consent, 46 completed the baseline assessment (one participant was unable to complete the trial due to time constraints and two others were unresponsive to our attempts to contact them). Of the 46 participants who completed the baseline assessment and constituted the “intent-to-train” sample, 43 completed the posttraining assessment. Of the 33 participants assigned to the BA OLT condition,Footnote 1 29 completed the one-week follow-up assessment. Participant characteristics are summarized in Table 2 and participant flow is depicted in Figure 1. No differences were found between the BA OLT and control OLT conditions at the baseline assessment for any of the outcome measures, including demographic variables (see Table 2).
Study 2 sample demographics
Study 2 participant flow
Satisfaction
We examined the course relevance and usability of the BA OLT. We considered relevance and usability as acceptable if the mean scores for these variables equalled or exceeded the a priori levels of 3.5 for relevance (relevance items were rated using a 5-point Likert scale), and 5.0 for usability (usability items were rated using a 7-point Likert scale). Mean scores were calculated for the course relevance items (BA OLT: M = 3.88, SD = 0.70) and course usability (BA OLT: M = 5.54, SD = 0.96). BA OLT participants significantly exceeded the a priori criterion for both relevance and usability, t(78) = 2.05, p < .05; t(78) = 2.24, p < .05, respectively.
Knowledge
As shown in Table 3, all participants correctly answered one-third or less of the knowledge test items at baseline, indicating a low level of BA specific knowledge. A significant effect for Time, F(2, 78.5) = 35.31, p < .0001, indicated that the sample as a whole improved in BA knowledge across time, while a significant Time by Condition interaction, F(2, 78.5) = 4.79, p < .05, revealed differential change between conditions over time. BA OLT participants significantly outperformed control participants at the postassessment, t(75.4) = 3.80, p < .001, with a large between-condition effect size (Cohen's d = 1.16). BA OLT participants maintained their knowledge gains; outperforming control participants at follow-up, t(72.1) = 4.00, p < .001), again with a large between-condition effect size (Cohen's d = 1.31).
Study 2 results
a In a row, when means share the subscript this means that they are significantly different (in post hoc comparisons, *p < .05, **p<.01, ***p<.001)
Recall test
As shown in Table 3, all participants correctly generated, on average, less than one BA term or concept at the baseline assessment, further indicating a lack of familiarity with BA. A significant effect for Time, F(2, 36.5) = 37.22, p < .0001, indicated that the sample as a whole improved in their ability to spontaneously generate BA terms and concepts, while a significant Time by Condition interaction, F(2, 36.5) = 33.14, p < .0001, revealed differential change between conditions over time. BA OLT participants significantly outperformed the control participants at the post- and follow-up assessment, t(35.9) = 8.06, p < .0001; t(36.0) = 6.30, p < .0001, respectively, with very large between-condition effect sizes at each time point (Cohen's d = 2.99 and 2.34; respectively).
Self-efficacy
A significant effect for Time, F(2, 42.3) = 17.77, p < .0001, indicated that all participants generally improved in perceived self-efficacy over time (Table 3). The Time by Condition interaction, F(2, 42.3) = 13.70, p < .0001, was significant, with BA OLT participants significantly outperforming control OLT participants at the post- and follow-up assessments, t(34.4) = 5.30, p < .0001; t(42.1) = 6.48, p < .0001, respectively. Large between-condition effects were found between BA OLT and Control participants at both post- and follow-up assessments (Cohen's d = 2.41 and 2.07; respectively).
Conclusion
All planned hypotheses were supported. BA OLT participants rated the course favourably and found it relevant to their clinical work. Compared to control participants, BA OLT participants recalled significantly more BA terms and concepts, scored significantly higher on a BA knowledge test, and reported greater self-efficacy in treating adults with depression using BA. Additionally, there was no evidence of decrements among participants in the BA OLT condition in knowledge, self-efficacy, or motivation during the follow-up period; however, participants did decrease in their ability to correctly recall BA terms and concepts.
Discussion
Based on an extension of the stage model of treatment development to training (Rounsaville, Carrol and Onken, Reference Rounsaville, Carroll and Onken2001), we conducted a pilot RCT to assess preliminary efficacy of BA OLTs before investing the substantial resources needed to develop a comprehensive OLT for BA. The results of the two preliminary studies reported indicate that there exists strong interest in learning BA via OLT (Study 1) and that core BA elements can be taught in an online framework (Study 2).
Given the evidence that the majority of depressed clients fail to receive adequate treatment (Wang et al., Reference Wang, Lane, Olfson, Pincus, Wells and Kessler2005), these findings are important in that they highlight the potential of disseminating BA with OLT. Although there are multiple factors that contribute to this shortage of effective treatments for depression, it is widely recognized that the dissemination and reach of evidence-based practices in routine clinical settings has been minimal (Baker et al., Reference Baker, McFall and Shoham2008). A promise of BA is that it is a relatively less complex treatment compared to other standard psychotherapies for depression and that it can be easily learned and implemented in routine practice settings. The results of the first study suggest that this perception is evident among practising clinicians, who report strong interest and confidence in learning BA and using it with their depressed clients as well as the perception that BA would be relatively simple to master.
The results of the second study also address the striking lack of attention in the field to methods of training clinicians in evidence-based practices. Results from survey studies suggest that lack of quality training is one important barrier to the dissemination and reach of evidence-based practices (Weissman et al., Reference Weissman, Verdeli, Gameroff, Bledsoe, Betts and Mufson2006). Fairburn and Cooper (Reference Fairburn and Cooper2011) encourage OLT as an important direction for the field given benefits of standardized training quality, convenience, self-pacing, flexibility in pedagogical style, and cost. In this preliminary study, results suggest that participants found OLT acceptable and relevant and that participants acquired basic BA knowledge and perceptions of self-efficacy in treating depressed clients with BA. Future tests of BA OLT will require assessment of how skillfully trainees implement BA and the extent to which clients treated by clinicians trained with this method demonstrate enhanced treatment outcomes.
A number of important limitations should be noted. First, it is important to examine to what degree these findings generalize to larger samples of practising clinicians with diversity in theoretical orientation, previous clinical training and experience, and setting. Second, this preliminary OLT was an initial effort to translate core BA principles into an interactive online format; the degree to which these preliminary results are replicated with a fully developed OLT is unknown. Third, the follow-up assessment interval was only one week. Future studies of BA OLT would benefit from extended follow-up periods to determine the extent to which information is retained over time. Fourth, it will be important to further validate these training outcome measures and use assessment instruments with sound psychometric properties in subsequent studies. Fifth, we did not assess whether raters who coded the recall tests were blind to treatment condition. It is possible that they could have ascertained participants’ training assignment and biased their ratings in favour of those in the BA OLT condition. Finally, it is critical that future training studies assess behavioural outcomes such as clinicians’ ability to deliver quality BA and whether clients experience reductions in depressive symptoms and improve functional capacity. Like face-to-face training, OLT provides no guarantee that what is learned in the OLT will transfer to clinical proficiency in the context of treating depressed clients, and the degree to which training in BA facilitates therapist learning is only of interest to the extent that such training improves quality of depression care.
The gap between clinical science and practice has been decried repeatedly. There is a critical need for our field to turn its attention to investigating methods to train clinicians as one approach to close the gap between science and practice. Evidence based psychological treatments, like BA, which may be amenable to widespread application, should be investigated in the context of rigorous randomized controlled training studies. In addition, the use of online training methods appears to represent a promising direction for the dissemination of evidence based psychological treatments.
Acknowledgments
Data collection and preparation of this article was supported by a Small Business Innovation Research grant (1R43MH081388-01) from the National Institute of Mental Health awarded to Sona Dimidjian and Linda A. Dimeff.
Comments
No Comments have been published for this article.