Introduction
Interest in contextual therapies, such as Acceptance and Commitment Therapy (ACT), has been steadily growing over the past 15 years. Developed by Hayes et al. (Reference Hayes, Strosahl and Wilson1999), ACT is a mindfulness-based behaviour therapy that aims to increase psychological flexibility: ‘the ability to contact the present moment more fully as a conscious human being, and to change or persist in behaviour when doing so serves valued ends’ (Hayes et al. Reference Hayes, Luoma, Bond, Masuda and Lillis2006, p. 7). The therapy has been used successfully with a wide range of clinical problems, including chronic pain, anxiety and depression.
The growing recognition and flexible application of ACT treatment principles render it a useful tool for psychological therapists. Additionally, there is a formal recognition of the effectiveness and demand for contextual therapies in the UK: NICE (2009) have included mindfulness-based cognitive therapy in their guidelines for the treatment of recurrent depression. Thus, training to boost awareness, interest and knowledge of ACT is a timely and important step in furthering expertise in the mental health field.
The clinician training literature shows that experiential teaching increases learning outcomes relative to didactic methods alone. Bennett-Levy et al. (Reference Bennett-Levy, Turner, Beaty, Smith, Paterson and Farmer2001) demonstrated that with self-practise and self-reflection, cognitive therapy trainees were increasingly able to put themselves ‘in the client's shoes’ regarding understanding of techniques, develop a deeper understanding of techniques and develop a deeper understanding of the self and cognitive therapy as a tool for personal change. Bennett-Levy et al. (Reference Bennett-Levy, Turner, Beaty, Smith, Paterson and Farmer2001) hypothesized that experiential training triggers the latter, enabling emotional processing and deeper encoding.
Given the experiential nature of ACT as a treatment modality, experiential training methods are an apt way of modelling therapeutic techniques. Additionally, given the processes of psychological flexibility and experiential avoidance inherent in clinicians and patients alike, experiential training may provide a means to address experiential avoidance processes that may hinder learning.
Literature has shown that when clinicians themselves received ACT to overcome psychological barriers to learning new treatment methods, they demonstrated increased psychological flexibility and were more likely to adopt the new treatment methods (Varra et al. Reference Varra, Hayes, Roget and Fisher2008).
This study sought to evaluate ACT training workshops, with respect to both the impact of the training and the suitability of the process of the training. Of particular interest was whether the workshops instigated any behavioural change in clinical practice. Evaluations of actual behavioural changes to practice are particularly difficult and time consuming. As such, self-report on these outcomes was relied on.
Method
Design
Three 1-day workshops were delivered to three groups of psychologists working in a South London mental health Trust. The workshops were facilitated by three clinical psychologists working in the Trust who were experienced in using ACT. It addressed the rationale and development of the ACT model, key processes and therapeutic techniques. Psychological processes pertinent to the ACT model and therapeutic techniques were illustrated with metaphor and experiential exercises. Prior to experiential exercises, the notion of taking risks in an accepting and respectful environment was discussed to make the experiential exercises less threatening.
The factors assessed were: ACT knowledge, interest in using ACT as a therapeutic approach, experiential willingness and its effect on knowledge and interest, evaluation of structural workshop aspects and impact of the workshop on service delivery. Pre- and post-questionnaires were administered immediately prior to and after the workshop. Additional follow-up questionnaires were administered 12 months later.
Participants
A range of participants attended the workshops, including clinical and counselling psychologists, trainee psychologists and assistant psychologists, spanning a range of specialities (adult, child, older adult, learning disability, forensic services). In total, 73 participants completed the pre-and post-questionnaires and therefore these responses comprise the analysis. Of those who completed pre- and post-questionnaires, 24 (33%) additionally completed 12-month follow-up questionnaires.
Materials
ACT knowledge was assessed using the ACT Knowledge Questionnaire (AKQ; J. B. Luoma, personal communication), which assessed theoretical knowledge of ACT processes with 16 multiple choice questions. The AKQ is currently subject to reliability and validity research.
Interest in using ACT as a therapeutic approach was assessed with the following three questions: ‘How likely are you to read more about ACT?’; ‘How likely are you to seek further training in ACT?’; ‘How likely are you to use ACT techniques in your practice?’ For each question, responses were made on a visual analogue scale, where 0 = very unlikely and 10 = extremely likely.
Experiential willingness and the effect on knowledge and interest
Experiential willingness was assessed with one pre-workshop question and several post-workshop questions. The pre-workshop question read: ‘It can sometimes be hard to share difficult experiences in the context of a group. How difficult do you expect this to be for you in this workshop?’ The post-workshop questions read: ‘It can sometimes be hard to share difficult experiences in the context of a group. How difficult was this for you in the group today?’; ‘To what extent did you feel coerced or pressured to share difficult experiences with the group today?’; ‘To what extent did you disclose difficult thoughts and feelings today to one or more members of the group?’; ‘In relation to your private thoughts and feelings, how open were you?’; ‘If you did share difficult experiences with the group today, how did it feel?’ Responses were made on a visual analogue scale.
Clinician's own levels of psychological flexibility were measured using the Acceptance and Action Questionnaire-II (AAQ-II, F. W. Bond et al. unpublished data). The aim was to gauge the effect of psychological flexibility on willingness to engage in experiential exercises and workshop effectiveness.
Impact on service delivery
Participants indicated the extent to which their practice had been directly or indirectly affected by the workshop. At 12 months, further follow-up questionnaires were sent to the participants.
Results
Knowledge: AKQ scores
AKQ scores (sum of correct responses) were submitted to a paired t test with time as the factor (with the levels pre- and post-workshop). This yielded a main effect of time (t 73 = −7.5, p < 0.001, r = 0.48), with more correct answers post-workshop (mean = 8.5) compared to pre-workshop (mean = 5.8).
Interest
Level of interest post-workshop was indexed using a visual analogue scale and mean responses were calculated. Results indicated that participants generally considered themselves likely to read about ACT, likely to seek further training in ACT and likely to use techniques in clinical practice.
Experiential willingness and the effect on knowledge and interest
Experiential factors were indexed using a visual analogue scale and mean responses were calculated. ‘Anticipated difficulty sharing’ was rated pre-workshop and all other factors were rated post-workshop. Participants found it easier to share experiences during the workshop than had been anticipated before the workshop. This was confirmed with a paired t test which demonstrated a main effect of time (t 72 = 4.9, p < 0.001, r = 0.27), with rated difficulty sharing higher before the workshop (mean rating = 5.1) compared to afterwards (mean rating = 3.8).
Participants also indicated that in general they had not felt coerced or pressured to share (mean = 1.9), had felt able to disclose ‘some difficult thoughts and feelings’, had been ‘about as open as usual’ and indicated that sharing feelings during the workshop had generally been positive (mean rating = 6).
The relationship between the experiential factors was explored with Pearson's correlation coefficients. A significant correlation was found between anticipated and experienced difficulty sharing (r = 0.27, p < 0.05), with participants anticipating difficulty sharing also experiencing difficulty. The relationship between experienced difficulty sharing and coercion or pressure to share also reached significance (r = 0.50, p < 0.01), indicating that participants who found it difficult to share also felt more coerced or pressurized to share. Finally, a significant correlation was found between experienced difficulty sharing and feelings towards sharing (r = 0.33, p < 0.05), with participants who found it difficult to share, reporting more negative feelings towards sharing. None of the experiential factors were found to correlate with changes on the AKQ.
The AAQ-II data were entered into a correlation matrix with the experiential measures (anticipated difficulty sharing, experienced difficulty sharing, coercion, disclosure, openness and feelings towards sharing), measures of interest generated by the workshop (likelihood of further reading, further training, use of ACT techniques in practice) and changes in AKQ scores. The AAQ was found to correlate only with the AKQ change score (r = 0.24, p < 0.05), suggesting that individuals with higher psychological flexibility tended to increase their AKQ score more.
Impact on service delivery – 12-month follow-up
A total of 29 participants responded to the 12-month follow-up questionnaire. Of those responding, 24 (83%) participants stated that the workshop had influenced their clinical work. Twenty (80%) respondents indicated that the workshop had influenced their work with more than three clients. Fifteen (58.6%) participants stated that the workshop had influenced others’ clients through their supervision. Of those, 50% indicated that three or more clients had been affected in this way.
Twenty-six (90%) respondents stated that they planned to use ACT in the future. Most indicated that they intended to integrate ACT with another treatment approach (n = 24) or attend further training (n = 21). Of note, 27 respondents (93%) said that they would recommend this workshop to a colleague.
Discussion
This research was conducted to investigate the effectiveness and usefulness of 1-day ACT workshops aimed at training clinicians in this treatment modality.
An analysis of various ACT knowledge measures demonstrated significant differences between knowledge pre- and post-workshop. Participants’ theoretical understanding of core ACT processes improved, as indexed by the ACT Knowledge Questionnaire. Furthermore, an assessment of interest in ACT indicated that the workshop had stimulated interest for the majority of participants. Follow-up at 12 months suggested that interest in ACT was sustained for the majority of participants. Furthermore, a large proportion of participants reported that the workshop had lead to a behavioural change in their practice, through both direct clinical work and through supervision of others. Nearly all of the participants said that they intended to use ACT in their practice in the future.
The clinician training literature shows that experiential teaching increases learning outcomes relative to didactic methods alone (Bennett-Levy et al. Reference Bennett-Levy, Turner, Beaty, Smith, Paterson and Farmer2001). The use of experiential methods also accord with the very nature of ACT itself. Therefore, the workshop drew heavily on experiential teaching methods. However, as Bennett-Levy et al. (Reference Bennett-Levy, Turner, Beaty, Smith, Paterson and Farmer2001) suggests, these methods can be perceived as threatening by participants. Assessment of experiential outcomes indicated that participants tended not to feel coerced or pressured to engage with the group, they felt able to disclose difficult thoughts and feelings, they felt they were open and reported positive feelings towards sharing difficult experiences with the group. Moreover, anticipated difficulty sharing ideas was significantly higher than the actual difficulty experienced during the workshop. Further, where there were feelings of coercion and negativity towards sharing difficult experiences with the group, these were significantly correlated with experienced difficulty sharing. In turn, experienced difficulty sharing was significantly correlated with anticipated difficultly sharing. This may suggest that where there are difficulties engaging with the group, this was less to do with the workshop itself and more to do with the clinician's own anticipatory anxieties or difficulties. This is speculative, however, and it is worth emphasizing that for the most part, few difficulties were reported.
Participants’ own level of psychological flexibility did not significantly correlate with willingness to engage with experiential exercises. It did, however, correlate with changes in knowledge of ACT. This suggests that individuals who are more psychologically flexible are perhaps more open to learning about ACT. Because of the cross-sectional nature of the study, causality cannot be inferred here; however, it does raise the interesting possibility about factors that may broadly facilitate learning, and specifically about new treatment modalities. There is a significant body of data now describing effective treatment protocols to increase psychological flexibility (see Hayes et al. Reference Hayes, Luoma, Bond, Masuda and Lillis2006 for a review) and there may be scope to include these procedures to explicitly target and boost learning outcomes.
The implications of this evaluation for health service practice operate at several levels. First, evaluation of the workshop offers the facilitators direct feedback regarding its effectiveness and potential changes that can be made to optimize effectiveness. This enables the development of increasingly effective workshops. Second, by conducting an evaluation, the usefulness of the workshops can be quantified. Indications that clinician knowledge and interest increased following the training, suggested that the workshop does contribute to continuing professional development in the workforce.
That knowledge increased after the workshop is a particularly important finding. Many workshops tend to evaluate the level of interest generated or satisfaction with the workshop format. However, these measures do not necessarily tap actual behavioural change and measures of interest can be difficult to validate. Participants may find a workshop interesting or well constructed but this may not translate into knowledge change; participants may demonstrate demand effects in questionnaire completion and may not have had their interest stimulated. Conversely, measures of knowledge change suggest that the workshop has effected behavioural change. That direct and indirect clinical work may have also been influenced indicated an impact on health service provision. The latter, in particular, provides a compelling justification for future ACT workshops. Third, running an effective and useful workshop raises the profile of ACT as a treatment modality. Whether or not ACT is viewed as an extension of CBT approaches (Hofmann & Asmundson, Reference Hofman and Asmundson2008) or an entirely new third-wave treatment (Hayes et al. Reference Hayes, Luoma, Bond, Masuda and Lillis2006) it stimulated interest in clinicians from a predominantly CBT background. Clinicians from this alternative theoretical persuasion were able to integrate ACT techniques or functional analytical thinking into their practice or supervision after brief training alone. This indicates that ACT can be applied flexibly, not simply as the sole treatment modality, but as an adjunct to other treatment modalities.
Limitations and improvements
The study relied on self-report data to measure improvements in knowledge and interest related to ACT. A more rigorous test of the effectiveness of the workshops would have used behavioural measures of therapist competence in the application of new clinical skills. As suggested by Rakovshik & McManus (Reference Rakovshik and McManus2010), this evaluation would be best conducted using randomization, blind assessment and measures of known reliability and validity.
Ideally the follow-up questionnaires could have measured knowledge and interest to establish whether or not any gains had been maintained. This was the initial intention of the evaluation but longer versions of the follow-up questionnaires yielded very poor response rates, hence use of the shortened version. However, although the response rate achieved with the shortened questionnaire was adequate (33%), it is possible that a response bias operated so that only those clinicians with positive views about ACT responded. As such, some caution should be exercised in drawing conclusions from the follow-up data.
Future evaluations of such workshops may be improved by restructuring the questions that tapped experiential aspects of the training: given the lack of significant associations with the measure of experiential avoidance it is possible that the current measures were not sufficiently sensitive to be used in a correlational analysis with the AAQ-II.
Despite the methodological points, the data from this evaluation suggest that these ACT workshops are effective in disseminating knowledge and stimulating interest in this treatment modality. Moreover, the results suggest the professional development of clinicians is being translated into clinical practice. Finally, this evaluation indicates that the essential experiential methods of teaching ACT are broadly acceptable to participants, suggesting that such future workshops could usefully incorporate such methods to enhance learning outcomes.
Conclusions
This study sought to evaluate ACT training workshops to NHS staff, examining changes in participant interest and knowledge. In addition, both the modality of training and participant variables were explored. The study found that the training increased participant self-reported interest in ACT and this was found to be sustained at 12 months follow-up. The majority of participants reported that their practice had been influenced by the workshops at follow-up, either through direct clinical work or supervision. Nearly all participants said that they planned to use ACT in some form in the future.
Participant knowledge in ACT was also found to significantly increase after the workshop. Participants did not find the experiential exercises aversive and there was no evidence that participants’ own levels of experiential avoidance affected training outcomes. However, there was evidence to suggest that participants’ level of psychological flexibility facilitated learning.
Overall, the study provides good evidence that this type of workshop, in terms of both content and format, is useful and beneficial to clinicians working within the NHS and should be offered more widely.
Acknowledgements
We are grateful for the advice and feedback provided by Andrew J. Hart in the preparation of this article.
Learning objectives
(1) To understand the impact of different approaches to teaching behavioural and cognitive therapies, such as the use of experiential exercises, on participant-reported outcomes.
(2) To consider the potential for training workshops involving new behavioural and cognitive therapy modalities, such as Acceptance and Commitment Therapy, to increase knowledge and generate sustained interest.
(3) To consider the impact of attendee variables on engagement with the workshop materials.
Comments
No Comments have been published for this article.