Introduction
More people than ever are training to become cognitive behavioural therapy (CBT) therapists, and there is a growing number of training institutions. Many components contribute towards CBT skill development, and feedback is one of these. The topic of assessment feedback has a long history in academic research, and it is a real concern in education research and practice (Boud and Molloy, Reference Boud and Molloy2013; Evans, Reference Evans2013; Nicol, Reference Nicol2010). The National Student Survey (NSS) is undertaken annually for all undergraduate students at UK institutions and consistently highlights that students are dissatisfied with assessment and feedback compared with other learning experiences. The Postgraduate Taught Experience Survey (PTES) invites postgraduate taught students to comment on their teaching and learning experience; questions elicit information on whether feedback received allowed improvement on postgraduate students’ next piece of work, and whether there was enough detail (Leman et al., Reference Leman, Turner and Bennett2013). It is unclear whether the PTES is being used by CBT training institutions, and there is little evidence for the types of feedback that CBT students value most for their development. Furthermore, CBT programme teams invest considerable time and resources into providing many forms of feedback to facilitate students’ learning, without knowing which are most helpful.
Definitions of feedback
There are many ways to define feedback. Hattie and Timperley (Reference Hattie and Timperley2007) highlight students as recipients of feedback, defined as ‘information from teachers, supervisor or peers on a piece of work, with instructions on how to improve’. There has been much emphasis on what educators should do in order to provide ideal written or oral feedback (Nicol and Macfarlane-Dick, Reference Nicol and Macfarlane-Dick2006), assuming a transmission-focused approach. The downside to this approach is that students are viewed as passive recipients. More recently, authors have emphasized the importance of a student’s proactive recipience of feedback (Reeve and Tseng, Reference Reeve and Tseng2011; Winstone et al., Reference Winstone, Nash, Rowntree and Menezes2016) and highlight the two-way engagement that is necessary for maximum learning to occur. Whichever definition we use, it is clear that students, supervisors, lecturers and peers all play a role in the feedback process.
Students engage in both formative and summative feedback. Formative feedback is used in the improvement of skill, such as a supervisor commenting on the strengths and weaknesses of a video therapy excerpt, or a peer observing a role play. On the other hand, summative feedback is used in decision making about a final grade. For example, on formal CBT training programmes, decisions have to be made about whether a student has passed an assignment, or met a competency standard.
Theoretical models
Conceptual models have been developed in the educational literature to take account of research, and to help us understand how feedback should be undertaken. More recently (Boud and Molloy, Reference Boud and Molloy2013; Carless et al., Reference Carless, Salter, Yang and Lam2011), these have repositioned feedback as a co-productive process where learners and educators have roles to play. Feedback therefore involves information that is actually used (to facilitate future learning), rather than what is transmitted. Education is increasingly being judged not on what it delivers now, but on what it produces beyond training. Current models therefore identify the importance of feedback that develops students’ capacities to learn both now and in the future.
Feedback conditions that strengthen learning
Receiving feedback on one’s skill and understanding is a key part of the learning process (Hattie and Timperley, Reference Hattie and Timperley2007). Following a meta-analysis of studies involving college and undergraduate students, Gibbs and Simpson (Reference Gibbs and Simpson2004) proposed seven conditions that need to be met for feedback to influence learning. Firstly, sufficiently frequent feedback needs to be provided and needs to be specific, i.e. both often enough and in enough detail, to be useful. Trainers providing comments such as ‘good job’ do not address strengths or weakness of the work and therefore do not provide information about how to improve (Smith and Gorard, Reference Smith and Gorard2005). Secondly, feedback needs to be focused on student performance rather than their personal characteristics. Critical feedback on personal characteristics can be demotivating for students and can affect their self-efficacy (Schunk, Reference Schunk1985), which in turn could lead to less effort being put into tasks and poorer academic performance. Thirdly, feedback needs to be timely, i.e. received while it still matters and in time to pay attention to learning or receiving assistance. Students are less likely to return to a task if the feedback is late (Higgins et al., Reference Higgins, Hartley and Skelton2002). Fourthly, it is important that feedback is related to specific criteria. Providing and then checking that students understand guidelines and marking criteria for each assignment are likely to facilitate learning. Fifthly, feedback needs to indicate how as well as what a student needs to do. For example, Higgins et al. (Reference Higgins, Hartley and Skelton2001) found that feedback such as ‘more critical analysis … would have helped’ was not always beneficial if students didn’t know how to critically analyse. From a CBT training perspective, ‘try to be less didactic’ may be less useful than ‘take a more guided discovery approach by asking questions about how the anxiety feels in her body, her thoughts and fears in these moments, and how she copes’. The final two conditions, according to Gibbs and Simpson (Reference Gibbs and Simpson2004), are that feedback needs to be received and it needs to be acted upon by the student.
Rather than be seen as the end point in the learning process, feedback is increasingly being characterized as the start point (Burke, Reference Burke2009). By playing active roles in the feedback process, learners can develop self-regulation skills and engage in a dialogue. For example, CBT students can be encouraged to elicit specific feedback from lecturers, supervisors or peers, report on how this has been helpful, and how this has shaped subsequent clinical or academic practice. This process is dependent upon a student receiving, understanding, and implementing feedback into their practice.
Student preferences for feedback
What do we already know about student preferences around feedback? MacLellan (Reference MacLellan2001) asked 130 undergraduate students and 80 lecturers about aspects of assessments and feedback. Their results showed wide discrepancies between student and lecturers on which feedback characteristics were beneficial. The majority of lecturers assumed that their feedback was frequently helpful for student understanding and learning. However, students did not share this view. Students responded that feedback was only sometimes helpful, with 30% of students stating that it never helped them understand. Mulliner and Tucker (Reference Mulliner and Tucker2015) also found a discrepancy between staff and student views on some aspects of feedback practice relating to vocational training programmes. They explored perceptions on different types of feedback, its timeliness, students’ engagement with and interest in feedback, its quality, and satisfaction with current practice. One example of discrepancy is that 86% of staff in this study felt that individual (face-to-face) feedback was effective compared with 63% of students. The majority of students (96%) indicated that they always access marked assignments, whereas only 38% of staff shared this view. On the other hand, both students and staff in this study did agree that ‘peer evaluation or discussion of work in peer groups’ was not an effective form of feedback. This is further supported by Dawson et al. (Reference Dawson, Henderson, Mahoney, Phillips, Ryan, Boud and Molloy2019), who found that students and educators rarely mentioned peer feedback as being effective and beneficial.
The question that arises is: what feedback do students most value? Dawson et al. (Reference Dawson, Henderson, Mahoney, Phillips, Ryan, Boud and Molloy2019) conducted a large-scale qualitative investigation of what educators and students think the purpose of feedback is, and what makes it effective. Students saw feedback as a way to improve, and perceived it was most effective when it was usable, detailed enough, personalized (versus generic) and when the narrative had a reasonable emotional tone. In line with these values, earlier studies confirm that undergraduate students want feedback that promotes self-efficacy, is prompt, suggests ways of improving, and that is motivating (Gibbs and Simpson, Reference Gibbs and Simpson2004; Hattie and Timperley, Reference Hattie and Timperley2007; Higgins et al., Reference Higgins, Hartley and Skelton2002). Less is known about postgraduate training or professional practice programmes, although Ferguson (Reference Ferguson2011) found that teacher training students preferred feedback that commented on what they did well, provided guidance on how to improve, and clear referencing of the mark they got to the marking criteria.
The theme of necessities and luxuries in relation to feedback qualities has more recently been taken up by Winstone et al. (Reference Winstone, Nash, Rowntree and Menezes2016). They wanted to identify the relative importance of different types or characteristics of feedback. They used a microeconomic budgeting method devised by Li et al. (Reference Li, Bailey, Kenrick and Linsenmeier2002), further applied by Senko et al. (Reference Senko, Belmonte and Yakhkind2012) to teacher characteristics. This method provides participants with different budgets from which to ‘purchase’ characteristics. Spending patterns then allowed authors to work out necessities from luxuries. In the study of Winstone et al. (Reference Winstone, Nash, Rowntree and Menezes2016), psychology undergraduates received fictional budgets ranging from £20 to £60 from which they were able to ‘purchase’ different qualities of feedback. Results showed that in the smaller budgets of £20 the most necessary feedback quality was ‘highlights the skills I need to improve for future assignments’. Students reliably purchased this quality even when resources were limited. They only valued other characteristics when they had more ‘money’ to spend. This echoes some of the research mentioned earlier, which states that students value feedback which provides them with resources they can use to improve their future work.
CBT training and types of assessment and feedback
Formal CBT training programmes include many types of assessment and feedback, with both formative and summative components. These have been summarized by a number of authors (Branson et al., Reference Branson, Shafran and Myles2015; Fairburn and Cooper, Reference Fairburn and Cooper2011; Keen and Freeston, Reference Keen and Freeston2008; McManus et al., Reference McManus, Westbrook, Vazquez-Montes, Fennell and Kennerley2010). They typically include ratings of clinical performance using a validated scale, written feedback on clinical case reports and essays, written feedback on experiential tasks, reports on use of clinical supervision, as well as reports on practice placements. Other types of feedback include verbal feedback on role plays, and supervisor and peer feedback on therapy excerpts. CBT programmes invest a lot of resources into training their core and wider staff to provide what they judge to be high quality and useful feedback, but nothing is known about what the students themselves most value and, by implication, what they most associate with their skill development.
Current research
Therefore, the rationale for this study is to determine which types of feedback CBT (professional practice) students most value during training. This question is particularly important to address with IAPT (Increasing Access to Psychological Therapies) students, given the significant national investment and intensity of training; we need to be clearer about what is most useful to these students’ learning, to support safe and effective therapy practice. Knowing this is a move towards recent models of feedback that emphasize the co-productive process (Boud and Molloy, Reference Boud and Molloy2013; Carless et al., Reference Carless, Salter, Yang and Lam2011).
Method
Participants
Forty-three postgraduate CBT students participated across six different university-based training programmes. Programme directors of IAPT high-intensity CBT programmes were contacted to recruit their students. Females represented 79% of participants, which is broadly representative of the gender demographic of CBT postgraduate courses. To be eligible for the study, participants needed to have completed at least 6 months of their high-intensity CBT training, so that they were accustomed to engaging in a range of feedback types.
Procedure
Budgeting task
This was an online study. Following Winstone et al. (Reference Winstone, Nash, Rowntree and Menezes2016), we used a budgeting methodology to establish the importance placed on different feedback types. The online task involves providing participants with fictitious monetary budgets from which they can ‘purchase’ nine different feedback types across three different conditions: £20, £40 and £60. Participants can spend their budget in each condition on the feedback types that they perceive most benefits themselves and their professional practice. Any amount between £0 and £10 can be spent on any given feedback type. The more money participants spend on a feedback type, the more valuable they consider it. The advantage of this method with the three conditions is that it prevents participants from rating every quality as equally important.
As the budget increases, participants will be able to ‘purchase’ more of what they consider luxuries.
Table 1 shows the types and characteristics of feedback included. The list represents what is typically built into postgraduate IAPT high-intensity CBT training, and what is influenced by BABCP’s minimum training standards.
Percentage of available money allocated to each feedback characteristic in the Necessity and the luxury budget, and the difference between the two proportions
a, b, c: within each column, means without shared superscripts differ at p < .05. In the ‘Difference’ column: *p < .05; **p < .01; ***p < .001 with Bonferroni correction applied.
Participants accessed the online study via a web link attached to an email. After consenting, participants were given a (fictional) budget of £20 and their task was to purchase the types of feedback that they valued the most. They were able to allocate money (units of £1) across all nine feedback types, or they could invest more money into fewer choices. After completing the £20 condition, they repeated the task with a £40 and then a £60 budget. Feedback types were presented in random order across the three conditions.
Results
Data analysis
We adopted the same analytic approach as Winstone et al. (Reference Winstone, Nash, Rowntree and Menezes2016) to allow comparison across studies. First, we took the £20 budget and calculated the percentage of money allocated to each of the nine feedback types. This is called the necessity budget. Second, we subtracted the money allocated to the £40 budget from the money allocated to the £60 condition, and converted the result to a percentage. This is called the luxury budget; it represents the proportion of the final £20 assigned to each of the nine feedback types (Table 1).
We conducted a repeated-measures univariate analysis of variance (ANOVA) with the independent variable being the feedback type, and the dependent variable being the proportion of money spent. Within the necessity budget, there was a statistically significant difference in the importance placed on each feedback type, F (8,336) = 11.83, p < .001, w 2 = .22. In decreasing order of importance, the biggest necessities were Written feedback on full session video recordings, and Written feedback on essays. For the luxury budget, there was also a statistically significant difference between the qualities chosen, F (8,336) = 2.70, p = .007, w 2 = .06. In this condition, the type of feedback that attracted the most spending was Verbal feedback by supervisor on therapy excerpts.
As mentioned earlier, a feedback type is defined as a luxury if there is a significant increase in the proportion of spending between the necessity and luxury budgets. We therefore conducted paired t-tests and found two luxuries by this standard: Verbal feedback by peers on clinical role play and Verbal feedback from peers on therapy excerpts (see right-hand column of Table 1). The two qualities assigned most in the necessity budget were both assigned significantly smaller proportions of the luxury budget.
Discussion
The objective of this study was to examine which types of feedback CBT students perceived as being most helpful: which did they judge as necessities, and which as luxuries? Results of the budgeting analysis clearly revealed that written feedback on full session video recordings of own therapy, and written feedback on essays, were identified as necessities. They were allocated significantly more money in the necessity budget than the luxury budget. These findings are consistent with current research on the benefits of video recording live sessions to improve the quality of professional practice as perceived by teachers in training (Eroz-Tuga, Reference Eroz-Tuga2012). It is of note that both of these highest-rated necessities were in the form of written feedback, suggesting that students prefer this over verbally presented material. Research has found that verbal feedback does help to some extent; however, with the addition of written feedback it can significantly improve a student’s work (Van den Berg et al., Reference Van den Berg, Admiraal and Pilot2006).
Two questions arise from the current study. Firstly, to what extent do students actually use feedback to address future activity, i.e. ‘feed-forward’ (Evans, Reference Evans2013)? For example, is a CBT student using written feedback on a video recording to shape current supervision questions, and to strengthen clinical practice? And how can trainers facilitate this level of engagement and part of the learning process? Such practice would then take account of current models of feedback (Boud and Molloy, Reference Boud and Molloy2013; Carless et al., Reference Carless, Salter, Yang and Lam2011) that emphasize learner agency which not only equips students during training, but also beyond. Secondly, does what students value actually benefit their performance? To answer this we would need to measure the impact of feedback on future learning, and this is a challenging task (Price et al., Reference Price, Handley, Millar and O’Donovon2016).
When resources were less scarce, CBT students prioritized both verbal feedback by peers on clinical role play, and verbal feedback by peers on therapy excerpts. These two types received a statistically significant increase in proportion of spending between the necessity and luxury budgets. It is encouraging to see that students value peer opinion because this is readily available as formative feedback.
Limitations
There are some limitations to the current research. The data come from self-selected samples of CBT students, so those who chose not to participate may have held different views about feedback. Another limitation is that student preferences may change during training, so eliciting these at the end of each training programme would be preferable. For example, it is possible that CBT students value peer opinion towards the later stages of training. A limitation of the budgeting methodology is that participants were limited to a maximum of £10 to each characteristic, so restricting their allocation.
Conclusion
This study aimed to identify which types of feedback are valued most by CBT students in training. Results revealed that, out of nine different types of feedback provided, written reports on full session video recordings of their own therapy, and written feedback on essays were identified as necessities. Knowing this justifies the workload involved in trainers providing detailed formative feedback on student work. So far, literature on CBT training is limited to overall effectiveness of programmes, as well as links to client outcomes. The current study is a move towards identifying what are the most effective training components.
Acknowledgements
None.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of interest
The authors have no conflicts of interest with respect to this publication.
Ethical statements
The authors have abided by the Ethical Principles of Psychologist and Code of Conduct as set out by the APA. Ethical approval was granted by the University of Southampton Psychology Ethics Committee (ERGO number: 31195).
Key practice points
(1) CBT training programmes provide many types of feedback for learning, which take time and resources.
(2) Written feedback on full session video recordings was most highly valued by 43 CBT students across a number of IAPT training programmes.
(3) A ‘budgeting methodology’ is a useful paradigm to explore learner preferences because this forces a trade-off between necessities and luxuries.
Comments
No Comments have been published for this article.