Introduction
Routine outcome monitoring (ROM) to systematically assess the individual impact of psychological therapies has long been regarded as a key part of the scientist-practitioner strategy integral to cognitive-behavioural therapies (Salkovskis, Reference Salkovskis1984). However, implementing this in clinical practice is often more difficult. The national rollout of the improving access to psychological therapies (IAPT) initiative has seen the prioritization of ROM in line with government policies (NIMHE, 2005), creating a climate where it is a fundamental requirement of clinical practice (Johnston & Gowers, Reference Johnston and Gowers2005). ROM measures clinically relevant areas of functioning across therapy, through assessments at baseline and treatment completion/discontinuation. Further monitoring throughout an intervention of an individual's goals, symptoms, and feedback from sessions may supplement this (Johnston & Gowers, Reference Johnston and Gowers2005).
ROM is required to evaluate an intervention's effectiveness and allow commissioners to evaluate a service's clinical value (Law, Reference Law2012). Johnston & Gowers (Reference Johnston and Gowers2005) further advocate ROM as having the potential to enable service user involvement and service development. Fundamentally, however, the most important reason for ROM must be to responsively enhance clinical practice (Law, Reference Law2012). By obtaining information about what appears to be helpful and unhelpful, clinical decision-making and responsiveness to clients can be enhanced (Lambert & Shimokawa, Reference Lambert and Shimokawa2011). Although important across all services, the use of ROM has traditionally tended to be stronger within adult populations, for example, as demonstrated by the interval between the development of the Clinical Outcomes in Routine Evaluation – Outcome Measure (CORE-OM; Barkham et al. Reference Barkham, Margison, Leach, Lucock, Mellor-Clark, Evans, Benson, Connell, Audin and McGrath2001; Evans et al. Reference Evans, Connell, Barkham, Margison, McGrath, Mellor-Clark and Audin2002) and the Young Person CORE (Twigg et al. Reference Twigg, Barkham, Bewick, Mulhern, Connell and Cooper2009).
The IAPT initiative for adults experiencing anxiety and depression successfully incorporated session-by-session ROM (Clark, Reference Clark2011). Clark et al.'s (Reference Clark2011) evaluation of two pilot IAPT services demonstrated the importance of a session-by-session system in informing routine clinical services of the significance of missing post-treatment data. By comparing a session-by-session system with a conventional pre-post system, their results suggested that individuals who fail to provide post-treatment data demonstrated significantly less improvement, highlighting the risk of services overestimating their effectiveness when working this way. This initial evaluation also indicated that a session-by-session system can successfully be implemented within clinical services and achieve high levels of data completeness.
Alongside adult IAPT, literature demonstrates the benefits of frequent client feedback of symptoms and satisfaction on treatment outcomes and retention rates (Lambert et al. Reference Lambert, Whipple, Hawkins, Vermeersch, Nielsen and Smart2003; Shimokawa et al. Reference Shimokawa, Lambert and Smart2010), particularly for cases identified as ‘not-on-track’ (Lambert & Shimokawa, Reference Lambert and Shimokawa2011). Research suggests that when therapists use ROM their clients are significantly more likely to improve (Whipple et al. Reference Whipple, Lambert, Vermeersch, Smart, Nielsen and Hawkins2003; Miller et al. Reference Miller, Duncan, Sorrell and Brown2005), and highlights the importance of frequent feedback of progress to both client and clinician (Knaup et al. Reference Knaup, Koesters, Schoefer, Becker and Puschner2009).
The IAPT programme has recently been extended to children and young people with the aim of re-designing existing Child and Adolescent Mental Health Services (CAMHS), incorporating session-by-session ROM by both IAPT- and non-IAPT-trained clinicians (Wolpert et al. Reference Wolpert, Fugard, Deighton and Görzig2012). Currently, it is unclear to what extent ROM findings with adults can be generalized to children and young people; however, the first randomized trial examining this suggests that when clinicians had weekly feedback young people improved faster than those whose clinicians did not (Bickman et al. Reference Bickman, Kelley, Breda, de Andrade and Riemer2011). Other findings indicate that families reporting discussion of weekly feedback at higher rates also described enhanced therapeutic relationships and child functioning (Stein et al. Reference Stein, Kogan, Hutchison, Magee and Sorbero2010).
The Children and Young People (CYP)-IAPT model emphasizes the valuable contribution ROM can make to clinical work through developing clinically meaningful conversations, and promotes a curious and reflective mind-set by practitioners (Law, Reference Law2012). While these potential benefits have been discussed, there is also evidence of both service user and professional concern about this practice. Both advocates of CYP-IAPT (Law, Reference Law2012) and young people, parents and carers (Moran et al. Reference Moran, Kelesidi, Guglani, Davidson and Ford2012) recognize that ROM could potentially be viewed as a ‘tick-box’ exercise and highlight concerns that, if not used sensitively, measures could have a negative effect on clinical interactions (Moran et al. Reference Moran, Kelesidi, Guglani, Davidson and Ford2012).
A common theme across discussions around implementing ROM is the importance of professionals’ endorsement of this way of working (Ford et al. Reference Ford, Tingay and Wolpert2006; Knaup et al. Reference Knaup, Koesters, Schoefer, Becker and Puschner2009). Some suggest that implementing session-by-session ROM within CAMHS may require a culture shift for some clinicians (Law, Reference Law2012; Wolpert et al. Reference Wolpert, Fugard, Deighton and Görzig2012), as therapists often display confidence in their ability to monitor their clients progress (Hatfield & Ogles, Reference Hatfield and Ogles2006) and a process of formally monitoring therapists’ outcomes may understandably evoke anxiety due to its public and transparent nature (Lambert, Reference Lambert2007). Johnston & Gowers (Reference Johnston and Gowers2005) found that lead clinicians within CAMHS named staff resistance and resource shortfalls as frequent obstacles to ROM. Staff resistance included worries that using quantitative measures advocates a medical model and would reduce the value of clinical judgements, while resource shortfalls incorporated concerns about IT deficits, staff training, funding, and the need for staff ownership.
Within cognitive behavioural therapy, the influence of thoughts and beliefs is assumed to apply as much to therapists as to clients (Westbrook et al. Reference Westbrook, Mueller, Kennerley, McManus, Mueller, Kennerley, McManus and Westbrook2010). Research investigating this hypothesis in relation to therapist beliefs about the use of homework demonstrated a clear link between therapists’ self-reported beliefs and practices (Fehm & Kazantzis, Reference Fehm and Kazantzis2004; Kazantzis et al. Reference Kazantzis, Lampropoulos and Deane2005). Research has not yet examined the influence of therapist beliefs on use of ROM.
While previous studies give us some ideas of therapists’ attitudes towards ROM, they asked only for lead clinicians’ views (Johnston & Gowers, Reference Johnston and Gowers2005). With the expanding implementation of CYP-IAPT, the present study was designed to explore a range of CAMHS professionals’ beliefs that may act as both barriers and drivers in the use of session-by-session ROM. Session-by-session ROM, as opposed to the broader pre-post definition (Johnston & Gowers, Reference Johnston and Gowers2005) that has been the emphasis of previous research, was the specific focus of interest, as this clinical practice is expected as part of CYP-IAPT and may pose its own opportunities and challenges. It was hypothesized that clinicians currently using session-by-session ROM would describe stronger positive beliefs and weaker negative beliefs, compared to those not currently utilizing a session-by-session system. It is hoped that exploring these views can help inform the implementation of this way of working within CAMHS. Initially, focus groups were run to explore CAMHS clinician's beliefs about session-by-session ROM. These findings then informed the development of a questionnaire distributed within four CAMHS teams.
The service
The project developed from a discussion around CYP-IAPT and how session-by-session monitoring might be received by CAMHS clinicians, particularly following experiences of the implementation of CAMHS Outcomes Research Consortium (CORC). The service had previously been part of a bid to become a new site for the CYP-IAPT programme; however, this was unsuccessful and a new bid is in process, with the service having a strong commitment to the model.
In developing the research, relevant profession (e.g. CAMHS Psychology meeting) and team meetings were attended by the lead researcher (K.J.) to further discuss the project design and feasibility. For example, there were discussions about carrying out initial interviews or focus groups to inform the development of a questionnaire. Further aspects were discussed such as the feasibility of visiting all CAMHS teams within the region. The project was also discussed with the Professional Lead for Psychology and those leading the CYP-IAPT bid for the host trust.
Study 1
Method
Focus groups were used to obtain a wide range of professionals’ views on session-by-session ROM (Nassar-McMillan & Borders, Reference Nassar-McMillan and Borders2002). The focus group structure enabled the researchers to introduce and explain the research area to several participants at once, and facilitated conversations between practitioners about their views on this practice.
Sample
The sample consisted of CAMHS professionals (including psychologists, psychiatrists, family therapists, primary mental health workers, and psychotherapists) at a team away day. Team members were informed about the research, provided with a consent form, and everyone present participated. Three focus groups were run comprising of a total of 12 participants (from a team of 15), with a mean of four participants per group.
Content of the focus groups
Each focus group provided an explanation of the research and an introduction to the CYP-IAPT session-by-session measures. Participants were asked to think about the potential positive and negative aspects of this practice, both for professionals and service users and their families, for approximately 30 min. The groups were facilitated by the researchers (K.J, S.E., T.H.), the first a clinical psychologist in training and the latter both experienced CAMHS clinical psychologists. All groups were audio-recorded and transcribed (Clausen, Reference Clausen2012) using thematic analysis (Braun & Clarke, Reference Braun and Clarke2006). The data from each transcribed focus group was reviewed by four of the researchers individually, with key themes within positive and negative views summarized by the first author and reviewed and amended by all.
Results
It was clear from the transcripts that the comments divided into perceived advantages and disadvantages (‘positive and negative aspects’) of ROM, and the results are structured accordingly. See Table 1 which details the main themes, the number of respondents who articulated each theme and illustrative quotes.
Focus groups themes, respondent numbers and illustrative quotes
CAMHS, Child and Adolescent Mental Health Services.
Potential positive aspects of session-by-session monitoring for clinicians
The most common theme when clinicians were asked to consider the potential pros of session-by-session monitoring was that this way of working provides a systematic and accurate view of a young person's experience, of progress made throughout therapy and can inform decisions around discharge (seven comments).
Session-by-session monitoring helping to provide focus (five comments) and being a collaborative process between clinicians and young people which encourages feedback (five comments) were other prominent themes. Responses in relation to this way of working providing focus included thoughts that the measures are ‘client-focused’ and ‘keeps clinicians on task’.
Further themes identified included that this way of working can encourage the client to take responsibility (three comments), is a way of enabling CAMHS to demonstrate change (three comments), and is a potential tool for engagement (two comments) and useful if quick and easy (two comments).
Potential negative aspects of session-by-session monitoring for clinicians
The most common theme when clinicians were asked to consider the potential cons of session-by-session monitoring was concern about how the information would be used (nine comments). Clinicians’ concerns around this included whether information would be used for performance management or comparison between therapists, and whether outcomes would be seen in isolation and take the complexity of therapeutic work into account. Session-by-session monitoring influencing the focus of therapeutic sessions by being goal-driven, direct and resulting in therapists not attending to other important issues was another prominent theme (six comments). Further themes identified included this way of working resulting in extra work for clinicians (five comments), being time consuming (four comments), and negatively impacting on or interrupting the development of a therapeutic relationship (four comments).
Potential positive aspects of session-by-session monitoring for young people
CAMHS professionals most commonly suggested that a potential pro of session-by-session monitoring, from a young person's perspective, could be its collaborative nature which empowers young people to feedback their views (six comments). Young people being able to see their progression over time and this being motivating and providing a sense of achievement was another prominent theme (four comments). Further themes identified included session-by-session monitoring being motivational for a young person (three comments), helping to provide a focus of therapeutic work (three comments), and potentially working better with appropriate technology (two comments).
Potential negative aspects of session-by-session monitoring for young people
CAMHS clinicians most commonly discussed the potential barriers that session-by-session monitoring may result in for young people (five comments). Within this theme, clinicians considered whether some young people may view this as not a collaborative way of working and could feel that they are being tested and not listened to. Further themes included session-by-session monitoring being perceived by young people as a ‘paper exercise’ (three comments), being demoralizing (three comments), and taking time away from talking about their difficulties (two comments).
Study 2
Method
Participants
Fifty-nine (78%) from a possible 76 CAMHS clinicians, who were all working clinically as therapists were recruited from four CAMHS teams in the South West region. At a team meeting, potential participants were provided with an introduction to the research and the CYP-IAPT session-by-session measures, alongside the consent form. Those who were happy to participate completed the consent form, followed by the session-by-session ROM questionnaire.
Measures
Session-by-session Outcome Monitoring Questionnaire
Item development
The identified themes (and detail within these) from study one were developed into possible questionnaire items by the first author (36 items) as far as appropriate using phrasing from the focus group transcripts. Items were designed to reflect both the positive and negative views expressed and covered practitioner's views on areas including: feasibility, impact on the therapeutic relationship, measures psychometrics, their clinical value, the role of technology, and service-related issues. Review of each item by all clinicians aimed to ensure clarity in individual items and prevent repetition. A meeting between the four clinicians further reviewed items, and agreed the questionnaire construction and format. The final questionnaire balanced the number of positive and negatively phrased statements.
Questionnaire description
The final questionnaire consisted of 34 self-report items assessing professionals’ demographic characteristics, attitudes towards, and use of, session-by-session ROM (copies available upon request from the corresponding author).
Twenty-six items were designed to explore professionals’ attitudes, phrased as statements about session-by-session ROM and its role in clinical practice. Although the term ‘routine outcome monitoring’ (Wolpert et al. Reference Wolpert, Fugard, Deighton and Görzig2012) is frequently used within the literature, the term ‘session-by-session outcome monitoring’ was used to reflect the nature of this practice expected as part of CYP-IAPT. Professionals were asked to rate their agreement on a 5-point Likert scale ranging from 1 (not at all) to 5 (totally), a rating scale consistent with existing CYP-IAPT measures (CORC, 2012).
In addition to items exploring professionals’ attitudes, participants were asked to provide an overall rating of how often they currently use session-by-session monitoring (i.e. ‘How often do you currently use session-by-session monitoring in your clinical practice?’) on a 5-point Likert scale ranging from 1 (never) to 5 (almost always). Professionals were also asked whether they have received any CYP-IAPT training (‘yes’ or ‘no’) and provided space for any comments on the questionnaire or session-by-session monitoring more generally.
As this questionnaire was designed specifically for this research, the reliability and validity has not been established.
Results
Sample characteristics
Table 2 outlines the demographic information for the 59 CAMHS professionals. Of those providing demographic information, the majority were white British (64%, n = 38), female (63%, n = 32), and within the age range 41–50 years (39%, n = 23). However, up to 24% of participants did not provide one or more of these details. A range of mental health professionals were represented within the sample, alongside a range of years’ experience working in CAMHS (see Table 2). The majority of participants (88%, n = 45) had not received any CYP-IAPT training, and 58% (n = 34) reported ‘never’ using session-by-session monitoring as part of their current clinical practice.
Demographic statistics
CAMHS, Child and Adolescent Mental Health Services; CYP-IAPT, Children and Young People's Improving Access to Psychological Therapies.
Items with a high proportion of missing data
Inspection of the raw data suggested high levels of missing data across two questionnaire items – item 18 (‘works well with technology to support it’) and item 21 (‘costs too much to use’). Twenty-two (37%) participants did not answer item 21, and 14 (24%) participants did not answer item 18. The low response rates on these two items suggested that many CAMHS clinicians did not feel able to answer them as they required factual information not accessible to them (e.g. the cost). Therefore, these items were removed from the dataset prior to any analyses.
Overall scale reliability
Based on the remaining 24 items, high internal consistency was indicated (Cronbach's α = 0.938).
Attitudes towards session-by-session monitoring
On the basis of the logic and face validity of the items, two subscales were developed. Six items from the questionnaire were selected to reflect beliefs about negative aspects around using session-by-session monitoring in CAMHS. Similarly, a further six items were chosen to represent beliefs about positive aspects of using session-by-session monitoring in this context. The questionnaire items, including those selected for the two subscales, are shown in Table 3.
Session-by-session outcome monitoring questionnaire items and subscales
a Negative impact of session-by-session monitoring subscale.
b Positive impact of session-by-session monitoring subscale.
Items 18 and 21 excluded due to high levels of missing data.
Subscale scores were calculated for each subscale, based on the total score of the items making up that subscale divided by the number of items (six for each). Higher scores indicate higher levels of agreement with the items. The negative impact subscale (n = 42) had a mean of 2.79 (s.d. = 0.95) and the positive impact subscale (n = 49) a mean of 3.52 (s.d. = 0.78).
Current use of session-by-session monitoring
To compare beliefs as a function of professionals’ current use of session-by-session monitoring, the frequencies of use of measures were examined. A pragmatic distinction of never used routine measures vs. used routine measures on some or many occasions was used. To distinguish those who have never used ROM from those who have, we used the item asking participants to illustrate how often they currently use this practice. Participants who indicated never using ROM (1) on this item were grouped, as were those who indicated using ROM at differing frequencies (2–5). This then formed the group of participants indicating varying use of session-by-session monitoring (n = 21) and those suggesting they ‘never’ use this way of working (n = 34), for the planned comparison described next.
Due to this data not being of normal distribution, as indicated by significant Kolmogorov–Smirnov tests, rank transformation was used. A two-way mixed-model analysis of variance (ANOVA) was then performed on the ranked scores. The repeated-measures factor (beliefs) was the negative and positive impact of session-by-session monitoring subscales. The between-subjects factor (use of session-by-session monitoring) was whether participants indicated that they did or did not currently use session-by-session monitoring. This analysis indicated a non-significant main effect for the within-subjects factor ‘beliefs’ (positive vs. negative beliefs about the impact of measures) (F 1,35 = 3.569, p = 0.067). Although non-significant, this suggested a trend that overall, regardless of group, participants were more in agreement with positive than negative beliefs. For the between-subjects factor ‘current use of session-by-session monitoring’ (F 1,35 = 5.830, p = 0.021) a significant main effect showed that those who used session-by-session monitoring were in stronger agreement with both positive and negative beliefs. There was no significant beliefs by current use of session-by-session monitoring interaction (F 1,35 = 0.030, p = 0.864, n.s.) (see Fig. 1).
Line graph to show mean value on positive and negative impact subscales for those who do and do not use session-by-session monitoring.
Demographic factors
To explore whether any other between-group differences might account for the belief ratings of analyses examining demographic factors were completed. Variables examined included age and years of CAMHS experience (as this may be linked to flexibility of beliefs), gender (as men and women may hold different beliefs), and profession (as some professions may be more likely to use outcome measures). Sample sizes in each analysis vary due to missing demographic data.
First, an independent samples t test indicated that there was no significant between-group difference for age (t 48 = -0.981, p = 0.331). χ2 analyses, comparing males and females who do and do not use session-by-session monitoring, suggested no significant association [χ2 1(42) = 1.235, p = 0.266]. Further χ2 analyses, investigating differing years of CAMHS experience [χ2 2 (45) = 1.263, p = 0.532] and professions [χ2 7(49) = 10.687, p = 0.153] in those who do and do not use session-by-session monitoring, also suggested no significant associations.
Additional comments
Clinicians were asked for further comments on session-by-session monitoring in an open-ended question. The most common theme (11 participants) was of being unsure about session-by-session monitoring due to too little knowledge or experience, and understanding of its impact on clinical work.
Concerns over the use of session-by-session monitoring with more complex cases (eight participants) and the importance of the therapeutic relationship (eight participants) were also prominent. Further themes identified included clinical practice already including session-by-session monitoring (seven participants), the meaningfulness of this practice (six participants), concerns over the limitations of measures (five participants), and the particular therapeutic model involved (four participants).
Discussion
Surprisingly, the present study is the only one we are aware of which has examined beliefs and actual practice of session-by-session ROM in CAMHS. The results of the focus groups suggested prominent themes that session-by-session ROM provides systematic and accurate accounts, and the view that it can be both collaborative and empowering. Other themes identified concerns over how the information would be used by managers and how the measurement process might adversely influence therapeutic sessions. Drawing on a quantitative analysis in a larger group, it was found that only 6.8% of participants reported ‘almost always’ utilizing session-by-session ROM, and that only 7% had received CYP-IAPT training. This suggests that session-by-session ROM is not current practice within these CAMHS teams. The results of the questionnaire based on these focus groups identified six items as reflecting the perceived negative impact of session-by-session ROM, and six items the perceived positive impact. It was also found that there was a trend towards stronger positive than negative beliefs about session-by-session ROM within the sample, with those who currently use this way of working holding stronger positive and negative beliefs than those who do not. Age, gender and profession were not associated with the strength of these beliefs.
Although research assessing the effectiveness of session-by-session ROM in children and young people is still developing, the first RCT examining this suggests that when clinicians had weekly feedback young people improved faster than those whose clinicians did not (Bickman et al. Reference Bickman, Kelley, Breda, de Andrade and Riemer2011). The finding that only a minority of clinicians reported ‘almost always’ utilizing session-by-session ROM could suggest that, without this regular feedback, clinicians are not able to be as responsive to young people and support them to improve faster. These findings were identified despite stronger positive than negative beliefs about this practice within the sample. The outcome that only 7% of clinicians had received CYP-IAPT training highlights that this initiative is still in the early phases of its development within these CAMHS teams. It could be hypothesized that further training and experience of utilizing session-by-session ROM within everyday clinical practice may influence clinicians’ beliefs about this way of working.
The finding that experience of using session-by-session monitoring was associated with both higher positive and negative beliefs about its utility within clinical practice was unexpected. It may be that those using this practice are more aware of both positive and negative aspects of this way of working or may reflect that those currently utilizing it are tending to do so in the absence of support structures. Interestingly, the findings indicated that there was a trend towards stronger positive beliefs in both those who do and do not currently use this practice. These findings fit with recent research (Thew et al. Reference Thew, Fountain and Salkovskis2015) in secondary care adult mental health services which found that clinicians generally endorsed positive beliefs about measures more strongly than negative ones.
Limitations
Within the centre where the research was carried out, not all team members’ participated (22% and 20%, respectively). Similarly, the teams involved may not be representative of CAMHS nationally, though they did cover a diverse range of populations amongst them. The study, therefore, would clearly benefit from replication across a bigger sample of services.
A strength of the present study was its use of mixed methods, there being complementarity between the use of focus groups and a questionnaire. Some clinicians, however, described feeling unable to complete the questionnaire due to their lack of experience with session-by-session ROM, potentially creating a bias in the final sample. However, the final open question within the measure enabled them to express this. In addition, as the questionnaire was developed specifically for this study, the intervals both within and between the subscale items may not be the same. Further exploring professionals’ views following the implementation of session-by-session ROM could be of value.
Implications for practice and training
This study is the first to examine CAMHS clinician's use of and beliefs about session-by-session ROM, in line with the CYP-IAPT ethos. There is a previous study in this area; however, this looked at ROM from the view of lead clinicians within CAMHS (Johnston & Gowers, Reference Johnston and Gowers2005), and indicated that resource shortfalls were the main obstacle to this way of working. Clearly this is from a different perspective and within the current study the items related to cost and technology could not be answered. The Johnston & Gowers study was, however, conducted 9 years ago and the additional investment within mental health services since may have been influential in relation to these findings. Studies with other professional and patient populations can also inform our understanding of attitudes towards outcome measures. Research with General Practitioners (GPs) and their patients (Dowrick et al. Reference Dowrick, Leydon, McBride, Howe, Burgess, Clarke, Maisey and Kendrick2009; Leydon et al. Reference Leydon, Dowrick, McBride, Burgess, Howe, Clarke, Maisey and Kendrick2011), for example, has found that patients generally favoured the use of measures assessing depression severity and viewed these as evidence of a thorough assessment by their GP. GPs, on the other hand, were more cautious about the validity and utility of such measures. Furthermore, research (Hatfield & Ogles, Reference Hatfield and Ogles2007; Gyani et al. Reference Gyani, Shafran, Myles and Rose2014) highlights the importance of a therapists theoretical orientations in influencing the likelihood of using outcome measures.
Since completion of the study, the results have been fed back to the services involved and there is now a pilot project around session-by-session ROM in place. This pilot study was developed with the aim of providing therapists with an opportunity to trial session-by-session ROM within their everyday clinical practice and the recognition that concerns that arise will be considered and addressed. It is hoped that this process can lead to a wider introduction of session-by-session ROM with the support of clinicians. A key part of this pilot will also be seeking feedback from young people and their families about their experience of session-by-session ROM, as many clinicians in the current study identified that these views would be key in informing their attitudes towards this clinical practice.
The present study, being cross-sectional, does not allow for a judgement to be made about causal relationships. Consequently, it may be hypothesized that the beliefs measured motivate ROM, the use of ROM may impact on beliefs, or, as seems more likely, both may be true as part of a reciprocal relationship. Therefore, a model creating a virtuous circle whereby positive beliefs motivate more frequent use of session-by-session ROM, which consequently reinforces these beliefs, would be most useful. This would be in preference to a similar inhibitory process whereby negative beliefs motivate reduced use of session-by-session ROM. A model facilitating positive beliefs may be made more likely through a process of enabling and supporting CAMHS teams to develop the use of session-by-session ROM within their clinical practice, as opposed to directives to implement this within routine work. Alternatively, it could be that factors other than beliefs play an important role in CAMHS clinicians’ practice of session-by-session ROM. This possibility would be in line with the current findings that despite positive beliefs in the sample overall, few clinicians currently use this clinical practice. Both these models and the processes best utilized to facilitate the use of session-by-session ROM in CAMHS needs to be further researched.
In terms of frameworks to understand this, psychological models of decision-making suggest that it is usually important to, firstly, understand why individuals hold positive and negative beliefs about a particular issue, and, secondly, increase the weight of positive beliefs and decrease negative beliefs by addressing important factors relevant to the decision. Wroe & Salkovskis (Reference Wroe and Salkovskis1999) suggest that in addition to a balance of pros and cons, decision making is linked to the accessibility of information at the time the decision is made, while Kahneman (Reference Kahneman2011) highlights further biases that can cause people to deviate from balanced decision making. Theoretically, when applied to session-by-session monitoring, this process of understanding and shifting the decisional balance should lead to better implementation of this way of working.
Beacon projects may be a helpful way of initially introducing session-by-session ROM, in order for services to learn and develop the most effective ways of using this clinical practice and understanding clinicians concerns further. This can then inform the on-going development and implementation of training and support systems to keep clinicians on board with this way of working. There is, however, a risk of tokenism within services required to adopt this approach and it will therefore be even more important for clinicians concerns to be further understood and addressed.
Suggestions for future research
Further research, with larger projects involving multiple CAMHS services, to explore clinician's beliefs about session-by-session ROM in relation to their current practice would be of benefit. Furthermore, experimental studies utilizing cluster randomization to focus on addressing clinician's beliefs about session-by-session ROM and the impact of implementing ROM within CAMHS may help define the best process for facilitating the use of this way of working in everyday clinical practice. Future research exploring the impact of CYP-IAPT training on the use of ROM and clinicians beliefs, in comparison with those who have not received such training, would also be of value.
This study could also be extended by exploring children, young people, and their families’ views and experience of session-by-session ROM. Only one previous study (Moran et al. Reference Moran, Kelesidi, Guglani, Davidson and Ford2012) has explored this, highlighting the importance of their involvement in the process of outcome measurement. Similarly, recent research in adult services (Thew et al. Reference Thew, Fountain and Salkovskis2015) has suggested that service users perceptions of how well measures were used and integrated into therapy were strongly associated with how helpful they found measures as part of therapy overall. Future research further exploring young people's views of session-by-session ROM, the acceptability of this clinical practice and individual measures, and service user experience in relation to clinician's beliefs will be of value.
Conclusion
Despite its limitations, these findings suggest that those clinicians who currently use session-by-session ROM may hold stronger positive and negative beliefs than those who do not. This, therefore, highlights the importance of considering how this practice can be best implemented within this setting.
Summary
There is increasing emphasis within CAMHS on measuring progress with children, young people and their families.
-
• This study aimed to understand what clinicians think about and how often they utilize session-by-session ROM to help inform its implementation.
-
• Themes which emerged from focus groups included this way of working providing objectivity, and being collaborative and empowering. Themes also illustrated concerns over how the information would be used and measures may influence therapeutic sessions.
-
• Questionnaire responses of 59 CAMHS professionals found that only 6.8% of participants reported ‘almost always’ utilizing session-by-session ROM and that only 7% had received CYP-IAPT training.
-
• Questionnaire responses also suggested that clinicians who currently use session-by-session ROM hold stronger positive and negative beliefs than clinicians who do not.
-
• This study highlights the importance of considering how this practice can be best implemented within CAMHS services.
Ethical considerations
Service evaluation approval for both studies was granted by the hosting trust's audit committee, which was endorsed by the University of Bath Ethics Committee (13–015).
Acknowledgements
The authors thank all of the CAMHS teams who participated in this research.
Declaration of Interest
None.
-
(1) To understand CAMHS clinicians’ attitudes and beliefs about the use of session-by-session routine outcome monitoring (ROM) in clinical practice.
-
(2) To explore clinicians’ views about session-by-session ROM through focus groups.
-
(3) To assess the characteristics of a questionnaire developed to elicit information about professionals’ demographic characteristics, attitudes towards, and current use of, session-by-session ROM.
-
(4) To consider the impact clinicians’ beliefs may have on the implementation of ROM.
Comments
No Comments have been published for this article.