Introduction
Case conceptualization, or formulation, is the process in which unique client experience is blended with a psychological theory or model with the purpose of understanding the origins, development and maintenance of a presenting problem (Tarrier and Calam, Reference Tarrier and Calam2002). The conceptualization is used to select the optimal interventions in order to alleviate the person's distress and help keep him or her well in the future (Kuyken, Padesky and Dudley, Reference Kuyken, Padesky and Dudley2008). Given the central role of formulation in treatments such as Cognitive Behavioural Therapy (CBT), there is a surprising lack of research into this process (Bieling and Kuyken, Reference Bieling and Kuyken2003). Within this scant evidence base, much of the research to date has addressed whether clinicians agree with each other on the content of a cognitive case formulation.
This work has indicated that cognitive therapists tend to agree about clients’ overt presenting issues (i.e. the physical symptoms, and behavioural, and emotional difficulties). However, the intra-psychic aspects of the person's presentation, those where an inference is made based upon cognitive theory (i.e. a person's core beliefs or dysfunctional assumptions) leads to poor agreement (Persons, Mooney and Padesky, Reference Persons, Mooney and Padesky1995; Persons and Bertagnolli, Reference Persons and Bertagnolli1999; Mumma and Smith, Reference Mumma and Smith2001; Kuyken, Fothergill, Musa and Chadwick, Reference Kuyken, Fothergill, Musa and Chadwick2005).
This low rate of agreement poses a challenge as formulation is considered to be a lynchpin process in effective CBT (Butler, Reference Butler1998). Hence, if therapists do not agree on the content of a formulation then it is unlikely they will agree on what interventions will be of most help to the person. Discrepancies in understanding arising from different formulations could have a profound impact upon the treatment a client receives in therapy. Formulation is therefore an important skill to understand and for cognitive therapists to develop. Consequently, research has considered which factors are associated with better rates of agreement between therapists. Persons et al. (Reference Persons, Mooney and Padesky1995) reported that PhD level training was the only predictor of the ability to identify presenting problems, and that training or experience did not relate to the ability to identify inferential aspects of the formulation. Kuyken et al. (Reference Kuyken, Fothergill, Musa and Chadwick2005) noted that formal training, greater experience and accreditation as a cognitive therapist were associated with higher quality CBT formulations. Eells, Lombart, Kendjelic, Turner and Lucas (Reference Eells, Lombart, Kendjelic, Turner and Lucas2005) compared novice, experienced, and expert clinicians in their ability to formulate a series of cases. The formulations of experts were more comprehensive, elaborated, complex, and systematic than both of the other two groups. In addition, they found that the treatment plans of experts were more elaborated and linked better to the formulations. Hence, it was not experience per se that predicted best performance but some level of expert knowledge. In this case, “expertise” was defined in terms of those clinicians who had devised and published formulation systems or led workshops on the topic. This clearly implies that formulation is a skill that requires time, effort and resource to develop.
One key function of formulation in CBT is to provide a shared understanding of what may have led to, and what may be maintaining the current difficulties (Dudley and Kuyken, Reference Dudley, Kuyken, Johnstone and Dallos2006). Hence, developing a formulation is of potentially great value when working with people with psychotic illness who often report experiences that may, at first, appear difficult to understand. Consequently, CBT for Psychosis (CBTp) treatment manuals place great emphasis on the importance of formulation as a way to help see the world from the person's point of view (Kingdon and Turkington, Reference Kingdon and Turkington1994; Fowler, Garety and Kuipers, Reference Fowler, Garety and Kuipers1995; Chadwick, Birchwood and Trower, Reference Chadwick, Birchwood and Trower1996; Morrison, Renton, Dunn, Williams and Bentall, Reference Morrison, Renton, Dunn, Williams and Bentall2004).
To our knowledge there are only two studies directly investigating formulation in psychosis (Chadwick, Williams and Mackenzie, Reference Chadwick, Williams and Mackenzie2003; Dudley, Siitarinen, James and Dodgson, Reference Dudley, Siitarinen, James and Dodgson2009) neither of which investigated reliability or agreement between therapists. Dudley et al. (Reference Dudley, Siitarinen, James and Dodgson2009) studied how people with psychosis made sense of the onset of their psychosis. In effect, it was a study of the client's own formulation as participants were asked about which stressors and/or vulnerabilities they felt led to the onset of their breakdown (as opposed to whether or not there was agreement between therapists).
Chadwick et al. (Reference Chadwick, Williams and Mackenzie2003) investigated the value and impact of formulation in a case series of people in receipt of CBTp. The conceptualization was developed over two sessions and then summarized in a formulation letter that incorporated an understanding of maintenance and development based on standard cognitive therapy models (Beck, Reference Beck1995). The results were somewhat equivocal, in that a formulation session had no impact on symptoms. Therapists perceived the formulation as helpful and the formulation session was associated with an increase in working alliance from the therapists’ perspective, but not by the client. At first glance the results of this study appear to be at odds with the functions attributed to formulation (Kuyken et al., Reference Kuyken, Padesky and Dudley2008). To understand this mismatch it is important to consider the potential limitations of the work before we discount the value of formulation in CBTp. First, the work was undertaken with a small number of participants and to demonstrate that a feature of treatment (be it homework, therapeutic alliance) is a mediator of a good outcome typically requires large scale highly powered studies (Borkovec and Castonguay, Reference Borkovec, Castonguay, Norcross, Beutler and Levant2006). Second, in our view there is no reason that a formulation developed in this way will naturally lead to the change expected in the study. In CBT a formulation is developed collaboratively with the client rather than presented to them. In addition, a conceptualization is developed over the course of therapy according to the level of understanding that is necessary for that stage of treatment (Kuyken, Padesky and Dudley, Reference Kuyken, Padesky and Dudley2009). Third, formulation in and of itself is not a treatment intervention. CBT as an approach does not consider the development of insight or understanding in itself, to be sufficient to lead to a reduction in distress (Beck, Reference Beck1995). Hence, the real value of formulation is in helping to create a shared understanding of the difficulties. This understanding is then actively subjected to empirical testing by the therapist and client, through the subsequent selection of treatment options. Formulation guides the selection of treatment interventions that are chosen to help alleviate a person's distress. The success or value of the intervention in turn acts as a test of value of the formulation.
As can be seen, the evidence base is extremely limited with regards formulation in general (Kuyken, Reference Kuyken and Tarrier2006) and particularly so in CBTp (Dudley and Kuyken, Reference Dudley, Kuyken, Johnstone and Dallos2006). Nevertheless, NICE guidelines for schizophrenia (2002) recommended CBTp be available for people with treatment resistant symptoms. The training of therapists to use CBTp almost invariably emphasizes the need to develop a coherent formulation of the person's presenting issues, based on a cognitive model. Hence, how we formulate psychotic illness, and the extent to which clinicians agree with each other, is still a highly relevant question.
The current study investigates what features of a formulation are identified by mental health professionals, when asked to produce a cognitive behavioural case formulation of an individual's delusional beliefs, within the context of a psychotic illness. It is the first study that investigates this rate of agreement of formulation, for psychotic illness. Additionally, we are able to investigate the relative contribution of different forms of experience, qualification and knowledge in the development of a conceptualization of delusional beliefs.
Despite the evidence that CBTp is helpful for people with psychotic illness (Turkington, Dudley, Warman and Beck, Reference Turkington, Dudley, Warman and Beck2004) there is still relatively little access to this form of help (Berry and Haddock, Reference Berry and Haddock2008). Rather, within the UK there has been an emphasis on training in Psycho-Social Intervention (PSI) that focuses on good case management, family and individual work (including CBT) in the context of increased understanding of the processes involved in psychotic illness such as the role of stress vulnerability models (Brooker and Brabban, Reference Brooker and Brabban2004). Hence, there is rich clinical experience of working with psychosis, but perhaps less specific CBT psychosis knowledge.
This study targets clinicians with CBT experience and those with psychosis experience in order to help understand the relative contributions of the two factors to the ability to produce cognitive conceptualizations. Previous research studies have not considered experience of CBT separately from the experience of the disorder. Thus, there were two main research questions. The first considers the reliability or rate of agreement between the therapists, and the second considers factors that may lead to lesser or greater agreement in the content of a formulation. In keeping with the previous research, it was hypothesized that participants would be better able to identify the overt symptoms and problems, and less able to identify the inferential aspects of a formulation, such as core beliefs or dysfunctional assumptions. It was also hypothesized that the higher level of academic qualification (Persons et al., Reference Persons, Mooney and Padesky1995), and greater clinical experience (Kuyken et al., Reference Kuyken, Fothergill, Musa and Chadwick2005) would lead to greater levels of agreement with an expert derived benchmark formulation. In addition, it was hypothesized that high levels of experience in either CBT or psychosis would be related to different patterns of strength in formulation ability. In particular, therapists with greater experience in CBT would be better at identifying the inferential aspects of the formulation (core beliefs, dysfunctional assumptions and thoughts). In contrast, therapists with greater experience of working with people with psychosis would be better at identifying the “stress-vulnerability” aspects of the presentation (i.e. individuals’ vulnerabilities, stressors and behaviours/coping strategies). The latter are aspects routinely taught on PSI courses and are compatible to some extent with the cognitive model that specifies psychological vulnerabilities. One key difference however, is the use of stress vulnerability models that often emphasize social and physical stressors in the onset of psychosis, and tend to not address the meaning of events so much as the quantity of events (Brabban and Turkington, Reference Brabban, Turkington and Morrison2002).
When studying reliability of formulation it is vital to consider a number of key methodological challenges. The first concerns the materials that clinicians draw upon when developing formulations. To help ensure the validity of the formulation, it should ideally be derived from multiple sources of information, such as the person's self report, the results of assessment measures, thought records, and response to interventions. Previous researchers have used written case vignettes (Eells et al., Reference Eells, Lombart, Kendjelic, Turner and Lucas2005), or audio (Persons et al., Reference Persons, Mooney and Padesky1995) or video tapes of excerpts from sessions to provide the clinical material (Mumma and Smith, Reference Mumma and Smith2001; Kuyken et al., Reference Kuyken, Fothergill, Musa and Chadwick2005). The client information in the current study is presented in the form of a video of an assessment session, in an attempt to emulate as closely as possible the information available to a clinician in their own practice. In addition, a timeline is used as a means of eliciting key historical events. This is a common technique utilized by clinicians when working with people with psychotic illness, within a PSI framework.
A second vital consideration when studying reliability of formulation is to consider what the clinician is meant to agree with. The benchmark formulation ideally will be high quality, parsimonious, logically coherent and, if possible, be agreed as acceptable by the client, therapist, supervisor, and the broader peer group of therapists (Butler, Reference Butler1998). At worst, a formulation may just be one person's opinion and, hence, be subject to bias (Kahenman, Reference Kahneman2003). To help reduce possible bias, we drew on a case that was treated by one of the research group, and we consulted three experts to provide an agreed formulation of the case based on the timeline, and viewing of the video. This reduced the potential subjectivity of the benchmark case formulation that was used to compare the responses of the clinicians against.
A third important matter is the format of the cognitive model on which to base the formulation. There are many variants on the cognitive model that are proposed to account for the emergence and maintenance of psychotic symptoms (i.e. Bentall, Reference Bentall2003; Chadwick et al., Reference Chadwick, Birchwood and Trower1996; Freeman, Garety, Kuipers, Fowler and Bebbington, Reference Freeman, Garety, Kuipers, Fowler and Bebbington2002; Freeman, Reference Freeman2007; Morrison, Reference Morrison1998). These vary in the amount of empirical support for the mechanisms described in the models, but generally lack evidence of their effectiveness (see Trower, Birchwood and Meada, Reference Trower, Birchwood and Meada2004 for an obvious exception in treating command hallucinations). To date, no specific cognitive models have shown to improve treatment for delusions above and beyond a generic cognitive model (Wykes, Steel, Everitt and Tarrier, Reference Wykes, Steel, Everitt and Tarrier2008). Moreover, these models share in common the notion that appraisals of events affect feelings and behaviour and, therefore, are all compatible with more generic cognitive models. Given the absence of a validated “symptom” specific model for delusions, it is assumed that more generic cognitive behavioural frameworks (i.e. Beck, Reference Beck1995; Persons and Bertagnolli, Reference Persons and Bertagnolli1999) are commonly applied to understand and treat delusional beliefs. Therefore, in this research we chose to use a generic cognitive model based on maintenance cycles (Beck, Reference Beck1995) and predisposing factors to difficulties described in terms of early life experiences leading to beliefs about self, and others that are managed by rules, and assumptions with distress activated by a triggering event that breached these rules or assumptions.
Method
Participants
Eighty-five mental health clinicians were recruited from postgraduate and professional training courses in CBT, and Psychosocial Interventions. Clinical psychologists undertaking doctoral training were also recruited. All participants were familiar with CBT, having undertaken a least a 4-day basic course in CBT. All participants completed a demographic questionnaire eliciting information about professional background, professional and academic qualifications, number of years of clinical experience, as well as specific experience in psychosis and CBT, including an estimate of the number of cases seen for CBT work, and the number of people with psychosis with whom the person has worked. Three people did not participate owing to choosing to use the time for other purposes. Hence, the rate of participation (82/85) was high.
Demographic characteristics of the sample and training information are shown in Table 1. Participants had a variety of professional backgrounds, with almost half comprising of mental health nurses. Almost two-thirds were educated to at least Bachelors’ degree level, and had an average of 8½ years clinical experience. They reported a rich experience of working with people with psychotic illness, having worked with many cases, but generally having less CBT case experience.
Participants’ demographic characteristics and training information
Measures
The measures employed were specifically developed for the study in light of the methodological challenges outlined in the introduction. A video of a clinical session was the basis on which participants constructed their formulations. Their formulations were then compared to a benchmark formulation previously developed by an expert panel of therapists. A scoring manual was used to assess how well the participant's formulation corresponded with the template, or benchmark formulation.
Video vignette
The case was presented as a video of a 30 minute CBT assessment session. The material was based on an actual client, who had given his consent, subject to protecting his anonymity. As a real case, the material contained many of the features that are emphasized within the disorder specific models (i.e. Freeman, Reference Freeman2007) of delusions that are considered important in the formation and maintenance of delusions (e.g. substance misuse, social isolation, disrupted sleep, jumping to conclusions reasoning style, bullying and negative beliefs about others). The client was role-played by a clinical psychologist who is also an experienced actor. Additional information was provided in the form of a timeline that detailed the client's history. In brief, the person developed a psychotic illness whilst working away from home. He was experiencing a number of stressors including being away from his supportive family, taking cannabis, working shifts, experiencing disturbed sleep, and working in a loud environment (an engine room). He became very concerned that his colleagues would discover some pornography he had brought with him that he had not realized contained homosexual images. He feared he would be ridiculed if this was discovered. During this period he watched a number of films with a common theme of aliens (i.e. Close Encounters of the Third Kind, ET). One evening whilst working in the engine room he became convinced that aliens were coming to harm him.
Formulation template
As stated, there is no unifying model of psychosis and there is only preliminary evidence of a symptom specific model for delusional beliefs, particularly in relation to treatment (Freeman, Reference Freeman2007). Despite their differences, such frameworks share many similarities with the generic cognitive model (Dudley and Kuyken, Reference Dudley, Kuyken, Johnstone and Dallos2006) in that they aim to account for: the presenting problems, maintenance features, precipitants or triggers, and predisposing factors that may have left this person vulnerable to the stressors at the time of onset. In choosing a formulation template, these criteria were considered and agreed as present via a process of consultation with an expert panel.Footnote 1 Beck's (Reference Beck1976) cognitive behavioural therapy framework, originally developed for depression, was used as the basis for the formulation template as it accounted for the common features in cognitive models of psychotic experiences.
To ensure the formulation format was appropriate, the courses from which the participants were recruited were contacted to ensure that the students would have had training in using these models. Whilst the CBT diploma courses emphasized disorder specific models for emotional disorders, they also included introductory training using generic formulation models. The Doctorate of Clinical Psychology courses and PSI courses also taught CBT using the generic model.
A pilot study (n = 5) of the materials led to clarification and definition of language in the template. The revised template was approved by the expert panel. The pilot data were not included in subsequent analyses.
Benchmark formulation
The expert panel individually watched the video and used the timeline to develop a cognitive formulation. Subsequent group discussion led to an agreed benchmark formulation. Only items agreed by all three members of the panel were included in the final formulation. The case material was rated by the expert panel for its likeness to a “typical case”, on a 0 to 5 (“nothing like” to “very much like” a typical case) likert scale, with an average rating of 4 indicating an acceptable level of face validity.
Scoring manual
The scoring manual was developed based on the experts’ benchmark formulation, and was refined further using three completed formulation templates that were randomly selected from the entire sample (described below). These cases were subsequently excluded from the data set. The research team scored the templates individually using the expert formulation as a guide, and discussed any discrepancies in the scores. This enabled precise guidelines to be written for the scoring of each item.
Each item of the formulation was scored of either 0 (inaccurate), 1 (theme identified) or 2 (accurate). Scores for each item were summed to produce a score for each component level of the formulation (i.e. Early experiences, Core beliefs, Dysfunctional assumptions, Stressors/triggers, Maintenance cycle). These components were summed to produce a total score. Components that were considered to be specific features of CBT formulations (Core beliefs, Dysfunctional assumptions, and the item “Thoughts” from the Maintenance cycle) were summed to create an “Inferential” subscale. The components of the formulation most consistent with stress-vulnerability models (Early experiences, Stressors, and the item “Behaviours” from the Maintenance cycle) were also identified, and labelled Stress-vulnerability subscale. Three completed formulations were selected at random from the data set. Each was scored independently by three of the researchers to establish inter-rater reliability of the scoring, using these guidelines. An acceptable level of inter-rater reliability for the total score, and the Inferential and Stress vulnerability subscales (kappa > 0.85) was achieved. For an illustration of the items and components within the formulation see Table 2.
Percentage agreement on each formulation aspect (overall and by level of training)
*Agreement for item > 70%
A level of 70% demonstrates good reliability (Luborsky and Diguer, Reference Luborsky, Diguer, Luborsky and Crits-Christoph1998)
Procedure
Having consented to participate, participants were tested in group settings, usually as part of scheduled training. Once familiar with the formulation template and the timeline materials, the video was presented, after which the participants were asked to complete the formulation template, without conferring about the case. Participants then completed the demographic questionnaire. Feedback on the expert formulation was provided at the end of the training session.
Ethics procedures
The study was subject to independent peer review, registered with a Trust R and D department and received Local Regional Ethics approval.
Data analysis
Data were analyzed using version 15 of SPSS (Statistical Package for the Social Sciences, 2007). In order to consider the first research question - what elements of a formulation are identified and to what extent it agrees with an expert formulation - the percentage agreement among participants was calculated, which is a recognized method of establishing reliability (Luborsky and Diguer, Reference Luborsky, Diguer, Luborsky and Crits-Christoph1998). By convention, a level of 70% demonstrates good reliability. To address which factors were associated with the ability to formulate, a series of regression analyses were undertaken in which qualification, years of and type of experience were entered as predictor variables. Three regressions were undertaken in which the criterion variable was total score, inferential score, and stress vulnerability score allowing consideration of whether particular CBT or psychosis experience was linked to these aspects of formulation.
Results
Table 2 provides the percentage agreement on each item (Kuyken et al., Reference Kuyken, Fothergill, Musa and Chadwick2005) and component of the benchmark formulation overall, by level of training. The figures shown represent the proportion of clinicians who correctly identified the item for each aspect of the benchmark formulation – i.e. scoring 1 or 2 on that item.
It is evident that as a component of formulation, behaviours are the most readily agreed items (overall agreement 91.6%). Physical symptoms are the next most readily identified (70.3%), followed by stressors or triggers (64.4%). This is consistent with previous research indicating that the overt presenting issues are better identified than the theory based inferential components of a formulation. However, it is also evident that there is great variability in the individual items of the formulation that are identified. Clinicians identify some of the more overt historical events (e.g. being bullied, 83.5%), as well as stressors/triggers (e.g. drug use, 91.1%), behaviours (e.g. carrying a knife, 88.6%) and physical symptoms of anxiety (77.2%) and one feeling (i.e. anxiety, 83.5%) with a high level of agreement. There was much less agreement for the inferential aspects of “core beliefs” and “dysfunctional assumptions” (between 36.7% and 65.8%). There was particularly low agreement with respect to identifying thoughts (mean overall agreement of 32.2%).
In keeping with previous research, there was only moderate agreement with the experts’ formulation amongst all of the participants. The mean score of 26.8 (SD = 5.88) was about half of the maximum score of 54. Scores for the inferential components (mean of 6.5 SD = 2.84, maximum possible = 20) were lower than that of the scores of the stress vulnerability components (mean = 16.1, SD = 3.7, maximum score possible = 26) once again reflecting the greater identification of more overt current and precipitating difficulties and less agreement on more theory based components.
To investigate potential predictors of formulation ability, a series of multiple regressions were performed in which the criterion variable was total score, inferential score, and stress vulnerability score. As some of the independent variables were significantly and positively skewed, transformations were employed as suggested by Tabachnick and Fiddell (Reference Tabachnick and Fiddell1996 p.174). “Years overall experience” and “number of CBT cases” were successfully transformed using the square root and log (base 10) transformations respectively. Transformations of the “number of psychosis cases” led to a bimodal distribution with one of the modes at zero. This variable was successfully transformed by taking the cube root. However, there still remained 18 participants or nearly 25% of the group who reported that they had had no experience of working with people with psychosis. Consequently, all analyses were run twice, once when cases where the experience of psychosis was zero were removed from the analysis, resulting in a reduced sample size of 58, and a second time, with all participants categorized into groups on the basis of their experience of working with psychosis (none, some, considerable) and the categorical variable was used as a predictor variable (only the reduced sample size is reported as it is the most conservative analysis, unless there were differences). The removal of cases did not impact significantly on the distributions for the other predictor variables. Bi-variate correlations (Tabachnick and Fiddell, Reference Tabachnick and Fiddell1996) and checks for multicollinearity indicated the variables were sufficiently independent.
The first multiple regression was for total formulation score as the outcome variable. Age and gender were entered next as control variables, followed by overall experience (years), highest qualification, number of CBT cases, and number of psychosis cases as the predictor variables. The overall model was significant and accounted for 21% of the variance (R2 = 0.21, F(4,53) = 3.49, p < .01). Two of the predictor variables contributed significantly to the overall model, (square root) overall years experience (β = 0.42, t = 2.85, p < .01) and (cube root) number of psychosis cases (β = −0.37, t = −2.38, p < .05). It should be noted that the “number of cases of psychosis seen” was inversely related to the outcome variable. This suggests that therapists with a greater number of years of clinical experience produce more reliable formulations, whereas therapists who have seen a high number of cases of psychosis specifically produce formulations that are less in agreement with those of an expert group of clinicians.
A second multiple regression, with the outcome variable being the score for the inferential components of the formulation, was undertaken. The predictor variables were entered as before. The overall model was significant (R2 = 0.17, F(4,53) = 2.79, p < .05) with the (log10) “number of CBT cases seen” making the only significant contribution to the prediction of the outcome variable (β = 0.28, t = 2.17, p < .05). This suggests that individuals with greater experience in CBT are better able to reliably formulate inferential aspects of a formulation.
A third multiple regression was performed with the stress-vulnerability component score as the outcome variable, and the predictor variables included previously. The overall model was not significant and accounted for only 14% of the variance (R2 = 0.14, F(4,53) = 2.15, p = .09). Effect sizes were calculated using R2 for each of the regression equations, using the convention stated by Cohen (Reference Cohen1988). The effect size for total score as the dependent variable was 0.27 (small effect size), which resulted in adequate power (0.87). With “Inferential aspects”, the effect size was 0.21 (small effect size), indicating reasonable power (0.77). For the stress-vulnerability components effect size was 0.16 (small), resulting in low power (0.63).
Additional analyses to investigate features considered to be very important in the development and maintenance of delusional beliefs were not possible owing to the lack of specificity in the formulation template. Hence, the identification of features such as jumping to conclusions reasoning style, or the presence of anomalous experiences (see Freeman, Reference Freeman2007) was not possible. However, we also considered if participants identified material not present in the expert formulation. In brief, no additional information was revealed, indicating that people did not include material not incorporated by the experts.
Discussion
This study investigated which features of a formulation were identified by clinicians with varying levels of experience of working with people with psychosis, and of CBT. We also investigated the rate of agreement with an expert generated formulation to help determine whether clinicians can produce reliable CBT formulations of an individual's delusional beliefs. Consistent with previous research on the reliability of CBT case formulation, there was greater identification of aspects of the formulation such as overt behaviours, physical symptoms, and triggers. There was less identification of the inferential aspects, such as core beliefs and dysfunctional assumptions.
The overall identification of CBT elements of formulation was low, with only 10 out of 27 elements (37%) achieving over 70% agreement when considering the entire sample (see Table 2). This illustrates the wide variation in features identified within the formulation. However, the participant sample was relatively inexperienced and had on average less than 2 years experience practising CBT. Despite this, the level of reliability was comparable to previous research on the reliability of CBT formulations (Persons and Bertagnolli, Reference Persons and Bertagnolli1999; Mumma and Smith, Reference Mumma and Smith2001), and was consistent with research examining the reliability of psychodynamic case formulation (Luborsky and Diguer, Reference Luborsky, Diguer, Luborsky and Crits-Christoph1998). This is particularly encouraging as clinicians were asked to generate items of the formulation independently, rather than be given a list of attributes as in some previous CBT reliability studies, which may have helped to increase reliability (Persons et al., Reference Persons, Mooney and Padesky1995; Persons and Bertagnolli, Reference Persons and Bertagnolli1999).
The identification of thoughts was particularly poor. There are a number of possible reasons for this. The thoughts may have been poorly defined by the experts, making this aspect of the formulation difficult to assess. Also, the process of identifying thoughts is likely to be associated with more CBT specific knowledge, rather than identifying more descriptive items such as behaviours, and CBT knowledge was generally low in the participants. However, there was a reasonable level of agreement for other components that involved making inferences based on cognitive theory (i.e. core beliefs and dysfunctional assumptions) amongst the more qualified group, suggesting that the low agreement on thoughts is unlikely to be owing to a lack of knowledge of cognitive theory per se.
Alternatively, it is possible that this aspect of Beck's (Reference Beck1976) generic CBT model does not transfer well to the formulation of delusional beliefs in psychosis. As yet, there is no widely accepted cognitive model of psychosis (Wykes et al., Reference Wykes, Steel, Everitt and Tarrier2008), nor of delusional beliefs. In one model (Freeman, Reference Freeman2007), persecutory delusions are conceptualized as threat beliefs, which are maintained by similar processes to those implicated in the maintenance of anxiety disorders. Beck's (Reference Beck1976) model was originally developed for conceptualizing depression, which is characterized by a negative cognitive triad (Beck, Reference Beck1976). The low level of agreement on thoughts observed in this study could reflect subtle differences in the formulation of thoughts within cognitive models for different disorders, or a lack of knowledge about the specific processes that are thought to be involved in the development of delusional thoughts.
It was hypothesized that the level of qualification, amount of overall clinical experience, and experience in CBT and psychosis would be related positively to the level of agreement with the benchmark formulation. Two variables, “years overall experience” and “number of psychosis cases seen” predicted the level of agreement to the benchmark formulation. Contrary to the Persons and Bertagnolli (Reference Persons and Bertagnolli1999) study, level of academic qualification did not make a significant contribution to the total formulation score, whereas overall experience did predict the total formulation score. This suggests that the ability to formulate improves with clinical experience, and is not significantly influenced by the level of qualification. This is somewhat at odds with the results of Eells et al. (Reference Eells, Lombart, Kendjelic, Turner and Lucas2005), who noted that it was not “experience” but “expertise” that was associated with better formulation. However, we did not specifically recruit experts, nor did we include a variable measuring expertise, and hence are unable to differentiate the effect of experience vs. expertise. However, the sample we recruited were undertaking training in PSI or CBT and were not targeted as potential experts in these approaches; hence the findings need to be considered against the moderate level of experience of the groups.
It is relevant to note that there was a negative relationship between the “number of psychosis cases seen” and the agreement with the benchmark formulation. In attempting to account for this finding it could be hypothesized that individuals with many years of experience in working with psychosis found it difficult to make the conceptual shift from their existing views to thinking psychologically about the development of psychosis, particularly within a cognitive framework. This is because, in the above sample, the majority (nearly 60%) of individuals who had experience with psychosis were nurse clinicians who had worked with a high number of people with psychotic illness. Presumably, one could predict that we would not see this negative relationship if we were to recruit participants at the end of their PSI training.
The hypothesis that therapists with greater experience in CBT are more able to formulate the inferential aspects of a formulation was supported. This finding is consistent with the idea that making inferences about a client's beliefs (e.g. core beliefs and assumptions) is a skill that develops through specific CBT experience. CBT case experience seems to be related to the reliability to be able to formulate the inferential aspects. This has implications for training, indicating the potential value of using supervised practice to develop clinical experience and greater competence in formulation work.
This study did not support the hypothesis that therapists who had seen more cases of psychosis could more reliably identify the stress-vulnerability components of a formulation. However, the level of power was low (63%), making small effects difficult to detect.
The findings need to be considered in the context of the potential limitations of this and similar research. A stark limitation of this work is that the client was not an active contributor to the process. Whether therapists agree with each other is perhaps less important a question than whether a client and therapist can agree together on the content of a formulation. For clinical, ethical and pragmatic reasons it was not possible to ask the client to meet with the participants to allow them to generate a formulation. However, without the involvement of the client, the resulting formulation will always be limited in its ability to capture the richness of the clinical material. Given this, the actual formulation developed with the client by the therapist (who was CBT trained) was very similar to that produced by the experts. This adds some support for the validity of the expert generated formulation.
A further limitation of this work is that we used a “static” or “snapshot” formulation, which is one in which no information was provided about the client's reaction to the formulation, or of the results of interventions guided by the formulation. In reality, formulation is a dynamic, collaborative, shared, tentative clinical process and one that changes and alters over the course of therapy. For these reasons, it could be argued that there is never a “right” formulation that accounts for all aspects of a person's distress and development. Moreover, they may be “good enough” for the therapeutic task at hand, and at early stages of treatment may not need to be comprehensive and all encompassing. We very much recognize that formulations are always subject to revision in the light of new information. Nevertheless, this does not mean that we cannot examine the process of formulation, but it does mean that we should be appropriately cautious in our interpretation of findings.
Further limitations of this work include the fact that the formulation template had to be derived de novo. There is at present no consensus on the most appropriate formulation model for psychosis. This may have led to a neglect of important disorder specific information. However, the generic model adopted in this study has been used elsewhere when working with psychosis (Chadwick et al., Reference Chadwick, Williams and Mackenzie2003) and is taught on the courses from which the participants were recruited.
There was great variance in the sample in relation to experience in CBT versus psychosis, with only a few people having a great deal of experience in CBT. The sample may not have had the experience of CBT that would enable them to produce coherent and high quality formulations. They would not likely meet criteria for BABCP accreditation which Kuyken et al. (Reference Kuyken, Fothergill, Musa and Chadwick2005) found was a predictor of formulation quality. Whilst an obvious potential limitation for this research, the amount of training and experience required to undertake CBT for Psychosis is unclear. Turkington et al. (Reference Turkington, Kingdon and Turner2002) demonstrated that valuable help was provided by community nurses with 10 days of training in CBT and family work. Hence, the threshold for skills development is not clearly defined. Participants in this study had been on the training courses for several months, and had completed at least 4 days of training in CBT. However, it is the case that the sample size was small for the regression analysis (Tabachnik and Fiddell, Reference Tabachnick and Fiddell1996, p.132). Also, the study primarily considered the rate of agreement using a quantitative approach. It would be interesting to consider whether qualifications or experience impacted upon the quality of the formulations, which may be assessed on criteria such as the degree of parsimony or elegance of the formulations. We have not considered the overall quality of the formulation, or whether the formulations are “good enough”.
Finally, with regards to future research directions there are naturally many questions that arise regarding the objective “truth” of a formulation, and it can be argued that formulation is an idiosyncratic and non replicable process. However, it would seem that the emphasis within cognitive approaches is to base formulations on empirically validated models accounting for the specific features of the presenting problem (Salkovskis, Reference Salkovskis and Salkovskis1996). Whilst the models may have limitations, we should expect some level of consistency if we are basing our understandings on such models. There are also additional features of cognitive therapy that, if implemented effectively, should help increase the usefulness or utility of the formulation. For instance, the collaborative approach in CBT means that the client is closely involved in the process of generation of the formulation, and this acts as a natural check or balance on the process. Also, placing an emphasis on incorporating a person's strengths into a formulation can help emphasize how much the person has coped with effectively, and helps draw attention to the aspects of the person that may serve as a resource in coping with the current difficulties. It will be helpful to consider future research that incorporates collaborative approaches, and an emphasis on strengths in their findings (Kuyken et al., Reference Kuyken, Padesky and Dudley2009).
Acknowledgements
We would like to thank Dr Alison Brabban, Pauline Callcott and Dr Douglas Turkington for providing the expert formulation. We would also like to acknowledge Kevin Gibson who kindly acted in the video. Helen Spencer kindly provided feedback on an earlier draft. We extend our appreciation to the course leaders for allowing us to recruit from their courses. We would like to express our thanks to the participants in the research and to the client who allowed us to use his story in the video.
Comments
No Comments have been published for this article.