Introduction
Recent developments in cognitive–behavioural therapy (CBT) in psychosis
CBT is recommended for psychosis in many national guidelines (e.g. National Institute of Clinical Excellence, Reference Naughton, Nulty, Abidin, Davoren, O'Dwyer and Kennedy2009). Meta-analyses have demonstrated modest but robust positive effects in blinded CBT trials on psychotic symptoms with small to moderate effect sizes: effect size positive symptoms = 0.29 (Zimmermann et al. Reference Zimmermann, Favrod, Trieu and Pomini2005); effect size positive symptoms = 0.23 (Wykes et al. Reference Wykes, Steel, Everitt and Tarrier2008); effect size positive symptoms = 0.43 (Burns et al. Reference Burns, Erickson and Brenner2014); effect size delusions = 0.24 and hallucinations = 0.46 (van der Gaag et al. Reference van der Gaag, Valmaggia and Smit2014). One meta-analysis produced a non-significant effect size in blinded studies: effect size positive symptoms = 0.08 (Jauhar et al. Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014). Moreover, CBT was superior to any other psychosocial intervention in reducing positive symptoms (Turner et al. Reference Turner, van der Gaag, Karyotaki and Cuijpers2014) and yielded robust results in all sensitivity analyses for risk of bias. It only disappeared in allegiance sensitivity analysis because of lack of power, as only three studies were non-alleged. The focus recently moved to examination of the working mechanisms and cognitive biases. The biases associated with data gathering and the appraisal and processing of information are associated with psychosis in general and, in particular, to positive symptoms such as (persecutory) delusions (van der Gaag, Reference van der Gaag2006; Freeman, Reference Freeman2007). The focus of therapies and training has moved from a predominantly content-oriented focus (what is the patient thinking?) towards a process-oriented focus on cognitive biases (Garety et al. Reference Garety, Kuipers, Fowler, Freeman and Bebbington2001; Morrison, Reference Morrison2001; Bentall et al. Reference Bentall, Rowse, Shryane, Kinderman, Howard, Blackwood, Moore and Corcoran2009; Bennett & Corcoran, Reference Bennett and Corcoran2010).
Cognitive biases associated with delusions
Several cognitive biases, such as jumping to conclusions (JTC), belief inflexibility, problems in theory of mind and externalizing attributions, are hypothesized to be associated with the pathogenesis and maintenance of delusions. The JTC bias refers to a tendency to gather less data or evidence than healthy controls in order to reach a decision or accept a hypothesis (Garety et al. Reference Garety, Hemsley and Wessely1991; Fine et al. Reference Fine, Gardner, Craigie and Gold2007). The JTC bias has also been found in individuals at risk for psychosis (Colbert & Peters, Reference Colbert and Peters2002; Van Dael et al. Reference Van Dael, Versmissen, Janssen, Myin-Germeys, van Os and Krabbendam2006) and in highly deluded and remitted patients (Moritz & Woodward, Reference Moritz and Woodward2005). Belief inflexibility refers to a bias against disconfirmatory evidence (Woodward et al. Reference Woodward, Moritz, Menon and Klinge2008) and is particularly related to delusional preoccupation and conviction (Garety et al. Reference Garety, Freeman, Jolley, Dunn, Bebbington, Fowler, Kuipers and Dudley2005; Colbert et al. Reference Colbert, Peters and Garety2010). Furthermore, problems in theory of mind (i.e. the inability to represent the beliefs, thoughts and intentions of others), which is known to be related to symptoms of disorganization, may also contribute to paranoid delusions (Craig et al. Reference Craig, Hatton, Craig and Bentall2004; Versmissen et al. Reference Versmissen, Janssen, Myin-Germeys, Mengelers, Campo, van Os and Krabbendam2008; Abdel-Hamid et al. Reference Abdel-Hamid, Lehmkamper, Sonntag, Juckel, Daum and Brune2009); however, there is mixed evidence on this topic, with some studies finding no associations (Fernyhough et al. Reference Fernyhough, Jones, Whittle, Waterhouse and Bentall2008) or an intact theory of mind during a delusional state (Walston et al. Reference Walston, Blennerhassett and Charlton2000). Finally, there is evidence that an externalizing attribution style, with patients making external (personal) attributions for negative events and internal attributions for positive events, is associated with delusions (Kaney & Bentall, Reference Kaney and Bentall1989; Kinderman & Bentall, Reference Kinderman and Bentall1997; Janssen et al. Reference Janssen, Versmissen, Campo, Myin-Germeys, van Os and Krabbendam2006). Again, inconsistent findings have been reported, with some studies finding no differences between early psychosis patients and controls in the tendency to externalize or personalize (Langdon et al. Reference Langdon, Still, Connors, Ward and Catts2013) and others concluding that the link between persecutory ideation and attribution biases only manifests when persecutory ideation is of delusional intensity, and that it is confined to only a personalizing bias (McKay et al. Reference McKay, Langdon and Coltheart2007). Generally speaking, the above-mentioned cognitive biases are of interest since they are assumed to mediate (or moderate) treatment response in delusional symptoms (So et al. Reference So, Garety, Peters and Kapur2010).
Metacognitive training in schizophrenia
Moritz & Woodward (Reference Moritz and Woodward2007) were the first to translate theoretical results on cognitive biases and processes into a series of training modules called metacognitive training (MCT). Furthermore, sessions on overconfidence in memory errors and depressive cognitive patterns were added. MCT aims to increase the patient's knowledge about cognitive biases and to raise (metacognitive) awareness of the dysfunctional nature of these biases by means of exercises. It adopts a ‘back-door approach’ by first addressing cognitive biases instead of directly aiming at core delusional beliefs. MCT is group-wise training for 3–10 patients and is comprised of eight different modules targeting cognitive biases. Exercises that demonstrate the fallibility of human cognitive apparatus are discussed in the group. Participants are encouraged to express personal examples of these biases, and discussion of ways to counter them, serving to provide corrective experiences in a supportive atmosphere. This approach has obvious advantages over mere didactic providing of information. Patients are taught to recognize and confront the biases that are important in schizophrenia, thus allowing them to arrive at more appropriate inferences.
The published results on MCT are inconsistent and the evidence for efficacy is still undecided. At the same time there is a widespread dissemination and the MCT modules are available in 33 languages and are used all over the globe. Although the number of studies is relatively small for properly powered meta-analysis (n = 11), we considered that it was necessary to systematically review the current literature and to conduct a meta-analysis on the effects of MCT compared with treatment-as usual or active controls on data-gathering bias, delusions and positive symptoms of psychosis in patients with positive symptoms of schizophrenia.
Method
Data collection
Eligibility criteria
Studies had to meet the following criteria for inclusion: (a) the experimental treatment was MCT (Moritz/Woodward approach); (b) the study had to be a comparative trial with or without randomization; (c) reporting both pre- and post-test measures; (d) any control condition was accepted; (e) at least 75% of the patients were diagnosed with schizophrenia spectrum disorders; (f) only published in peer-reviewed journals (conference abstracts were excluded); and (g) the study used data gathering, delusion ratings and/or positive symptom ratings as an outcome measure. Although there were no language restrictions, all studies were in the English language. The PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines for systematic reviews and meta-analyses were followed (Liberati et al. Reference Liberati, Altman, Tetzlaff, Mulrow, Gotzsche, Ioannidis and Moher2009).
Information sources
Studies were selected by various methods. First, a systematic search was made (from 2002 to 1 July 2014) in Medline, PsycINFO, EMBASE, the Cochrane Central Register of Controlled Trials and the Cochrane Database of Systematic Reviews, to detect studies describing metacognitive therapy or training in patients with positive symptoms of psychosis and schizophrenia.
Articles were identified by combining terms indicative of metacognitive psychological treatment (search terms included ‘metacognitive training’ OR ‘MCT’) AND outcome research (search terms included ‘randomised controlled trial’, ‘randomized controlled trial’ OR ‘RCT) (for the algorithm, please contact the first author; B.v.O.).
Second, the search was supplemented by relevant papers identified by manual search of the reference lists of the identified articles. Finally, leading researchers in the field of CBT and MCT (Professor S. Moritz, Professor P. Garety, Professor D. Freeman and Professor A. Morrison) were asked to make suggestions regarding relevant literature.
Data extraction
The titles of the 611 retrieved papers were screened for eligibility by the first author (B.v.O.). A first selection on the topic of ‘MCT in psychosis’ resulted in 22 potential papers.
A few studies were excluded based on the aforementioned criteria: Aghotor et al. (Reference Aghotor, Pfueller, Moritz, Weisbrod and Roesch-Ely2010) (based on criterion c); Erawati et al. (Reference Erawati, Keliat, Helena and Hamid2014) (based on criterion c); Favrod et al. (Reference Favrod, Maire, Bardy, Pernier and Bonsack2011) (based on criterion b); Ferwerda et al. (Reference Ferwerda, de Boer and van der Gaag2010) (based on criteria b); Moritz & Woodward (Reference Moritz and Woodward2007) (based on criterion g); Moritz et al. (Reference Moritz, Kerstan, Veckenstedt, Randjbar, Vitzthum, Schmidt, Heise and Woodward2011a ) (based on criterion g: no reports on total scores, only subscales).
Finally, 11 studies were included in this meta-analysis (Fig. 1). One study had delusional symptoms as the primary outcome (van Oosterhout et al. Reference van Oosterhout, Krabbendam, de Boer, Ferwerda, van der Helm, Stant and van der Gaag2014), two had positive symptoms as the primary outcome (Naughton et al. 2012; Balzan et al. Reference Balzan, Delfabbro, Galletly and Woodward2014) and five papers dealt with both delusional symptoms and positive symptoms (Kumar et al. Reference Kumar, Zia Ul Haq, Dubey, Dotivala, Veqar Siddiqui, Prakash, Abhishek and Nizamie2010; Moritz et al. Reference Moritz, Veckenstedt, Randjbar, Vitzthum and Woodward2011b ; Briki et al. Reference Briki, Monnin, Haffen, Sechter, Favrod, Netillard, Cheraitia, Marin, Govyadovskaya, Tio, Bonin, Chauvet-Gelinier, Leclerc, Hode, Vidailhet, Berna, Bertschy and Vandel2014; Favrod et al. Reference Favrod, Rexhaj, Bardy, Ferrari, Hayoz, Moritz, Conus and Bonsack2014; Kuokkanen et al. Reference Kuokkanen, Lappalainen, Repo-Tiihonen and Tiihonen2014). One paper dealt with both data-gathering bias and positive symptoms (Rocha & Queiros, Reference Rocha and Queiros2013), one paper dealt with data-gathering bias, positive symptoms and delusional symptoms (Moritz et al. Reference Moritz, Veckenstedt, Bohn, Hottenrott, Scheu, Randjbar, Aghotor, Kother, Woodward, Treszl, Andreou, Pfueller and Roesch-Ely2013), and one paper had data-gathering bias as the primary outcome (Ross et al. Reference Ross, Freeman, Dunn and Garety2011).
Flowchart.
Table 1 lists the characteristics of the 11 selected studies. Studies differed in sample size from 16 (Kumar et al. Reference Kumar, Zia Ul Haq, Dubey, Dotivala, Veqar Siddiqui, Prakash, Abhishek and Nizamie2010) to 150 (Moritz et al. Reference Moritz, Veckenstedt, Bohn, Hottenrott, Scheu, Randjbar, Aghotor, Kother, Woodward, Treszl, Andreou, Pfueller and Roesch-Ely2013). Four trials had in-patients, four had out-patients and three had both. Training was either eight or 16 sessions in most trials, but only short modules of only a maximum 60 min in two trials. Several researchers had made small adaptations to the MCT package. The trial by Moritz et al. (Reference Moritz, Veckenstedt, Randjbar, Vitzthum and Woodward2011b ) embedded the MCT within an individual CBT; Rocha & Queirós (Reference Rocha and Queiros2013) added training in social cognition; Balzan et al. (Reference Balzan, Delfabbro, Galletly and Woodward2014) had a single module focusing on data gathering and bias against disconfirmation, and Ross et al. (Reference Ross, Freeman, Dunn and Garety2011) had a cut-down single module version partially (2/3) based on MCT. So, there was quite some heterogeneity in the training format.
Description of the interventions, patient characteristics, quality of the studies and location
s.d., Standard deviation; CTAM, Clinical Trial Assessment Measure; MCT, meta-cognitive training; n.a., not applicable; TAU, treatment as usual; CBT, cognitive–behavioural therapy; MSCT, MCT plus social cognition training; MCT-JTC, 45-min single session partially based on MCT; MCT-T, 60-min single session focusing on data-gathering bias and confirmation bias modules; SC, supportive counselling.
a Selected outcome was the algorithm of van der Gaag (van der Gaag et al. Reference van der Gaag, Hoffman, Remijsen, Hijman, de Haan, van Meijel, van Harten, Valmaggia, de Hert, Cuijpers and Wiersma2006).
b Patients with scores of 6 or 7 on the Positive and Negative Syndrome Scale paranoia/suspiciousness subscale were excluded.
c Removed from analysis due to attending fewer than eight sessions.
Quality assessment
The methodological quality of the eligible studies was reviewed using the Clinical Trial Assessment Measure (CTAM; Tarrier & Wykes, Reference Tarrier and Wykes2004). Quality ratings were based on the following criteria: sample characteristics (i.e. is the sample a convenience sample or a geographically representative cohort?; sample size); allocation procedures (i.e. valid randomization procedure); assessment of outcomes (i.e. standardized assessment method used); control condition (i.e. has a credible control condition been implemented?); analysis (i.e. appropriate statistical analysis given the design and type of outcome); description of treatment (i.e. has the treatment been sufficiently described or manualized?).
The maximum achievable score on the CTAM is 100. Similar to Wykes et al. (Reference Wykes, Steel, Everitt and Tarrier2008) we adopted an arbitrary cut-off score of 65 to denote either high- or low-quality studies. Two experienced independent raters (S.C. and A.B.P.S.) performed the screenings. A consensus meeting was held to resolve differences in scores and ratings.
Data analysis
Meta-analyses were conducted for the end-of-treatment effects for each of the available outcome measures separately. The outcomes at the end of treatment across the trials were synthesized meta-analytically using Comprehensive Meta-Analysis version 2.2 (www.meta-analysis.com). Post-hoc power analysis for random-effects models in meta-analysis resulted in a detectable small to moderate pooled effect size of 0.37 (Hedges’ g; two-sided, power = 0.80, α = 0.05) in both positive symptoms and delusions.
The studies in this meta-analysis examined different samples and used various control interventions. Therefore, differences between the effect sizes are likely to reflect these sources of heterogeneity. A random-effects model for meta-analytic synthesis of effect sizes across the primary studies was conducted. Most studies used small samples. We decided to use Hedges’ g as the effect size, which is corrected for small sample bias.
Heterogeneity is always a matter of concern in meta-analysis. Therefore, we evaluated whether the variability in the outcomes across the studies could be attributed to random sample error alone, or might be attributed to systematic factors, such as type of intervention. We tested heterogeneity with a χ 2 test and degrees of freedom (df) set at the number of primary studies in the meta-analysis minus one. We also report the I 2 statistic, which is easier to interpret: when I 2 = 0, 25, 50 or 75%, then no, low, moderate or high heterogeneity, respectively, is assumed (Higgins et al. Reference Higgins, Thompson, Deeks and Altman2003).
Meta-analysis may be subject to publication bias. When publication bias was likely, then Duval and Tweedie's trim-and-fill procedure was used; this yields an adjusted estimate of the pooled effect size after publication bias has been taken into account (Duval & Tweedie, Reference Duval and Tweedie2000).
Sensitivity analyses
The inclusion of studies was relatively liberal. High- and low-quality studies with various types of statistical analyses and procedures to correct for unblinding were selected. To examine the effects of study quality, we conducted additional sensitivity analyses in which we successively included high-quality, low-quality and blinded studies, as well as studies using proper intention-to-treat analysis. In order to correct for allegiance bias we used the criteria of the Researcher Allegiance Assessment Tool (Cuijpers et al. Reference Cuijpers, Driessen, Hollon, van Oppen, Barth and Andersson2012) which accounts for the following criteria: (1) only one of the interventions was mentioned in the title; (2) one of the two interventions was explicitly mentioned as the main experimental intervention in the introduction section of the study; (3) one intervention was explicitly described as a control condition included to control for the non-specific components of the other therapy; and (4) there was an explicit hypothesis that one comparison therapy was expected to be more effective than the other. When these criteria were applied none of the studies was non-alleged. Therefore we could not run analyses on allegiance bias.
Results
Characteristics of the included studies
Table 2 presents the results of the primary studies measuring positive symptoms analysis (upper part), delusions analysis (middle part) and data-gathering bias (lower part).
Random effect sizes, heterogeneity and publication bias in the main and sensitivity analyses
CI, Confidence interval; Q, value for heterogeneity tested by χ 2; df, degrees of freedom; I 2, degree of heterogeneity; MOD, moderate heterogeneity; CTAM, Clinical Trial Assessment Measure; n.a., not applicable; LOW, low heterogeneity; HIGH, high heterogeneity; NO, no heterogeneity.
Overall analysis on primary outcome measures
Positive symptoms
Overall analysis of the effects of MCT on positive symptoms
The results on the positive symptoms are presented in Table 2 (upper panel) and Fig. 2. The effect size (g = 0.26) showed a statistical tendency. But correction for publication bias reduced the effect size to non-significant (g = 0.21). Heterogeneity was moderate. Both the high- and low-quality studies were non-significant. Four blinded studies had significant results (g = 0.36), but correction for publication bias reduced the effect size again to non-significant (g = 0.22). If proper intention-to-treat statistics were used (in one study only) the effect size was very small and non-significant (g = 0.10). For funnel plots, see Supplementary Figs S1 and S2.
Forest plot of the effect on positive symptoms of psychosis. CI, Confidence interval; MCT, meta-cognitive training.
Delusions
Overall analysis of the effects of MCT on delusions
The results on the delusions are presented in the middle panel of Table 2 and Fig. 3 and showed a non-significant effect size (g = 0.22), which was further reduced after correction for publication bias (g = 0.03). The level of heterogeneity was moderate. High-quality studies (g = 0.11), blinded studies (g = 0.17) and studies with intention-to-treat statistics (g = −0.02) were all non-significant. Just the low-quality studies showed a significant effect. One-by-one removing of a single trial resulted in significance for the effect of MCT on delusions. This was the case if the van Oosterhout et al. (Reference van Oosterhout, Krabbendam, de Boer, Ferwerda, van der Helm, Stant and van der Gaag2014) trial was removed: a trial with zero findings and high CTAM score compared with the other trials. For funnel plots, see Supplementary Figs S3, S4, S5 and S6.
Forest plot on the effect on delusions. CI, Confidence interval; MCT, meta-cognitive training.
Data-gathering bias
Overall analysis of the effects of MCT on delusions
The results on the data-gathering bias are presented in the lower panel of Table 2 and Fig. 4; results showed a non-significant effect (g = 0.31). The level of heterogeneity was moderate.
Forest plot on data-gathering bias. CI, Confidence interval; MCT, meta-cognitive training.
Conclusions
Main findings
Currently, the evidence of this meta-analysis does not support the efficacy of MCT for any of the outcomes selected. All main analyses on positive symptoms, delusions and data-gathering bias yielded non-significant effect sizes. Corrections for publication biases using the trim-and-fill procedure further reduced the effect sizes. In general, the effect sizes were further reduced in high-quality studies, blinded studies and studies using proper intention-to-treat analysis. The exception was the results of the blinded studies measuring positive symptoms, with results resembling the effects of CBT. Nevertheless, significance disappeared after correction for publication bias.
In almost all analyses there was a moderate to high level of heterogeneity, which makes it difficult to interpret the findings and increases the risk of bias. This raises the question to what extent other (methodological and clinical) trial characteristics may contribute to the effects in the various trials and whether positive or negative effects were exerted on the true effect sizes. Differences in types of patients, levels of delusional symptomatology at baseline, treatment dosage and lack of randomization and blindness are probably causing the heterogeneity in the results of the studies.
In our trial we found no effect on data gathering or on delusions and a non-specific effect on positive symptoms in high-quality studies (which disappeared after correction for publication bias). Regarding the data-gathering bias, only the Rocha trial (Rocha & Queiros, Reference Rocha and Queiros2013) found a significant effect, but also showed a worsening of positive symptoms instead of a reduction. The largest trial, from Moritz et al. (Reference Moritz, Veckenstedt, Bohn, Hottenrott, Scheu, Randjbar, Aghotor, Kother, Woodward, Treszl, Andreou, Pfueller and Roesch-Ely2013), found no effect on data gathering and only small non-significant effects on positive symptoms. Freeman et al. (Reference Freeman, Startup, Dunn, Cernis, Wingham, Pugh, Cordwell, Mander and Kingdon2014) reported that only 24% of delusional patients showed a data-gathering bias and that this was associated with deficits in working memory, lower intelligence quotient, lower levels of tolerance for uncertainty, and lower worry. The data gathering-bias was not associated with psychopathology in that study. Thus, data gathering might better be addressed by retraining working memory rather than by education. Also, the association between data-gathering bias and delusions is not very robust, as some studies found no such associations (Young & Bentall, Reference Young and Bentall1997; McKay et al. Reference McKay, Langdon and Coltheart2007). This raises the question as to what extent data gathering (JTC) and delusions are causally linked, or whether JTC is an epiphenomenon related to psychosis and if making patients aware of cognitive biases and its negative consequences (aim of MCT/back-door approach) is necessary to achieve symptom reduction.
One of the most successful studies was conducted by Moritz et al. (Reference Moritz, Veckenstedt, Randjbar, Vitzthum and Woodward2011b ). This was in fact MCT plus CBT (mentioned as ‘MCT+’). The positive effect might be due to the effective CBT part of the intervention, rather than to the MCT part. The individual study was marginally significant on positive symptoms and showed a tendency on delusions. However, as this was not compared with CBT alone, the addition of MCT to the effective CBT cannot be evaluated at this moment. Furthermore, it was observed that the developers of MCT have found positive results, but so far independent testing by other research groups has not indicated significant change in positive symptoms (with the exception of a small study; Briki et al. Reference Briki, Monnin, Haffen, Sechter, Favrod, Netillard, Cheraitia, Marin, Govyadovskaya, Tio, Bonin, Chauvet-Gelinier, Leclerc, Hode, Vidailhet, Berna, Bertschy and Vandel2014) or in delusions (with the exception of a small study; Favrod et al. Reference Favrod, Rexhaj, Bardy, Ferrari, Hayoz, Moritz, Conus and Bonsack2014). Moreover, in our study we found the latter studies to have relatively low CTAM scores, reflecting lower methodological quality. More recent findings on MCT (Moritz et al. Reference Moritz, Veckenstedt, Andreou, Bohn, Hottenrott, Leighton, Kother, Woodward, Treszl, Menon, Schneider, Pfueller and Roesch-Ely2014) reported consolidation of delusion scores and consolidation of no effect on JTC. In the completer analysis with 40% drop-out the Positive and Negative Syndrome Scale positive symptoms deteriorates in the control, probably due to an increase in hallucinations. The intention-to-treat analysis reported a group effect over all time moments and no group x follow-up interaction. We think these results are hard to interpret. Independent research indicating positive change is necessary for any treatment to be added to evidence-based guidelines for routine care.
Strengths and limitations
The present study has several strengths and limitations. A strength is that separate meta-analyses were conducted on different outcome measures such as data-gathering bias, delusions and positive symptoms.
Another strength was the statistical power to detect small to medium effect sizes. There is little chance of making a type 2 error and incorrectly reject the hypothesis that MCT is efficacious.
At the same time the power is a limitation. Nevertheless, a cumulative analysis on the positive symptom and delusion outcomes showed that the effects stabilized at small and non-significant effect sizes after five trials.
General conclusions
This is the first meta-analysis on MCT. Currently, we can state that the studies do not support a positive effect for MCT on positive symptoms, data gathering and delusions. The methodology of most studies was poor and sensitivity analyses with blinded studies, high-quality studies, and studies that used intention-to-treat analyses reduced the effect sizes even further. Also correction for publication bias reduced the effect sizes considerably. Dissemination of MCT in routine care cannot be recommended at this moment. More rigorous research would be helpful in order to create enough statistical power to detect small effect sizes and to reduce heterogeneity.
Supplementary material
For supplementary material accompanying this paper visit http://dx.doi.org/10.1017/S0033291715001105
Acknowledgements
We kindly thank all fellow researchers for providing us with relevant papers.
Declaration of Interest
None.
