Introduction
There is increasing evidence for the effectiveness of cognitive behavioural therapy for psychosis (CBTp) as an add-on therapy to pharmacotherapy (Gould et al. Reference Gould, Mueser, Bolton, Mays and Goff2004; Wykes et al. Reference Wykes, Steel, Everitt and Tarrier2008). CBTp has evolved over the years, whereby the initial focus on the content of dysfunctional thinking has broadened to an additional focus on cognitive processes and biases (Garety et al. Reference Garety, Kuipers, Fowler, Freeman and Bebbington2001; Morrison, Reference Morrison2001). Cognitive processes and biases, responsible for distortions in the gathering, appraisal and processing of information, are linked to psychosis in general, and to positive symptoms such as (persecutory) delusions in particular (van der Gaag, Reference van der Gaag2006; Freeman, Reference Freeman2007). The most prominent biases and processes are the jumping to conclusions (JTC) bias (Fine et al. Reference Fine, Gardner, Craigie and Gold2007; So et al. Reference So, Freeman, Dunn, Kapur, Kuipers, Bebbington, Fowler and Garety2012), problems in theory of mind (Brüne, Reference Brüne2005) and false-negative and false-positive errors in memory (Aleman et al. Reference Aleman, Hijman, de Haan and Kahn1999; Moritz et al. Reference Moritz, Woodward, Cuttler, Whitman and Watson2004), together with overconfidence in errors (Moritz et al. Reference Moritz and Woodward2006b ), a bias against disconfirmatory evidence (Moritz et al. Reference Moritz, Woodward, Burlon, Braus and Andresen2006; Woodward et al. Reference Woodward, Moritz and Chen2006a ,Reference Woodward, Moritz, Cuttler and Whitman b ) and biases in attributional style (Bentall et al. Reference Bentall, Kinderman and Kaney1994; Moritz et al. Reference Moritz, Woodward, Burlon, Braus and Andresen2006; Lincoln et al. Reference Lincoln, Mehl, Exner, Lindenmeyer and Rief2010).
Based on this research on cognitive processes and biases, Moritz & Woodward (Reference Moritz and Woodward2007) developed metacognitive training (MCT). MCT is a group training of eight sessions based on two principles. The first principle is knowledge translation: cognitive biases are explained in a comprehensible way and are linked to delusion formation. The second principle is teaching awareness of the possible negative consequences of cognitive biases. The aim is to make patients aware of these biases.
The development of MCT is a good example of translational research in which knowledge about the previously mentioned biases is converted to a teaching module. However, although the assumptions may be valid, the question remains whether MCT is in itself effective. Previous uncontrolled studies showed promising results on delusion scores (Ferwerda et al. Reference Ferwerda, de Boer and van der Gaag2010; Favrod et al. Reference Favrod, Maire, Bardy, Pernier and Bonsack2011) whereas controlled but underpowered studies showed inconclusive results. Relative to an active control, Moritz et al. (Reference Moritz, Veckenstedt, Randjbar, Vitzthum and Woodward2011c ) with 2 × 24 patients found significant results on JTC and positive symptoms of the Positive and Negative Syndrome Scale (PANSS) but not on the Psychotic Symptom Rating Scales (PSYRATS) total score; it should be mentioned that, in their study group, MCT was complemented by individual CBT. Another study by Moritz et al. (Reference Moritz, Kerstan, Veckenstedt, Randjbar, Vitzhum, Schmidt, Heise and Woodward2011a ) with 2 × 18 patients (MCT versus wait-list control) with co-morbid substance disorder found no effects on the PANSS, PSYRATS total score and JTC, although significant change was found on PSYRATS item level (‘intensity of delusional distress’) and changes in JTC approached trend level, both in favour of the MCT group. The trial of Kumar et al. (Reference Kumar, Zia-ul-Haq, Dubey, Dotivala, Siddiqui, Prakash, Abhishek and Nizamie2010) with 2 × 8 patients reported better performance of MCT but the group × time effects were non-significant. Aghotor et al. (Reference Aghotor, Pfueller, Moritz, Weisbrod and Roesch-Ely2010) with 16 versus 14 patients found no effects on PANSS and JTC, although MCT did slightly better, approaching a medium effect size for positive symptoms. Ross et al. (Reference Ross, Freeman, Dunn and Garety2011) modified the JTC modules and improved the didactic and change-inducing characteristics. They tested the efficacy with 2 × 17 patients and detected significant effects on the 60:40 JTC task but not on the 85:15 JTC task; the patients with a severe JTC bias did not change. Overall, the available data are indecisive, mainly because most prior trials were underpowered.
In preparation of this trial we conducted an uncontrolled pilot/feasibility study with patients scoring ⩾68 on the Green Paranoid Thoughts Scale (GPTS; Green et al. Reference Green, Freeman, Kuipers, Bebbington, Fowler, Dunn and Garety2008), which means they were having a paranoid psychotic episode (Ferwerda et al. Reference Ferwerda, de Boer and van der Gaag2010). Florid psychotic symptoms did not disturb the atmosphere in the group. The patients participated and were interested. They evaluated the training as very positive and 93% would recommend the training to others. In the pilot study we found large and significant effects on delusions [Delusions Rating Scale (DRS; Haddock et al. Reference Haddock, McCarron, Tarrier and Faragher1999)], suspicious thoughts and delusions of reference (GPTS) and improved self-reflectiveness [Beck Cognitive Insight Scale (BCIS; Beck et al. Reference Beck, Baruch, Balter, Steer and Warman2004]. This influenced our decision to run a trial with moderately to severely deluded patients (GPTS score 50).
The current study was sufficiently powered to assess relevant differences in efficacy between MCT + treatment as usual (TAU) and TAU alone. In addition, the economic consequences of MCT + TAU were evaluated.
The hypotheses examined in this study were: (1) MCT would reduce delusions compared to TAU; (2) MCT would reduce subjective ideas of social reference and persecutory ideas compared to TAU; (3) MCT would reduce the subjective experience of cognitive biases and dysfunctional metacognitive beliefs compared to TAU; (4) MCT would improve cognitive insight compared to TAU; and (5) MCT would be cost-effective.
Method
Trial design
This study was a multi-centre, single-blind, parallel-group randomized clinical trial conducted in The Netherlands. It was registered in the Dutch Trial Register (NTR 2307). The study was approved by the local ethics committee (NL28883.097.09). Measurements took place at baseline, at 8 weeks at the end of training and at follow-up 24 weeks after baseline.
Participants
Eligible participants were adults aged 18–65 years with a psychotic disorder in the DSM-IV schizophrenia spectrum (APA, 2000). Based on positive results of the pilot study in paranoid patients (Ferwerda et al. Reference Ferwerda, de Boer and van der Gaag2010), in the current trial participants were selected who met the criteria for at least moderate delusional symptoms, that is ideas of social reference and/or persecutory ideas on the GPTS score 50. The diagnosis was established by the Schedules for Clinical Assessment in Neuropsychiatry (SCAN; WHO, 1999). Exclusion criteria were primary addiction, insufficient understanding of the Dutch language and an IQ < 70. The study was conducted at six psychiatric hospitals in The Netherlands between April 2010 and February 2012. Participating hospitals were Reinier van Arkel group (n = 23), GGZ Noord-Holland-Noord (n = 22), Parnassia Psychiatric Institute (n = 55), GGZ Drenthe (n = 16), GGZ Delfland (n = 16) and Yulius (n = 22).
Interventions
In the experimental condition, in addition to TAU, patients received MCT, a group intervention intended for 3–10 patients (Moritz, Reference Moritz2009). Each of eight sessions was conducted either by a clinical psychologist, psychiatrist, occupational therapist or psychiatric nurse. In any case at least one of the trainers was a psychologist with more than 2 years of experience as a clinician treating psychotic patients. Although most contributing therapists were already successful trainers in the pilot study, all of them were trained by an experienced trainer (J.F.; acknowledged as such by S. Moritz, the developer of MCT) during an 8-h training course. During the trial, each trainer attended two or more supervision sessions. MCT comprises eight highly structured modules presented by using powerpoint presentations, and diversion from the correct order of slides and group activities is almost impossible, thereby enforcing treatment adherence and fidelity. Small exercises characterize the modules. Patients practised to counteract cognitive biases such as JTC. The recommended dosage of two parallel sessions in 1 week is suitable for in-patient programmes. However, most of our patients were out-patients, so we decided to have therapy sessions once a week because most out-patients considered two times a week as involving too much effort and travelling.
In the TAU condition, patients received standard treatment for psychotic patients, which consists of medication prescribed by a psychiatrist and/or out-patient treatment by a social psychiatrist nurse and/or psychologist.
Outcomes
The primary outcome was delusions measured with the PSYRATS DRS (hypothesis 1). This instrument is a well-known semi-structured interview that measures qualitative and quantitative aspects of delusions and has good inter-rater and retest reliability. The validity is considered good, as assessed by internal consistency, sensitivity to change and in relation to the PANSS (Drake et al. Reference Drake, Haddock, Tarrier, Bentall and Lewis2007). In this trial, Cronbach's α was 0.83.
The secondary outcome measures were as follows. The GPTS (hypothesis 2), a questionnaire with 32 items on a five-point Likert scale, was used to measure ideas of social reference (part A) and persecutory ideas (part B). The internal consistency of the GPTS is good, with a Cronbach's α > 0.70, and the test is considered valid and sensitive to change.
The subjective experience of cognitive biases and metacognitive beliefs (hypothesis 3) were tested respectively with the Davos Assessment of Cognitive Biases Scale (DACOBS; van der Gaag et al. Reference van der Gaag, Schütz, Ten Napel, Landa, Delespaul, Bak, Tschacher and de Hert2013) and the 30-item version of the Metacognitive Questionnaire (MCQ-30), which follows the metacognitive approach by Wells et al. (Reference Wells and Cartwright-Hatton2004). The DACOBS is a questionnaire that measures the subjective experience of cognitive bias using 42 items on a seven-point Likert scale. In the present study we used the subscale ‘subjective problems in (social) cognition’. The DACOBS is considered a reliable and valid instrument for use in clinical practice and research (Cronbach's α = 0.90; van der Gaag et al. Reference van der Gaag, Schütz, Ten Napel, Landa, Delespaul, Bak, Tschacher and de Hert2013). The MCQ-30 measures metacognitive beliefs on a four-point Likert scale, and distinguishes between cognitive self-confidence, positive views, cognitive self-awareness, uncontrollability and danger and need for control. Its validity and reliability are satisfactory (Cronbach's α = 0.72–0.92; Spada et al. Reference Spada, Mohiyeddini and Wells2008).
The BCIS (Beck et al. Reference Beck, Baruch, Balter, Steer and Warman2004) was used to measure aspects of cognitive insight (hypothesis 4). The BCIS is a 15-item self-report scale measuring two constructs: the ability to acknowledge fallibility (labelled self-reflectiveness) and certainty about belief and judgments (labelled self-certainty). The BCIS has demonstrated good convergent, discriminant and construct validity with in-patients (Beck et al. Reference Beck, Baruch, Balter, Steer and Warman2004) and improvement in cognitive insight and delusional beliefs are correlated (Riggs et al. Reference Riggs, Grant, Perivoliotis and Beck2012).
The EQ-5D is a standardized measure of health status developed by the EuroQoL Group (1990) to provide a simple, generic measure of health for clinical and economic appraisal. The results of the EQ-5D were used to calculate quality-adjusted life years (QALYs), which were included in the cost–utility analysis (hypothesis 5).
A detailed questionnaire on cost aspects (Hakkaart-van Roijen et al. Reference Hakkaart-van Roijen, van Straten, Donker and Tiemens2002) was used to measure health and societal costs. This instrument focused on health care consumption (including hospital admissions, contacts with health care professionals and medication use) and societal aspects (e.g. informal care and productivity losses). In addition, all costs of providing MCT were documented in detail.
Sample size
To detect a medium effect size (power of 0.80 and α = 0.05), a sample size of 64 participants per condition (n = 128) is required. Considering that attrition rates of 15–20% are relatively common, we aimed to include 154 patients.
Randomization and blinding
After providing informed consent, patients were randomly allocated to either MCT + TAU or TAU alone. The random allocation lists were generated by a web-based automated randomization system. The randomization was stratified to a research site in blocks of 10. The allocation list was kept in a remote secure location and the different sites confirmed the randomization status to the randomization bureau. Independent research assistants who were blind to condition conducted the assessments. The assessments were conducted at locations other than the training locations. Assistants were asked to report any unblinding of the assessments.
Statistical analysis
The results were analysed on an intention-to-treat (ITT) basis using SPSS version 19 (SPPS Inc., USA) with linear mixed models (LMMs). The LMM procedure is the recommended method in longitudinal studies, as it uses all available data without deleting subjects with missing data. A cost–utility analysis was also conducted (hypothesis 5). Unit prices per cost type are based on standard Dutch prices for the year 2011. The bootstrap method (Efron & Tibshirani, Reference Efron and Tibshirani1993) was applied to provide information on the uncertainty of the results of the economic evaluation.
Results
Figure 1 shows the flow of participants: 154 patients were randomized and measured at baseline (meansite = 26 participants, range 16–55; meanexp = 8.3 participants, range 6–9). None of the participants were excluded. There were no adverse events and no unblinding was reported.
Flow diagram of the study population and data analysis. MTC, Metacognitive training; TAU, treatment as usual; ITT, intention-to-treat.
At the end of treatment, 128 participants were available for measurement (58 MCT + TAU and 70 TAU), and at follow-up, 111 were available (51 MCT + TAU and 60 TAU). No site differences were found on attrition rates. Table 1 presents baseline data for the study population. Significant differences were found on BCIS self-reflectiveness, MCQ-30 cognitive self-consciousness, MCQ-30 beliefs about uncontrollability and MCQ-30 beliefs about need to control. For each prescribed antipsychotic medication, chlorpromazine equivalences (Woods, Reference Woods2003) were calculated and, at baseline, no differences in medication levels between groups were found (meanexp = 379.5, s.e. = 70.5; meancontrol = 284.0, s.e. = 38.0; t 151 = 1.206, p > 0.05). In general, the level of medication did not differ between groups over time.
Characteristics of the experimental group (MCT + TAU) and the control group (TAU only) at baseline
MCT, Metacognitive training; TAU, treatment as usual; AP, antipsychotic; MS, mood stabilizer; PSYRATS, Psychotic Symptom Rating Scales; GPTS, Green Paranoid Thought Scales; DACOBS, Davos Assessment of Cognitive Biases Scales; BCIS, Beck Cognitive Insight Scale; MCQ-30, 30-item Metacognitions Questionnaire; NOS, not otherwise specified.
Values given as number or mean (standard deviation).
a 0–1: no education to primary education; 2–4: low-to-medium (vocational) education; 5–7: higher education.
* Significant at p < 0.05.
Table 2 shows the means and standard deviations of primary and secondary outcome measures at baseline, at end of treatment and at follow-up, and the group × time interactions (p value). No significant group × time interactions were found in favour of hypotheses 1–4. The only statistically significant finding was that the results on paranoid delusions (GPTS-B) was in favour of the control group. In all cases, after adding all significant differences at baseline as covariates, the results remained non-significant. There was insufficient statistical power to assess site effects. In prior MCT research it has been usual to consider item scores on the DRS as outcome measures. Inspection of the items did not lead to other conclusions.
Data on primary and secondary outcome measures at baseline, and at 8 weeks after end of training (T1) and at follow-up 24 weeks post-baseline (T2)
MCT, Metacognitive training; TAU, treatment as usual; DRS, Delusions Rating Scale; GPTS, Green Paranoid Thought Scales; DACOBS, Davos Assessment of Cognitive Biases Scales; BCIS, Beck Cognitive Insight Scale; MCQ-30, 30-item Metacognitions Questionnaire; NOS, not otherwise specified.
Values given as mean (standard deviation).
Group × time interactions on outcome variables: p values at T1 and at follow-up (intention-to-treat basis).
* Significant at p < 0.05.
In addition, for the primary outcome measure (PSYRATS-DRS), for each protocol, generalized linear model (GLM) analyses were conducted with baseline covariates. There were no significant differences for ‘condition’ at the end of treatment or at follow-up.
Finally, an analysis with only those patients who attended at least six of the eight sessions found no differences from the group who missed three or more sessions.
Economic evaluation
Table 3 shows the various medical and non-medical costs generated by both groups during the 6-month study period. The mean total cost of providing the MCT was €143 per patient, and was largely related to the cost of the group sessions provided by the psychologists. In both groups, the costs of hospital admission, sheltered accommodation, homecare and other informal care had the largest impact on the total amount of societal costs. The mean total societal costs (based on all cost types in Table 3 and patients available for the cost–utility analysis) were estimated to be €13 325 in the MCT + TAU group and €12 827 in the TAU group. Differences in mean total costs were not statistically significant [95% confidence interval (CI) –€4464 to +€5563]. However, these results should be interpreted with caution as the study was powered to demonstrate differences in health outcomes and not in costs (as is the case for most economic evaluations).
Medical and non-medical costs (in euros) during the 6-month study period
MCT, Metacognitive training; TAU, treatment as usual; s.d., standard deviation.
a Percentage of patients using the cost types concerned.
b Consultation Office for Alcohol and Drug Addiction.
Figure 2 presents results of the cost–utility analysis, showing the bootstrap simulations based on the EQ-5D results. Mean costs were slightly higher and QALYs were significantly lower in the MCT+TAU group. About 57% of the bootstrap simulations were located in the northwest quadrant, indicating that TAU dominated MCT+TAU. Sensitivity analyses were conducted to assess the robustness of the current results. These analyses examined completers only, leaving out ‘other informal care’ and adding ‘time costs’ for the MCT + TAU group. The impact of these sensitivity analyses on the overall economic outcomes was very small and did not change any of the results. This provides additional support for the conclusion that MCT + TAU was not cost-effective. An additional cost-effectiveness analysis focusing on the balance between costs and paranoid symptoms (GPTS) showed similar results.
Results of the cost–utility analysis (quality-adjusted life years; QALYs).
Discussion
In the present study, although both groups showed a decrease of symptoms in general, improvement in the experimental group could not be attributed to MCT. Moreover, MCT did not affect subjective experience of cognitive biases, dysfunctional metacognitive beliefs and cognitive insight. As a result, MCT did not prove to be cost-effective.
Similar to other uncontrolled findings (Ferwerda et al. Reference Ferwerda, de Boer and van der Gaag2010; Favrod et al. Reference Favrod, Maire, Bardy, Pernier and Bonsack2011), we found small to medium within-subject effect sizes on symptom reduction in the experimental group (Cohen's d ranging from 0.29 to 0.64). The control group improved even further, reflecting a strong time effect.
In the pilot study (Ferwerda et al. Reference Ferwerda, de Boer and van der Gaag2010), participants with florid psychosis were included and delusions and self-reflectiveness improved after MCT, but the pilot also found that the Beads Task improved only partially: there was only an effect on the easy version and not on the more difficult version. There were no effects on the Hinting Task, either on memory corruption or on self-esteem. Regression to the mean may explain the symptom reduction in MCT and TAU in this trial and in the pilot study, whereas the other measures showed no effect.
Another explanation for the current lack of effect of MCT might be that MCT only uses education and does not personalize the information sufficiently to arouse emotion and shape the conditions for emotional learning. As stated by Beck & Weishaar (Reference Beck, Weishaar, Freeman, Simon, Bentler and Arkowitz1989), it is necessary to arouse personal emotional meaning because otherwise the cognitive constellations underlying affect do not become accessible and modifiable. Awareness about a disorder such as psychosis does not necessarily lead to a decrease in symptoms (Cunningham Owens et al. Reference Cunningham Owens, Carroll, Fattah, Clyde, Coffey and Johnstone2001) or a reduction in relapse (Bechdolf et al. Reference Bechdolf, Knost, Kuntermann, Schiller, Klosterkötter, Hambrecht and Pukrop2004). In the UK National Institute of Clinical Excellence (NICE, 2009) guidelines, there is no evidence for the efficacy of psycho-education in the treatment and management of schizophrenia in primary and secondary care, although these recommendations were partially refuted in later research (Xia et al. Reference Xia, Merinder and Belgamwar2011). Furthermore, homework assignments to improve generalization to daily life were lacking.
A third reason might be that MCT does not affect patients with moderate to severe delusions. The inclusion of these deluded cases did not have a negative influence on the group cohesion, but no effects were found. Encouraged by our pilot study findings (Ferwerda et al. Reference Ferwerda, de Boer and van der Gaag2010), only participants with at least moderate delusional symptoms were included (PSYRATS DRS meanexp = 13.5). Other studies have included only mild delusions (meanexp = 8.71 in Moritz et al. Reference Moritz, Kerstan, Veckenstedt, Randjbar, Vitzhum, Schmidt, Heise and Woodward2011a ; meanexp = 5.50 in Moritz et al. Reference Moritz, Kerstan, Veckenstedt, Randjbar, Vitzhum, Schmidt, Heise and Woodward2011a ). In addition, Ross et al. (Reference Ross, Freeman, Dunn and Garety2011) found that the effect of training on biases was limited to patients with low baseline scores and that more deluded cases did not benefit. Ross and colleagues suggested that a lengthier training package (focusing on generalizing to delusional thinking, which proceeds from the engaging materials to stimuli related to interpersonal judgments and then to materials more directly relevant to the content of delusions, such as interpersonal threat) may have a greater impact on the more extreme JTC reasoning bias, and on belief flexibility and delusional conviction. Moritz et al. (Reference Moritz, Veckenstedt, Randjbar and Vitzhum2011b , Reference Moritz, Veckenstedt, Randjbar, Vitzthum and Woodward c ) have recommended a combination of MCT and CBT to meet these goals, and developed an individualized MCT programme. A pilot trial on this matter was found to be partially successful (Moritz et al. Reference Moritz, Veckenstedt, Randjbar, Vitzthum and Woodward2011c ); positive symptom scores improved on the PANSS but not on the PSYRATS.
The present study has some limitations and strengths that need to be addressed. One limitation is that only the PSYRATS is a rated measure whereas the other measures are self-rated. However, Liraud et al. (Reference Liraud, Droulout, Parrot and Verdoux2004) found that ratings and self-rating of symptoms were highly correlated independent of insight. A second limitation is that no pre-, post- and follow-up measurements of cognitive biases, depression, anxiety or self-esteem were included. Future research should include measures of cognitive biases as these are the focus of MCT and symptom changes are assumed to be secondary to changes in cognitive biases.
A third limitation was the number of patients lost to follow-up. At the end of treatment the drop-out rate was 11%, which is common in psychosocial treatment (Villeneuve et al. Reference Villeneuve, Potvin, Lesage and Nicole2010). However, at follow-up another 17% were lost to follow-up. These findings could not be attributed to the study condition or research site and it is unclear what caused drop-out other than participants withdrawing consent.
A fourth limitation concerns the blinding procedure by which assistants were asked to report unblinding during assessment. Even though no unblindings were reported, a more assertive check on unblinding would have been appropriate. Because self-report and interview-obtained assessments of delusion were very similar, we cautiously assume that unblinding did not affect the results in a dramatic manner.
Finally, this trial was based on the original MCT manual dating from 2007, without the combination with CBT. Whether this addition of CBT will exceed the effects of CBT alone still needs to be established.
The strengths of the study include the randomized design, rigorous randomization procedures, generously formulated inclusion criteria, intention-to-treat analysis, assessment of comprehensive primary and secondary outcomes, well-trained and motivated trainers, and outcomes assessed by researchers blinded to treatment allocation.
The conclusion is that this study does not demonstrate the efficacy of MCT on researcher-rated delusions and self-reported symptoms, subjective experience of cognitive biases and metacognitive beliefs in at least moderately deluded patients. MCT did not prove to be cost-effective.
Acknowledgements
We thank all of the patients who participated, and the trainers, coordinators and research assistants at Reinier van Arkel group, ‘s-Hertogenbosch; GGZ Noord-Holland-Noord, Alkmaar; Parnassia Psychiatric Institute, The Hague; Yulius, Dordrecht; GGZ Delfland, Delft; and GGZ Drenthe, Assen. This work was supported by The Netherlands Organization for Health Research and Development (Zon-Mw), grant no. 80-82305-97-10045.
Declaration of Interest
None.
