Study design

We carried out a single-blind, randomised controlled trial of a new therapy to improve cognition in people with schizophrenia by comparing a group receiving 40 sessions of therapy with a group who received only usual treatment. We tested whether cognitive skills improved in the intervention group and whether this improved cognitive skill led to improvements in symptoms, social functioning and self-esteem. After baseline assessment, participants were randomised to either treatment or control and were then assessed at 14 weeks (post-therapy) and 40 weeks (6 months after therapy discontinuation). The trial registration number is ISRCTN44277627.

Inclusion and exclusion criteria

We recruited participants from local community mental health teams in the South London and Maudsley National Health Service Trust in a structured geographical rotation from February 1999 to December 2002. Patients were included if they had been in contact with the services for at least 1 year, were at least 17 years old, had a diagnosis of schizophrenia based on DSM–IV (American Psychiatric Association, 1994) and evidence of both social functioning, defined as a problem on the Social Behaviour Scale (SBS; Reference Wykes and SturtWykes & Sturt, 1986), and thinking difficulties. Thinking difficulties were defined as a poor memory score on the Rivermead scale (Reference Wilson, Clare and CockburnWilson et al, 1999), and/or cognitive flexibility on the Wisconsin Card Sorting Test (WCST; Reference Heaton, Chelune and TalleyHeaton et al, 1993) below the 16th centile, and/or a poor score on the Hayling Sentence Completion Test (Reference Burgess and ShalliceBurgess & Shallice, 1996).

Therapy

Several programmes are available to test, but the one chosen was first developed in Australia (Reference Delahunty and MoriceDelahunty & Morice, 1993) and incorporates the teaching of strategic information processing, which has been identified as the training more likely to produce larger cognitive benefits following analysis in the most comprehensive review (Reference Krabbendam and AlemanKrabbendam & Aleman, 2003). This is a promising programme because, unlike the others, it has been shown to have specific effects when tested in a randomised controlled trial against another psychosocial programme (Wykes et al, Reference Wykes, Reeder and Corner1999, Reference Wykes, Reeder and Corner2003).

Therapy consisted of 40 face-to-face sessions, each involving a number of paper and pencil tasks that provide practice in a variety of cognitive skills that are set out in a manual (Reference Delahunty, Reeder and WykesDelahunty et al, 2002). Therapy was delivered to individuals on at least 3 days per week until 40 sessions were completed. The therapists were graduate psychologists who had followed a dedicated training programme involving theory, observation and supervised practice sessions. The therapy is based on three general clinical principles:

1. (a) teaching (or facilitating learning of) new efficient information processing strategies;

2. (b) individualising therapy;

3. (c) aiding the transfer of cognitive gains into the real world.

The programme consists of three modules: cognitive flexibility, working memory and planning (Reference Delahunty, Reeder and WykesDelahunty et al, 2002; Reference Reeder, Newton and FrangouReeder et al, 2004). In each module there is a series of tasks, graded from ‘extremely easy’ to ‘easy’, so that an errorless learning environment can be provided. In the cognitive flexibility module, patients are given practice in engagement, disengagement and re-engagement activities for a particular cognitive set or between two sets. The working memory module requires the person to maintain two sets of information simultaneously and to carry out transformations on a held information set. The planning module consists of tasks in which the participant has to plan a sequence of moves to acquire a goal. The emphasis in this module is to organise information and to create and use sub-goals. One major change from the therapy as administered in previous studies (e.g. Reference Wykes, Reeder and CornerWykes et al, 1999) was the emphasis of therapists on the possible uses of the strategies being taught within the participants' own lives, for example in going shopping. This was achieved by encouraging the participants to reflect on how the skills learnt in therapy might be used to achieve real-life goals (see Reference Wykes and ReederWykes & Reeder, 2005 for further details).

Therapist fidelity was checked against the records completed at the end of each session, the task sheets produced during the sessions and by direct observation. Participants did receive therapy that complied with the manual, and the majority of the tasks were delivered for most participants. These high levels of fidelity were maintained and supported by weekly supervision.

Outcome measures

The three main outcome measures were:

1. (a) cognitive flexibility – categories achieved from the Wisconsin Card Sorting Test;

2. (b) planning – the profile score from the Behavioural Assessment of the Dysexecutive Syndrome (BADS; Reference Wilson, Alderman and BurgessWilson et al, 1996);

3. (c) working memory – total raw score on the Digit Span test of the Wechsler Adult Intelligence Scale III (WAIS–III; Reference WechslerWechsler, 1981).

In addition to the main outcomes we also collected data on symptoms from the Positive and Negative Syndrome Scale (PANSS; Reference Kay, Fizbein and OplerKay et al, 1987), a self-esteem score from the Rosenberg Self-Esteem Scale (SES; Reference RosenbergRosenberg, 1965) and level of social functioning from the Social Behaviour Scale (Reference Wykes and SturtWykes & Sturt, 1986). Health, social care and criminal justice system resource use were assessed using the Client Service Receipt Inventory (Reference Beecham, Knapp, Thornicroft, Brewin and WingBeecham & Knapp, 1992) retrospectively from healthcare staff or records, and/or by participant self-report for the relevant assessment intervals. Unit costs (at 2000–2001 levels) based on national statistics were attached to all resource use to calculate total health and societal costs.

Procedure

All participants gave written informed consent prior to inclusion in the trial. After baseline assessment participants were randomly allocated by an independent statistician using a concealed randomisation method. Participants assigned to the cognitive remediation condition received therapy within 2 weeks of randomisation. Therapy continued for 40 sessions (approximately 12 weeks). In addition to the assessments on outcome measures, data were also collected on clinical history, demographic characteristics and premorbid IQ as assessed on the National Adult Reading Test (NART; Reference Nelson and WillisonNelson & Willison, 1991).

Sample size and power of the study

Previous studies of this programme have suggested that there will be improvement in both groups with repeated testing over time. We have therefore used the outcome data reported by Wykes et al (Reference Wykes, Reeder and Corner1999) to define a clinically significant difference as 71% of the experimental group improving compared with 31% of the control group. This difference is considered to be a clinically significant difference in proportions considering the amount of therapy time that would need to be allocated. This is an odds ratio of 0.184. We estimated that a sample of 29 people per group would have 80% power at the 5% significance level to detect this difference. The sample size was increased to 42 to take into account a possible 30% withdrawal rate.

Statistical analyses

Participants were analysed in the treatment group to which they were randomised irrespective of whether they adhered to their treatment. All outcome measures were analysed using linear mixed modelling with models fitted using restricted maximum likelihood methods based on the assumption of normality for the error terms. Models included baseline values of the outcome measure, and symptoms considered possibly to affect cognitive outcome following therapy as explanatory variables. The experimental factors, randomisation group (therapy or control) and time (post-treatment or follow-up) were included in the model as fixed main effects and a group × time interaction. In addition, random effects for participants were included. A significant interaction term implies a differential intervention effect at the two post-therapy time points. Where the interaction effect was not significant, the corresponding model was refitted excluding this term, to assess the overall group effect. A main effect of randomisation group would then be interpreted as an effect of the intervention therapy consistent across both time points.

We also chose to investigate whether the effects of treatment meant that cognitive scores were then within the normal range. This differential improvement rate was tested by chi-squared tests for each main outcome measure, by investigating the changes to normal scores of all those who had abnormal scores at baseline.

Outcomes of therapy

Table 3 shows the group comparison results for all the outcomes. All statistically significant changes show an advantage for cognitive remediation. Working memory shows an improvement across both post-treatment time points and cognitive flexibility an improvement at the follow-up time point. Both differences are small to moderate effects. For working memory change the number needed to treat (NNT) is 3.1 to produce a clinical change of at least 2 points on the Digit Span test. For cognitive flexibility the NNT is 6.7 to improve by at least two categories on the WCST at follow-up.

Table 3

Mixed effects models analysis

Outcome measures	Interaction	Group effect (excluding non-significant interaction)	Estimated advantage to CRT 1 (95% CI)	Standardised effect size 2 (95% CI)
Primary outcomes
Working memory (Digit Span)	F (1,72)=0.21, P=0.84	F (1,71)=5.82, P=0.019	1.33 (0.43 to 2.16)	0.34 (0.1 to 0.55)
Cognitive flexibility (WCST)	F (1,61)=6.924, P=0.011	NA
Post-treatment			0.17 (–0.64 to 0.98)
Follow-up			1 (0.17 to 8.0)	0.47 (0.08 to 3.77)
Planning (BADS)	F (1,67)=0.315, P=0.576	F (1,72)=2.9, P=0.092	1.1 (–0.18 to 2.4)
Secondary outcomes
Self-esteem (SES)	F (1,71)=4.44, P=0.039	NA
Post-treatment			1.05 (–0.3 to 2.42)
Follow-up			–0.57 (–1.99 to 0.85)
Symptoms (PANSS)	F (1,72)=3.55, P=0.06	NA
Post-treatment			–4.68 (–10.81 to 1.44)
Follow-up			1.59 (–4.8 to 8.04)
Social functioning	F (1,76)=0.438, P=0.51	F (1,74)=0.008, P=0.929

BADS, Behavioural Assessment of Dysexecutive Syndrome; CRT, cognitive remediation therapy; NA, not applicable; PANSS, Positive and Negative Syndrome Scale; SES, Self-Esteem Scale; WCST, Wisconsin Card Sorting Test

1. Number of points on outcome measure scale

2. Effect/baseline s.d.

Using the same mixed models analyses, drug effects were investigated using drug type as an additional explanatory variable. Neither working memory nor WCST outcomes were related to the type of medication prescribed (defined as either typical v. atypical or when depot preparations were considered). However, there was a significant drug × group interaction (F _(2,70)=4.4, P=0.016) for planning scores. Further investigation of these effects suggests that cognitive remediation therapy had an overall effect on planning for those who received either clozapine or typical medication that was absent for those who received other atypical medications.

Normal score attainment following therapy

The normal range for the main outcome tests was conservatively estimated from the test manuals. For working memory this was within 1 standard deviation of the mean normal score, for cognitive flexibility it was above the 5th percentile and for planning it was above the low average score.

For working memory there were 21 participants in the therapy group and 18 in the control group who had abnormal working memory scores at baseline. Following the intervention, there was an advantage to therapy which was significant at the post-therapy assessment but failed to reach significance at follow-up (post-treatment: 43 therapy v. 11% control, Fisher's exact test P=0.037; follow-up: 32 therapy v. 7% control, Fisher's exact test P=0.10). For cognitive flexibility there was no difference at either the post-treatment or the follow-up assessment (post-treatment: 15 therapy v. 17% control; follow-up: 17 therapy v. 21% control). For planning, although almost double the number of people in the therapy group had a normalised score, there was no statistically significant effect (post-treatment: 32 therapy v. 17% control; follow-up: 36 therapy v. 19% control).

Other outcomes

These analyses are designed to detect whether there is a direct effect of cognitive remediation therapy on functioning outcomes, irrespective of the level of cognitive improvement detected. The results of the analyses are shown in Table 3. For both symptoms and self-esteem the results were in the expected direction, with the therapy group improving compared with the control group at post-treatment. There was evidence of an interaction such that any differential improvement at the post-treatment assessment disappeared at follow-up. However, there was no evidence of a direct effect of therapy on social behaviour scores.

Economic outcomes

Total overall health and societal costs are shown in Table 4. There is an advantage (although with highly skewed confidence limits) for the treatment period, with a difference of UK £1086 in healthcare costs and £1284 in societal costs in favour of the therapy group, but costs are higher at follow-up. The intervention dominates at the post-treatment evaluation with both lower costs and a greater proportion of participants showing cognitive improvement (mean difference 21%, 95% CI 0–41). At follow-up, the cognitive advantage of the intervention therapy (mean difference 21%, 95% CI 2–41) needs to be considered against the additional costs. The cost-effectiveness ratios showed that for each additional 1% of ‘improvers’ in Digit Span (see Method) there was an additional cost of £46 in healthcare and social care costs and £24 in societal costs.

Table 4

Baseline-adjusted mean costs and mean cost differences, including the cost of CRT

	CRT			Usual care			Mean difference 1
	n	Cost, £ Mean (s.d.)	n	Cost, £ Mean (s.d.)	£	(95% CI)
Post-treatment
Health/social care costs	41	7756 (5936)	39	8271 (7494)	–1086 (–3146 to 1152)
Societal costs	41	8868 (5849)	39	9497 (7413)	–1284 (–3348 to 942)
Follow-up
Health/social care costs	41	15 639 (12 453)	37	13 426 (12 852)	975 (–3330 to 5255)
Societal costs	41	17 586 (12 197)	37	15 735 (12 654)	494 (–3564 to 4577)

CRT, cognitive remediation therapy

1. Adjusted for the baseline values of equivalent cost categories and baseline total Positive and Negative Syndrome Scale score

Effects of cognitive improvement

Since group could be shown to affect working memory, the effect of working memory change over the treatment period on social functioning outcome to follow-up was investigated using regression. There was no evidence of a group-dependent effect of cognition (F=0.996, d.f.=1.65, P=0.32), but after excluding the interaction term there was a significant effect of cognitive change (F=4.78, d.f.=1,66, P=0.03), suggesting that improvements in cognition were associated with improvements in social behaviour. When the effects were investigated in each group independently there was a significant effect for the therapy group (F _(1,35)=9.2, P=0.005) but not for the control group (F _(1,34)=0.06, P=0.16).

Methodological considerations

Data from other studies demonstrate high levels of stability of cognitive functioning without an intervention (Reference Greig, Nicholls and WexlerGreig et al, 2004) and it might be proposed that these cognitive effects result only from attention and increased social contact. However, they are similar to those found in a previous trial in which there was an attention control group (Wykes et al, Reference Wykes, Reeder and Corner1999, Reference Wykes, Reeder and Corner2003), and although levels of social contact were higher in the therapy group there was no direct effect on social functioning outcomes. It therefore seems parsimonious to assume that the intervention therapy did produce the beneficial cognitive effects.

It is also vital to consider any possible challenges to the study validity, which for studies of psychological interventions in particular include rater bias. Although procedures were in place to reduce the chance of unmasking group allocation this did happen in some cases when cognitive data were collected. However, data quality checks (double scoring, data entry checking, etc.) were carried out each month to ensure that data were not obviously biased for any one rater or for any one participant. It was not possible to mask group allocation for the social functioning informants, but in this case there was no measurable effect of therapy, only interactions with cognitive improvements that were unknown to the key informants when they were providing the relevant information. It therefore seems likely that these acknowledged methodological difficulties did not compromise the study validity.

The sample size was small (although larger than any prior studies of this therapy) and so the power to investigate subtle effects was low. However, if this therapy is to be provided by health services it needs to show at least moderate effects and the current effect sizes are similar to those attributed to cognitive–behavioural therapy (Reference Tarrier and WykesTarrier & Wykes, 2004) suggesting that they may have some clinical relevance.

The effects of improving working memory

Poor memory has been highlighted in a number of studies as predicting poor overall outcome (Reference Mueser, Bellack and DouglasMueser et al, 1991; Reference Green, Kern and BraffGreen et al, 2000). It was assumed that cognitive improvements would lead to functional change, and this is one of the reasons that cognitive remediation therapy was developed. However, few studies have measured the functional effects at a time when it might be possible for cognitive improvement to have had time to translate into functional changes. In this study there was support for a model in which a change in working memory had a beneficial effect on social behaviour 6 months after the end of therapy. This effect was only significant in the therapy group. The intervention was associated with lower costs at post-treatment assessment, and even when there were higher costs at follow-up these were small in relation to the beneficial outcome in working memory. The therapy itself has been estimated to cost £546.97 at 2000–2001 prices per patient (Reference Wykes, Reeder and CornerWykes et al, 2003). The cost-effectiveness analysis has demonstrated that this translates into a small price to pay for memory improvements which are likely to produce further benefits on social behaviour.

The mechanisms through which cognitive change leads to functioning change have been somewhat elusive. Wykes & Reeder (Reference Wykes and Reeder2005) have suggested that for change to occur in routine behaviours, cognitive capacity of the sort measured by neuropsychological tests must change. This increases the efficiency with which routine cognitive schemas are implemented or held online. However, most functioning is not routine and must be flexible, so for novel behaviours it is necessary to change a further aspect of thinking – metacognition. Metacognitive processes and knowledge are required for the replacement of inefficient existing routine schemas and for the production of new, temporary high-level schemas. Current measures of cognition do not allow the separation of cognitive from metacognitive processes, and current measures of functioning also do not allow the differentiation of routine from novel actions. It is possible that the incomplete translation of working memory improvements into social behaviour change is explained by the fact that it is not currently possible to differentiate the two forms of action. Working memory improvements are likely to translate into routine actions immediately, but not necessarily directly to non-routine actions unless metacognition has also improved. Future studies need to be able to differentiate behaviour change into routine behaviour efficiency and novel behaviour flexibility.