Trial design

Problem-solving therapy concentrates on counteracting impulsivity, defining problems, generating solutions, encouraging consequential thinking and developing means–end action planning (Reference D'Zurilla and NezuD'Zurilla & Nezu, 1999). The intervention studied here is an extension of a particular problem-solving programme (Stop & Think!) that has been shown to produce significant improvements on self-assessed problem-solving ability in a secure setting (McMurran et al, Reference McMurran, Richardson and Egan1999, Reference McMurran, Fyffe and McCarthy2001). In this community setting where attrition rates are considerable (Reference Skodol, Buckley and CharlesSkodol et al, 1983; Reference Gunderson, Frank and RonningstamGunderson et al, 1989), group sessions were preceded by brief individual psychoeducation (Reference Banerjee, Duggan and HubandBanerjee et al, 2006) to inform patients about their diagnoses, to prioritise problems identified by the personality assessment, to clarify links between diagnosis and social problem-solving difficulties, and hence to highlight the relevance of the treatment to follow and encourage engagement.

The study was a pilot, as it was the first time that this therapeutic combination had been tested for personality disorder in the community. The design was shaped by the practical clinical trial model (Reference Hotopf, Churchill and LewisHotopf et al, 1999) while conforming to as many of the CONSORT (Consolidated Standards of Reporting Trials; Reference Begg, Cho and EastwoodBegg et al, 1996) guidelines as possible.

The study was approved by regional and local research ethics committees and conducted across five sites in the East Midlands region of England, encompassing four counties and a mix of urban and rural settings (total catchment area 11 937 km²; total population 2.41 million). Local services were asked to identify potential volunteers, who were then given written information about the trial. All volunteers were offered assessment unless there was previous indication that any of the inclusion criteria would not be met. The inclusion criteria comprised: presence of at least one DSM–IV (American Psychiatric Association, 1994) personality disorder, absence of a major functional psychosis, and age between 18 and 65 years. Literacy and cognitive functioning sufficient to cope with assessment and allow engagement with the intervention was established through discussion with the referrer before accepting a nomination. There was no preferential selection of individuals who appeared highly motivated. All participants provided written informed consent and received no payment for taking part, although travelling expenses were reimbursed.

Diagnosis of DSM–IV personality disorder based on the interview version of the International Personality Disorder Examination (IPDE; World Health Organization, 1995). Interviews were carried out by one of six clinicians who were experienced in working with people with personality disorder and trained in the use of the IPDE. Interrater reliability was checked by one of the authors, who observed 16 randomly selected assessments and scored responses independently. Demographic and historical data were obtained from reviews of participants’ records.

Randomisation

Randomisation was to one of two conditions: intervention, in which participants were offered problem-solving therapy plus psychoeducation in addition to their usual treatment; or waiting-list control, in which participants received only their usual treatment. Randomly permutated blocks based on computer-generated random numbers were provided by an independent statistician. Block size was not revealed to any research or clinical staff. Allocation codes were pre-sealed into identical, sequentially numbered, opaque envelopes which were opened in sequence by research staff with the person responsible for recruitment (the trial coordinator), masked to allocations. A sealed summary of participants’ names and allocations was retained by an impartial custodian until the end of the trial. As an aid to recruitment, all those allocated to the control condition were offered the intervention directly after the corresponding intervention-arm therapy group had concluded.

Delivery of the intervention

Participants initially attended an individual psychoeducation programme, typically three 1-h sessions, where they learned about personality disorder and the nature of their own diagnosis as derived from their IPDE assessment. This was followed by 16 weekly group-based problem-solving sessions, each lasting approximately 2 h. Groups started with no more than eight members, with men and women in separate groups. Depending on individual need and staff availability, additional support sessions were available to some participants on request. These focused solely on progress with problem-solving steps, and were fortnightly or less frequent.

At each site, and for each gender, intervention and waiting-list conditions were managed as a pair. Each treatment group was facilitated by two qualified mental health professionals experienced in working with adults with personality disorder and seconded from their usual posts within the participating National Health Service (NHS) trusts. Of a total of 21 facilitators, 8 were psychologists and 11 were community psychiatric nurses. No facilitator had previous experience with Stop & Think! and each attended a 2-day training course. Therapy adherence was checked by supervision and inspection of facilitators’ logs of group and individual sessions.

Outcome measures

Primary outcome measures were scores on two self-report instruments assessing social problem-solving ability and overall social functioning. The Social Problem Solving Inventory–Revised (SPSI–R; Reference D'Zurilla, Nezu and Maydeu-OlivaresD'Zurilla et al, 2002) has five sub-scales (Positive Problem Orientation, Negative Problem Orientation, Rational Problem Solving, Impulsivity/Carelessness Style and Avoidance Style) and also gives a total score in the range 0 to 20, with higher values indicating greater problem-solving ability. The Social Functioning Questionnaire (SFQ; Reference Tyrer, Nur and CrawfordTyrer et al, 2005) contains eight items relating to difficulties completing tasks, financial problems, problems with close relationships and sex life, relations with relatives, feelings of loneliness and isolation, and enjoyment of spare time. Responders rate the extent to which they have experienced problems in each area over the past 2 weeks. Scores range from 0 to 24, higher values indicating greater social dysfunction.

Secondary outcome measures were scores on four additional self-report instruments measuring anger, impulsiveness, shame and dissociation–the State–Trait Anger Expression Inventory–2 (STAXI–2; Reference SpielbergerSpielberger, 1999), the Barratt Impulsiveness Scale (BIS; Reference Patton, Stanford and BarrattPatton et al, 1995), the Experience of Shame Scale (ESS; Reference Andrews, Qian and ValentineAndrews et al, 2002) and the Dissociative Experiences Scale (DES; Reference Bernstein and PutnamBernstein & Putnam, 1986). All these instruments have well-established validity and reliability.

Participants allocated to the intervention arm were asked to complete all six psychometric measures at baseline (i.e. before commencing group work) and again at end point (defined as the time at which each treatment group concluded). Elapsed time from baseline to end point varied as therapy groups did not always complete their allocated quota of 16 sessions at the same time, and it was judged that assessments should be made only after completion, not at a uniform time. Although the mean period to end point was 24 weeks, this ranged from 21 to 28 weeks, and included some groups that terminated early or started late and were influenced by practical factors such as school and public holidays. Participants allocated to the control arm were asked to complete the SPSI–R and the SFQ measures immediately following randomisation (baseline) and to complete all six measures as they reached the end of their time on the waiting list (end point). Use of services was recorded for each participant, over the period during which the intervention arm received treatment. Details of in-patient admissions, accident and emergency department visits, and contacts with mental health staff were obtained by inspecting hospital records and health service databases.

Calculation of sample size

Assuming equal numbers of participants in intervention and control arms, a total sample size of 128 was calculated for an effect size of 0.50, an alpha level of 0.05 and a power of 0.80 (GPOWER Software; Reference Faul and ErdingerFaul & Erdinger, 1992).

Statistical analysis

Basic statistical analyses were performed using the Statistical Package for the Social Sciences software for Windows (version 12.0). Analyses of covariance (ANCOVAs) and bootstrapping were conducted in R 2.2.1 (R Development Core Team, 2005). A significance criterion of P<0.05 and two-tailed tests were used throughout. Categorical comparisons were made using chi-square tests with Yates's correction. The planned primary outcome analyses were by intention-to-treat, i.e. analysed by randomisation arm irrespective of attendance or treatment compliance. Missing end-point data were replaced using last-observation-carried-forward (LOCF) baseline data on a case-by-case basis where available. No other method of imputation was deployed. Intervention and control arms’ end-point outcome measure scores were compared using ANCOVA with baseline values as the covariate. Where the regression of end point on baseline scores was statistically significantly different by group, we estimated a model incorporating separate slopes and report interaction significance as well as the main effect for randomisation group. For secondary psychometric measures, the two allocation conditions were compared at end point using one-way analysis of variance, or Mann–Whitney tests where score distributions were clearly not Gaussian.

For measures of service use, comparisons were made in two ways: on observed rates over the time period (from randomisation to end point) over which the intervention arm received treatment and changes in rates from baseline reported with bootstrapped 95% confidence intervals; and by comparing mean survival times until next episode, calculated using Kaplan–Meier survival analysis, for accident and emergency department visits and in-patient admissions.

Recruitment and allocation

Referrals were sought between September 2002 and March 2004. Of 464 individuals identified as potentially suitable, 126 (59 men; 67 women) chose not to volunteer, and 21 were screened out for reasons that included evidence of functional psychosis, not understanding that the intervention involved group work, expectation that groups would be run out of hours, and insufficient literacy or cognitive functioning. Of the 317 who volunteered, 255 attended for assessment, of whom 14 did not meet the criteria for personality disorder (Fig. 1). Of the 241 volunteers (117 men; 124 women) meeting the inclusion criteria, 176 entered the trial and were randomly allocated to intervention (n=87) or to waiting-list control (n=89). Those allocated to intervention were offered participation in one of the 13 treatment groups running between March 2003 and December 2004. Rates of recruitment were not uniform between sites, however; in cases where slow recruitment was delaying the allocation process unreasonably, a decision was made on ethical grounds to offer the affected volunteers the intervention without further delay. A total of 65 individuals were assigned to non-randomised treatment for this reason; their outcome is not reported here.

Fig. 1

Participant flow through the trial. Although suitable, 65 volunteers were not randomised because of slow recruitment, leading to unethical delay in offering treatment. PD, personality disorder.

Reliability of IPDE diagnosis

Interviewer–observer agreement from 16 double-rated interviews was calculated on 3×3 tables; Cohen's kappa ranged from 0.69 to 0.88 (mean 0.83, s.d.=0.05). There was no disagreement in the assignment of research diagnosis of personality disorder in any of these cases.

Result of allocation

The two allocation groups appeared well-matched on baseline characteristics, Axis II diagnosis and rates of contact with services in the 6 months before entering the trial (Table 1). However, those in the intervention arm were significantly more likely than the control group to have been admitted to hospital at some time in their life(49.4% v. 33.7%; χ² ₍₁₎=3.85, P=0.049) but were not significantly more likely to have been admitted to psychiatric hospital in the previous 6 months (16.1% v. 7.9%; χ² ₍₁₎=2.10, P=0.147).

Table 1

Baseline characteristics and Axis II diagnoses of participants

Baseline characteristic	Intervention (n=87)	Control (n=89)
Age in years, mean (s.d.)	36.2 (9.69)	36.2 (9.31)
Male	42 (48%)	44 (49%)
Married or living with partner	29 (33%)	21 (24%)
Education beyond secondary level	16 (18%)	15 (17%)
Currently unemployed	69 (80%)	62 (70%)
Criminal conviction, ever	28 (32%)	37 (42%)
Notes record information suggesting1
Alcohol misuse, ever	29 (33%)	31 (35%)
Substance or drug misuse, ever	21 (24%)	24 (27%)
Self-harm, ever	69 (79%)	60 (67%)
Psychiatric hospital admission, ever	43 (49%)	30 (34%)
Compulsory admission, ever	12 (14%)	5 (6%)
Use of services in previous 6 months
A&E visit (any reason), any	25 (29%)	24 (27%)
A&E visit (self-harm), any	10 (12%)	15 (17%)
Psychiatric hospital admission, any	14 (16%)	7 (8%)
Psychiatrist, mean contacts/month (s.d.)	0.21 (0.30)	0.27 (0.44)
Other mental health staff, mean contacts/month (s.d.)	0.63 (1.09)	0.83 (1.81)
Axis II diagnosis2
With one personality disorder	37 (43%)	38 (43%)
With personality disorder in one cluster	46 (53%)	43 (48%)
With personality disorder in two or more clusters	23 (26%)	27 (30%)
Recorded personality disorders (code)2
Paranoid (30 1.0)	7 (8%)	15 (17%)
Schizoid (30 1.20)	1 (1%)	1 (1%)
Schizotypal (30 1.22)	0	1 (1%)
Antisocial (30 1.7)	13 (15%)	11 (12%)
Borderline (30 1.83)	37 (43%)	32 (36%)
Histrionic (30 1.5)	2 (2%)	2 (2%)
Narcissistic (30 1.81)	0	3 (3%)
Avoidant (30 1.82)	40 (46%)	31 (35%)
Dependent (30 1.6)	3 (3%)	5 (6%)
Obsessive—compulsive (30 1.4)	9 (10%)	16 (18%)
Not otherwise specified (30 1.9)	18 (21%)	19 (21%)

We attempted to define ‘treatment as usual’ for those 89 individuals allocated as controls, by examining their records while they remained on the waiting list. Over this period, 42 (47%) had no recorded contact with any mental health professional, although 1 individual was supported by a probation officer. Of 47 who did have contact with mental health services, 32 visited a psychiatrist, 25 received home-based support from a community nurse, 3 attended a day hospital and 10 had at least 2 sessions with a psychologist or substance misuse therapist.

Allocation concealment

In an attempt to test the degree to which the trial coordinator had remained masked to the allocation process, a randomly selected list of 86 names was prepared just before the end of the trial. The coordinator was asked to decide the randomisation of each. When checked against the list held by the impartial custodian, the allocation was correct for 53.5% of cases, indicating that guessing allocation was no better than chance.

Outcome on primary outcome measures

End-point means include LOCF substitution, and the intention-to-treat analysis required 20 SPSI–R and 16 SFQ baseline scores to substitute for missing end-point data. Table 2 shows the group parameters and tests of group differences. The number for the baseline is lower than that for the end point, as some participants did not complete the measures at baseline but agreed to do so at end point. The ANCOVA of SPSI–R data showed a statistically significant difference in the regression of end point scores on baseline scores between the groups (slopes: intervention =0.61; control=0.93; P=0.02), indicating that those who scored lowest on the SPSI–R at baseline in the intervention group improved markedly more than those with similar impairments in the control group, and there was a highly significant simple effect of treatment as well (t=4.4, P<0.001). For SFQ scores, there was no significant difference in slopes between the groups (P=0.62), but there was again a significant simple effect of group (t=1.06, P=0.031). In summary, those allocated to the intervention condition had significantly better social problem-solving skills and significantly higher overall social functioning at end point in comparison with those allocated to the waiting-list control condition (Table 2).

Table 2

Intention-to-treat analysis of psychometric scores: mean (s.d.) unless otherwise specified

	Baseline				End point1
	Intervention group	n	Controls	n	Intervention group1	n	Controls	n	Difference2	95% CI3	Statistic4	P	d5
Social problem-solving (SPSI—R)	7.72 (3.50)	74	6.98 (3.41)	63	9.62 (4.05)	75	7.12 (3.42)	70	2.09	1.21 to 2.97	t=4.4	<0.001	0.56
Social functioning (SFQ)	13.81 (4.41)	74	14.52 (3.83)	64	12.09 (4.54)	74	13.74 (3.78)	72	-1.05	-1.99 to -0.18	t=1.06	0.031	-0.25
Anger expression (STAXI-AX)					45.1 (16.5)	76	51.2 (16.5)	55	-6.1	-15 to -2.0	F=4.34	0.039	-0.37
Impulsiveness (BIS)					75.1 (13.8)	76	75.8 (13.4)	55	-0.7	-8.7 to 1.1	F=0.08	0.774	-0.05
Shame (ESS)					69.8 (95% CI 64.7-74.1) (median 74.0)	76	69.2 (95% CI 64.0-74.5) (median 69.0)	55	0.6	-9.3 to 6.5	Z=-0.26	0.794	-0.03
Dissociation (DES)					23.3 (95% CI 19.1-27.5) (median 16.4)	75	25.7 (95% CI 21.1-30.3) (median 24.0)	54	-2.4	-10.4 to 3.2	Z=-1.12	0.265	-0.13

Correlation between SFQ and SPSI–R scores was moderate and significant (Pearson's r=–0.49, P<0.001) at baseline, and similar at end point. To explore for the possibility that outcome was dependent on geographical site, the ANCOVA was repeated with site as an additional fixed factor (five categories). No significant group-by-site interaction was detected for either SPSI–R scores (F=0.62, P=0.65) or SFQ scores (F=0.88, P=0.48).

This analysis was repeated for each SPSI–R sub-scale. Adjusted mean differences between intervention and control conditions reached statistical significance (P<0.05) for all five scales. Effect size ranged from +0.31 to +0.71.

Outcome on secondary outcome measures

Intention-to-treat comparisons on secondary psychometric outcome measures also are given in Table 2. No significant difference was detected between intervention and control arms on impulsiveness (BIS), dissociation (DES) or shame (ESS) scores. One-way analysis of variance of data from the STAXI–2 instrument indicated that the intervention arm scored significantly lower on overall anger expression (P=0.039) in comparison with the controls.

Outcome on service use measures

Rates of service use were calculated as the mean contacts per 30 days over the time period of interest. Compared with the control group at end point, those in the intervention arm had a greater mean rate of contact with a psychiatrist (0.17 v. 0.15 visits per month) but a lower mean rate of contact with other mental health staff (0.57 v. 0.89 visits per month), with all mental health staff (0.74 v.1.04 visits per month), with accident and emergency departments for self-harm (0.03 v. 0.07 visits per month), with accident and emergency departments for any reason (0.09 v. 0.13 visits per month), and for in-patient admissions (0.02 v. 0.13 visits per month). None of these differences was significant. Mean survival times until next episode were not significantly different between arms for visits to accident and emergency departments (log-rank statistic 0.80, P=0.37) or for in-patient admissions (log-rank statistic 1.51, P=0.22).

Attendance and attrition

Of the 87 individuals randomised to intervention, 11 (13%) never attended (Fig. 1). All but one of those who did attend completed the psychoeducation component. There were 42 (48%) completers who were still attending at the penultimate or final group session, and 33 (38%) non-completers who dropped out of treatment before the penultimate group session; the latter group comprised 18 (21%) who discontinued after fewer than five group sessions and 15 (17%) who attended at least five sessions.

Regular attendance was encouraged, although commitment to attend every group session was not an entry criterion. Participants knew that exclusion would only occur on the third consecutive missed session, which implied a minimum acceptable attendance rate of 6 out of 16 group sessions. Mean number of sessions attended was 12.1 for the intervention overall and 9.1 for group sessions alone. Of the intervention group, 50% were still attending at the 11th group session. We were unable to follow up each individual who discontinued early, but the feedback that was available suggested that some participants discontinued for negative reasons such as not liking anything about the programme (n=2) or feeling conflict with other group members (n=3), some for positive reasons such as starting employment (n=2), commencing dynamic psychotherapy (n=5) or feeling they had gained as much as they could from the programme; and some for neutral reasons such as moving to another area (n=3).

In an attempt to discern factors that might predict attrition, we first calculated the total number of sessions available to each participant, to allow for the fact that 5 of the 13 treatment groups did not run for the full 16 weeks. We then compared those who attended less than 50% of available sessions (n=34) with those who attended at least 50% of sessions (n=53) on Axis II diagnosis, service use history, attendance at the initial assessment and baseline psychometric scores. Members of the high-attrition subgroup were more likely to have a forensic history (χ² ₍₁₎=9.51; P=0.002), to have personality disorder in more than one cluster (χ² ₍₁₎=5.05; P=0.025), to have personality disorder in cluster B (χ² ₍₁₎=3.88, P=0.049), and to have greater impulsivity scores at baseline (t=2.62; P=0.011) than those attending more than 50% of sessions. The poor attenders were also less likely to have attended the first IPDE assessment appointment offered (χ² ₍₁₎=3.70, P=0.054). The two subgroups were not significantly different by site or distance travelled to the group venue.