Aims

We aimed to identify international differences in home treatment services for mental health problems through a systematic review of studies using Cochrane methodology. We aimed to answer the question, ‘Does the effectiveness of home treatment services vary internationally in terms of reducing days in hospital?’ This analysis was an a priori aim of the review.

Specific hypotheses, focused on Europe compared with North America, were that:

1. (a) North American home treatment studies find greater reductions in days in hospital than European studies (difference between experimental and control arm days in hospital greater for North American than European studies);

2. (b) patients in the experimental arms of North American home treatment studies spend fewer days in hospital than do those in the experimental arms of European studies;

3. (c) North American home treatment studies are more likely than European studies to use in-patient treatment as the control service.

Literature search

Five databases — CINAHL (1982 to October 1999), the Cochrane Schizophrenia Group Register (up to September, 1999), EMBASE (1980 to October 1999), MEDLINE (1966 to December 1999) and PsychLIT (1887 to September 1999) — were searched systematically using terms for mental health problems and home treatment (defined above). Studies of day, foster and residential care were excluded. Studies were included as randomised controlled trials (RCTs) if they met Cochrane standards (Reference Mulrow, Oxman, Clarke and OxmanMulrow & Oxman, 1997). Non-randomised studies (non-RCTs) comparing two or more services were also included. RCTs with flawed randomisation were relegated to the non-randomised study group. Non-RCTs were only included in the meta-analysis if they were prospective and provided evidence for baseline comparability of groups. The outcome measure was days in hospital per patient per month. Full details of the search strategy and methodology for the whole review are reported elsewhere (Reference Catty, Burns and KnappCatty et al, 2002a ).

Follow-up

A questionnaire was sent to each author, asking about 20 possible components of home treatment, derived from a Delphi exercise with leading psychiatrists (Reference Burns, Knapp and CattyBurns et al, 2001). This was to ascertain the components of the experimental service and of the control service, if the latter was described as a community mental health team (CMHT). Missing data were supplemented from the papers where possible. The components were all considered item by item in the subsequent analyses, rather than as a scale score.

In response to the perception that North American studies are more likely than European studies to find significant differences in hospitalisation in favour of the experimental services, we noted the numbers of studies in each location finding significant results and tested these proportions using χ -squared tests. We used days in hospital where they had used this measure; if they had not, we used another hospitalisation measure if reported. This was a crude measure of differences in study findings, reflecting both the number of patients in the study and the size of any differences found. It is intended merely as a description, to shed some light on the common perception.

Meta-analysis

Studies were divided into ‘North American’, ‘European’ and ‘Other’ and their service components, study design and hospitalisation findings compared. In practice, there were insufficient studies from the ‘Other’ locations to include them, so the results focus only on North American and European studies. Studies were designated either in-patient control (where the control service was an initial period of in-patient treatment, with discharge when appropriate) or community control (control service not in-patient treatment).

Studies were only included in the outcome meta-analysis if data were available in the form of mean hospital days. The primary outcome measure was mean monthly hospital days for the entire study. Studies of less than a year's duration were excluded.

Two analytical strategies were used:

1. (a) comparative analysis: this addressed hypothesis (a) by measuring the difference in hospital days between the experimental and control groups for the North American and European studies. The location of the study (Europe/North America) was tested for association with the difference in hospital days, using a weighted unpaired t-test. In-patient-control and community-control studies were analysed separately. The analysis was performed on RCTs initially and then repeated including non-RCTs. It was weighted by the total number of patients in the study with hospital days data.

2. (b) Experimental services analysis: this addressed hypothesis (b) by including only experimental arm data and testing the location of the study for association with the overall days in hospital for experimental patients. This ascertained whether patients in North American studies' experimental services spend fewer days in hospital than patients in European studies' experimental services. Because no control service data were used, all studies could be analysed together, regardless of whether they were inpatient control or community control. This was not a randomised comparison; its validity was based on the methodological rigour of the study data and for this reason, only RCTs were included. We attempted to control for illness severity by adjusting for ‘high service use’ (using studies' inclusion criteria). The analysis was weighted by the number of patients in the experimental arm with hospital days data (Reference Burns, Knapp and CattyBurns et al, 2001).

In the main analyses for the review, key service components were tested for association with the outcome measure, using weighted regression analyses (Reference Catty, Burns and KnappCatty et al, 2002a ). These analyses were not repeated for each location but their results will be used to illuminate the findings of the meta-analyses reported here.

Studies

We found 91 studies, 59 (65%) of which were conducted in North America, 25 (28%) in Europe and 7 (8%) elsewhere (for full details, see Reference Burns, Knapp and CattyBurns et al, 2001). Most North American studies were from the USA (four were Canadian). Of the 25 European studies, 21 were British, 3 were Scandinavian and 1 was German. Four of the other studies were Australian, one was from New Zealand, one was from India and one from China.

There were no significant differences between European and North American studies in the proportion of studies using in-patient treatment as the control service, refuting hypothesis (c). There were also no differences in the control service being a CMHT, or in the year of publication (Table 1).

Table 1

Study characteristics

	North American		European		Other		P 1
All studies, n (%)	n=59		n=25		n=7
RCTs	38	(64)	16	(64)	2	(29)	1.0
In-patient-control	9	(15)	6	(24)	3	(43)	0.36
CMHT control service	20	(34)	7	(28)	2	(29)	0.73
Median year of publication	1993		1995		1995		0.14
All/predominately psychotic	56	(95)	21	(84)	7	(100)	0.19
Author responded	35	(59)	18	(72)	2	(29)	0.33
RCTs included in meta-analyses	n=19		n=9		-	-
In-patient-control	3	(16)	1	(11)	N/A	N/A	0.74
CMHT control service	6	(32)	5	(56)	N/A	N/A	0.67
Median year of publication	1995		1995		N/A	N/A	0.96
All/predominately psychotic	18	(95)	7	(78)	N/A	N/A	0.29
Author responded	13	(68)	9	(100)	N/A	N/A	0.06

Study findings

More North American studies (18 studies: 31%) than European ones (5 studies: 19%) found significant reductions in hospitalisation, but this difference was non-significant (Table 2). It was still non-significant when RCTs and non-randomised studies were treated separately. European studies had larger mean sample sizes (224 compared with 160), but this was non-significant.

Table 2

Studies' reported hospitalisation findings¹

	North American n=58		European n=26		Other n=7
Sample size, mean (s.d.)	160	(145)	224	(292)	172	(173)
Significance favours experimental service, n (%)	18	(31)	5	(19)	4	(57)
Significance favours control service, n (%)	2	(3)	1	(4)	0	(0)
Non-significant, n (%)	24	(41)	10	(38)	2	(29)
Not applicable/significance not reported, n (%)	14	(24)	10	(38)	1	(14)

Follow-up

Authors of 55 studies (60%) responded to follow-up: 18 European (15 UK, 2 Scandinavian and 1 German), 35 North American (33 US and 2 Canadian) and 2 ‘Other’ (both Australian). Responders were more commonly European than were non-responders and less likely to be from ‘Other’ countries, but this was non-significant. The response rate was higher from authors of RCTs (77%).

Service characterisation

The findings below are for all studies for which we had information. For differences in components when only considering those studies in the meta-analyses, see below.

Experimental services

As Table 3 shows, significantly more European teams had occupational therapists. All the North American services had ‘in-service training’, compared with 62% of the European services (Fisher's exact P < 0.001). The mean ‘average’ contact frequency for the North American teams was 2.5 times the European mean contact frequency (P=0.003).

Table 3

Experimental team characteristics¹

	North American		European		P
Staff, n (%)
Multi-disciplinary teams	31	(80)	19	(86)	0.70
Psychiatrist on the team	30	(88)	12	(67)	0.554
Psychiatric nurse on the team	24	(83)	17	(94)	0.43
Occupational therapist on the team	9	(32)	13	(72)	0.002
	[3]	[27]	[5]	[63]	[0.092]
Social worker on the team	26	(86)	11	(61)	0.097
	[12]	[92]	[4]	[50]	[0.047]
Psychologist on the team	7	(18)	7	(35)	0.209
Psychiatrist an integrated member of team (% of services with a psychiatrist)	21	(70)	10	(83)	0.464
In-service training	33	(100)	11	(61)	<0.001
	[14]	[100]	[7]	[88]	[0.36]
Psychiatrist hours per week, mean (s.d.)	18.9	(11.8)	34.6	(43.0)	0.234
Number of staff on team, mean (s.d.)	7.2	(3.2)	9.6	(6.6)	0.540
Patients and contacts, mean (s.d.)
Target number of patients	82.3	(73.6)	117.2	(114.4)	0.314
Target individual case-load size	14.3	(7.7)	18.1	(11.2)	0.271
Maximum contact frequency/month	32.5	(32.8)	17.3	(11.9)	0.245
Average contact frequency/month	11.0	(12.8)	4.5	(4.7)	0.003
	[6.6]	[3.6]	[2.7]	[2.0]	[0.018]
Proportion of visits made at home	55.2	(26.6)	63.1	(22.3)	0.552
Home treatment function, n (%)	33	(97)	16	(94)	0.84
Regularly visit patients at home, n (%)	32	(94)	17	(94)	0.94
Service procedures, n (%)
Responsibility for health and social care	34	(87)	14	(78)	0.442
Crisis element to the team	22	(76)	9	(53)	0.124
Multi-disciplinary review at least weekly	33	(97)	13	(72)	0.022
	[11]	[100]	[5]	[63]	[0.058]
Protocol for meeting carers' needs	13	(41)	3	(18)	0.143
Days per week, mean (s.d.)	5.6	(1.1)	5	(0.5)	0.01
	[6.2]	[1.0]	[5.0]	[0.0]	[0.009]
Hours per weekday, mean (s.d.)	11.4	(5.4)	8.7	(1.5)	0.013
	[13.9]	[7.4]	[8.1]	[0.4]	[0.10]

Table 4

Community Mental Health Team (CMHT) control team characteristics¹

	North American		European		P
Staff, n (%)
Multi-disciplinary teams	10	(83)	6	(86)	0.82
Psychiatrist on the team	11	(55)	7	(100)	0.226
Psychiatric nurse on the team	8	(57)	7	(100)	0.239
Occupational therapist on the team	3	(20)	6	(86)	0.017
	[3]	[100]	[3]	[75]	[0.143]
Social worker on the team	13	(77)	4	(57)	0.255
Psychiatrist integrated member of team (services with psychiatrist)	8	(67)	6	(86)	0.110
In-service training	6	(32)	5	(71)	0.205
Psychiatrist hours per week, mean (s.d.)	19.5	(25.3)	31.4	(32.0)	0.354
Number of staff on team, mean (s.d.)	6.4	(2.4)	9.7	(2.6)	0.033
	[6.3]	[2.5]	[9.0]	[3.6]	[0.24]
Patients and contacts, mean (s.d.)
Target number of patients	98.1	(60.3)	247.8	(199.1)	0.212
Target individual case-load size	33.9	(26.3)	21.5	(15.5)	0.559
Maximum contact frequency/month	21.2	(22.9)	8.4	(7.5)	0.298
Average contact frequency/month	8.0	(10.7)	1.5	(0.4)	0.018
	[3]	[1.0]	[1.5]	[0.5]	[0.68]
Proportion of visits made at home	29.5	(25.7)	48.8	(19.3)	0.185
	n	(%)	n	(%)
Home treatment function, n (%)	9	(53)	7	(100)	0.121
Regularly visit patients at home, n (%)	8	(47)	7	(100)	0.052
	[4]	[80]	[4]	[100]	[1.0]
Service procedures, n (%)
Responsibility for health and social care functions	7	(47)	5	(71)	0.381
Crisis element to the team	8	(50)	2	(29)	0.320
Multi-disciplinary review at least weekly	5	(33)	5	(71)	0.314
Protocol for meeting carers' needs	3	(19)	1	(14)	0.73
Days per week, mean (s.d.)	5.3	(0.8)	5.0	(0)	0.16
Hours per weekday, mean (s.d.)	9.5	(4.4)	8.0	(0)	0.27

Table 5

Meta-analyses: comparative (difference in mean hospital days) and experimental (mean hospital days)

	North American1 (n)	Days per patient per month	European (n)	Days per patient per month	Test statistics
Comparative analysis: mean difference in days (control—experimental)
Randomised controlled trials (RCTs)2	16	0.78	8	-0.303	t=2.79, d.f. 22, P=0.014
RCTs and non-RCTs	18	0.57	9	-0.283	t=0.63, d.f. 25, P=0.0564
Experimental services analysis: mean days5
RCTs only	19	1.57	9	1.75	t=1.93, d.f. 26, P=0.0646

Having a ‘home treatment function’, ‘regularly’ visiting patients at home and the proportion of contacts made at home did not differ between groups. North American services operated significantly longer hours (per weekday) than European ones (11.4 compared with 8.7 hours; U=250.5, P=0.01) and more days, although the difference was small. More North American services had weekly multi-disciplinary review (P=0.022; Table 3).

Control services

Twenty North American services (57% of responders) and seven European (39%) described themselves as CMHTs. The following findings pertain only to them.

More European control services had an occupational therapist (P=0.017). More European control services had in-service training, but this was not statistically significant. North American control services had a significantly higher average contact frequency, with eight contacts per month compared with 1.5 for European control services (U=11.0, P=0.018).

All seven of the responding European control services that were CMHTs visited patients at home, compared with under half of the North American services (Fisher's exact P=0.052; Table 4).

Meta-analyses

Despite the extensive follow-up, only 28 studies could be used in the meta-analyses by location because of insufficient mean hospital days data for a minimum of 1 year's follow-up. Characteristics of these studies are shown in Table 1.

Comparative analysis

There were insufficient in-patient-control studies with available data to test for differences in location (only one out of the four eligible studies was European). For the 24 community-control RCTs for which we had appropriate available data, the results were significantly different for North American compared with European studies. North American RCTs found an overall reduction of 0.8 hospital days per patient per month in favour of the experimental services, whereas the European studies found an increase of 0.3 days. We were unable to put confidence intervals on the within-group overall mean differences because of lack of standard deviation data. The difference between the two locations (1.1 days), however, was significant (t=2.79, d.f. 22, P=0.01). When we included the three community-control non-RCTs for which we had data, the difference between the two locations became smaller and failed to reach significance (Table 5).

Experimental services analysis

This included both in-patient-control and community-control RCTs (28 studies). The difference between North America and Europe in mean days spent in hospital by experimental patients was only 0.2 per patient per month and non-significant (Table 5). The difference between locations became even smaller after adjusting for whether or not the study stated that it was specifically for patients with ‘high service use’.

Service characteristics of meta-analysed studies

To explore possible reasons for the difference in hospital days reduction found between locations, we considered again the service characteristics of the 24 RCTs included in this analysis. For their experimental services, having an occupational therapist on the team, in-service training and hours of operation were no longer significant. Average contact frequency was still significantly different, and weekly multi-disciplinary review approached significance. The days of operation for this group were significantly higher for North American services, and more North American services had a social worker (Table 3).

Only 10 of these RCTs said that their control service had been a CMHT. For these studies, there was no significant difference in having an occupational therapist. The difference in average contact frequency was no longer statistically significant; furthermore, regular visits at home and the number of staff on the team also ceased to be significant, possibly because of the small number of studies tested here.

Limitations

There were very few studies from outside North America and Europe, and for this reason they could not be included. Moreover, US and UK studies dominated the two groups. The meagre number of studies from other areas could have been a result of an English-language bias in our search strategy, although several non-English-language studies were found but did not meet our inclusion criteria (Reference Kluiter, Giel and NienhuisKluiter et al, 1992; Reference Otero and RebolledoOtero & Rebolledo, 1993; Reference De CangasDe Cangas, 1994; Reference Hu, Xiong and FeiHu et al, 1994; Reference Van Minnen, Hoogduin and Broekmanvan Minnen et al, 1997).

The availability of hospital days data limited the analysis. We also attempted to collect service utilisation data to ascertain cost differences (Reference Burns, Knapp and CattyBurns et al, 2001; Reference Catty, Burns and KnappCatty et al, 2002b ), but insufficient data were available.

The controversy about relative effectiveness has focused on assertive community treatment. Although studies of assertive community treatment form a substantial proportion of the studies included and meta-analysed here, our definition of home treatment was deliberately broad and not limited to such treatment. It should also be noted that we could not include in-patient-control studies in the comparative meta-analysis of hospitalisation reduction, and this group of course includes the original Madison study (Reference Stein and TestStein & Test, 1980); they were, however, included in the experimental services analysis. Exclusion of studies from elsewhere than North America and Europe, moreover, meant that the findings of another study that has featured in the controversy (Reference Hoult, Reynolds and Charbonneau PowisHoult et al, 1983) could not be included.

Study design and services

The North American studies were no more likely than the European studies to have used in-patient treatment as the control service, so this could not in itself account for their perceived greater success rate. Despite the perception that North American services work with smaller case-loads, the difference in case-load size between locations was not significant. European CMHT control services were more likely to visit patients at home ‘regularly’, although the difference compared with the North American services just failed to reach significance. This difference would seem in line with the finding of the UK700 study of intensive case management that 76.5% of control service (standard CMHT case management) contacts were delivered in non-service settings (Reference Burns, Fiander and KentBurns et al, 2000).

Perceptions of difference

Although more North American than European studies have demonstrated significant reductions in hospitalisation, this is at least partly because of the relatively low number of European studies conducted at all (25 compared with 59). The difference in proportions here was not significant, although the proportions of studies reporting an advantage to the control service were actually the same. European studies had larger sample sizes than North American studies, which might imply that they had greater power to detect significant differences, but the difference in average sample sizes was non-significant. It could be that a few high-profile studies on either side of the Atlantic have given rise to the impression that North American studies find greater differences. A basic comparison of hospitalisation results does not support this idea.

Reducing hospitalisation

There was clear evidence to support the hypothesis that North American studies are more successful at reducing hospitalisation than are European ones. This finding could have been affected by the heterogeneity of the studies included. North American studies seem, nevertheless, to find a greater difference between experimental and control patients than European ones do.

One explanation offered for this disparity has been that European services have implemented home treatment poorly (Reference TyrerTyrer, 2000). Our analysis of experimental service components yields inconclusive results in this respect. Six components differed between the two locations, of which three (having an occupational therapist, in-service training and hours of operation) became non-significant when we analysed only the 24 RCTs used in the meta-analysis. Average contact frequency, however, remained significantly higher for North American teams, even when only these 24 RCTs were analysed. It is possible that this difference might account for some of the difference in reduction in hospitalisation. The difference between experimental and control North American services in this respect, however, was small; this makes this interpretation less plausible. Having a social worker on the team was significantly more common for the North American teams in the meta-analysis, but this seems unlikely to account for the difference found. Finally, North American teams operated 6-day weeks, on average, compared with the European 5-day week and also compared with the average 5-day weeks of North American control services. This might seem a more plausible explanation. None of these components, however, was associated with reduction in hospital days in our meta-analysis of service components across all studies (Reference Catty, Burns and KnappCatty et al, 2002a ).

European control services: too close to the experimental services?

An alternative suggestion has been that European control services might be more effective than North American ones, that is, closer to the experimental services (Reference Burns, Fahy and ThompsonBurns et al, 1999a ). Our experimental services analysis suggests that this could indeed be the case. This analysis found that patients in North American experimental services spent no fewer days in hospital than those in European services. Differences in hospitalisation policies would have a potential influence on this finding, as well as differences in severity of illness of patients included. We made some adjustment for the latter, but it might not have adjusted fully for differences in severity. Nevertheless, the lack of a difference found between experimental services is consistent with the hypothesis that it is the control services that differ between North America and Europe.

Our analysis of components yields contradictory evidence to support this conclusion. North American control services had significantly higher contact frequencies than European ones across the board: a difference which would seem to be in their favour. This difference was not statistically significant for the studies in the meta-analysis, however, possibly because of the small number of studies. European control services were significantly more likely to visit their patients at home ‘regularly’, although this only approached significance across all studies and was non-significant within the 24 RCTs used in the meta-analysis. Although these findings are thus equivocal, it is possible that the tendency for European control services to visit patients at home could have at least partially accounted for their studies' failure to demonstrate differences in hospitalisation relative to North American studies. ‘Regularly visiting at home’ was associated with reducing hospital days across all the studies (Reference Catty, Burns and KnappCatty et al, 2002a ), so this component could be particularly meaningful.

For the two-thirds of control services that were not CMHTs, we have scant information. For the 24 RCTs meta-analysed, only 10 used CMHTs as their control services. We cannot therefore rule out substantial differences in the control services of these studies between North America and Europe. It is plausible that such differences too could account for the differences in the overall mean reductions in hospital days between North American and European studies.

The conundrum

Although it seems clear that North American studies find greater reductions in hospitalisation than do European ones, determining the cause of this difference is difficult. Our evidence is that patients in North American experimental services were spending no fewer days in hospital than were patients in European services; that is, that experimental and control services differ in North America to a greater extent than in Europe. Our interpretation of this is necessarily tentative, given the difficulties of obtaining full data and methodological limitations. Nevertheless, it seems plausible to suggest that European control services are close to their experimental counter-parts — perhaps particularly in home visiting — and that this, or other unknown control service features, could account for the failure of European studies to replicate the findings of certain North American studies. The implications of this are farreaching and should be an important check on current European service initiatives. Future research is likely to need greater design sophistication and power than that of many of the studies included in this review if it is to yield convincing answers to the questions raised here.