Outcome measures

The principal outcome measure of depression was the 21-item Beck Depression Inventory (BDI; Reference Beck, Ward and MendelsonBeck et al, 1961), and the results are based upon this instrument. We also included MADRS–S (9 items), the 21-item Beck Anxiety Inventory (BAI; Reference Beck, Epstein and BrownBeck et al, 1988) and the Quality of Life Inventory (QoLI; Reference Frisch, Cornell and VillanuevaFrisch et al, 1992). The QoLI includes 16 dimensions of life (e.g. health, economy); for each dimension a rating is made regarding importance (scored 0 to 2) and of how pleased the person is with that dimension (scored –3 to +3, but with no 0 alternative). The QoLI has been reported to have satisfactory reliability and validity (Reference Frisch, Cornell and VillanuevaFrisch et al, 1992). All outcome measures were administered using the internet.

Treatment conditions

The cognitive–behavioural self-help treatment was based on Beck's cognitive therapy, as presented in numerous sources (e.g. Reference BurnsBurns, 1999), and on behavioural activation (Reference Lewinsohn, Munoz and YoungrenLewinsohn et al, 1986; Reference Martell, Addis and JacobsonMartell et al, 2001). The material (presented in Swedish) consisted of 89 pages of text, divided into five modules: introduction; behavioural activation; cognitive restructuring; sleep and physical health; and relapse prevention and future goals. The sleep module was based on a programme for insomnia (Reference Ström, Pettersson and AnderssonStröm et al, 2004). Each module ended with a quiz, with questions on the content of the module. Responses were automatically sent to the therapist, who in turn gave e-mail feedback on the answers and gave the participant access to the next treatment module within 24 h. Each module was available on the website in hypertext markup language (HTML) format. The website was built by JavaServer Pages (JSP) programming and MySQL databases. The participant could also print each module by first downloading them as rich text format or as portable document format documents (PDFs). The amount of time advised for completion of all five modules was 8 weeks. However, the mean time for completion was 10 weeks. The time spent on each participant for completion of treatment was estimated to be 2 h in total, including screening, responding to e-mails and monitoring the discussion group. In total 506 messages were sent to the participants, which included a few e-mails sent to the control group.

The discussion groups were separate and differed in their content, since the groups had different topics to discuss. For example, the treatment group could discuss the contents of the self-help material, whereas the control group was more likely to bring up topics such as sick leave and the experience of being depressed. All activity in the discussion groups was closely monitored, with the possibility of deleting inappropriate postings. However, this never occurred. In addition, the therapists in the study answered some of the questions posed by members of the discussion groups when appropriate, for example questions regarding the website.

Each time a participant in either group logged on to the website, the MADRS–S was automatically administered, with the restriction that at least 7 days had to have passed since the previous form was completed. This was done in order to monitor depression levels – and in particular zest for life – on a regular weekly basis.

Follow-up

For ethical reasons the control group members were given access to the treatment modules after the intervention group had finished their treatment. Participants were contacted by e-mail and asked to fill in the questionnaires again on the internet 6 months after the treatment had ended.

Analysis

All randomised participants with follow-up data were included in the analyses regardless of how many treatment modules they had completed. This could also be referred to as ‘intention to treat’, as we included all those who provided post-treatment data. However, for the main outcome measure we also calculated results on a last observation carried forward basis, replacing missing values post-treatment with pre-treatment values. Since this procedure assumes that values remain frozen in time, we did not report this for all measures. Confidence intervals, analysis of variance and t-tests were used for outcome analyses. Significance was set at 0.05 and all tests were two-tailed. Power was estimated by assuming an effect size (Cohen's d, defined as the standardised difference between groups obtained by calculating the mean difference and dividing by their pooled standard deviation) of 0.80, which would require 52 participants to obtain a power of 80% with a conventional alpha level of 0.05. The actual power for the main outcome measure with 85 participants was over 95%.

Outcome on self-report measures

Table 2 shows results on the outcome measures, including change scores with 95% confidence intervals. Analyses of variance with a 2×2 design (one group factor and one repeated-measures factor) resulted in significant interactions for the BDI (F _(1,83)=14.22; P<0.001), MADRS–S (F(_1,83)=7.77; P=0.007) and BAI (F _(1,83)=5.72; P=0.019). These interactions reflect differences in change scores between the active treatment and the control condition. The corresponding effect sizes (Cohen's d between groups at post-treatment) were 0.94 for the BDI, 0.79 for the MADRS–S and 0.47 for the BAI. There was no statistically significant interaction on the QoLI (mirrored by a low effect size of 0.32). In order to check for potential confounding by medication status pre-treatment, medication status was entered as a between-group factor in the analysis. This did not affect the outcome (e.g. no significant main effect of interaction with medication status), but we acknowledge that testing for medication interaction effects in this study is unreliable, given the small sample size.

Table 2

Self-reported outcomes: pre-treatment, post-treatment and at 6-month follow-up

Scale	n	Pre-treatment score Mean (s.d.)	Post-treatment score Mean (s.d.)	Pre—post difference Mean (95% CI)	n	Follow-up score1 Mean (s.d.)	Pre-treatment to follow-up difference Mean (95% CI)
BDI (range 0-63)
Treatment	36	20.5 (6.7)	12.2 (6.8)	8.3 (5.7 to 10.9)	36	13.1 (9.1)	7.8 (4.6 to 11.3)
Control	49	20.9 (8.5)	19.5 (8.1)	1.4 (-1.1 to 3.9)	35	13.1 (7.6)	7.4 (4.0 to 10.7)
MADRS—S (range 0-54)
Treatment	36	20.1 (5.7)	12.7 (8.3)	5.5 (4.6 to 10.1)	36	14.6 (9.2)	6.3 (3.2 to 9.3)
Control	49	21.6 (7.2)	19.0 (7.6)	2.6 (-0.4 to 4.8)	35	14.5 (9.3)	6.8 (3.9 to 9.7)
BAI (range 0-63)
Treatment	36	17.1 (8.2)	14.1 (8.4)	3.1 (1.2 to 4.9)	36	15.1 (9.3)	2.8 (0.3 to 5.4)
Control	49	17.6 (8.5)	17.8 (9.4)	-0.2 (-2.2 to 1.7)	35	12.8 (8.4)	4.6 (2.1 to 7.2)
QoLI (range -6 to 6)
Treatment	36	-0.1 (1.1)	0.5 (1.6)	0.6 (0.2 to 1.1)	36	0.7 (1.7)	0.9 (0.4 to 1.4)
Control	49	-0.2 (1.6)	0.0 (1.5)	0.2 (-0.2 to 0.6)	35	0.9 (1.8)	1.0 (0.5 to 1.4)

Further analysis of the BDI data, replacing missing values post-treatment with pre-treatment values, also resulted in a significant improvement, with a mean reduction in score of 5.2 (95% CI 3.2–7.1) in the treatment group and 1.5 (95% CI –0.9 to 3.2) in the control group post-treatment. The same analysis of the follow-up data (bringing last observation forward for missing data) showed a mean pre-treatment to follow-up reduction in score of 7.2 (95% CI 4.4–10.5) in the treatment group and 5.2 (95% CI 2.5–7.9) in the control group. Hence, replacing missing values with the last observation available for the full sample of 117 participants did not alter the results on the main outcome measure.

Adherence and modules completed

Participants in the intervention group normally reached at least the fourth module, with 65% completing all modules. The average number of modules completed was 3.7 (s.d.=1.9). The number of modules completed was weakly correlated with post-treatment BDI scores (Spearman's r=–0.33, P<0.05).

Activity in discussion groups

Activity in the discussion group was not correlated with improvement in the treatment group. However, there was a marked difference in activity between the discussion groups, with a total of 233 postings in the treatment discussion group and 842 postings in the control discussion group, which was also reflected in the mean difference between the groups of –11.0 (95% CI) –21.5 to –0.6. Overall, the form of activity differed between the groups, as the control group tended to discuss their own problems more, whereas the treatment group leaned more towards discussing the treatment.

Follow-up

At the 6-month follow-up, 71 participants (all in the treatment group and 35 in the control group) completed the questionnaires again, yielding a 16% rate of withdrawal from post-treatment to follow-up (0% in the treatment group and 29% in the control group). Table 2 shows the outcomes at follow-up for each group separately. At this stage the control group also had received the treatment.

Analysis of the difference between the groups at follow-up revealed no statistically significant difference. Improvements between pre-treatment and follow-up were, however, found for both groups on the BDI, MADRS–S, BAI and the QoLI (see Table 2 for change scores and confidence intervals). As the control group had received treatment, we expected changes between post-treatment and follow-up for this group, but no difference for the treatment group. This assumption was confirmed by means of paired t-tests for the BDI, MADRS–S, BAI and QoLI (all P values were less than 0.05 in the control group and more than 0.05 in the treatment group).

Adherence

Although self-administered treatments have shown promising results in many studies, a crucial problem is how well participants adhere to the treatment. For example, in a recent study only 41 out of 139 randomised participants were available for assessment at the 3-month follow-up (Reference Richards, Barkham and CahillRichards et al, 2003), which makes it likely that some failed to complete the self-help material. Internet-based self-help facilitates monitoring of adherence to treatment, because modules are provided only when the previous module has been completed. There was, however, a differential rate of withdrawal between the two groups, and judging from the comments we received, some perceived the text and the exercise as too demanding. A solution to this is to adjust the text, and to allow a longer treatment period.

Implications for cognitive–behavioural therapy

Self-help treatment of depression is an attractive treatment option, as practitioners often wish to offer their clients effective psychosocial interventions, but hesitate to do so because of lengthy waiting lists (Reference Williams and WhitfieldWilliams & Whitfield, 2001). Indeed, developing self-help approaches has been recommended several times (Reference Hollon, Munoz and BarlowHollon et al, 2002). Our study was preceded by other applications of internet-based self-help treatments (Reference Carlbring, Westling and LjungstrandCarlbring et al, 2001), and differs from other applications of internet-based treatments of depression (Reference Clarke, Reid and EubanksClarke et al, 2002). First, we divided the material into modules to be provided on a consecutive basis dependent on progress. Second, individualised feedback was given by a therapist who was clearly identified with a name and a photograph on the website. In a recent review it was concluded that self-help results in effect sizes roughly equivalent to the average effect size obtained in psychotherapy studies (Reference McKendree-Smith, Floyd and ScoginMcKendree-Smith et al, 2003). In common with our study, most self-help studies on depression would be better described as testing minimal therapist contact treatments, as it is common to have either meetings or telephone calls to monitor progress and adherence. Internet-based self-help does not therefore exclude clinician input and can be demanding for the therapist. However, given that responses are not given directly in ‘real time’, colleagues can be consulted and specific questions can be directed to the specialist, all being done within 24 h. For example, in our study the psychiatrist was consulted about some of the participants’ questions, whereas the psychologists handled other questions dealing with the contents of the programme. It is, however, interesting to compare our findings with the results of Proudfoot et al (Reference Proudfoot, Goldberg and Everitt2003), who used a stand-alone computer in a general practice setting, and Christensen et al (Reference Christensen, Griffiths and Jorm2004), who used an open web page, both finding promising results.

Interestingly, participation in the discussion group only did not confer any immediate benefits; this is in contrast to an observational study in which benefits were found (Reference Houston, Cooper and FordHouston et al, 2002). Findings in the latter study were attributed to the effects of breaking down social isolation by participation in the discussion group. As we did not include any measure of social isolation, we cannot exclude the possibility that members of our study group were less socially isolated. On the other hand, participants in the waiting-list discussion group spent more time with the discussion group compared with the therapy group, most probably because the therapy group members were occupied with the treatment. Although our study indicates that adding discussion group activity to cognitive–behavioural therapy does not yield incremental improvements, this cannot be directly inferred, given that a therapy-only group was not included. A plausible explanation for the lack of an effect in the waiting-list discussion group could be that the patients were aware of being placed on a waiting list, and hence were not expecting any change from participation in their group.

Limitations

Although self-report was used to obtain a likely diagnosis using DSM criteria, no formal diagnosis was made in an interview. Hence, it is possible that people with depression were excluded and people without depression were included. However, this is not very likely, particularly the latter possibility of including people who would not fulfil DSM depression criteria in a structured interview. Internet administration of both interviews and questionnaires is a research area on its own that needs further investigation. Independent ratings by clinicians would have strengthened the self-reported findings, but was not done, given that participants were not requested to attend a research clinic.

Confounding with respect to medication status cannot be ignored. First, self-report was used to ascertain medication use. Second, those with ongoing but stabilised medication regimens were not excluded. Although no effect of medication status was found, in line with other research (Reference Oei and YeohOei & Yeoh, 1999), a better approach would have been to control for medication status in the first place in order to enable investigation of drug–therapy interactions.

The study period was relatively short, and it would have been preferable to have had a control group that had not received any self-help or psychotherapy at the 6-month follow-up, or at least data from a less specific attention control treatment. This was not possible in the study protocol for ethical reasons.

Limited access to the internet is often put forward as an argument against internet treatments, and although a large proportion of the Swedish population does have access to the internet (about 70%), there is still a significant minority who do not, and this is even more the case in countries outside northern Europe.

Further directions

Research on internet-based self-help for depression would benefit from clear-cut diagnoses before initiation of treatment; in our study, we did not use a clinician-administered interview. However, one of the potential benefits of internet-delivered treatments is that geographical distances are immaterial. Requiring participants to come in for a clinical assessment would therefore introduce a limitation. It is possible that the internet could be used for diagnoses in the future, perhaps complemented with web-camera technology or video conferencing. The validity of such procedures has yet to be assessed. Internet technology might also be used in the future for preventing relapse, perhaps in combination with medication. All these suggestions point to the importance of evaluating the cost-effectiveness of internet treatments. No attempt was made here to do this, as a proper assessment of costs would include the costs of programming and computer equipment, as well as therapist time devoted to writing the self-help material and processing the participants’ responses to the modules. Finally, effective mechanisms are yet to be disclosed, as most studies of cognitive–behavioural therapy include packages of treatment ingredients. Our study was no exception in this respect.

CLINICAL IMPLICATIONS

1. • Internet-based self-help facilitates the spread of cognitive–behavioural therapy for depression to those who usually do have access to this form of treatment.

2. • Outcome with internet-based therapy resembles that in controlled studies of clinician-delivered therapy.

3. • Participation in a discussion group within a controlled study does not seem to confer any benefits.

LIMITATIONS

1. • No structured clinical interview was conducted with a clinician to secure diagnoses.

2. • Medication status was not controlled.

3. • Lack of access to the internet might be a problem in future dissemination of the therapy.