Introduction
The question whether depressed and anxious patients are capable of applying a cognitive behavioural intervention to themselves, with only minimal support from a professional therapist, has been examined for more than 40 years (Kahn & Baker, Reference Kahn and Baker1968; Hogan & Kirchner, Reference Hogan and Kirchner1968; Donner & Guerney, Reference Donner and Guerney1969; Watkins & Clum, Reference Watkins and Clum2008). Since then, dozens of randomized trials and meta-analyses have shown that guided self-help is effective in reducing depression, panic disorder, phobias and other anxiety disorders (Cuijpers, Reference Cuijpers1997; Gregory et al. Reference Gregory, Canning, Lee and Wise2004; Hirai & Clum, Reference Hirai and Clum2006; Gellatly et al. Reference Gellatly, Bower, Hennessy, Richards, Gilbody and Lovell2007; Menchola et al. Reference Menchola, Arkowitz and Burke2007; Spek et al. Reference Spek, Cuijpers, Nyklíček, Riper, Keyzer and Pop2007). Guided self-help can be defined as a psychological treatment, where the patient or client takes home a standardized psychological treatment and works through it more or less independently (Marrs, Reference Marrs1995; Cuijpers & Schuurmans, Reference Cuijpers and Schuurmans2007). In the standardized psychological treatment, the patient can use step-by-step instructions on how to apply a generally accepted psychological treatment procedure to himself. The standardized treatment can be written down in book form, or be made available through other media, such as the Internet, a stand-alone personal computer, television, video or audio. Guided self-help can be distinguished from other self-help interventions by the support that is given by a professional therapist or coach to the patient when working through the standardized treatment. The support given by the therapist should primarily be of supportive or facilitative nature, and is meant to support the patient in working through the standardized psychological treatment. Interaction between patient and therapist can take place through face-to-face contact, by telephone, by email, or any other communication method. An important distinction between guided self-help and face-to-face treatment is the amount of contact which is minimized in guided self-help.
Although a considerable number of studies have examined whether guided self-help is effective in the treatment of depression and anxiety disorders, it is not clear whether they are equally effective as face-to-face treatments. Most studies on guided self-help have found positive effects of these interventions compared with control conditions, and meta-analyses in this field have found large effect sizes (Gregory et al. Reference Gregory, Canning, Lee and Wise2004; Hirai & Clum, Reference Hirai and Clum2006; Gellatly et al. Reference Gellatly, Bower, Hennessy, Richards, Gilbody and Lovell2007; Menchola et al. Reference Menchola, Arkowitz and Burke2007) which are comparable with those found for face-to-face treatments of depression and anxiety. However, these meta-analyses may very well be influenced by factors that differed among the various studies, such as length of treatment, type of treatment, or initial symptom severity (Shadish & Sweeney, Reference Shadish and Sweeney1991; Spielmans et al. Reference Spielmans, Pasek and McFall2007). Hence, possible differences between the effects of the two treatment formats may very well be artifacts, which do not reflect true superiority of one of the types of treatment over the other (Spielmans et al. Reference Spielmans, Pasek and McFall2007). Direct comparisons of guided self-help and face-to-face treatments, in which patients are assigned to one of the two treatments in the same study, are better equipped to rule out the influence of study characteristics, and they certainly provide more reliable evidence about a possible superiority of one type of therapy over the other (Spielmans et al. Reference Spielmans, Pasek and McFall2007).
Although a considerable number of studies have compared guided self-help and face-to-face treatment directly with each other (i.e. in the same study), no meta-analysis of these studies has been conducted. Such a meta-analysis is, however, important, because most individual studies in this field do not have sufficient statistical power to detect a significant difference between the two treatments. In fact, many studies could be better described as equivalence studies in which a null finding is the expected outcome (Piaggio et al. Reference Piaggio, Elbourne, Altman, Pocock and Evans2006). If there is a difference between the two treatments it can be assumed that this difference is relatively small, and in order to detect a small effect, a large number of participants is needed. Most studies in this field have not included sufficient participants to detect a possible difference. This could suggest that the two treatments are equally effective, while in fact they are not. A meta-analysis can solve this problem because the results of multiple studies are combined, and sufficient statistical power is available to detect a possible difference.
One recent meta-analysis of studies examining self-help interventions for anxiety disorders has included a subset of studies in which guided self-help was compared directly with face-to-face treatments for anxiety disorders (Hirai & Clum, Reference Hirai and Clum2006). This meta-analysis did not find a significant advantage of face-to-face treatment over guided self-help (effect size d=0.11), or the other way around. This meta-analysis included, however, only a selection of the currently available studies (nine of the 15 studies on anxiety disorders we included in the current meta-analysis). Furthermore, no power calculation was conducted to assess whether the available studies had sufficient power to detect a significant effect. Finally, sources of heterogeneity were not examined, and nor was publication bias.
We decided to conduct a new meta-analysis of randomized controlled trials in which the effects of guided self-help on depression and anxiety were compared directly with face-to-face psychotherapies for depression and anxiety disorders.
Method
Identification and selection of studies
We used several methods to identify studies for inclusion. First, we used a database of 1036 papers on the psychological treatment of depression, which includes studies comparing guided self-help and face-to-face therapies. This database has been described in detail elsewhere (Cuijpers et al. Reference Cuijpers, van Straten, Warmerdam and Andersson2008) and has been used in a series of earlier meta-analyses (www.evidencebasedpsychotherapies.org). It was developed through a comprehensive literature search (from 1966 to January 2009) in which we examined 9011 abstracts in PubMed (1629 abstracts), PsycINFO (2439), EMBASE (2606) and the Cochrane Central Register of Controlled Trials (2337). These abstracts were identified by combining terms indicative of psychological treatment and depression [both medical subject heading (MeSH) terms and text words]. For this database, we also collected the primary studies from 42 meta-analyses of psychological treatment for depression (www.evidencebasedpsychotherapies.org). Second, we conducted additional searches in bibliographical databases, in which we combined search terms indicative of guided self-help and each of the disorders we examined, and randomized controlled trials as a limit. An example of a search string can be found in Supplementary Appendix B of this paper (available online).
Third, we examined the reference lists of all meta-analyses examining guided self-help interventions for depression and anxiety mentioned earlier. Fourth, we checked the references of the included primary studies. We did not contact study authors for additional data, unpublished studies and studies in press.
We included randomized studies in which guided self-help was compared with a face-to-face treatment for depression or anxiety, and which reported outcomes on depression and/or anxiety. Guided self-help was defined as a treatment (1) in which the procedures were written down (or presented in an audio or video file) and (2) the patients had to work through these procedures more or less independently, while (3) the therapist gave support in working through the procedures (4) with limited contact. Limited contact could be delivered during brief personal contacts, by telephone or by email. We allowed for a maximum of 12 contacts, with a maximum of 20 min each. Face-to-face treatments had to use the same format and contents as the guided self-help, but also had to use full individual or group treatment sessions to deliver the treatment.
We included studies in which a diagnostic interview was used to establish the presence of depression or anxiety disorders, but we also included studies which used other inclusion criteria. We excluded studies in children and adolescents, as well as studies examining virtual reality treatments (Côté & Bouchard, Reference Côté and Bouchard2008). No language restrictions were applied.
A detailed list of the variables we extracted from each study can be found in Supplementary Appendix C (available online). Data abstraction from the studies was conducted by the first author (P.C.) and checked by the third author (A.v.S.).
Quality assessment
We assessed the validity of included studies using four criteria of the ‘risk of bias’ assessment tool, developed by the Cochrane Collaboration (Higgins & Green, Reference Higgins and Green2008). This tool assesses possible sources of bias in randomized trials, including the adequate generation of allocation sequence; the concealment of allocation to conditions; the prevention of knowledge of the allocated intervention; and dealing with incomplete outcome data. The two other criteria of the ‘risk of bias’ assessment tool were not used in this study, because we found no clear indication in any of the studies that these had influenced the validity of the study (suggestions of selective outcome reporting; and other problems that could put it at a high risk of bias).
We also rated the quality of the treatment implementation using three criteria which were based on an authoritative review of empirically supported psychotherapies (Chambless & Hollon, Reference Chambless and Hollon1998): (1) the study referred to the use of a treatment manual (either a published manual, or a manual specifically designed for the study); (2) the therapists who conducted the therapy were trained for the specific therapy, either specifically for the study or as a general training; (3) treatment integrity was checked during the study (by supervision of the therapists during treatment or by recording of treatment sessions or by systematic screening of protocol adherence by a standardized measurement instrument).
Meta-analyses
For each comparison between guided self-help and a face-to-face treatment, we calculated the effect size indicating the difference between the two groups at post-test (Cohen's d). Effect sizes were calculated by subtracting (at post-test) the average score of the guided self-help group from the average score of the face-to-face treatment group, and dividing the result by the pooled standard deviations of the two groups. Effect sizes of 0.56 to 1.2 can be assumed to be large, effect sizes of 0.33 to 0.55 are moderate, and effect sizes of 0 to 0.32 are small (Lipsey, Reference Lipsey1990).
In the calculations of effect sizes we only used those instruments that explicitly measured symptoms of depression, and in the studies that examined anxiety, instruments that explicitly measured symptoms of anxiety. If more than one measure was used, the mean of the effect sizes was calculated, so that each study provided only one effect size. If means and standard deviations were not reported, we used the procedures of the comprehensive meta-analysis software (see below; Biostat, Inc., USA) to calculate the effect size using dichotomous outcomes. In two studies, it was only reported that there was no significant difference between the two conditions (Baker et al. Reference Baker, Cohen and Saunders1973; Ghosh et al. Reference Ghosh, Marks and Carr1988). In these studies, the effect size was assumed to be zero.
To calculate pooled mean effect sizes, we used the computer program comprehensive meta-analysis (version 2.2.021; Biostat, Inc., USA). As we expected considerable heterogeneity among the studies, we decided to calculate mean effect sizes using a random-effects model.
As a test of homogeneity of effect sizes, we calculated the I 2 statistic, which is an indicator of heterogeneity in percentages. A value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high heterogeneity (Higgins et al. Reference Higgins, Thompson, Deeks and Altman2003). We also calculated the Q statistic, but only report whether this was significant or not.
Publication bias was tested by inspecting the funnel plot on primary outcome measures, and by Duval & Tweedie's trim and fill procedure (Duval & Tweedie, Reference Duval and Tweedie2000) which yields an estimate of the effect size after the publication bias has been taken into account (as implemented in comprehensive meta-analysis, version 2.2.021).
Apart from the outcomes on depression and anxiety, we also calculated the relative risk of dropping out from the treatments. Again, we conducted all meta-analyses with the random-effects model and we calculated the Q statistic and the I 2 statistic to estimate heterogeneity between study outcomes.
In order to examine possible predictors of the difference between guided self-help and face-to-face treatment, we conducted a multivariate meta-regression analysis with the effect size as the dependent variable. As predictors, we used the characteristics of the populations (disorder: depression, panic disorder, phobia; community versus other type of recruitment; whether or not a diagnostic interview was conducted; aimed at adults in general or a more specific target group); the interventions (personal contact support versus other support during guided self-help; book versus other medium; individual versus group treatment) and the quality of the studies (met more than three quality criteria). In order to avoid collinearity among the predictors that were entered in the regression model, we first examined whether high correlations were found among the variables that could be entered into the model. The correlations between all entered characteristics were calculated. We found that none of the correlations was higher than 0.50. The multivariate meta-regression analyses were conducted in Stata SE/8 for Windows (StataCorp LP, USA).
Power calculation
We assumed the two treatment formats to be equally effective if the differential effect size was small. Although there is no clear agreement on what should be considered to be a small effect size, we used the definition of Lipsey (Reference Lipsey1990), which says that effect sizes of 0.3 can be considered as small. To have sufficient statistical power in our meta-analysis to be able to detect a small effect size, we conducted a power calculation according to the procedures described by Bohrenstein et al. (Reference Borenstein, Hedges, Higgins and Rothstein2009). These calculations indicated that we would need to include at least 14 studies with a mean sample size of 50 (25 participants per condition), to be able to detect an effect size of d=0.3 (conservatively assuming a high level of between-study variance, τ2, a statistical power of 0.80, and a significance level, α, of 0.05). Alternatively, we would need 18 studies with 40 participants each to detect an effect size of d=0.30, or 24 studies with 30 participants.
Results
Selection and inclusion of studies
We examined a total of 2761 abstracts, and 91 publications were retrieved for possible inclusion. We excluded 70 studies, because no guided self-help was examined (33 studies), because guided self-help was not compared with a face-to-face treatment (17 studies), because the studies examined virtual reality (nine studies), because the studies were not randomized trials (four studies), because they examined children or adolescents (three studies), or because of other reasons (four studies). The remaining 21 studies met all inclusion criteria and were included in our meta-analysis. A flowchart describing the inclusion of studies is presented in Fig. 1.
Flowchart of inclusion of studies. GSH, Guided self-help; FTF, face-to-face psychotherapy.
Characteristics of included studies
The 21 studies included a total of 810 participants (429 in face-to-face conditions and 381 in the guided self-help conditions). Selected characteristics of the included studies are presented in Table 1.
Selected characteristics of studies comparing face-to-face therapies with guided self-help for depression or anxiety disorders
ADIS, Anxiety Disorders Interview Schedule; ADIS-IV, Anxiety Disorders Interview Schedule for DSM-IV; ADIS-R, Anxiety Disorders Interview Schedule Revised; AQ, Acrophobia Questionnaire; AU, Australia; BA-f, frequency of panic attacks; BA-g, goals (blind assessor); BA-mp, main problem (blind assessor); BAI, Beck Anxiety Inventory; BAT, Behavior Approach Test; BCL; Timed Behavioral Checklist for Performance Anxiety; BDI, Beck Depression Inventory; BDI-II, Beck Depression Inventory, 1996 Revision; BSQ, Body Sensations Questionnaire; CA, Canada; CBT, cognitive behaviour therapy; CES-D, Center for Epidemiological Studies – Depression scale; CIDI, Composite International Diagnostic Interview; Clin, recruitment from clinical samples; Comm, community recruitment; Comp, computer; CR, clinician rating of depression; CR-a, clinician-rated agoraphobia; CR-p, clinician-rated panic; DASS, Depression, Anxiety, Stress Scales; DPAR, Daily Panic Attack Record; DSM-III, Diagnostic and Statistical Manual of Mental Disorders, third edition; DSM-III-R, Diagnostic and Statistical Manual of Mental Disorders, third edition revised; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, fourth edition; DYS, dysthymia; FNE, Fear of Negative Evaluation Scale; FQ, Fear Questionnaire; FSS, Fear Survey Schedule; FT, Fear Thermometer; FTF, face-to-face therapy; GDS, Geriatric Depression Scale; Grp, group; GSH, guided self-help; GSS, global symptom severity; HAMD, Hamilton Rating Scale of Depression; HSCL-a, Hopkins Symptom Checklist, anxiety subscale; ICD-9, International Classification of Diseases, 9th revision; Ind, individual; LSAS, Liebowitz Social Anxiety Scale self-report version; MDD, major depressive disorder; MI, Mobility Inventory; minD, minor depression; MMPI, Minnesota Multiphasic Personality Inventory; MMPI-d; Minnesota Multiphasic Personality Inventory Depression Scale; NR, not reported; PACQ, Panic Attack Cognitions Questionnaire; PASQ, Panic Attack Symptoms Questionnaire; PDSS, Panic Disorder Severity Scale; POMS, Profile of Mood States; RDC, Research Diagnostic Criteria; SAD, Social Avoidance and Distress Scale; SCID, Structured Clinical Interview for DSM Disorders; SCID-II, Structured Clinical Interview for DSM-IV Axis II Personality Disorders; SCID-RV, Structured Clinical Interview for DSM Disorders Research Version; SCL-90-d, Symptom Checklist-90 depression scale; SDS, Self-rating Depression Scale; SIAS, Social Interaction Anxiety Scale; SNAQ, Snake Attitude Questionnaire; SPS, Social Phobia Scale; SPSQ, Social Phobia Screening Questionnaire; SR, self-rating of depression; SR-g, goals (self-rated); SR-mp, main problem (self-rated); SUD, Subjective Units of Disturbance; SW, Sweden; w/wo, with or without.
a Quality assessment: a, allocation sequence adequately generated; b, allocation adequately concealed; c, knowledge of the allocated interventions adequately prevented (blinding; ± indicates that only self-report measures were used); d, incomplete outcome data adequately addressed; e, manual available; f, therapist trained; g, integrity check of intervention.
Most studies (14 out of 21) were aimed at adults in general, five were (largely) aimed at student populations, and two were aimed at more specific populations (one at older adults, and one at adults with co-morbid substance-use problems). Of the studies, six studies were aimed at patients with depression, seven at panic, three at social phobia, two at specific phobias, and three at phobias in general. In 15 of the 21 studies, a diagnostic interview was used to establish the presence of the depressive or anxiety disorder. Patients were recruited from the general population through media announcements in 17 studies, while three studies recruited people from clinical populations (one study did not report the recruitment method).
In the 21 studies, a total of 24 comparisons were made between a guided self-help condition and a face-to-face therapy. All treatments were cognitive and behavioural in nature. In the 24 comparisons, 16 of the face-to-face treatments used an individual treatment format while the remaining eight used a group treatment format. The number of treatment sessions in the face-to-face treatments ranged from four to 16. In the 24 guided self-help conditions, 15 used a self-help book, four a stand-alone computer program, three used an Internet-based intervention, and two used audio recordings to deliver the treatment. Support was given by brief personal contact in five comparisons, brief personal contact plus telephone calls in seven comparisons, by telephone calls only in five comparisons, and by email in three comparisons. In the remaining four comparisons the patient worked alone on a personal computer at the therapist's office and received brief face-to-face support.
Of the studies, 13 studies were conducted in the USA, three in the UK, and the remaining five in other countries (Australia, Sweden, Canada).
Quality of included studies
The quality of the studies was not optimal. Of the 21 studies, 16 gave insufficient information whether the allocation sequence was generated adequately. Also, 17 studies gave insufficient information about whether the allocation was adequately concealed. We assessed whether incomplete outcome data were adequately addressed, by conducting intention-to-treat analyses with all randomized subjects being included in the analyses. This was the case in only five studies. However, in 18 studies knowledge of the allocated interventions was adequately prevented by blinding of the assessors or because only self-report measures were used (and blinding of assessors was not relevant). Only one study met all of the four quality criteria (Carlbring et al. Reference Carlbring, Nilsson-Ihrfelt, Waara, Kollenstam, Buhrman, Kaldo, Söderberg, Ekselius and Andersson2005), and four studies met three of the four criteria (Sharp et al. Reference Sharp, Power and Swanson2000; Marks et al. Reference Marks, Kenwright, McDonough, Whittaker and Mataix-Cols2004; Kiropoulos et al. Reference Kiropoulos, Klein, Austin, Gilson, Pier, Mitchell and Ciechomski2008; Kay-Lambkin et al. Reference Kay-Lambkin, Baker, Lewin and Carr2009).
The quality of the treatment implementation was good in most studies. In 19 studies a treatment manual was used, in 15 studies the therapists were specifically trained for the intervention, and in 12 studies an integrity check of the interventions was conducted. In seven studies all three criteria for the quality of treatment implementation were met (Rosen et al. Reference Rosen, Glasgow and Barrera1976; Schmidt & Miller, Reference Schmidt and Miller1983; Brown & Lewinsohn, Reference Brown and Lewinsohn1984; Floyd et al. Reference Floyd, Scogin, McKendree-Smith, Floyd and Rokke2004; Carlbring et al. Reference Carlbring, Nilsson-Ihrfelt, Waara, Kollenstam, Buhrman, Kaldo, Söderberg, Ekselius and Andersson2005; Tillfors et al. Reference Tillfors, Carlbring, Furmark, Lewenhaupt, Spak, Eriksson, Westling and Andersson2008; Kay-Lambkin et al. Reference Kay-Lambkin, Baker, Lewin and Carr2009).
Differences between guided self-help and face-to-face treatments
The mean effect size indicating the difference between guided self-help and face-to-face psychotherapy was d=−0.02 [95% confidence interval (CI) −0.20 to 0.15, n.s.], in favour of guided self-help. Heterogeneity was low (I 2=26.34) and not statistically significant (Table 2). The effect sizes and 95% CIs of the studies are plotted in Fig. 2. Our sample of studies had sufficient statistical power to detect a differential effect size of d=0.30.
Meta-analyses of studies comparing the effects of guided self-help and face-to-face psychotherapies for adult depression and anxiety disorders
CI, confidence interval; n.s., non-significant; ES, effect size.
a The p values in this column indicate whether the Q statistic is significant (the I 2 statistics does not include a test of significance).
b Hecker et al. (Reference Hecker, Losee, Roberson-Nay and Maki2004).
c In these analyses only one comparison from each study was used.
Differences between guided self-help (GSH) and face-to-face (FTF) treatments for depression and anxiety disorders: standardized effect sizes. Std diff, Standardized difference; CI, confidence interval; I, individual; G, group; sG, small group; lG, large group.
The 95% CI of the effect size found for one study (Hecker et al. Reference Hecker, Losee, Roberson-Nay and Maki2004) did not overlap with the CI of the pooled effect size, and may be an outlier. However, removal of this study hardly affected the outcome (d=0.02, 95% CI 0.14–0.18, n.s., I 2=12.19, n.s.).
We also examined the influence of the two studies in which the effect size was assumed to be zero because no significant difference was found between guided self-help and face-to-face therapy, by removing them from the analyses. The results of these analyses, however, were virtually the same as the overall analyses (d=−0.01, 95% CI −0.20 to 0.19, I 2=26.34).
In this meta-analysis we included two studies in which more than one comparison was made between guided self-help and a face-to-face treatment (Schmidt & Miller, Reference Schmidt and Miller1983; Brown & Lewinsohn, Reference Brown and Lewinsohn1984). This means that multiple comparisons from these two studies were included in the same analysis. These multiple comparisons, however, are not independent of each other, which may have resulted in an artificial reduction of heterogeneity and may have affected the pooled effect size. We examined the possible effects of this in two different ways. First, we divided the number of respondent over the different comparison groups in the studies with multiple comparison groups, so that we included each respondent only once in the overall analyses. The results of these analyses are almost the same as in our main analyses (d=0.00, 95% CI −0.17 to 18, I 2=19.35). In our second approach, we included only one comparison per study in the analysis (Table 2). First we conducted an analysis in which we included only one effect size per study and chose the one with the largest effect size from the studies with multiple comparisons. Then we conducted another analysis in which we included only the smallest effect size. As can be seen from Table 2, the resulting effect sizes were almost the same as in the overall analyses. Heterogeneity did not increase very much, and was not statistically significant in any analysis.
In our analyses, we included studies with more than one outcome measure and we calculated the mean of the effect sizes based on these outcome measures. However, this approach assumes that the correlation between the measures within the same study is 1.00, which is not necessarily the case. Therefore we conducted another meta-analysis in which we assumed that all effect sizes within one study were independent (and the correlation would be zero). In Appendix A we have presented the forest plot of these analyses. The resulting overall effect size was almost the same as found in the main analyses (d=−0.03 in favour of guided self-help, with very low heterogeneity, I 2=12.85).
In order to examine the influence of individual studies, we examined which study had the largest impact on the overall effect size. Removal of the study by Sharp et al. (Reference Sharp, Power and Swanson2000) resulted in the largest decrease of the effect size (the resulting effect size was d=−0.06). After the removal of this study, we repeated this procedure and examined which study should be removed in order to realize the next largest decrease of the effect size. This was the study by Kiropoulos et al. (Reference Kiropoulos, Klein, Austin, Gilson, Pier, Mitchell and Ciechomski2008), and the meta-analysis resulted in an effect size of d=−0.09. Repeating this procedure a third time resulted (after removal of the study by Kay-Lambkin et al. Reference Kay-Lambkin, Baker, Lewin and Carr2009) in a mean effect size of d=−0.12. None of the three resulting effect sizes were significantly different from zero. A comparable procedure to examine whether individual studies resulted in an increase of the effect size (in favour of face-to-face treatment) indicated that removal of the study by Hecker et al. (Reference Hecker, Losee, Roberson-Nay and Maki2004) resulted in the largest increase (resulting effect size d=0.02), followed by the study of Schmidt et al. (Reference Schmidt and Miller1983) (effect size d=0.06), and the study by Floyd et al. (Reference Floyd, Scogin, McKendree-Smith, Floyd and Rokke2004) (effect size d=0.08). These analyses did not suggest that removal of individual studies resulted in major changes in the overall effect size.
We could calculate effect sizes indicating the difference between face-to-face therapies at follow-up in 17 studies. We divided the effect sizes according to the follow-up period (1–3 months, 4–6 months, 12 months). As can be seen in Table 2, none of the effect sizes differed significantly from zero.
Neither the funnel plot nor Duval & Tweedie's trim and fill procedure pointed to a significant publication bias. The effect size indicating the difference in reduction of depressive symptomatology between guided self-help and face-to-face treatments did not change after adjustment for possible publication bias (the observed and adjusted effect sizes were exactly the same, and the number of imputed studies was zero).
We examined possible predictors of the difference between guided self-help and face-to-face treatment, with multivariate meta-regression analysis. The results of these analyses are presented in Table 3. As can be seen, none of the predictors was significantly associated with the effect size.
Regression coefficients of study characteristics in relation to the effect size of guided self-help versus face-to-face treatments of depression: multivariate meta-regression analyses
CI, Confidence interval.
Drop-out
The definition of drop-out differed considerably between studies, ranging from drop-out from the interventions to drop-out from the studies, and many variations in between. In 18 studies (21 comparisons) data on drop-out were presented. We calculated the relative risk of dropping out from guided self-help and face-to-face treatments in these studies. One study reported no drop-out in both conditions, and was not included in these analyses (Selmi et al. Reference Selmi, Klein, Greist, Sorrell and Erdman1990). The pooled relative risk of dropping out was 1.14 (95% CI 0.77–1.67), indicating that the drop-out rate was somewhat higher in the guided self-help conditions, but that was not significant (p=0.52). Heterogeneity (I 2) was zero and not significant (Q=17.05, p>0.1).
Discussion
In this study, we found no indication that the effects of guided self-help and face-to-face treatments differ significantly from each other, although we had sufficient statistical power to detect small differences. This was also true at follow-up periods of up to 1 year. Furthermore, we found no indication that drop-out rates differed between the two treatment formats.
Although this meta-analysis found support for the hypothesis that guided self-help and face-to-face treatments for depression and anxiety do not significantly differ in effectiveness, this does not imply that guided self-help is effective in all patients seeking help in mental health care or primary care. The studies examined in this meta-analysis only included patients who were willing to be randomized to both conditions. People who are not interested in guided self-help have probably not participated in these trials. This means that guided self-help and face-to-face treatments may indeed be equally effective for many, but not for all people with mood or anxiety disorders. Future research should be conducted to examine who is willing to participate in guided self-help treatments and who is not, and if there are differential predictors and mediators of outcome (Andersson et al. Reference Andersson, Carlbring and Grimlund2008).
Most research included in this meta-analysis was conducted with people who are recruited from the community by media announcements. More research is needed to examine how guided self-help can be used in clinical practice. There are some indications that patients referred to guided self-help by their general practitioner benefit most from these treatments, compared with patients who are referred by mental health professionals and self-referrals from the community, while self-referred patients improve more than patients referred by mental health professionals (Mataix-Cols et al. Reference Mataix-Cols, Cameron, Gega, Kenwright and Marks2006).
An interesting issue that results from this study is that apparently the patient–therapist relationship can be realized with minimal contact with the therapist. This may suggest that it is not so much the intensity of the contact that makes a well-functioning relationship possible (Knaevelsrud & Maercker, Reference Knaevelsrud and Maercker2007), but more the contact between the two in itself. It is also possible that the patient–therapist relationship is not needed at all in these treatments, although there is evidence that unguided self-help is significantly less effective than guided self-help, both in depression and anxiety disorders (Hirai & Clum, Reference Hirai and Clum2006; Spek et al. Reference Spek, Cuijpers, Nyklíček, Riper, Keyzer and Pop2007). This study has several limitations. One important limitation is the relatively small sample of studies, which limits the possibility to explore differences between subgroups. For example, we did have sufficient power to detect a small differential effect size for the whole sample of studies and for the studies on anxiety disorders, but not for the studies on depression. Second, the quality of the included studies was not optimal in many studies. Third, we may have missed studies that were unpublished or in the press. Fourth, we included a broad range of disorders in our meta-analysis. Depression and anxiety disorders are different categories of mental disorders and within the category of anxiety disorders, we included studies focusing on panic disorders, phobias in general and specific categories of phobias. However, we found very low levels of heterogeneity, suggesting that the different diagnostic categories did not result in different answers to the question whether the two treatments are equally effective. Furthermore, co-morbidity between anxiety disorders and depression is very high, suggesting that they are closely related to each other, and cognitive and behavioural treatment strategies for depression and anxiety share many common elements.
Despite these limitations, it seems safe to conclude that guided self-help and face-to-face treatments for depression and anxiety have comparable effects, and that there is no evidence that one or the other is significantly larger than the other. There is no reason not to consider using guided self-help as a complement in clinical practice, and we suspect that face-to-face treatment and guided self-help will blend in with each other increasingly in the near future, for example by using computer assistance when providing psychoeducational material (Craske et al. Reference Craske, Rose, Lang, Welch, Campbell-Sills, Sullivan, Sherbourne, Bystritsky, Stein and Roy-Byrne2009).
Note
Supplementary material accompanies this paper on the Journal's website (http://journals.cambridge.org/psm).
Acknowledgements
None.
Declaration of Interest
None.
All effect sizes included from studies comparing guided self-help with face-to-face psychotherapy for depression and anxiety disorders
AQ, Acrophobia Questionnaire; BAI, Beck Anxiety Inventory; BAT, Behavior Approach Test; BCL; Timed Behavioral Checklist for Performance Anxiety; BDI, Beck Depression Inventory; BDI-II, Beck Depression Inventory, 1996 Revision; Blind assess, Blind assessor; BSQ, Body Sensations Questionnaire; CES-D, Center for Epidemiological Studies – Depression scale; CI, confidence interval; clin, clinician; CR, clinician rating of depression; DASS, Depression, Anxiety, Stress Scales; DPAR, Daily Panic Attack Record; FNE, Fear of Negative Evaluation Scale; FQ, Fear Questionnaire; FSS, Fear Survey Schedule; FT, Fear Thermometer; FTF, face-to-face therapy; G, group; GSH, guided self-help; GSS, global symptom severity; HAMD, Hamilton Rating Scale of Depression; HSCL anx, Hopkins Symptom Checklist, anxiety subscale; I, individual; lG, large group; LSAS, Liebowitz Social Anxiety Scale self-report version; MI, Mobility Inventory; MI-ac, Mobility Inventory (avoidance when accompanied by a trusted person); MI-al, Mobility Inventory (avoidance when alone); MMPI, Minnesota Multiphasic Personality Inventory; MMPI-d; Minnesota Multiphasic Personality Inventory Depression Scale; PA, panic attack; PDSS, Panic Disorder Severity Scale; POMS, Profile of Mood States; SAD, Social Avoidance and Distress Scale; SCL-90-d, Symptom Checklist-90 depression scale; SDS, Self-rating Depression Scale; sG, small group; SIAS, Social Interaction Anxiety Scale; SNAQ, Snake Attitude Questionnaire; SPS, Social Phobia Scale; Std diff, standardized difference; SPSQ, Social Phobia Screening Questionnaire; SR, self-rating of depression; SUD, Subjective Units of Disturbance.
