In a recent meta-analysis entitled ‘Was Eysenck right after all?’, Cuijpers et al. (Reference Cuijpers, Karyotaki, Reijnders and Ebert2018) propose a sobering reassessment of the effects of psychotherapy for adult depression. Their approach consists of a series of sequential sensitivity analyses adjusting the pooled effect size of psychotherapies for biasing factors, such as the use of waiting list (WL) arms as controls, trial risk of bias and publication bias. They evidence an overall effectiveness of psychotherapies for depression, though with more modest estimates, reduced to half (Hedges’ g  =  0.31) after excluding studies affected by biases. Some aspects of the methodology, like the use of the trim and fill to provide an estimate deemed free from publication bias, are less than ideal (Peters et al., Reference Peters, Sutton, Jones, Abrams and Rushton2007), but the overall findings have a familiar resonance. The notion that the effects of psychotherapy for depression are overestimated due to biases is supported by previous similar work from the same research group (Cuijpers et al., Reference Cuijpers, Cristea, Karyotaki, Reijnders and Huibers2016), as well as by independent research (Dragioti et al., Reference Dragioti, Karathanos, Gerdle and Evangelou2017).

In reply, Munder et al. (Reference Munder, Flückiger, Leichsenring, Abbass, Hilsenroth, Luyten, Rabung, Steinert and Wampold2019) challenge this assessment by proposing a thought-provoking re-analysis. While Cuijpers et al. (Reference Cuijpers, Karyotaki, Reijnders and Ebert2018) appeal to Eysenck's famous dictum about psychotherapy not being effective seemed mostly rhetoric, Munder and colleagues opt for an almost verbatim interpretation, focusing their entire rebuttal on establishing whether this historical claim is upheld. They object to several methodological choices in the original meta-analysis, but the crux of their case focuses on trials with a WL arm. Claiming that WL can be beneficial or at least not harmful, Munder et al. (Reference Munder, Flückiger, Leichsenring, Abbass, Hilsenroth, Luyten, Rabung, Steinert and Wampold2019) go on to conclude that it represents an appropriate control group by which to weigh the effectiveness of psychotherapy. Consequently, they recalculate effect estimates contrasting psychotherapy with WL and report a reassuring g  =  0.71, double the effect in the original meta-analysis.

The plea for WL-controlled trials is in stark contrast to the negative attention this type of control has garnered over the recent years. In an oft-cited network meta-analysis, Furukawa et al. (Reference Furukawa, Noma, Caldwell, Honyashiki, Shinohara, Imai, Chen, Hunot and Churchill2014) branded the WL a ‘nocebo’, after showing that the odds of response for patients in the psychological placebo, and, more strikingly, even in the no treatment (NT) control groups were significantly higher than for those in the WL groups. Munder et al. (Reference Munder, Flückiger, Leichsenring, Abbass, Hilsenroth, Luyten, Rabung, Steinert and Wampold2019) dispute these results as not providing ‘persuasive evidence’ (p. 3) that WL is an inappropriate control, but their main refuting argument is a conjectural post hoc ergo propter hoc. They pinpoint factors, such as the inclusion of studies on participants who were mildly or moderately depressed or not seeking treatment, which might have confounded the comparison between NT and WL and may consequently account for the observed differences. Evidently, this is a possibility, but until demonstrated empirically, on actual data, it remains solely a supposition.

Moreover, converging evidence that WL is inferior to other types of control arms comes from a network meta-analysis involving main author Thomas Munder (Barth et al., Reference Barth, Munder, Gerger, Nuesch, Trelle, Znoj, Juni and Cuijpers2013), as well as from another meta-analysis (Khan et al., Reference Khan, Faucett, Lichtenberg, Kirsch and Brown2012), both showing diminished response on depressive symptoms for WL participants, as well as significant inferiority to other control conditions, such as treatment as usual (TAU) or placebo. Undoubtedly, all meta-analytic methods have limitations, and underscoring them, as Munder and colleagues do, is important. Yet these general limitations do not obliterate the consistent finding that not only are the effects of active psychotherapies disproportionately higher when contrasted to WL than to other control groups, but also its corollary that WL is conducive of worse outcomes than other control conditions.

Furthermore, other lines of research questioned the adequacy of WL control arms. One meta-analysis (Palpacuer et al., Reference Palpacuer, Gallet, Drapier, Reymann, Falissard and Naudet2017) with a meta-regression analysis showed that the effects of psychotherapies for depression compared to WL were rendered non-significant after controlling for non-specific factors, such as recruitment method, provenience, number of treatment sessions, length of follow-up and researcher allegiance. Another study (Cooper and Conklin, Reference Cooper and Conklin2015) showed that in trials of psychotherapy for depression, inactive control conditions (a category inclusive of WL and placebo, but not of TAU) were associated with higher overall drop-out rates than active ones. As a minimum, this implies that a relevant proportion of patients is not interested or willing to remain in an WL-controlled trial. Relatedly, disappointment about being randomised to the control group has been repeatedly reported in intervention research, and related to participant drop-out (Lindström et al., Reference Lindström, Sundberg-Petersson, Adami and Tönnesen2010; Skingley et al., Reference Skingley, Bungay, Clift and Warden2014).

These findings add to the fact that psychotherapy trials with WL arms are in conspicuously large numbers (Barth et al., Reference Barth, Munder, Gerger, Nuesch, Trelle, Znoj, Juni and Cuijpers2013; Cuijpers et al., Reference Cuijpers, Cristea, Karyotaki, Reijnders and Huibers2016; Dragioti et al., Reference Dragioti, Karathanos, Gerdle and Evangelou2017; Palpacuer et al., Reference Palpacuer, Gallet, Drapier, Reymann, Falissard and Naudet2017). Investigators might consider only opting for these designs on a case-by-case basis, after a careful and judicious cost-benefit analysis, weighing both patient benefit and potential harms, along with trial validity. WL-controlled trials should probably not be routine, at the very least in a field as saturated with research as psychotherapy for depression. If publication bias could be taken out of the equation and we could reasonably assume all conducted trials are published, pitting new interventions against WLs could have the benefit of screening out ineffective interventions (i.e., that don't outperform WL) before they are tested in large-scale trials, preventing the squandering of resources on treatments that don't work (Ioannidis, Reference Ioannidis2016). With publication bias pervasive in clinical research, the informational value of WL-controlled trials is limited. Published trials comparing psychological interventions to WL systematically produce very large effects (Cuijpers and Cristea, Reference Cuijpers and Cristea2016; Fodor et al., Reference Fodor, Cotet, Cuijpers, Szamoskozi, David and Cristea2018). Patients extract little benefit from being on a WL, and other, more beneficial, control arms are available (e.g., TAU). In this regard, several recent trials employed ingenious recruitment strategies aimed at maximising the time patients spend on a WL for active treatment by randomising them to low-intensity interventions during the waiting period (Lovell et al., Reference Lovell, Bower, Gellatly, Byford, Bee, McMillan, Arundel, Gilbody, Gega, Hardy, Reynolds, Barkham, Mottram, Lidbetter, Pedley, Molle, Peckham, Knopp-Hoffer, Price, Connell, Heslin, Foley, Plummer and Roberts2017). Equipoise-stratified designs (Lavori et al., Reference Lavori, Rush, Wisniewski, Alpert, Fava, Kupfer, Nierenberg, Quitkin, Sackeim, Thase and Trivedi2001; Shalev et al., Reference Shalev, Ankri, Israeli-Shalev, Peleg, Adessky and Freedman2012), allowing participants to refuse undesired treatment options and still be randomly assigned to the other arms, could represent other options to counteract participant loss and the ensuing selection bias at recruitment, in those cases where a WL control is deemed necessary by investigators, or where other types of control arms are difficult to implement or pose other risks. Researchers should routinely report the uptake of delayed treatment for participants on the WL, as well as changes in the primary and secondary outcomes for this group. If patients who receive treatment after the WL period experience less improvement than those receiving treatment immediately after recruitment, this would provide further clues regarding the possible iatrogenic effects of this type of control condition.

Ultimately, the debate over whether Eysenck was right or not in a contention dating more than 50 years back has a grating academic resonance. This matter is entirely inconsequential for patients, caregivers or clinicians looking into systematic reviews and meta-analyses for guidance in selecting a course of treatment. The emphasis should be on up-to-date evidence as to what types of control arms are the most advantageous options in terms of maximising patient welfare and the internal and external validity of a trial.