1. Introduction

It seems rather obvious that when our experiments are potentially confounded, the prudent response is to take better precautions. Yet given the mounting empirical evidence that randomized clinical trials (RCTs) designed to be “double-blind” fail to remain so, the CONSORT 2010 guidelines have dropped their recommendation to report whether blinding was successful. They reason that if a treatment has a significant therapeutic effect, maintenance of the blind may be impossible. I will argue that this change represents a step backward and ultimately results from an inadequate definition of “placebo.”

While Cartwright’s (Reference Cartwright2010) analysis has illuminated the experimental logic undergirding ideal RCTs, worldly vagaries put this logic in tension with the standard philosophical account of placebos offered by Grünbaum (Reference Grünbaum1986). Accordingly, the first tasks of this article will be explicating Grünbaum’s conception of a placebo and substantiating the claim that, defined as such, placebos are incompatible with Cartwright’s ideal. Next, I will provide a methodological definition of a placebo that resolves the incompatibility. Finally, I will explore two ways in which RCTs could be altered to bring practice in line with this resolution. Regardless of how this incompatibility is resolved, clarifying the nature of the tension is salutary. It both identifies the situations where medical trials fail to warrant the conclusions they are standardly taken to demonstrate and suggests how they might be amended so as to yield more reliable results.

2. Grünbaum’s Placebo

Behind Grünbaum’s (Reference Grünbaum1986) work on placebos was an internecine debate in the mental health community over the efficacy of various schools of psychotherapy. With the example of psychotherapy in the foreground, it was clear that a placebo is not just any treatment that relies on psychological effects. It is similarly flawed to identify placebos with inert substances. Water, salt, and sugar are paragons of therapeutically inert substances. But while inert generically, they can be effective treatments for dehydration, hyponatremia, and hypoglycemia, respectively. Such modest examples elucidate Grünbaum’s central insight: that a treatment t is a placebo with respect to some disease D.

More specifically, Grünbaum divides a treatment as follows: a therapy (t) is explained by a theory ψ as being composed of (a) the characteristic factors F that are purportedly responsible for improvement and (b) the incidental factors C that are not (see fig. 1). For example, suppose that a therapeutic theory holds that the chemical fluoxetine is the proper treatment for depression. Since Prozac and Sarafem are chemically identical (they are both fluoxetine), they have identical characteristic factors. Yet while they share some incidental factors (e.g., they are both manufactured by Eli Lilly), there are others they do not share (e.g., a pill of Sarafem is pink and purple, while Prozac is green and white; Sarafem is roughly 15 times more expensive).

Figure 1.

An example of Grünbaum’s model used to specify drug/placebo effects.

Next, Grünbaum divides the effects of the treatment on the patient (he calls the patient a “victim”) into (i) the intended effects on the target disorder D and (ii) other aspects of the patient’s life (i.e., side effects). Staying with the example above, Prozac might cause improvement in mood, as intended, but unintentionally cause sexual dysfunction.

A treatment t is “an intentional placebo” (a placebo intended for treatment) for victim V suffering from disorder D treated by practitioner P iff (1) none of the characteristic factors F positively affect D, (2) P believes that the first condition is true, (3) P believes that some incidental treatment factors C will positively affect D for V, and (4) P generally allows V to believe that t has remedial efficacy for D because of F. More plainly, condition 1 entails that a sugar pill is a placebo as long as the patient was not helped by the sugar. Conditions 2–4 specify that the patient, but not the doctor, believes that the treatment is effective because of some characteristic factor. By these conditions, most sugar pills are placebos.

3. The Crystal Palace

To highlight potential problems with Grünbaum’s conception of a placebo, consider the following thought experiment. Suppose that crystal healing is the standard form of treatment, but we were inclined to doubt the efficacy of crystals. Crystal healers (and thus, ex hypothesi, the population at large) believe that sandstone is the appropriate treatment for some disorder. To assuage our concern, a crystal healer performs an RCT. In the experimental group, sandstone is used to treat the patients. In the control group, amethyst (which has no expected effect) is used instead. The amethyst and the sandstone are enclosed in a device that allows for the appropriate skin contact while preventing the patient and the healer from seeing which is being used. In follow-up evaluations, while both groups have improved considerably, the treatment group has significantly better outcomes. Suppose further that this finding is repeatedly confirmed. Healers conclude that this constitutes scientific evidence to support the widely accepted claims of efficacy.

Such trials would meet the general standards of evidence-based medicine and even the requirements of the Food and Drug Administration. Standard practice allows that the difference in outcomes between patients who received sandstone and patients who received amethyst can be attributed to the effectiveness of the sandstone treatment. Amethyst also meets Grünbaum’s (Reference Grünbaum1986) definition of a placebo control (a placebo intended for a trial) iff (1) none of its characteristic factors positively affect D, (2) P believes that amethyst is harmless for V with D, and (3) P wishes to know whether any of the observed improvement can be attributed to the characteristic factors.

Yet we can grant each of these conditions while maintaining that the test was not properly controlled. Suppose I hypothesized that it is possible to determine patients’ group assignment during the trial. To support this, I produce evidence that patients, doctors, and independent evaluators are all able to make this discrimination at levels above chance. There are two possible explanations for this. It could be because they correctly believe that sandstone works. For example, subjects who recover reason that since they recovered, they must have been treated with sandstone. Subjects who don’t recover reason the same way (mutatis mutandis) concerning amethyst. Alternatively, it could be the case that after the groups were randomized, amethyst did not mimic “nontherapeutic” aspects (e.g., side effects) of sandstone effectively, allowing subjects to discriminate between treatments based on properties incidental to treatment.

In line with the latter hypothesis, suppose that when sandstone is rubbed on the body, it causes abrasions, whereas amethyst does not. Further, suppose I show that patients guessed their treatment assignment on the basis of abrasions, not improvement. Finally, suppose we find the following: when you statistically control for which group patients believed they were in, patients are no better off if they were actually exposed to sandstone. In light of these facts, I believe that a reasonable conclusion is that sandstone was not an effective treatment despite its apparent superiority. Its only independent effect was to cause abrasions. The presence of abrasions signaled to subjects that they were receiving the culturally accepted medical practice, which in turn resulted in improvement via an expectancy effect. Because the abrasions were a side effect (not a characteristic or incidental factor), such an expectancy effect does not realize Grünbaum’s conception for either a placebo effect or a nonplacebo effect (see fig. 1). Let us call this “the sandstone effect.”Footnote ¹

4. The Placebo Control Revisited

The methodological definition of a placebo is determined by the underlying logic of ideal RCTs. As articulated by Cartwright and Hardie (Reference Cartwright and Hardie2012), the ideal RCT is a manifestation of Mill’s method of difference. Patients are randomized into two groups. The control group receives precisely the same care as the treatment group except for x, where x is what is being evaluated for efficacy. Given that patients receiving treatment t improve and the only difference between t and t* is x, x must be the cause of the improvement.

Faced with the situation above, it seems that we must either give up the standard definition of placebo or conclude that an RCT is insufficient for establishing efficacy. In the remainder of the article I will parse out the consequences for the former. In place of the traditional definition, I will propose the methodological definition of a placebo. In essence, the methodological definition turns “placebo” into a success term. As a first pass, a treatment t* is a placebo in an RCT testing t for D iff it plays the methodological role required to determine whether t is a treatment for D. As an example, let’s consider arthroscopic knee surgery for osteoarthritis of the knee.

Moseley et al. (Reference Moseley, O’Malley, Petersen, Menke, Brody, Kuykendall, Hollingsworth, Ashton and Wray2002) randomized patients into one of three groups. In the incision group, a patient was locally anesthetized and several incisions were made, but nothing more. The second group also had their joint lavaged (washed out), and the final group had their knee lavaged and debrided (surgeons removed damaged tissue). If the actual procedure was not performed, the surgeon still went through the motions of asking for the instrument, manipulating the patient’s leg, even splashing saline to simulate the sounds of lavage. In this procedure a number of factors are clearly incidental: the fact that Moseley was a celebrated surgeon, the sound of splashing water as the joint is rinsed, and so on. Other factors are less clearly categorized.

Surely, the group we consider to be the placebo group will depend on what our theory ψ determines to be the characteristic factors of t, but much can hang on such designations. Surgeons diverge in whether they attribute the therapeutic effect to the debridement, the lavage, or both (all agree that the incision is incidental). According to ψ₁, the debridement is the characteristic factor and the lavage is merely an incidental prelude. In contrast, ψ₂ supposes that the lavage is the characteristic factor of the procedure and the debridement is both incidental and otiose. Finally, ψ₃ posits that both the lavage and the debridement are characteristic factors, each of which makes a contribution to the therapeutic effect. The correct theory of arthroscopic knee surgery classifies as characteristic factors all and only the causally relevant factors of the treatment that exert a therapeutic effect independent of the patient’s expectations. The choice is not conventional; the other theories are false.

The definition of a characteristic factor above implies that any factor that does not independently improve therapeutic outcomes is an incidental factor. In the sandstone parable, the amethyst group improved (though to a lesser extent than the sandstone group); however, according to the society’s prevailing theory, mere exposure to amethyst outside of the healing ritual would not have resulted in any therapeutic effect. Accordingly, any therapeutic effect that did occur must be the result of the patient’s expectations.

Beyond their therapeutic effects, characteristic factors may also cause side effects (i.e., effects unrelated to the disorder). Side effects might be caused independently of patient belief, as when the incision during knee surgery results in soreness or, contra Grünbaum, as a result of expectancy effects (Kirsch Reference Kirsch2010). For the sake of clarity I’ll use “expectancy effects_t” for therapeutic effects and “expectancy effects_se” for side effects (see fig. 2).Footnote ²

Figure 2.

Grünbaum’s basic model stays intact; however, expectancy effects are not limited to therapeutic effects, and side effects may impact D.

Returning to arthroscopic surgery and pace the subsequent controversy, we are now in a position to spell out the methodological role required to determine whether t is a treatment for D. With regard to ψ₁, only the lavage group serves as a suitable control. If the debridement group were compared with the incision group, a superior efficacy would not have established that the debridement alone was responsible for the improvement. According to ψ₁, the lavage group contains all of the incidental factors C of t and thus controls for any expectancy effects_t caused by C.

As it turns out, Moseley et al. (Reference Moseley, O’Malley, Petersen, Menke, Brody, Kuykendall, Hollingsworth, Ashton and Wray2002) found no difference between the lavage and the debridement groups. Nevertheless, this does not show that the procedure is ineffective; perhaps the lavage has an independent effect on osteoarthritis. If so, the lavage was not a placebo control but an effective therapy. Such a possibility can be assessed by examining the difference between the lavage and the incision group. This comparison is complicated by the fact that these groups had different levels of pain shortly after the surgery, allowing for the possibility of a sandstone effect. Fortunately, Moseley et al. assessed patients’ beliefs about which group they were in, as well as their expectations for a successful recovery, and found no differences in either. Thus, the incision treatment serves as a placebo control for the lavage treatment. Furthermore, as Moseley found no difference between the incision and the lavage, the latter was also a placebo for debridement.Footnote ³

In summary, the methodological definition of a placebo is as follows: t* is a placebo in an RCT testing t for D iff (1) t* methodologically controls for the expectancy effects_t caused by t,Footnote ⁴ (2) none of the incidental factors of t* have an independent therapeutic effect, and either (3a) t* produces all of the same side effects as t or (3b) despite the failure of 3a, patients’ beliefs about which treatment they are receiving are the same for both t and t*. It is worth noting a number of nonstandard implications of this definition. First, whether t* succeeds in serving as a placebo for t should be supported by evidence. Second, contra Grünbaum, it is not enough for the placebo to simply lack the characteristic factors of t; it must share the incidental factors. Moreover, a placebo can, and in many cases must, cause side effects independent of any expectancy effects. For example, if patients in the control group did not have to heal from the incision, the procedure would fail to serve as a placebo control by virtue of the fact that it caused no side effects. I think that one can, if one wishes, maintain that a “placebo effect” that is independent of a patient’s expectation violates one’s concept of a placebo (i.e., placebos should be inert); however, as alluded to above, I do not think that one can maintain that position and continue to have placebos serve the methodological role they currently occupy in medical research.

5. An Alternative to Placebos

The view rehearsed above cuts against current trends in medical research. Recently, in light of the fact that blinding consistently fails in practice, the CONSORT 2010 guidelines retracted their previous recommendation for trials to test to see whether blinding was successful, claiming, “We now believe the interpretation of a test of its [blinding] success to be dubious, if not impossible” (Schulz et al. Reference Schulz, Altman, Moher and Fergusson2010, 1145). They claim that because blinding might fail owing to patient improvement, such a test may simply reflect the efficacy of the drug instead of the methodological failure of the trial. Recall that the logic of RCTs is that to attribute the efficacy to the drug, the only difference must be characteristic factors. Thus, the CONSORT 2010 guidelines essentially confront a possible failure of the central assumption required by the statistical analysis used to assess efficacy with the unsubstantiated hope that everything will work out.

Fundamentally, this problem is an outgrowth of the fact that inert substances often fail to serve as adequate placebos. If RCTs used a placebo in the sense I propose, no such ambiguity would arise, as the therapeutic effect would be the only possible cause of unblinding. However, since most drugs have side effects, so would most placebos. Nevertheless, for either practical or ethical reasons, it may not be possible to give patients a substance that causes all of the negative effects known to be caused by treatment, but without any of the characteristic factors hypothesized to be beneficial. An alternative to the methodological control is to administer an inert substance (i.e., not a placebo) and attempt to control for these effects statistically.

Consider a trial on antidepressants carried out by the National Institute of Mental Health that used an inert substance in the control group.Footnote ⁵ Given the methodological definition of a placebo, whether the trial was in fact placebo controlled must be substantiated. Since the control group was given a lactose pill that was identical in appearance to the antidepressant, the design of the trial provides prima facie evidence that the placebo group controlled for all of the incidental factors of antidepressant treatment without containing any independent therapeutic effects. However, while the first two conditions of a placebo were met, patients in the antidepressant group experienced far more side effects. Moreover, patients were able to guess which group they were in at levels far above chance. In short, the trial was not a placebo-controlled experiment, and the possibility that the observed superiority is due to a sandstone effect cannot be ruled out.

The ability of participants to correctly identify which group they were in does not entail that the drug was ineffective, but the standard practice of comparing group means is not justified. Instead, a more complicated procedure should be employed to achieve statistically what a placebo is intended to achieve methodologically. The null hypothesis is that there is no therapeutic effect of the drug. Accordingly, if patients were able to identify which group they were in on the basis of therapeutic improvement, then the null hypothesis is false (the treatment is effective) and the logic of the RCT is not threatened (Howick Reference Howick2012). In line with such reasoning, proper analysis must ascertain the cause of patients’ beliefs about group assignments. If, as in the antidepressant trial, patients’ guesses were the result of differences in side effects, a sandstone effect cannot be ruled out and we cannot reject the null hypothesis.

In such cases there is a solution that can be carried out with the type of data already collected in many trials. Instead of testing group differences, structural equation modeling (SEM) or regression analysis can be used to test a proposed model of symptomatic improvement. Specifically, statistical analysis can be used to ascertain whether one variable acts as a mediator between two other variables (Holmbeck Reference Holmbeck1997; MacKinnon et al. Reference MacKinnon, Lockwood, Hoffman, West and Sheets2002).

First, suppose that we have three variables A (treatment), B (belief about group assignment), and C (symptomatic improvement) that are each highly correlated, and we wish to determine whether A has an independent effect on C. This results in a model structure with three paths as in figure 3. The arrows represent possible effects in the model. We have evidence of the sandstone effect if four conditions hold: (1) A is a significant predictor of C, (2) A is a significant predictor of B, (3) B is a significant predictor of C after controlling for effect of A on C, and (4) A is a significantly poorer predictor of C when B is controlled for.

Figure 3.

Possible causal model in which the treatment has both a direct effect on symptomatic improvement (the arrow between A and C) and an indirect effect via the treatment’s effect on patients’ beliefs concerning group assignment (the arrows from A to B and B to C).

To get a feel for the model, consider the crystal palace thought experiment. To assess crystal healing, let A be the actual treatment, let B be which treatment the patient believes they are receiving, and let C be the degree of symptomatic improvement. In the supposed situation, sandstone causes an expectancy effect, so there will be a relation between A and C. Further, since patients used abrasions to discriminate between “real crystal healing” and placebo crystals, there will be a relationship between A and B. Further, all of the variance in improvement is explained by people’s confidence that they are receiving “real crystal healing,” so there will be an effect of B on C (since crystals do not actually heal, controlling for their contribution to healing [A] will not change the effect of B on C). Finally, the effect of crystals on healing disappears when we control for belief, so A is a significantly worse predictor of C when B is controlled for. Thus, all four conditions are met.

With this more sophisticated analysis, we can see that the pessimism of CONSORT 2010 is unfounded. With sufficient data, it is entirely possible to identify the cause of unblinding and subsequently tease apart the direct effect of the treatment from the expectancy effect caused by side effects. In the trial of antidepressants, it turns out that patients’ guesses are driven by side effects and not improvement. Patients who received antidepressants improved more than the control group, and patients who believed they were receiving antidepressants improved more than patients who believed they were receiving placebos. Crucially, once what a patient believes is accounted for, there is no added benefit to actually receiving antidepressants. In short, this trial, once properly controlled for statistically, provided no evidence that antidepressants were an effective treatment. The apparent superiority of antidepressants in this case is caused by the sandstone effect. Given that the patients (and doctors) could determine which group the patients were in by the presence of side effects, the lactose pill does not meet the methodological criteria to be considered a placebo; however, we might reasonably call trials analyzed in this fashion statistically controlled RCTs.

6. Conclusion

The standard meaning of “placebo” stands in need of revision. All too often trials are called “double-blind” or “placebo controlled” on the basis of their design. There is now significant evidence that for any condition that is susceptible to expectancy effects treated by a drug that causes side effects, providing an inert substance to the control group will fail to warrant the assumptions that are standardly used to analyze the data. The same rationale that motivated the introduction of inert substances into trial design can be marshaled to argue for a truly adequate placebo. Nevertheless, ethical or practical concerns may militate against the use of such placebos. In such cases, statistical controls can be used to attenuate methodological shortcomings. In either event, trials must be described not by their intended design but by what conditions actually obtain, and reporting guidelines should be changed accordingly. If researchers accepted such methodological strictures, then it is overwhelmingly likely that they will conclude that most sugar pills are not placebos.