1. Introduction

Shapiro and Morris (Reference Shapiro, Morris, Garfield and Bergin1978) make the point that for thousands of years, physicians prescribed medications that were useless and often dangerous, and that at the beginning of the twentieth century few medical interventions were more effective than a placebo. Proof of effectiveness was slow to come. In 1978, the U.S. Congress's Office of Technology Assessment (OTA) estimated that only 10% to 20% of all procedures currently used in medical practice had been shown to be of benefit by controlled clinical trial (1978, 94). Before the OTA closed its doors in 1995 due to a withdrawal of congressional funding, it issued one final report, concluding that much, if not most, of existing medical technology and practice had been inadequately evaluated with respect to its effectiveness (1994, 37). As to the percentage of procedures studied in formal trials, the report concluded: “The longstanding estimate that only about 10 to 20 percent of procedures have ever been formally evaluated for safety and efficacy … remains a rule of thumb” (21).

In the 1990s, more medical treatments were studied in Randomized Clinical Trials (RCTs), partly due to the influence of those, such as Archie Cochrane (Reference Cochrane1972), who advocated “evidence-based medicine” and strongly supported the use of RCTs in evaluating medical interventions.

The epistemic situation in psychotherapy, the discipline encompassing all forms of psychological treatment, mirrors medicine in some important respects. Despite Josef Breuer's contention that he had demonstrated the effectiveness of his talk therapy with his patient Anna O. and Freud's claim to have shown through his clinical case studies that psychoanalysis was effective, there was still no credible evidence even in the mid-twentieth century for the effectiveness of any form of psychotherapy (Eysenck Reference Eysenck1952). Some progress was made in the next several decades, due largely to improved research techniques developed by behavior therapists and cognitive therapists, but as late as 1997, the effectiveness of more than 90% of the over 400 psychotherapies remained in doubt (Erwin Reference Erwin1997, Chapter 8).

In the past 10 years, much progress has been made, but the effectiveness of most of the psychotherapies, on my view, is still not backed by firm empirical evidence. The cure, or at least an important part of the cure, for psychotherapy's problems might appear to be the same as in medicine: Use randomized clinical trials to identify effective techniques and weed out ineffective ones. Yet there are important differences between the two disciplines. Despite some recent doubts raised in the medical literature, there is widespread agreement that well designed RCTs (randomized clinical trials) provide the best form of evidence and, at least for pharmaceutical treatments, a necessary sort of evidence (but see the final section for some doubts). In contrast, many psychotherapists (such as Gergen Reference Gergen1994; Mahrer Reference Mahrer1988; Seligman Reference Seligman1995; Sandell et al. Reference Sandell, Blomberg, Lazar, Carlsson, Broberg and Schubert2000; Freedman et al. Reference Freedman, Hoffenberg, Vorus and Frosch1999) and some philosophers (Nagel Reference Nagel1995; Levy Reference Levy1996; Hopkins Reference Hopkins and Neu1991) have challenged either the necessity or sufficiency of well designed psychotherapy experiments, whether RCTs or not, in assessing psychotherapeutic effectiveness. There is more here, moreover, than mere clashes of opinion; there are impediments to using RCTs in psychotherapy research that are relatively inconsequential in medical research.

In this paper, I focus on several philosophical issues about employing RCTs in psychotherapy outcome research. I conclude with a brief assessment of the overall case for and against their use.

2. The Impermanence Problem

In his Outlines of Psychology (Wundt Reference Wundt[1897] 1969), Wilhelm Wundt argued that if psychology is to progress, it must become an experimental science. He rested his case not on the principle that all true sciences are experimental but on the impermanence of psychology's subject matter. Psychology, he argued, deals exclusively with processes not permanent objects; for this reason, we must be able to bring about their conditions at will and vary conditions of their occurrence. Doing these things, Wundt argued, requires experimentation (20). It is ironic, then, that some contemporary opponents of Wundt's experimental ideal appeal to the impermanence of human action to show just the opposite of what he intended: Because of impermanence, they argue, psychology, and by implication psychotherapy, cannot be an experimental science.

Kenneth Gergen (Reference Gergen1994) has given the most detailed and influential version of the impermanence argument. The major function of science, Gergen notes (7), is to construct general laws or principles governing the relationship among classes of observable phenomena, but the execution of this function depends on the stability of the relationship among events in nature. Without stability, the goal of establishing laws cannot be met. Yet molecules and electrons are also in constant flux. If instability is a threat to the social sciences, why is it not also a threat to the natural sciences? Gergen replies that there are several differences that explain why the threat is much greater with respect to the study of human conduct.

The first difference has to do with stereotypical and flexible responses. The behavior of lower organisms, say flatworms, is stimulus bound. Given the same class of stimuli, and same initial conditions, the same sort of response will be forthcoming. The result is that the behavior of flatworms is stereotypical. If a stimulus-response theory of human behavior were correct, human behavior would be stimulus bound in the same way, but there is good reason to reject that sort of approach. Even given the same external conditions, Gergen argues, two human beings can react quite differently. Human behavior, unlike that of flatworms, is flexible.

The standard cognitivist response to the lack of stimulus boundness on the part of humans is to appeal to the possibility of developing stable principles of cognition that would explain variation in responses to the same stimuli. Gergen (Reference Gergen1994, 16) discounts this possibility on empirical grounds, partly by appealing to the flexibility of humans in talking, eating, playing, etc., and in developing cognitive heuristics in solving problems.

Two other differences between natural science and social science phenomena concern the restructuring of views of reality and the erosion potential of common values. As to the first of these alleged differences, neither molecules nor insects develop elaborate conceptual schemes to transform the meaning of stimulus conditions, but humans, Gergen points out (170), are capable of doing so in an indeterminate number of ways, making existing behavioral regularities subject to change as interpretations of stimuli change. As to the erosion potential of common values, Gergen appears to have in mind the destructive effect of certain basic desires on existing patterns of behavior. He gives as an example (19–20) the desire for freedom from restraint and its potential for disrupting existing patterns of conduct. Gergen is unclear on this point, but the idea appears to be that as people become expected to “follow the common pattern” (20), they see their desire for freedom being thwarted, thus motivating them to upset existing patterns of behavior.

After making his case for radical instability, Gergen concludes:

Taken as an argument against the use of controlled experiments to establish truth claims in psychotherapy, a connection needs to be made between the impermanence thesis and the skeptical conclusion about experimental inquiry. Gergen draws the connection by claiming (7) that the major function of science is to construct general laws. If one accepts that this is the function of experimental science, then one can make the case for the futility of experimental inquiry by appealing to Gergen's thesis about the non-existence of laws of human action, assuming he has supported that thesis.

Gergen does not explain what he means by a “general law,” but he appears to be thinking of causal laws that do not take universal, non-statistical form: For any x, if F and certain initial conditions, I, then G. But why believe the sole function of experimental inquiry to establish general laws of this form? Gergen gives no reason except for a quotation from Braithwaite—“The function of science … is to establish general laws …” (Braithwaite Reference Braithwaite1953, 1).

If psychotherapy researchers could confirm hypotheses having a universally quantified form, they might be able to establish that certain types of events are causally sufficient for certain kinds of outcomes. For experimental inquiry to be useful, however, it is not necessary to establish causal sufficiency; it is enough that causal relevance can be established.

Psychotherapeutic outcome hypotheses, for example, can and often do take the following statistical form: Treatment T, under conditions C, makes a positive causal difference to the remediation of problem P in such and such a percentage of cases. There is nothing in Gergen's argument about human change to show that statistical hypotheses of this form cannot be true, or that experimental inquiry cannot be useful in assessing their truth value, or even that they cannot qualify as statements of laws of nature.

In brief, in order for experimental inquiry to be useful in assessing psychotherapy truth claims, it need not be used to establish exceptionless, non-statistical laws of human conduct. It is enough that such inquiry can establish or falsify hypotheses asserting that one kind of event is causally relevant to the occurrence of another.

Instead of making the argument hinge on the alleged purpose of scientific inquiry, some might fall back on an ontological principle once favored by Davidson (Reference Davidson and Crumley2000, 431): “[W]here there is causality, there must be a law: events related as cause and effect fall under strict deterministic laws.” Davidson later relaxed this strict deterministic requirement, but at the time he stated it, he may have been thinking of events that are causally sufficient. Once we switch to speaking of causally relevant factors, Davidson's principle becomes either implausible or useless to Gergen. There is plenty of evidence for the operation of psychological factors that are causally relevant to psychotherapy outcomes. This evidence undermines Davidson's principle if interpreted to mean that all causal relationships, even those involving merely causally relevant factors, presuppose strict deterministic laws. If interpreted to apply only to causally sufficient factors, the principle is of no help to Gergen's argument; the usefulness of RCT's does not require that the factors being studied be causally sufficient.

Gergen's argument about human change is relevant to the issue of whether there are laws in the social sciences and to the question of whether there can be a science of psychotherapy, but it is hardly decisive on either issue. Opposing arguments need to be considered (see especially Kincaid Reference Kincaid1996 and McIntyre Reference McIntyre1996 for a defense of the possibility of laws in social science). The issue here, however, is not the existence of social science laws, but the implications for RCT research in psychotherapy of saying that they do not and cannot exist. Even if Gergen were to prove that there are no social science laws waiting to be discovered, there is nothing in his argument about human change to show that experimental research is not useful in assessing the truth of psychotherapeutic hypotheses. Taken as an anti-RCT argument, Gergen's argument must then be judged a failure.

3. The Identification Problem

As many philosophers have pointed out, when people perform an action, they typically do more than one thing. I punch a voting card and thereby vote for a presidential candidate. But is that all I am doing? I am also likely to be voting for a particular candidate. Then, again, I may be protesting what my government has been doing, or voting to support my political party, or doing what my wife asked me to do, or perhaps doing all of these things, and other things, all at once. Unless an experimenter has some way of singling out one of these descriptions as being “the right one,” some argue, there is a problem in doing an experiment about my action or any other human action that admits of multiple descriptions. The problem arises in specifying the dependent variable when it is a human action.

Charles Taylor (Reference Taylor1985) argues that one way to solve this problem is to give “brute data descriptions”: “His hand appeared to be moving toward the punch card in a downward motion, etc.” But if we restrict descriptions of human action in this way, Taylor argues, we leave out too much; for on another level, actions have meaning for the agent that is not captured by brute data descriptions.

Gergen (Reference Gergen1994) uses the following example to illustrate the identification problem. Suppose that I see Ross touching Laura's hair. What, Gergen asks (60), have I observed? What does the action suggest about the relationship of the two people? I could limit myself to a physical description of what went on: At such and such a time, a hand moved in such and such a direction, and came in contact with hair, etc. But as Taylor had earlier pointed out, this level of description is virtually uninformative. It does not tell me what it means to engage in such an action, nor can the meaning be derived merely from the physical description.

The next step in Gergen's argument is to show that classification of the action will be relative to background beliefs, and different classifications will be warranted as I change my beliefs when new information is obtained. For example, if Ross had earlier told me that he was in love with Laura, I might confidently view his action as a sign of affection. However, suppose that I later learn from Laura that she had told Ross that he was cold and unexpressive. That bit of information changes things. Now I might see Ross's action not as a sign of affection but as an attempt to demonstrate that he is not so cold. Suppose that I obtain other new information to the effect that Laura and Ross had been lovers but that Laura had recently terminated the relationship. This piece of information might reasonably cause me to change my mind again. Gergen also shows how further changes in my background beliefs can warrant further revisions in my description of the action.

This is but a single case, but on the assumption that it typifies the problem in identifying human actions, what does it show? Gergen draws from it three propositions of “substantial significance” (62–64). The first is that the identification of any given action is subject to infinite revision (62); the second is that the anchor point for any given identification is not fundamentally empirical but relies on a network of interdependent and continuously modifiable interpretations (63); and the third is that any given action is subject to multiple identifications, the relative superiority of which is problematic (64).

Gergen's first principle, about the possibility of infinite revision, seems right, but not “momentous.” It is just the familiar point that observational evidence rarely if ever yields certitude, and empirical beliefs, consequently, may have to be revised as new evidence comes in. This is far different from saying that descriptions of actions can never be warranted empirically. In the case of Ross and Laura, we might, at first, be warranted in describing Ross's action as “his deliberately touching Laura's hair,” but new evidence indicating that the touching was accidental might warrant our rejecting our initial description. Explanatory descriptions may also be initially confirmed but subject to revision if disconfirming evidence were to be obtained. The possibility of multiple revisions exists, but by itself presents no special problem for an investigator trying to warrant some particular description of an action—not the description but just a description.

Gergen's second principle appears to be partly right and partly wrong (at least if we ignore for a moment his argument about “intelligibility rules”). As Gergen points out, the identification of an action relies on a network of interdependent and continuously modifiable interpretations, but, contrary to what his second principle says, this does not make identifications “not fundamentally empirical.” As I noted earlier, the identification of Ross's action as his deliberately touching Laura's hair might be supported by what I saw. I saw him, let us suppose, walk up to Laura's desk, take off his glove, and stroke Laura's hair. As to the further description, that he was trying to express his love, this too can be warranted by what he tells me. Here I am tacitly supposing that Ross is not lying and knows why he stroked Laura's hair, but these background assumptions can also be warranted empirically. There is nothing in Gergen's description of the case to show that this is not so.

Gergen does, however, have a separate argument. He says (62) that the potential for infinite revision is dependent not only on the continuously unfolding range of “contextual indicators” (which are nothing more than background assumptions and empirical observations), but also on “intelligibility rules.” Intelligibility rules are supposed to be rules of agreement concerning the meaning of one event in relationship to a second.

Ross's action, Gergen points out (62), bears no intrinsic relationship to his declaration of love. The declaration bears on our interpretation primarily because in our culture we share an “intelligibility rule,” Gergen claims, encouraging us to accept the proclamation of love and the touching as a signal of the underlying emotional state. Yet to those exposed to psychoanalytic thought, Gergen points out, it would seem plausible to accept a different intelligibility rule, one linking Ross's being weaned at an early age to his expressing his devotion by touching Laura's hair. Freud, Gergen claims, stated intelligibility rules that enable the connection to be made to infantile states. Because intelligibility rules vary from culture to culture, or from sub-culture to sub-culture, how people make sense of an action, Gergen argues, will vary with the culture.

Although I disagree with Gergen's interpretation of Freud (see below), he is right that someone exposed to psychoanalysis might argue that because Ross was weaned at any early age, the act of touching Laura's hair was one of devotion. But what is being claimed here? Is it that there is a causal connection between Ross's early weaning and his attempt to show devotion by touching Laura's hair? If the answer is affirmative, then we can ask what empirical evidence the observer has to support that belief. If the answer is “none,” then the person has no reason to give this particular interpretation; for all that the person knows, there is absolutely no connection between the early weaning and the touching of the hair.

Suppose, however, that the person withdraws the causal claim. In that case, the person is again left without grounds for saying that Ross was expressing devotion to Laura because he was weaned early. For all that Gergen has shown, people exposed to Freud's writings might have certain background beliefs, typically causal beliefs, which appear to them to justify certain descriptions of human actions, but unless they have evidence to back up these beliefs, their descriptions are unwarranted. I may make Ross's action intelligible to myself by saying that in touching Laura's hair he is expressing oral dependency, but if he is not oral dependent, or if he is but oral dependence played no causal role in what he did, my description is not merely unwarranted: It is wrong.

There is nothing in Gergen's talk of “intelligibility rules,” then, to support the idea that no particular description of an action can be justified empirically; speaking of such rules adds nothing of substance to the discussion. Freud does try to make human action intelligible, typically by appealing to his theory of repression, but the principles he states are causal principles outlining the effects of repressed infantile wishes on subsequent behavior; they are not intended to be relativistic; and their credibility, if any, rests on empirical observations and inferences.

Gergen's third, and final, principle says that any given action is subject to multiple identifications, the relative superiority of which is problematic (p. 64). What does this mean? Gergen explains the principle as follows: Given the ultimate lack of an empirical touchstone on which to rest competing interpretations, he contends, we cannot easily argue for the superior validity of one description as opposed to another. This claim about the lack of empirical evidence for competing descriptions, however, presupposes Gergen's earlier arguments which I have already answered. If I describe Ross's action of touching Laura's hair as “expressing affection for her” and someone else describes it as “not expressing affection for her but instead as an attempt to humiliate her in front of her co-workers,” we might both be wrong, but if one of us is right, there is nothing to rule out the possibility of validating the correct description by obtaining empirical evidence as to Ross's intentions.

The case is different, of course, if the descriptions are not in competition. Ross may be touching Laura's hair and expressing affection. Although neither possesses “superior validity” given that both are correct, one description might be preferable in a certain situation and the other in a very different situation. In brief, two descriptions of an action are either incompatible or compatible. If incompatible, we can decide which is correct by appeal to empirical evidence. If the descriptions are compatible, then both may be correct and both may be supportable by appeal to empirical evidence

In a psychotherapy outcome experiment, an experimenter can concede that there are multiple correct ways to describe what a subject was doing prior to receiving therapy and that none of the true descriptions possesses what Gergen calls “superior validity.” Yet the research hypothesis may dictate that a single description is of interest. An agoraphobic who never left her house for six months prior to undergoing therapy may have been doing many things while staying at home, but what might be of interest to the experimenter might simply be the accuracy of the description “staying at home.” Gergen gives no reason for thinking that the researcher can not establish empirically that this description is correct.

I conclude that there is nothing in Gergen's arguments to show that a researcher can never warrant any particular description of an action by appeal to empirical evidence or that a researcher needs to demonstrate that one and only one description of an action is the true one. As a further argument designed to cast doubt on the value of experimental research in psychotherapy, it too leads nowhere. There are problems in characterizing therapies and outcomes that tend to be more troublesome in using RCTs in psychotherapy research compared to medical research (see below the discussion of therapy integrity), but none of these problems presents a deep philosophical difficulty, or one that rules out the use of RCTs.

4. Idiographic Problems

The distinction between two kinds of sciences, the nomothetic and the idiographic, was introduced by Wilhelm Windelbrand in his 1894 lecture “History and Natural Science” (Reference Windelband1998). Those who work in the nomothetic (or natural) sciences, according to Windelbrand, seek to establish general laws of nature, whereas in the idiographic (or human) sciences, attempts are made to explain unique events. Although Windelbrand did not specifically address the status of psychotherapy research, he held that psychology—hence by implication psychotherapy—is an idiographic discipline and hence not a natural science.

Discussions about the idiographic have now gone far beyond the issue of whether psychotherapy can be a natural science. Hundreds of journal articles on the idiographic have appeared in the last dozen years, raising a bewildering number of issues about idiographic factors and problems, the nature of psychotherapy, the usefulness of the diagnostic categories employed by the American Psychiatric Association and used in much psychotherapy research, and the legitimate role, if any, of psychotherapy experiments (see, for example, Held Reference Held1995; Levine, Sandeen, and Murphy Reference Levine, Sandeen and Murphy1992; Grice, Jackson, and McDaniel Reference Grice, Jackson and McDaniel2006; Persons Reference Persons2006; and Philips Reference Philips2005). Those skeptical of randomized clinical trials often explain their lack of value by pointing to idiographic problems, specifically the uniqueness of the causal factors of interest to therapists or the uniqueness of the patients themselves.

Concerns about uniqueness in psychotherapy have a long history. More than forty years ago, the psychologist Gorden Allport wrote “… the dilemma of uniqueness haunts the house of clinical psychology” (quoted in Levine, Sandeen, and Murphy Reference Levine, Sandeen and Murphy1992, 410). More recently, Nagel (Reference Nagel1995) has pointed out that much of our mental life consists of multiple causes and background conditions that never will precisely recur in anyone else. This creates a problem, Nagel claims, in trying to use in psychotherapy research the experimental methods used in organic medicine: How are we to do repeatable experiments, Nagel asks, to gauge the effects of these unique events? To develop such evidence, he notes, we would have to apply general methods to the unique circumstances of the case.

Psychotherapists often disagree about how informative psychotherapy experiments typically are (for doubts about the value of RCTs in psychotherapy research, see Seligman Reference Seligman1995; Levine, Sandeen, and Murphy Reference Levine, Sandeen and Murphy1992; and Mahrer Reference Mahrer1988), but to the degree that they are less informative than typical medical experiments, the idiographic theory offers what appears to some to be a plausible explanation. With the possible exception of those doing group therapy, psychotherapists want information that is useful for treating an individual client having characteristics and problems that to some degree are different from every other client, but in RCTs, the focus is on mean outcome differences between treatment and control groups.

Idiographic problems also arise in diagnosing a client's problems. Suppose that a female client comes to me for treatment for her recurrent panic attacks, which are usually followed by persistent concern about having additional attacks. Her attacks appear unrelated to any general medical condition or to the ingestion of a drug. I decide that she meets the DSM IV criteria for Panic Disorder without Agoraphobia. However, she is also very depressed and, for all I know, the panic attacks may well be causally related to her depression. So, she is different in possibly crucial ways from at least some patients generally classified as having panic disorder. Moreover, she lives in a dangerous neighborhood and was recently attacked while walking home at night. In fact, there are many differences between her and the average sufferer of panic disorder. Some of these differences are likely to be causally irrelevant to treating her condition, but given the current state of clinical knowledge, how can I be sure that others are not very relevant to the understanding of her problem and to the likely success of standard treatments studied in RCTs?

Because of such doubts, as well as others, many psychotherapists are skeptical about the use in experimental research of DSM categories (the Diagnostic and Statistical Categories of the American Psychiatric Association). In a paper on the idiographic and a critique of DSM categories, Levine, Sandeen, and Murphy contend that “Careful adherence to diagnostic reasoning seems irrelevant for many cases and can obscure idiographic factors” (Reference Levine, Sandeen and Murphy1992, 411).

Problems with DSM categories do not per se present a problem for using RCTs, the use of which clearly does not commit one to any single diagnostic scheme, but in so far as DSM classifications are used in RCTs, their idiographic problems will affect the clinician's attempt to interpret the experimental results.

Supporters of the idiographic theory are right in claiming that there is a problem in translating experimental information from studies of groups into useful information that applies to a unique individual. This problem would not arise if the focus of treatment were not a unique individual. On this single point, then, the idiographic theory is correct. Yet the theory does not explain the comparative differences in information content between experimental outcome research in medicine and psychotherapy for the obvious reason that the subjects of medical research and the causal factors that account for their individual medical problems are also unique; this uniqueness does not prevent medical researchers from obtaining information that is of great value to the clinician.

A rival theory is more plausible. It attempts to explain the relative disutility of psychotherapy experiments—to the extent that there is a general difference in useful content between medical and psychotherapy experiments—primarily in terms of factors that have nothing to do with uniqueness or the complexity of the human mind. These factors pertain to the nature of psychotherapeutic treatments, outcome measures, the variability of patient responses, and the diagnostic categories used in the field of psychotherapy.

In medicine, the treatments that are researched are much easier to standardize. If researchers study the effects of a pill on hypertension, and I choose the same therapy for my client, I can be reasonably sure that I am using the same kind of pill. If, however, psychotherapy researchers tell me that the effects of Rational Emotive Therapy or Psychodynamic therapy on anxiety conditions are positive, there is an immediate problem in applying this information in my next case. These therapies come in different forms, and are used differently by different therapists. There may, consequently, be uncertainty about whether my use of such a therapy will not be different from that of a therapist participating in an RCT; if our respective treatments do differ, this can matter to the treatment of my particular client. This problem, known as the “therapy integrity” problem, is often a serious one in psychotherapy outcome research; it can arise in medical research too, but it appears to be less serious.

The outcomes in medicine are also generally easier to recognize and to calibrate relatively precisely. If it is predicted that a cancer patient receiving chemotherapy will survive five years after treatment, it is easy to tell if the prediction turned out to be right. The patient will be either dead or alive, although if the prediction proves false, the death might have nothing to do with the cancer. If the prediction is that her blood pressure will fall after treatment, a physician can measure the precise number of points that it falls. It is sometimes not so easy, in contrast, to determine whether predicted psychotherapeutic outcomes have even occurred. For that reason, one prominent researcher agrees that objective methods can be used to establish the efficacy of pharmacologic treatments, but notes that objective standards cannot be reliably established for authenticating such putative entities as “regressive flight from Oedipal relationships” or “a condensation of aggressive and anti-aggressive forces” (Holzman Reference Holzman and Erwin2002, 460). Reactions to psychotherapeutic interventions, moreover, tend to be more variable than reactions to drugs or surgery, even if there can be plenty of variance in the medical cases as well.

The differences cited so far are a matter of degree and their import can easily be exaggerated. Sometimes it is difficult to determine if a predicted medical outcome has occurred and sometimes it is comparatively easily to tell what has occurred in a psychotherapy experiment. As the late Hans Eysenck, once remarked, if a child suffers from enuresis, it is not beyond the wit of man to tell if he or she has stopped wetting the bed. Yet it does appear to be generally harder in psychotherapy than in organic medicine to apply experimental research findings to the individual case. Assuming that this is so, the main reason it is harder is due to contingent differences between psychotherapy experiments and medical experiments, differences that can be diminished as RCTs become more and more refined; in some cases, the differences can be eliminated.

Take, first, the difference between a pill and a complicated psychotherapy such as Rational Emotive therapy or Psychoanalytically Oriented Psychotherapy. As noted earlier, if I am prescribing a pill, I can be assured that it is the same sort of medicine studied in certain RCTs (although problems about dosages have to be considered). In contrast, how can I be sure that Rational Emotive Therapy or Psychodynamically Oriented Psychotherapy, as I use them, are the same procedures as that studied by researchers? There is a way, however, to deal with this aspect of the problem. The solution is to use standardized therapy manuals both in experiments and in clinical practice. A start has been made in doing this (see Wilson Reference Wilson1996 for a discussion of some of the issues this development raises).

A second difference is that outcomes are generally, even if not always, easier to identify and measure in medicine than in psychotherapy. There is no single solution to this problem, but neither is it hopeless. The development of better outcome measures is an important scientific problem, but the idiographic aspect of psychotherapy hardly renders the problem unsolvable.

We already have reliable ways to determine whether patients suffering from the most common clinical disorders, such as depression, phobias, and anxiety states, have improved and we have reliable scales to measure the degree of improvement. As to Holzman's (Reference Holzman and Erwin2002) psychoanalytic examples, they may show that certain psychoanalytic hypotheses cannot be tested by using RCTs, but that is a problem for one theory, not for psychotherapeutic outcome research in general.

Concerning the point argued by Levine, Sandeen, and Murphy (Reference Levine, Sandeen and Murphy1992)—that the use of DSM categories can “obscure idiographic factors”—all general diagnostic categories whether medical or psychotherapeutic will necessarily omit information about the particular patient. The category “clinical depression” fits my patient but tells me nothing about unique facts about her that might be causally relevant to whether a certain treatment will work in her case. The obvious way around this difficulty is to execute not a single RCT but multiple trials, making it more and more likely that the treatment will work whatever the unique characteristics of the patient. Using randomizing techniques to select patients for study, not merely to assign them to treatment and control groups, can also help with problems of external validity.

I conclude that the ontological fact of uniqueness of the patient and the cause of his or her problems does not explain the relative difficulty of executing informative RCTs in psychotherapy outcome research, nor does it present an insuperable barrier to doing such experiments. The main problems are epistemological and conceptual, and they can be remedied by devising better theories and better experimental designs.

5. The Case for RCTs in Psychotherapy Outcome Research

To make the case for RCTs in psychotherapy outcome research, it is necessary to answer all of the main arguments against their use and then to provide a positive argument. I have dealt here only with a certain class of philosophical anti-RCT arguments, but the important philosophical and empirical arguments not discussed here have been addressed elsewhere. On the Breuer and Freud arguments, see Grünbaum Reference Grünbaum1984, Reference Grünbaum1993, Reference Grünbaum and Erwin2002 and Erwin Reference Erwin1996; on the appeal by Levy Reference Levy1996, Hopkins Reference Hopkins and Neu1991, and others to meaning relations as a substitute for controlled experiments, see Grünbaum Reference Grünbaum1993 and Erwin Reference Erwin1996; on the Consumer Reports model used in recent psychoanalytic research, see Erwin Reference Erwin and Jokic2006a; on the ethical issues, see Erwin Reference Erwin1988; on empirical issues raised in the recent medical literature, see Erwin Reference Erwin2006b).

As to constructing a positive case, there are at least two ways to proceed. If one is interested mainly in the reliability of research designs, as is likely to be true of those who accept a reliabilist account of empirical justification, then the key issues are empirical. As of now, the traditional reliabilist argument for requiring the use of RCTs in medical research—namely, that there is a discrepancy between the results of RCTs and the seemingly less reliable observational studies, with the latter yielding overly optimistic results—is challenged by new findings (Concato, Shah, and Horowitz Reference Concato, Shah and Horowitz2000; Kaptchuk Reference Kaptchuk2001; Benson and Hartz Reference Benson and Hartz2000). Benson and Hartz (Reference Benson and Hartz2000) looked at 136 clinical reports about 19 diverse treatments and found that in most cases (17 out of 19) the estimates of treatment effects from observational studies and controlled trials were similar. They also report that the United Kingdom Health Technology Assessment Group found eight treatments that were the subject of a randomized clinical trial and of an observational study with a control group. In seven out of eight of these treatments, there were no differences between the results of the RCTs and the observational studies. Concato, Shah, and Horowitz (Reference Concato, Shah and Horowitz2000) examined 99 reports and found that the average results of the observational studies were remarkably similar to those of the RCTs.

To the extent that these recent findings hold up, the undermining of the discrepancy argument has implications for psychotherapy outcome research, but exactly what they are needs to be determined. In the review by Concato, Shah, and Horowitz (Reference Concato, Shah and Horowitz2000), for example, certain types of rigorous observational studies, those using either a cohort design or a case-control design, were found to be as reliable as RCTs; observational studies using historical controls, of the sort used in recent studies of psychoanalysis (Sandell et al. Reference Sandell, Blomberg, Lazar, Carlsson, Broberg and Schubert2000; Freedman et al. Reference Freedman, Hoffenberg, Vorus and Frosch1999), were excluded from the sample. If cohort designs or case-control designs prove to be as reliable as RCTs in studying psychotherapy outcomes, then, in so far as reliability is the sole criterion, RCTs would lose their edge and might be inferior on the basis of costs or ease of implementation, but the reliability of historical controls or uncontrolled clinical case studies would still be in doubt. Other issues about what lessons to draw for psychotherapy research from the recent medical findings also need to be examined (for further discussion, see Erwin Reference Erwin2006b).

Issues about reliability, however, are not the only epistemological issues. If it is stipulated that a belief is justified if and only if it results from a reliable belief-forming mechanism, then there remains an important epistemological issue about RCTs that does not involve justification in this reliablist sense: Are RCTs generally needed to provide firm evidence for psychotherapeutic outcome hypotheses? The answer might be affirmative even if some other type of evidence, such as expert opinion or findings from certain types of observational studies, were just as reliable as RCT-findings, but their reliability had not yet been demonstrated. Issues about reliability and evidence are related but not identical.

In pursuing the evidential question, it seems wise to give up at the outset a rule that says that RCTs are always required for obtaining firm evidence for effectiveness. In medical research, for example, certain types of dramatic and sudden effects can be attributed to surgical procedures without doing any controlled experiments. If a patient has a cancerous tumor and it suddenly vanishes after surgery, there is generally no reasonable doubt that the surgical procedure caused the elimination of the tumor, even if controlled studies are needed to see if there are effects on mortality rates. In psychotherapy outcome research, these kinds of sudden and dramatic effects are not very common, but where one finds them, a causal hypothesis might be confirmable without experimental evidence. Another problem is that exceptions need to be made for single subject experiments using the subject as both treatment and control, assuming that there is both internal validity and a satisfactory procedure for generalizing to other cases.

There are other problems in stating a general rule about RCTs, but even if it were impossible to state any informative general rule applicable to all psychotherapy procedures, generality can be found by relativizing rules to procedures and DSM diagnostic categories. For example, I believe a strong case can be made that without RCT-evidence, it is unlikely that strong empirical support can be found for psychological or pharmaceutical procedures used to treat clinical depression, anxiety states, or phobias. Part of the case is philosophical and part empirical.

One way to argue the philosophical part is to articulate and defend certain evidential principles. One of these, Salmon's (Reference Salmon1984, 185–186) principle of causal relevance (see Erwin Reference Erwin1996, 73–84), can be employed to show that controlled experiments of some sort or other are needed to confirm certain types of causal hypotheses. Although the principle needs to be fine-tuned, it says roughly this: Except where there are “countervailing causal factors” (see Erwin Reference Erwin1996, 82), to establish that X is generally causally relevant to the occurrence of Y, it is necessary to establish for some reference class, c, that the frequency of Y's is greater in the subset of c's in which X's occur than in the subset of c where X does not occur.

A second (“differential”) principle requires that all known credible rivals to the research hypothesis be discounted (Erwin and Siegel Reference Erwin and Siegel1989; Erwin Reference Erwin1996, 44–54). Other principles that are more controversial but, I believe, still defensible, concern negative checks on inferences to the best explanation and appeals to pragmatic factors such as simplicity and explanatory power (see Erwin Reference Erwin1996; Sober Reference Sober1990).

If the evidential principles are assumed, an empirical argument can then be made that for the diagnostic categories mentioned earlier, one or more of these evidential principles would be violated if the evidence did not come from RCTs. A further empirical argument can be made that other standard features of RCTs besides randomization, such as double blinding (or where feasible triple blinding) and the inclusion of a credible placebo control, also need to be included to satisfy the same evidential principles. However, even after we relativize rules about RCTs to certain procedures and diagnostic categories, we might still need room for very exceptional cases where the epistemological function served by randomization or other typical features can be served by some epistemological substitute (on exactly what function randomization serves, see the competing views of Worrall Reference Worrall2002; Abel and Koch Reference Abel and Koch1999).

In short, the arguments against using RCTs in psychotherapy outcome research are not convincing, and a firm positive case can be made for the necessity of their use provided that the requirement is relativized to certain types of procedures and diagnostic categories. Although the foregoing sketch is somewhat vague, more details are provided in Erwin Reference Erwin2006b, where the issue of the sufficiency of RCTs is also taken up.