Popper's philosophical principle of falsification is commonly represented as seeking and finding counter-examples to a hypothesis, as Oaksford & Chater (O&C) show in Bayesian Rationality (henceforth BR, Oaksford & Chater Reference Oaksford and Chater2007, p. 167). Indeed, general laws can be definitively falsified, but not definitively confirmed, according to logic. Hence, counter-examples give us sure knowledge that the putative hypothesis is false. Large parts of Popper's work are directed at uncovering the failure of confirmation theory. However, philosophers and psychologists rejected the principle of falsification as obviously incompatible with scientific practice and everyday reasoning. In a natural environment, inside or outside the scientific lab, we cannot live with falsified beliefs alone. Falsified ideas are not publishable and they give poor cues about how to cope with the everyday environment (Poletiek Reference Poletiek1996; Poletiek & Berndsen Reference Poletiek and Berndsen2000). However, as O&C show, old theories of confirmation have also failed to provide a rational account of hypothesis testing. It was not before the introduction of the Bayesian approach in confirmation theory that a coherent alternative to the falsification method could be advanced.Footnote ¹ O&C's information gain model provides such an account for hypothesis testing that predicts test preferences in accordance with the information they can provide for the truth and falsity of hypotheses.

There is, however, an interesting alternative version of the idea of falsification, hesitatingly proposed by Popper himself. It avoids the obvious problems of Popper's logical falsification principle, and fits in with O&C's Bayesian approach. (Poletiek Reference Poletiek2001). This alternative definition of falsification is putting hypotheses to severe tests. I argue here that Severity of Test (SoT) provides an intuitive statistical model for what people do when they test hypotheses. As Figure 1 below shows, the formal measure for SoT Footnote ² (Popper Reference Popper1963/1978; for a discussion, see Poletiek Reference Poletiek2001) for a piece of evidence e predicted by the hypothesis H is:

Figure 1.

What distinguishes SoT from logical falsification, and to what behavior does it correspond? First, SoT is about evidence that confirms the hypothesis put to the test, and not evidence that is expected to falsify it. The severity of a test increases as the prior probability of confirmation e is low and its likelihood under assumption of the hypothesis is high. Minimizing the prior probability of a confirmation corresponds to a falsifying strategy. Second, it can be shown that the measure S is mathematically equivalent to the traditional Bayesian measure for degree of confirmation, that is, the revision of belief in H, after observing e Footnote ³ (see Poletiek Reference Poletiek2001) (Fig. 2 below):

Figure 2.

This equivalence is probably unacceptable for a logical falsificationist, but for modeling hypothesis-testing strategies it is quite helpful. Indeed, it implies that increasing the severity of test, prior to the actual test, will result in an equal amount of increased confirmation after e has been observed. Third, the definition of test severity shows that the probability of a predicted confirmation e and its severity S are inversely related. Hence, everything being equal, as a confirming prediction is less expected a priori, it will provide a better confirmation of the hypothesis. This is in line with the information gain perspective. But, of course, the risk of not finding that confirmation increases as well. Furthermore, if we assume a test with two possible outcomes (a confirmation e and a falsification), the probability of the falsifying outcome increases as P(e) decreases. This uncovers a Popperian paradox: maximizing the probability of a falsification necessarily minimizes its power to reject the hypothesis (Poletiek Reference Poletiek2001).

Finally, SoT models hypothesis testing without specifying the choices people make, because these choices are assumed to depend on utilities regarding the values of test outcomes. Let us suppose a hypothesis H that generates several predictions with likelihoods 1 (P(e|H)=1). SoT suggests that choosing a test involves solving the trade-off between the chance of observing a prediction and the degree of confirmation this evidence may provide for H. Depending on considerations that are external to the test, people may either prefer low degrees of confirmation and avoid the risk of getting a falsification, or go for “risky” tests because they need to be very sure about H's truth. This representation of test choice has been shown to capture the intuition of everyday hypothesis testing (Poletiek & Berndsen Reference Poletiek and Berndsen2000).

As another example of this intuition, consider how SoT would account for the following adapted selection task. A participant is presented with four cards revealing an A, a K, a 2, and a 7, respectively. Every card has a number on one side and a letter on the other. She is asked to test the rule, if there is an A on one side of a card, then there is a 2 on the other side. In addition, she is told that the cards are sampled from a huge number of cards having either the letter A or the letter K on one side, and any number between 1 and 100 on the other. Which cards would she turn over? SoT predicts that she is interested in confirming results only (“A and 2” cards). The K and 7 cards are not selected. Under reasonable assumptions about the distribution of the letters and the numbers on the cards, it can be calculated that turning the A and finding a 2 card is a more severe test of the rule than turning the 2 and finding an A. But this difference is felt, as well. Indeed, finding a 2 (P(e)) is much less likely (given 100 different numbers) than finding the A, beforehand. Only if the rule is true would we expect a 2. Observing an A (given only two possibilities) on the 2 card, however, may just be chance. The the SoT model not only predicts that we should be more convinced by the first test outcome, we actually feel we should.

In summary, O&C's probabilistic model of hypothesis testing, which provides a sophisticated alternative for Popper's problematic logical falsification theory of testing, has an interesting precursor developed by Popper himself. However, this alternative falsification theory got little attention, and has been somewhat played down, both by Popper himself and by other falsificationist philosophers of science. Probably because it turned out to resemble too much its rival: confirmation theory.