1. Introduction

The requirement of total evidence (RTE), in its broadest meaning, says that in evaluating a hypothesis, you should take account of all the evidence you have. Formulated in this way, the RTE may sound like a truism, but it becomes more substantive when it is placed within a substantive epistemology. In a Bayesian setting, the basic idea (to be refined presently) is that in assessing how probable a hypothesis is, you should conditionalize on all the evidence you have.

Good (Reference Good1967) discusses the Bayesian RTE, provides references, and offers a defense. More recently, the RTE was introduced into discussions of the fine-tuning argument by White (Reference White2000), who made explicit the role that Hacking (Reference Hacking1987) assigned it in his discussion of the multiverse hypothesis. White invoked the RTE to rebut Leslie’s (Reference Leslie1989) variant of that argument. Subsequently, the RTE itself has been disputed by Manson and Thrush (Reference Manson and Thrush2003) and more recently by Epstein (Reference Epstein2017), who argues that the RTE generates mistaken conclusions about various examples that he constructs. This article provides a critical evaluation of the example of Epstein’s that we find most interesting. Here it is, mostly in his own words (we changed the notation slightly):

Fish. You know there are some large fish and some small fish in a lake, but you do not know the proportions: either there are mostly large fish (H_L) or mostly small fish (H_S). You know that you will catch one fish each time you stick in your net. You stick in your net and catch a large fish.

Let us describe your evidence in the following way:

E₁: You caught some large fish.

Evidence E₁ would seem to give you evidence in favor of H_L over H_S, since you are more likely to catch a large fish if the majority of the fish are large than if the majority of the fish are small:

(1)$\begin{array}{}& \mathrm{Pr}\left(E_1\right|H_L)>\mathrm{Pr}(E_1\left|H_S\right)\text{.}\end{array}$

Reasoning in this way seems like a paradigmatic example of empirically confirming an hypothesis, and, as such, it seems beyond reproach. But wait. The fish you caught, in addition to being large, is a particular fish (as fish tend to be). You name her Asha (ostensively, without need of any clever rigid designation). So you also have the evidence:

E₂. You caught Asha.

We can now ask whether E₂ is more likely given H_L than given H_S: were you more likely to catch Asha if there were mostly large fish, rather than mostly small fish?

It seems clear that you were equally likely to catch Asha on either hypothesis: whatever the size of the other fish, the likelihood that your net would happen upon the specific large fish Asha was exactly the same. Asha was swimming along and happened to be in the wrong place at the wrong time. The probability of that event is presumably sensitive to such factors as how frequently Asha swims in that area of the lake. But one thing that seems completely irrelevant is the size of the other fish around her. So, once again, the specific evidence fails to confirm the hypothesis:

(2)$\begin{array}{}& \mathrm{Pr}\left(E_2\right|H_L)=\mathrm{Pr}(E_2\left|H_S\right)\text{.}\end{array}$

(Epstein Reference Epstein2017, 643–44)

From this, Epstein concludes that “if we follow RTE, we will be led to the false conclusion that we get no confirmation of H_L over H_S” (644).

In what follows, we clarify the concept of confirmation that Epstein is deploying (sec. 2) and describe how the RTE should be understood so that it applies to his fishing example (sec. 3). Then we reformulate Epstein’s argument about fishing (sec. 4) to remove vague or distracting features of the original example and propose a reductio argument against his analysis of the example (sec. 5). After that, we discuss various replies that might be made to our reductio and argue that none of them underwrites a rejection of the RTE (secs. 7–11).

2. Confirmation and the Law of Likelihood

Epstein’s use of “confirm” and “confirmation” suggests he has in mind the Bayesian idea that E confirms H precisely when $\mathrm{Pr}\left(H\right|E)>\mathrm{Pr}(H)$ . However, he describes his example in terms of the likelihood inequality (1) and the likelihood equality (2), which each involve an evidence statement and two competing hypotheses, with no mention of hypotheses having posterior or prior probabilities. He also talks about evidence confirming or failing to confirm one hypothesis over another. This latter, comparative, judgment suggests that Epstein is using some measure of degree of confirmation, but no measure is stated in his article. More than a few measures of degree of confirmation have been discussed in the literature; all agree about the definition of yes/no confirmation, but they are not ordinally equivalent (Fitelson Reference Fitelson1999).

Epstein’s propositions (1) and (2) suggest that he is using the law of likelihood (Hacking Reference Hacking1965), which says that E favors H₁ over H₂ precisely when $\mathrm{Pr}\left(E\right|H_1)>\mathrm{Pr}(E\left|H_2\right)$ . That principle makes no mention of prior or posterior probabilities. However, the two-place relation of Bayesian confirmation and the three-place relation discussed in the law of likelihood are connected by a simple principle if the two hypotheses form a partition:

$\mathrm{Pr}\left(H\right|E)>\mathrm{Pr}(H\left)\text{ precisely when }\mathrm{Pr}\right(E\left|H\right)>\mathrm{Pr}\left(E\right|~H)\text{.}$

E confirms H in the Bayesian sense if and only if E favors H over ~H in the sense of the law of likelihood. Since Epstein stipulates that the two hypotheses H_L and H_S form a partition, we take it that the law of likelihood is the principle in play in his discussion of propositions (1) and (2).

3. The Requirement of Total Evidence

The RTE is a pragmatic principle that says, roughly, that you should use a logically stronger rather than a logically weaker formulation of the evidence you have in evaluating competing hypotheses. Bayesians make this idea precise by using their definition of confirmation and saying that you should conditionalize on all the evidence you have. However, if the law of likelihood, rather than Bayesian confirmation, is being used, the question arises of how the RTE should be stated. We answer this question by way of a simple example.

Suppose you are considering the following two hypotheses about an urn that contains a large number of balls, each of which has a single color:

You draw 100 balls at random from the urn and observe that

The law of likelihood tells you to interpret this result as favoring H₁ over H₂, which we hope the reader will find intuitive. However, there is another, logically weaker, description of what you observe:

Hypotheses H₁ and H₂ both confer probability 1 on E′, so the law of likelihood concludes that E′ does not favor either hypothesis over the other.

The RTE advises you to use E, not E′, to evaluate the hypotheses. We formulate the requirement like this:

We add the proviso in the last nine words because you may have noticed as you drew your sample from the urn that each of the balls weighs 2 ounces. You are not obliged to pay attention to this additional fact if the hypotheses say nothing about the probability of the balls’ weights (Sober Reference Sober2018).

Returning to fishing, we note that the RTE says that you should use E₂ rather than E₁ if and only if E₂ entails E₁ but E₁ does not entail E₂. Where there is no such entailment, the RTE says to use the conjunction E₁&E₂ if that is your total evidence. Since neither of Epstein’s E₁ and E₂ entails the other, we assume that Epstein wants to consider $E_T=E_1\&E_2$ instead of E₂ as the stronger evidence. This means that Epstein’s claim (2) should be rewritten as

(2′)$\begin{array}{}& \mathrm{Pr}\left(E_T\right|H_L)=\mathrm{Pr}(E_1\&E_2\left|H_L\right)=\mathrm{Pr}(E_1\&E_2|H_S)=\mathrm{Pr}(E_T\left|H_S\right)\text{.}\end{array}$

4. Reformulating the Fish Story

Epstein (Reference Epstein2017, 657) discusses the Asha example in connection with Kripke’s (Reference Kripke1980) thesis of the necessity of origins but notes that his argument does not depend on the Kripkean picture. He says that his argument goes through if, instead of talking about Asha, you talk about a specific genetic trait that the caught fish has. We do this in what follows, by saying that you observe that the caught fish has total genome g. Just as there is only one Asha in the Asha version of the story, there is only one fish with genome g in the non-Kripkean variant (assuming as we do that there are no identical twins among those fish). We prefer the genomic version of Epstein’s argument because it avoids complications arising from questions about naming and essentialism, but our analysis works for either version of the argument, as we later explain.

After presenting the Asha example, Epstein (Reference Epstein2017, 647) adds a detail to the example to pinpoint how he wants it to be understood:

Here Epstein brings his argument into contact with an important and much-discussed problem for Bayesian inference. In simple accounts of Bayesian updating, your observation at t₁ that proposition E is true allows you to update your credences in hypotheses H₁ and H₂ only if you know the values of Pr(E | H₁) and Pr(E | H₂) at an earlier time t₀. The Asha story violates this requirement, since you introduced the name “Asha” only after you observed the fish you caught. The Asha problem is thus a cousin to the “problem of old evidence” and the “problem of new theories” (Earman Reference Earman1992).

The usual strategy that Bayesians follow to address such problems is to construct the hypothetical probability distribution that you would have used earlier (at t₀) had you possessed the relevant concept. Bayesians then argue that the solution to the problem about that hypothetical past solves the problem about the actual present (t₁). In simple versions of this strategy, the model you construct to characterize your situation for the hypothetical past is the model you actually had in place, supplemented by further assumptions. Seeing Asha (or a fish with genome g) simply adds probability statements about Asha (or genome g) to what you believed earlier without modifying anything of relevance in what you then believed. Although this strategy requires a justification beyond the fact (if it is a fact) that it generates plausible results, we think that this is the strategy that Epstein uses; we will use it too.

There are two incidental details of Epstein’s example that do not affect the conclusion one ought to draw about it but that make the probability reasoning a bit blurry. The first detail concerns “big” and “small.” These terms are on the one hand vague and on the other hand composites of more specific size descriptions, such as “0.5 meters long.” To remedy this difficulty, we propose to substitute for “big” and “small” two species of lake fish, catfish species c and noncatfish species m.

To sidestep the second detail that might muddy the waters, we will talk about a different pair of hypotheses. We assume that there are 1,000 fish in the lake, and instead of discussing the hypotheses H_L (“mostly large”) and H_S (“mostly small”), we discuss the more specific hypotheses H₉₉₀ and H₁₀, where the former says that 990 of the 1,000 fish in the lake are in catfish species c, and the latter says that only 10 are in that species.Footnote ¹

5. A Reductio of Epstein’s Central Claim

The hypotheses are H₉₉₀ and H₁₀. The evidence is described by:

The total evidence is G&C. Epstein’s claim (2′), reformulated in terms of hypotheses H₉₉₀ and H₁₀, genome g, and catfish species c, is this:

(3)$\begin{array}{}& \mathrm{Pr}(G\&C|H_{990})=\mathrm{Pr}(G\&C\left|H_{10}\right)\text{.}\end{array}$

This is true precisely when

(4)$\begin{array}{}& \mathrm{Pr}\left(G\right|H_{990}\&C\left)\mathrm{Pr}\right(C\left|H_{990}\right)=\mathrm{Pr}\left(G\right|H_{10}\&C\left)\mathrm{Pr}\right(C\left|H_{10}\right)\text{.}\end{array}$

Since $\mathrm{Pr}\left(C\right|H_{990})=990/\mathrm{1,000}$ and $\mathrm{Pr}\left(C\right|H_{10})=10/\mathrm{1,000}$ , you can conclude that (3) is true if and only if

(5)$\begin{array}{}& \mathrm{Pr}\left(G\right|H_{10}\&C)=(99\left)\mathrm{Pr}\right(G|H_{990}\&C)\text{.}\end{array}$

That is, for the total evidence G&C to fail to discriminate between the two hypotheses, as (3) asserts, what is needed is this: if you catch a catfish, its chance of having genome g if there are 10 catfish in the lake is exactly 99 times greater than its chance of having that genome if there are 990 catfish in the lake. This is not impossible (so our reductio does not involve deriving a contradiction), but it strikes us as extremely implausible.

Actually, Epstein would achieve his purpose if he could show something weaker than (5), namely,

(5′)$\begin{array}{}& \mathrm{Pr}\left(G\right|H_{10}\&C)\ge (99\left)\mathrm{Pr}\right(G|H_{990}\&C)\text{.}\end{array}$

Then, the stronger and the weaker formulations of the evidence would disagree about which of the two hypotheses has the higher likelihood. Here is a scenario that justifies (5′): Suppose genome g causes the organisms that have it to decimate the catfish population in a lake when catfish are common but to desist from doing so when catfish are rare. This is enough for genome g to be evidence favoring H₁₀ over H₉₉₀.

Detailed ecological information might justify this theory about genome g and thereby justify inequality (5′). However, Epstein does not base his case on ecological data and theory; rather, he thinks that the meagre information he supplies about genome g is enough to justify this weak inequality. That is very far from being the case.

Like other reductio arguments, the one we just presented does not pinpoint where Epstein went wrong. We attempt to describe what better reasoning about Asha & Co. should look like in what follows.

6. What If G Is Independent of $H_{990}$ , $H_{10}$ , and C?

In Epstein’s example, after you observe that the fish you catch is big (or, in our reformulation, that it is a member of catfish species c), you learn that the fish before you has genome g and nothing more. You did not have the concept of genome g in your model before that point, so you are unprepared to say how this second observation should influence your assessment of the competing hypotheses. What you need to do is to modify the model you have, by incorporating assumptions about genome g. Here are some model guidelines that might seem natural:

Initially, you do not assume any logical relation between G and C. So the total evidence after observing G is $E_T=G\&C$ , which is stronger than G.

The independence assumptions sanctioned by M₃ entail that

$\mathrm{Pr}\left(G\right|C\&H_{990})=\mathrm{Pr}(G)=\mathrm{Pr}(G|C\&H_{10})\text{.}$

This, plus the fact that

$\mathrm{Pr}\left(C\right|H_{990})>\mathrm{Pr}(C\left|H_{10}\right)\text{,}$

entails that

$\mathrm{Pr}(G\&C|H_{990})>\mathrm{Pr}(G\&C|C\&H_{10})\text{.}$

So under the assumption of independence just mentioned, the strong evidence G&C and the weak evidence C agree on which of H₉₉₀ and H₁₀ is favored. There is no threat here to the RTE.

Epstein (Reference Epstein2017, 644) expresses sympathy with the independence assumption M₃ when he writes: “It seems clear that you were equally likely to catch Asha on either hypothesis: whatever the size of the other fish, the likelihood that your net would happen upon the specific large fish Asha was exactly the same. Asha was swimming along and happened to be in the wrong place at the wrong time. The probability of that event is presumably sensitive to such factors as how frequently Asha swims in that area of the lake. But one thing that seems completely irrelevant is the size of the other fish around her.” We, however, feel some hesitation in endorsing M₃. To infer independence from ignorance sounds suspiciously like inferring a flat probability distribution from ignorance, so our doubts about the principle of indifference carry over to this way of thinking about fishing. That said, the current point is that the independence assumption does not help Epstein’s case against the RTE.

7. What If G Entails C?

Of course, it may be true that although G is new to your model, you nonetheless possess considerable knowledge relevant to G before you make your observations at time t₁. For example, it is quite reasonable to suppose that you know at t₀ that genome g is a genome found only in catfish species c, even before genome g is identified as the genome of your caught fish, and to incorporate this knowledge into the revised model.

To evaluate the probabilities $\mathrm{Pr}\left(G\right|H_{990})$ and $\mathrm{Pr}\left(G\right|H_{10})$ in the revised model where you now assume that G entails C, we introduce the proposition

Then G is equivalent to L&G.Footnote ² We now can answer the titular question of this section by considering this proposition:

(6)$\begin{array}{}& \mathrm{Pr}\left(G\right|H_{990})=\mathrm{Pr}(L\&G\left|H_{990}\right)=\mathrm{Pr}\left(L\right|H_{990}\left)\mathrm{Pr}\right(G|L\&H_{990})\mathrm{.}\end{array}$

Let n be the total number of biologically possible genomes in catfish species c. Then plausibly

(6a)$\begin{array}{}& \mathrm{Pr}\left(L\right|H_{990})=\frac{990}{n}\text{,}\end{array}$

where we regard the set of catfish genomes present in the lake at t₀ as a random subset of the set of all biologically possible genomes of fish in catfish species c.Footnote ³

In evaluating proposition (6), one needs to be a little careful with $\mathrm{Pr}\left(G\right|L\&H_{990})$ . Its value is

(6b)$\begin{array}{}& \mathrm{Pr}\left(G\right|L\&H_{990})=\frac{1}{\mathrm{1,000}}\mathrm{.}\end{array}$

The value of this conditional probability is not 1/990, since any fish in the lake may be caught with equal probability, not just the catfish. The conditioning proposition L&H₉₉₀ describes the composition of the lake at t₀—it contains 990 catfish of species c and exactly one of them has genome g—but this does not constrain the catchings at t₁.

Putting (6a) and (6b) together yields

(7)$\begin{array}{}& \mathrm{Pr}\left(G\right|H_{990})=\frac{990}{n}\frac{1}{\mathrm{1,000}}=\frac{990/\mathrm{1,000}}{n}\text{.}\end{array}$

Similar reasoning applies to H₁₀:

(7′)$\begin{array}{}& \mathrm{Pr}\left(G\right|H_{10})=\mathrm{Pr}(L\&G\left|H_{10}\right)=\mathrm{Pr}\left(L\right|H_{10})\times \mathrm{Pr}(G|L\&H_{10})=\frac{10}{n}\frac{1}{\mathrm{1,000}}=\frac{10/\mathrm{1,000}}{n}\text{.}\end{array}$

Propositions (7) and (7′) show that the stronger evidence G (which now is assumed to entail C) favors H₉₉₀ over H₁₀. Since the stronger evidence G and the weaker evidence C agree about this, the supposition that G entails C does not change our verdict about Epstein’s example: it poses no threat to the RTE.

8. A Possible Reply to Our Analysis in the Previous Section

Our analysis of what happens when G is taken to entail C in the previous section relies on particular values for the likelihoods $\mathrm{Pr}\left(G\right|H_{990})$ and $\mathrm{Pr}\left(G\right|H_{10})$ that arise from conceiving of the set of catfish genomes in the lake as a random subset of the set of biologically possible catfish genomes. Using these values, we may derive another formula. Two propositions follow from (7) and (7′):

(8)$\begin{array}{}& \begin{array}{cc}\frac{990/\mathrm{1,000}}{n}& =\mathrm{Pr}\left(G\right|H_{990})\\ & =\mathrm{Pr}(C\&G|H_{990})=\mathrm{Pr}(C\left|H_{990}\right)\mathrm{Pr}\left(G\right|C\&H_{990})=\frac{990}{\mathrm{1,000}}\mathrm{Pr}(G|C\&H_{990})\mathrm{.}\end{array}\end{array}$

(8′)$\begin{array}{}& \begin{array}{cc}\frac{10/\mathrm{1,000}}{n}& =\mathrm{Pr}\left(G\right|H_{10})\\ & =\mathrm{Pr}(C\&G|H_{10})=\mathrm{Pr}(C\left|H_{10}\right)\mathrm{Pr}\left(G\right|C\&H_{10})=\frac{10}{\mathrm{1,000}}\mathrm{Pr}(G|C\&H_{10})\text{.}\end{array}\end{array}$

From these, we obtain

(9)$\begin{array}{}& \mathrm{Pr}\left(G\right|C\&H_{990})=\mathrm{Pr}(G|C\&H_{10})=\frac{1}{n}\text{.}\end{array}$

Proposition (9) is perhaps more intuitive than the derivation in the previous section. Given that you have caught a catfish of species c, the probability that it has genome g is simply $1/n$ (the probability of a randomly chosen genome from the set of all genomes that are possible in species c) and does not depend on the composition of the lake (H₉₉₀ or H₁₀).

It might be replied that it is implausible to think that the probability of G is determined by a random process of this sort. After all, since the caught fish has genome g, this provides information about the genomes possessed by its parents, its grandparents, and so on. The suggestion is that the probability of the caught fish having genome g should be conceptualized by conditionalizing on the biologically possible genomes of those ancestors.

We have no problem with this suggestion. Genome g was the result of sexual reproduction, and mutation, recombination, and migration may have played a role as well. There are literally millions of possible genomes that the caught fish’s parents might have had, conditional on the fact that their offspring has genome g, and it makes sense to describe some of these genomes as having higher probabilities than others, again conditional on the genome of the offspring. However, these considerations do not tell against equality (9). It does not matter whether there are 990 catfish in the lake or only 10—the probability that the fish you caught has genome g, given that it is a catfish, is the same.Footnote ⁴

9. What If G Is Otherwise Probabilistically Dependent on C, $H_{990}$ , or $H_{10}$ ?

There are other ways, apart from G’s entailing C, by which G could probabilistically depend on one or more of C, H₉₉₀, and H₁₀, and this could be known at t₀. We already considered one way in the reductio section 5, namely, the possibility that genome g makes the organisms that have this trait stupendous catfish killers (but to desist from killing catfish when catfish are rare), so that if genome g is found in the lake, that will be strong evidence that there are few catfish there. Here we have a case in which the catfish observation favors H₉₉₀ over H₁₀, whereas the genome g observation has the opposite evidential significance. When you put these two observations together, it may turn out that their conjunction favors H₁₀ over H₉₉₀, or favors the latter over the former, or does not discriminate at all between the two hypotheses. We see no threat to the RTE here. This case is analogous to conflicting testimony from reliable witnesses at a trial.

10. Back to Asha

We treated the “genomic” variant of Epstein’s example first in order to avoid distracting issues associated with the metaphysics of identity, but the genome version itself is an instance of a simple and general urn model with a second observable property, and our treatment applies to the “Asha” variant as well. To focus on Asha, our notation will be the same as that used earlier, except that we now replace proposition G with

Using A instead of G corresponds to our analysis in section 6, where we treated G and C as logically independent. Now we say, in parallel fashion, that A does not entail C; you do not know (at t₀) that Asha is a member of catfish species c. Indeed, at t₀ you know virtually nothing about Asha. You do not know for certain that you will catch Asha at t₁. Indeed, you do not even know for certain that Asha is in the lake at t₀, for the fish in the lake under hypothesis H₉₉₀ cannot all be identical to the fish that are in the lake under H₁₀. (A catfish cannot be identical to a noncatfish.) As far as you know at t₀, Asha is not special in any way, and this needs to be built into the modified model.

Given all this, it is reasonable (by guideline M₃) to take A and L to be independent of C, H₉₉₀, and H₁₀. In particular, $\mathrm{Pr}\left(A\right|C\&H_{990})=\mathrm{Pr}(A\left|H_{990}\right)=\mathrm{Pr}\left(A\right|H_{10})=\mathrm{Pr}(A)=q$ (say). In this, we agree with Epstein, who thinks that the probability of catching Asha does not depend on H₉₉₀ (or on H₁₀). Because A does not entail C, the total evidence is now C&A, not A alone, so

(10)$\begin{array}{}& \mathrm{Pr}(C\&A|H_{990})=\mathrm{Pr}(C\left|H_{990}\right)\mathrm{Pr}\left(A\right|C\&H_{990})=\mathrm{Pr}(C\left|H_{990}\right)\mathrm{Pr}\left(A\right)=\frac{990}{\mathrm{1,000}}q\text{.}\end{array}$

(10′)$\begin{array}{}& \mathrm{Pr}(C\&A|H_{10})=\mathrm{Pr}(C\left|H_{10}\right)\mathrm{Pr}\left(A\right|C\&H_{10})=\mathrm{Pr}(C\left|H_{10}\right)\mathrm{Pr}\left(A\right)=\frac{10}{\mathrm{1,000}}q\text{.}\end{array}$

As in the genomic variant, both the strong evidence and the weak evidence favor H₉₉₀ over H₁₀.

11. Conclusion

Epstein’s example and our modification of it are easily generalizable. For example, in place of the predicate “x has genome g,” you may consider any other observable predicate F(x) and the evidence E_F = “the fish x that you caught is/has F.” Depending on what is known or assumed about F at t₀, C may screen off the hypotheses from E_F as in (8), or it may not. If it does not, then the total evidence $E_T=E_C\&E_F$ may favor H₉₉₀ over H₁₀ or say the opposite or favor neither. We conjecture that all other such examples will challenge the RTE no more than the genome g version of the fishing example does.Footnote ⁵