1. SIMPSON'S PARADOX: THE USUAL DIALECTIC

There was a period in the 1970s when the admissions data for the UC–Berkeley graduate school (hereafter, BGS) exhibited some (prima facie) peculiar statistical correlations.Footnote ¹ Specifically, a strong negative correlation was observed between being female and being accepted into BGS. This negative correlation (in the overall population of BGS applicants) was (initially) a cause for some concern regarding the possibility of gender bias in the admissions process at BGS. However, closer scrutiny of the BGS admissions data from this period revealed that no individual department’s admissions data exhibited a negative correlation between being female and being admitted. In fact, every department reported a positive correlation between being female and being accepted. In other words, a correlation that appears at the level of the general population of BGS applicants is reversed in every single department of BGS. This sort of correlation reversal is known as Simpson's Paradox. Because admissions decisions at BGS are made (autonomously) by each individual department, the lack of departmental correlations seems to rule-out the gender bias hypothesis as the best (causal) explanation of the observed correlations in the data. As it happens, there was a strong positive correlation between being female and applying to a department with a (relatively) high rejection rate. That is, women tended to apply to departments within BGS that have low(er) acceptance rates. And, this is what explains the observed correlations.

This informal presentation is typical of (most) discussions of Simpson's Paradox that one finds in the literature. Typically, one sees causal explanations of how some set of (“Simpson's-Paradoxical”) statistical correlations could have arisen in the first place (i.e., causal “how-possibly” explanations of the “paradoxical” correlational patterns in the statistical data). In this paper, I will be doing something different. I will be discussing what I will call rationalizing explanations of the paradoxicality of Simpson's Paradox. That is, I will be examining some explanations of why the cases may seem paradoxical in the first place.Footnote ² In this sense, my treatment of Simpson's Paradox will be more similar to the way philosophers have approached (many) other “paradoxes” — by trying to explain (away) the sense of paradoxicality itself.Footnote ³ In the next section, I will provide some background on confirmation theory. In the remaining sections, I will (a) discuss some existing treatments (both causal and confirmation-theoretic) of Simpson's Paradox, and (b) offer a new (purely confirmation-theoretic) rationalizing explanation of the paradoxicality of Simpson's Paradox.

Here is a simplified, precise version of the BGS example, which will serve as our canonical example of Simpson's Paradox for the remainder of the paper. Let there be two departments: E and $\overline E $ (think: easy and not easy) and two genders (of applicants): M and $\overline M $ (think: male and not male).Footnote ⁴ And, let A ( $\overline A $ ) express the proposition than an applicant is accepted (not accepted). The following table gives all the relevant acceptance rates (i.e., statistical frequencies, which we will assume align with the objective probabilities/chances, in this situation) regarding this simplified characterization of BGS admissions (involving a total of 260 applicants, 130 of which applied to department E and 130 of which applied to $\overline E $ ).

These acceptance rates are to be understood as follows. For instance, $\Pr (A \vert E\& M) = {60}/{80}$ means that the probability of acceptance for male applicants to department E is $ {60}/{80}$ — i.e., $ {60}/{80}$ of the male applicants to department E were accepted (and this frequency accurately reflects the corresponding objective probability of acceptance).

The crucial thing to notice here is that $\overline M $ is negatively relevant to acceptance in the overall population, i.e., $\Pr (A \vert \overline M ) \lt \Pr (A \vert M)$ , while $\overline M $ is positively relevant to acceptance within each of the two departments, i.e., $\Pr (A \vert \overline M \& E) \gt \Pr (A \vert M\& E)$ and $\Pr (A \vert \overline M \& \overline E ) \gt \Pr (A \vert M\& \overline E )$ . This kind of reversal of correlations (between subpopulations and the overall population) is known as Simpson's Paradox.

In the next section, I will provide a bit of theoretical background on confirmation theory. Then, I'll examine two recent discussions of Simpson's Paradox. Finally, I will present a new rationalizing explanation of the (apparent) paradoxicality of Simpson's paradox.

2. SOME BACKGROUND ON CONFIRMATION THEORY

From a deductive-logical point of view, “argument strength” does not come in degrees. An argument is either deductively valid or it is not. Confirmation theory aims to generalize this purely qualitative deductive notion of “argument strength.” Ideally, it would be nice to have a numerical measure of the degree to which some premise P supports some conclusion C. At least, that was Carnap's (Reference Carnap1962) dream – to construct a confirmation function $\frak {\rm c}$ (C, P), which (a) generalizes deductive entailment (in a sense to be made precise, below) and (b) is defined using probabilistic concepts, where (c) the probabilities in terms of which $\frak {\rm c}$ is defined are “objective” (and, ideally, logical). For present purposes, we won't need anything like a Carnapian confirmation function. This is for two main reasons. First, we'll only be looking at cases in which there are (uncontroversial, well-defined and well-known) statistical probabilities (viz., objective chances). So, we won't need to worry about the “objectivity” of the probability assignments that will underlie our confirmation-theoretic relations (and we won't need to mess around with “logical” probabilities). Second, we'll only need qualitative confirmation-theoretic concepts in order to provide our analyses (i.e., numerical confirmation measures won't be required for our purposes). In this sense, we will be employing a more Bayesian (and less Carnapian) approach to confirmation. However, we will follow Carnap in making a key distinction between two kinds of probabilistic confirmation.

For present purposes, all of the conditional probabilities $\Pr ( \cdot \vert \cdot )$ will be given by a specification of some (uncontroversial) statistical probabilities (viz., objective chances). So, there will be no controversy about where the probabilities in question come from. Our focus here will be on using these two confirmation-theoretic relations to characterize Simpson's Paradox and explain why it seems paradoxical (but, in fact, is not).

One important theoretical similarity between these two confirmation relations is that they both constitute generalizations of deductive validity, in the following sense. If P entails C, then P confirms C, relative to any background supposition S. To see why, note that if P entails C, then $\Pr (C \vert P\& S)$ will be maximal (i.e., equal to 1), and so we will (generallyFootnote ⁶ ) have both $\Pr (C \vert P\& S) \gt t$ and $\Pr (C \vert P\& S) \gt \Pr (C \vert S)$ . That is, entailing premises will (generally) make their conclusions highly probable and raise the probabilities of their conclusions (above the values taken relative to whatever background suppositions are involved).

For our present purposes, the key theoretical differences between these two notions of confirmation can be seen by thinking of them as constituting (similar, but subtly different) explications of the concept of “argument strength.” If a premise P confers a high probability on conclusion C, then (at least in one sense) the argument can be said to be a “strong” one.Footnote ⁷ However, even if P confers a high probability on C, it may also be true that P lowers the probability of C (as compared with its initial/prior probability). In such cases, there also seems to be a sense in which P can not form the basis of a “strong argument in favor of” C. There are many examples of this sort in the literature. Some of the most interesting examples fall under the rubric “the base rate fallacy” (Kahneman and Tversky Reference Kahneman, Tversky, Kahneman, Slovic and Tversky1982; Levi Reference Levi1983).Footnote ⁸ For instance, let P be the claim that a positive test result has been obtained from an imperfect but reliable test for some rare disease (performed on a sample drawn from some individual a). And, let C be the claim that a does not have the rare disease in question. If the disease is sufficiently rare (i.e., if Pr(C) is sufficiently high), then $\Pr (C \vert P)$ will (still) be very high. Nonetheless, $\Pr (C \vert P)$ will be less than Pr(C), and so P will lower the probability of C. In such a case, it would be odd to say that P constitutes strong evidence in favor of C. The problem here is that “argument strength” has (at least) two probabilistic components: confirms _f and confirms _i , and these two components can run in opposite directions. This is what happens in “base rate fallacy” cases (and many other cases besides – see Fitelson (Reference Fitelson2015) for further discussion). Such cases reveal one important theoretical difference between confirms _f and confirms _i .

These two concepts also exhibit some more subtle theoretical divergences that will be important for our analysis of Simpson's Paradox. In order to grasp these more subtle theoretical differences, it is useful to consider the following principle regarding “argument strength.”

Basically, the SSTP involves a sort of suppositional “reasoning by cases.” It asks us to consider whether (a) the argument $P\therefore C$ is strong, given some (indicative) supposition S and (b) the argument $P\therefore C$ is strong, given the negation of that (indicative) supposition (viz., $\overline S $ ). And, if both of these assessments come out positive, then the SSTP implies that $P\therefore C$ is strong unconditionally (i.e., in the absence of background suppositions).

This kind of suppositional “reasoning by cases” is correct for some understandings of “argument strength” but not for others. Specifically, SSTP is satisfied by confirmation as firmness (confirms _f ), but not by confirmation as increase in firmness (confirms _i ). That is to say, we have the following two key theoretical facts.Footnote ¹⁰

Fact 2 is the theoretical property of confirms _i which makes Simpson's Paradox possible in the first place. Indeed, Simpson's Paradox cases just are the examples which establish the truth of Fact 2 (e.g., the simplified BGS example depicted in Table 1 above).

Table 1

Probabilistic structure of canonical (BGS) example of Simpson's Paradox.

While confirms _f and confirms _i disagree regarding the SSTP, they agree on a similar (but subtly different) theoretical principle regarding “argument strength.” To wit:

Like the SSTP, the CSTP also involves “reasoning by cases.” However, unlike the SSTP (which involves suppositional reasoning), the CSTP involves conjunctive (and non-suppositional) “reasoning by cases.” That is, the CSTP does not ask us to assess the strength of a single argument $P\therefore C$ , relative to a pair of suppositions (viz., S and $\overline S $ ). Rather, the CSTP asks us to consider the unconditional strength of a pair of arguments ( $P\& S\therefore C$ and $P\& \overline S \therefore C$ ). Interestingly, both confirmation _f and confirmation _i satisfy the CSTP. That is, we have:

The disagreement between confirms _f and confirms _i with respect to SSTP vs. CSTP can be understood in terms of their ability to distinguish conjunctive vs. suppositional support/confirmation. Confirmation as firmness simply cannot distinguish between the two. If you want to know whether P confirms _f C, on the supposition that S, you check whether $\Pr (C \vert P\& S) \gt t$ . And, this is exactly the same thing you must do if you want to check whether $P\& S$ confirms _f C, unconditionally. In other words, from the point of view of confirmation as firmness, there is no distinction between conjunctive and suppositional support (in general). In light of this, it is easy to see why Facts 1 and 3 both obtain (for confirms _f ) – they are actually one and the same fact, from the perspective of confirmation _f . This is not so for confirmation _i , which distinguishes suppositional and conjunctive support in such a way that allows for Simpson's-Paradoxical cases (i.e., violations of the SSTP) to arise.

Facts 1–3 are the key theoretical facts that will undergird my (ultimate) rationalizing explanation of Simpson's Paradox. In the next section, I will examine two recent discussions of Simpson's Paradox. This will lead naturally into a new rationalizing explanation, based on Facts 1–3 above.

3. TWO RECENT DISCUSSIONS OF SIMPSON'S PARADOX

It is clear that there can be nothing truly paradoxical about “Simpson's Paradox.” After all, examples like Cartwright's BGS example actually occur in nature (all the time). So, what needs to be explained (or perhaps explained away) is the appearance of paradoxicality in cases of Simpson's Paradox (i.e., their prima facie paradoxicality). In this section, I'll discuss two recent treatments of Simpson's Paradox. Then, I'll propose a new, purely confirmation theoretic rationalizing explanation.

4. A NEW RATIONALIZING EXPLANATION OF SIMPSON'S PARADOX

I think Kotzen's discussion leads us in the direction of a purely confirmation-theoretic rationalizing explanation of Simpson's Paradox. I propose that the prima facie paradoxicality of Simpson's Paradox can be explained (at least in part) by appealing to the (subtle) distinction between conjunctive confirmation (or support) and suppositional confirmation (or support) that was implicit in our discussion of SSTP vs. CSTP above. To wit, contrast the following:

I propose that there is a tendency for hearers of descriptions of Simpson's Paradox cases to conflate these two readings. Let's apply this to our BGS case. To make things more concrete, let us now suppose that department $\overline E $ is History. Then, we may describe the correlation between being female and being accepted to History, in the following two ways:

In our BGS example, it is only the Suppositional reading that is true.Footnote ¹⁵ Moreover, as we have seen, these two readings – while similar-sounding to the (untrained) ear – make all the difference, in terms of paradoxicality. If we slide from Suppositional to Conjunctive, then we are sliding from an interpretation on which the examples described are possible (because of Fact 2), to one on which they are impossible (because of Fact 3). And, this will be true for any example of Simpson's Paradox – even ones in which Pearl's causal independence assumption (†) fails. So, if my proposal is right, then we can exploit the fact that the conjunctive STP is a truism about confirmation (no matter how it is explicated) to furnish an explanation-away of the apparent paradoxicality of Simpson's Paradox.

Here's how that explanation would go. Examples of Simpson's Paradox involve violations of the SSTP. To the extent that hearers tend to conflate suppositional vs. conjunctive readings of the key locutions involved in descriptions of such cases, there will be a tendency to hear such cases as violations of the C STP. But, there can be no violations of the CSTP (since, as Fact 3 reveals, it is a general truism about confirmation — no matter how it is explicated). Therefore, to the extent that hearers tend to conflate suppositional vs. conjunctive readings of the key locutions involved in descriptions of such cases, there will be a tendency for hearers to interpret the cases as implying something paradoxical (i.e., as somehow contradicting a theoretical truism).Footnote ¹⁶ But, because there can be no such cases, the appearance of paradoxicality is a mere appearance (arising, at least in part, from said conflation).

Our rationalizing explanation has a couple of virtues, in comparison with Pearl's causal rationalizing explanation. First, ours is purely confirmation-theoretic – it makes no appeal to causal relations, which are inessential to the description of Simpson's Paradoxical cases. Moreover, Pearl's explanation relies on an implicit causal independence assumption [(†)] which is not stated in the descriptions of the Paradox. Our explanation applies to all instances of Simpson's Paradox – even ones in which Pearl's assumption (†) fails.Footnote ¹⁷ In any case, even if causal facts/presuppositions are playing a role in the (prima facie) paradoxicality of Simpson's Paradox, my proposal is that the suppositional/conjunctive distinction/conflation also has (independent, rationalizing) explanatory relevance/force.