1. Introduction

Auditors have found an irregularity in the books for Acme, Inc. Bungled paperwork explains most anomalies of this kind, but fraud cannot be ruled out, so teams are assembled to investigate. Team 1 learns that Carson, a longtime Acme bookkeeper, is divorcing his wife, which is sad news for the Carson family but of little interest to Team 1. Most people manage a divorce without defrauding their employer, after all. Team 2 learns that Carson has a Brazilian lover, but they merely pause to think, lucky Carson. Team 3 finds out that Carson took a flight to Rio de Janeiro seated beside a stunning but dodgy Brazilian financier.

Considered in isolation, each team's evidence signals little more than middle-life boredom. Yet, when taken together, the evidence suggests fraud. It is not the facts alone that warrant hauling Carson in for questioning but rather how those pieces of evidence fit together into a compelling case against him.

But what does it mean for evidence to ‘fit together’, and what is it about combined evidence that lends more support to a hypothesis than each piece of evidence on its own? One answer is to compare the likelihood of evidence occurring together to its co-occurrence given a cause or some unifying explanation. The events of the story are unusual on their own, but a conspiracy to defraud Acme involving Carson provides a unifying explanation for those events, one that would be alarming enough for Acme investigators to want a word with Mr. Carson.

The core idea animating this example is encapsulated in a measure called focused correlation (Myrvold Reference Myrvold1996; Wheeler Reference Wheeler2009), which is defined for n binary evidence variables and a single binary hypothesis variable as follows:

Focused correlation has been given a variety of interpretations: to capture the effect that diversity of evidence has on confirmation of some hypothesis (Myrvold Reference Myrvold1996); to provide a partial explication of the relationship between epistemic coherence and incremental confirmation of a hypothesis (Wheeler Reference Wheeler2009); and, in normalized form, to provide an account of unified evidence for a hypothesis (Myrvold Reference Myrvold2003). But, the basic idea is that by comparing the degree of association within an evidence set to the degree of association in that evidence set given some hypothesis, in certain circumstances several questions about confirmation relationships between evidence and that hypothesis can be answered.

The conditions under which focused correlation tracks confirmation, pace Myrvold (Reference Myrvold2003), depend on what type of logical or causal structure governs the relationships between evidence and hypothesis in a problem (Wheeler and Scheines Reference Wheeler and Scheines2011). For example, Erik Olsson (Reference Olsson2005) and Luc Bovens and Stephan Hartmann (Reference Bovens and Hartmann2006) have drawn attention to problematic features of models in which all binary evidence variables E_i of a problem are conditionally independent of the binary hypothesis variable H; that is, . The correlation between any distinct pair of evidence variables E_i and E_j in this class of models is the product of the correlations between each E and H (Danks and Glymour Reference Danks, Glymour, Breese and Koller2001). But this fact yields counterintuitive results if one interprets association as epistemic coherence since then association among the evidence is strictly less than the (nondeterministic) confirmation that each piece of evidence lends to the hypothesis.

In the case of focused correlation, it is easy to see from equation (1) that the focused correlation of an evidence set with respect to some hypothesis under this conditional independence assumption is strictly less than one, and thus is negative, whereas the confirmation of the hypothesis by that evidence may be positive. This conditional independence condition is central to the witness testimony models of Bovens and Hartmann (Reference Bovens and Hartmann2003) and Olsson (Reference Olsson2005), and it is the condition that drives their noted negative results concerning the construction of a probabilistic measure of coherence (Wheeler Reference Wheeler2009).

But, contra Olsson (Reference Olsson2005, 135), there are interesting cases outside of this class of models where a measure like focused correlation can be useful, and, pace Luc Bovens and Stephan Hartmann (Reference Bovens and Hartmann2006), there are conditions under which a difference in confirmation lent to some hypothesis by two evidence sets is tracked by a difference in their focused correlation, even within this obstinate class of models.Footnote ¹ To illustrate, consider again L'affaire de Carson. Suppose that a performance review of the Acme Investigation Department involves comparing the evidence set E, consisting of a divorcing employee who traveled with a dubious associate, to the evidence set E*, consisting of a divorcing employee who is having an affair. Acme is a big company with many cases on file, and their models suggest that E is a better indicator of fraud than E* because fraud is a slightly ‘better’ explanation for the co-occurrence of divorce and suspicious travel than it is for divorce and an adulterous affair. Under what conditions can Acme management draw this inference?

In Wheeler and Scheines (Reference Wheeler and Scheines2011), sufficient conditions for a wide variety of incremental confirmation measures are given, under which a difference in focused correlation entails a difference in confirmation lent to that hypothesis and vice versa. But their results are only trivially true in this problematic class of models because one of the conditions for the comparison result is that all individual items of evidence have equal relevance to the hypothesis. In effect, this restrictive condition, when combined with the conditional independence assumption mentioned above, deals with the problematic cases driving Bovens and Hartmann's counterexamples (Reference Bovens and Hartmann2003, Reference Bovens and Hartmann2006), by ensuring that at least one of the antecedent conditions is false. The motivation for this strong assumption is clear in their project, for they address whether a difference in association of two evidence sets induced by some hypothesis can track confirmation of that hypothesis. But including equal relevance as a ceteris paribus condition will not do for practical comparisons of evidence sets; it is much too restrictive.

In this article, we show that the equal evidence assumption can indeed be relaxed to identifiable cases involving unequal relevance. Our result is robust as well, and we retain conditions for bidirectional tracking between the class of incremental confirmation measures and focused correlation. The cases in which tracking fails are instructive, suggesting that focused correlation is a stronger measure of evidential support than classical incremental confirmation measures, and we illustrate this point through some examples.

The structure of the article is as follows. In section 2, we set up the machinery by making explicit our notation and the confirmation measures we consider. Section 3 presents our results, and in section 4, we explain why two common nonincremental confirmation measures are not tracked by focused correlation. This analysis offers further evidence for distinguishing between classical, incremental confirmation measures and other notions of confirmation. Proofs are set in the appendix.

2. Setup

In this section we specify notation and list the confirmation functions we will consider.

Define a probability space such that is a σ-algebra over a set Ω, and is a probability measure defined on the space satisfying

• P1..

• P2., when are countable, pairwise disjoint elements of .

A probability structure is a tuple , where is a probability space and V is an interpretation function associating each element with a truth assignment on the primitive propositions in such that for each and for every proposition in Φ.

For each primitive proposition in Φ, we define if and only if (iff) and proceed by induction on the structure of propositional formulas. Since P is defined on events in rather than propositions, let denote the set of outcomes within Ω in M where A is true, which will correspond to a subset of . The following equations make explicit the relationship between propositions and events for arbitrary propositional formulas A and B:

1. i) ,

2. ii) ,

3. iii) .

An evidence set is a set of propositions, written . We defined focused correlation in equation (1) with respect to random variables E_i and H each taking values 0 or 1. We use the abbreviation E for and for and likewise will use the abbreviation for . We will relax notation by expressing focused correlation of propositions as well as of variables, using the conventions here for distinguishing between propositions and variables to signal which is which.

2.1. Confirmation Measures

A statement E is confirmation to hypothesis H with respect to a classical probability model specifying the measure P just when E and H are positively correlated under P; that is, when . A confirmation function measures the degree to which evidence confirms a hypothesis, and there are several proposals:Footnote ²

3. Unequal Relevance

We consider, as before, two evidence sets and , but now we shall relax assumption A2: we no longer require that all propositions , , are equally confirmatory. That is, we allow for the degree of positive relevance of the individual pieces of evidence to vary.

What we would like to check now is whether, in this generalized situation, focused correlation continues to track confirmation, and vice versa. That is, we would like to explore under what conditions proposition 1 still holds. ‘Confirmation’ is here understood as quantified by the measures defined in section 2.1. We shall denote these measures collectively by in what follows; that is, jointly stands for any of the functions , , , , and .

A quick look at the definitions of these confirmation measures (sec. 2.1) shows that they are based only on the probabilities of H conditional on the full evidence set and on the shared evidence piece but not on the probabilities of H conditional on the distinct individual pieces and , that is, not on and . The appearance of and in equation (1) is therefore a particular feature of focused correlation. This feature exemplifies the additional input to the measure, which takes into account the conditional probabilities for all .

As an aside, we note that the relevance of the evidence will not enter into the following considerations, not withstanding its role in and . This is so because is shared among both evidence sets and , and we are here concerned only with a comparison of the degrees of focused correlation and confirmatory power of these two sets.

3.1. From Focused Correlation to Confirmation

Let us first describe the condition under which a larger degree of focused correlation for evidence set than for implies larger confirmation of the hypothesis by than by .

Proposition 2. If and

then also .

Thus, the confirmation measures track focused correlation if the bound (2) is fulfilled. In particular, this will be the case whenever , but the implication will also hold if by an amount that is a function of the difference in the focused correlations of the two sets and , as specified by (2). However, if drops below this bound, then the tracking fails, and the confirmation measures will indicate less confirmatory power for than , despite the fact that has larger focused correlation than .

This observation has an intuitive explanation, as is equivalent to when both A1 and A2 hold. Now, as mentioned above, focused correlation goes a step further than the measures because it weighs the joint relevance by how much each individual piece of evidence confirms H. If and individually confirm H very little compared to the confirmation of H provided by the conjunction , while the probability of H given does not exceed by much the product of the probabilities of H given and alone, then the set may have larger focused correlation than the set , even if . In other words, may arise from a situation in which is so much smaller than as to be able to counteract the greater confirmatory power of over as quantified by the measures .

We will illustrate our results by presenting alternative versions of Mr. Carson's scandal. In each of our examples, we shall let represent the fact that Carson is divorcing his wife, = Carson's trip abroad seated next to a shady financier, = Carson's affair with a Brazilian lover, and H = Carson swindled Acme. We begin by considering the case in which the confirmation measures fail to track focused correlation.

Example 1

Suppose that the evidence set has larger focused correlation than , with respect to the hypothesis H. We interpret this relation as saying that evidence of both Carson's divorce proceedings and his travel cohere better toward supporting the hypothesis of fraud than evidence of Carson's divorce and affair.

But this fact does not necessarily mean that the coincidence of divorce and travel is also more confirmatory of the hypothesis of fraud than the divorce-plus-affair scenario. Why not?

Imagine a backstory such that Carson's affair, considered in isolation, supports fraud much more strongly than a trip to Brazil, considered in isolation. Perhaps poor Carson is in over his head: his affair is a surprise to all and an expensive one, at that. His trip to Rio, although unusual, pales by comparison, and on its own there is no reason to think that his having a shady seat companion is anything but a coincidence. Relative to this background knowledge, A, . Now add the news of Carson's divorce proceedings. This would not dent the confirmatory power of the affair scenario since adding a divorce to that affair only piles on more expenses for Carson. But learning of the divorce would also do little to alter the confirmatory force of the travel scenario on judging whether Carson is complicit in defrauding Acme. Thus, .

But then there is little reason to believe that we need the coincidence of a divorce and an affair to send up a red flag in Acme's fraud department, as opposed to just knowing of Carson's liaison. The coherence of divorce and affair may therefore be less than the coherence of divorce and travel with respect to confirming the hypothesis of fraud, and still the conjunction of divorce and affair may be more confirmatory of fraud than the conjunction of divorce and travel. As we have seen, this happens when the confirmatory power of the affair (considered by itself) becomes sufficiently large in comparison with the confirmatory power of the travel (considered by itself). When coherence is measured by focused correlation, what qualifies as ‘sufficiently large’ is then precisely quantified by the bound (2).

3.2. From Confirmation to Focused Correlation

Let us now tackle the converse relation, namely, the implication from a difference in confirmation to a difference in focused correlation.

Proposition 3. If and

then also .

The bound (3) is a catchall condition that holds for all confirmation measures considered in section 2.1. It may be tightened by focusing on a particular measure. For example, for and , the bound is

which allows for within certain limits.

The existence of a bound on the ratio of to has a similar explanation as the bound (2). If by a sufficiently large amount, then the larger confirmatory power of over may be overshadowed by the fact that the confirmation of H by alone, with respect to the confirmation by the conjunction , is much less than the confirmation of H by alone, with respect to the confirmation by the conjunction . Or, put more plainly, the beliefs in the set cohere better toward supporting H than do the beliefs in the set .

Example 2

In the spirit of the introduction, suppose that Carson's affair is viewed as an ordinary disaster, and his travel plans are more suspicious—it is known that he booked both his seat and his companion's, let's say. So, suppose that the conjunction of divorce and travel is more confirmatory of the hypothesis of fraud than the conjunction of divorce and an affair on this background knowledge, labeled B: , and .

Now, what happens when we consider focused correlation instead of confirmation? Equation (1) weighs and by how much each of the individual pieces of evidence differing between the two evidence sets—that is, Carson's travel () and affair ()—confirms the suspicion of fraud. If condition (3) is violated, that is, if , this would mean that isolated knowledge of Carson's travel makes us more inclined to think of the possibility that he may be engaged in fraudulent business than if we simply heard that Carson is embroiled in an affair. It is the merit of focused correlation to now put up a cautionary flag when it comes to considering the implications for the relative degrees of coherence of the two evidence sets and . How so?

The argument is similar to the discussion of the previous example in section 3.2. Since according to background B the evidence of travel, considered in isolation, is already more supportive of the hypothesis of fraud than the evidence of an affair, also considered in isolation, and since the only other piece of evidence—namely, the fact that Carson is in the midst of divorce proceedings—is common to both evidence sets, then one should not automatically conclude that it is the coincidence of divorce and travel that boosts our confidence in fraud. Rather, it may be the case that the evidence of travel alone is what makes the hypothesis so eminently plausible, not the fact that it coincides with Carson's divorce. Conversely, while Carson's affair alone may barely raise suspicions of fraud, its coincidence with Carson's being in the midst of a messy—and potentially financially threatening—divorce may boost the likelihood of fraud by a large margin. In other words, although the coincidence of Carson's divorce and travel is more confirmatory of fraud than the coincidence of his divorce and affair, the latter evidence set may well be judged to have a larger degree of coherence. Focused correlation captures this subtlety.

Conversely, if condition (3) is obeyed, that is, if , focused correlation and confirmation will gravitate toward favoring the same evidence set. Together with the fact that , we can now safely conclude that divorce and travel cohere better toward supporting the hypothesis of fraud than the conjunction of divorce and affair: although evidence of travel alone is less (or equally) indicative of fraud compared with the evidence of an affair, when taken together with the evidence of an ongoing divorce, Carson's travel will ring Acme's alarm bells more readily than evidence of Carson's involvement with his Brazilian sweetheart.

3.3. Bidirectional Tracking

Propositions 2 and 3 show that when the ratio of to is within a certain range, focused correlation will be tracked by confirmation, and when it is within another range, focused correlation will track confirmation. It is important to note that these ranges overlap. Propositions 2 and 3 readily quantify this overlap, that is, the range of values of , for which bidirectional tracking holds. Combining propositions 2 and 3 yields:

Corollary 1. If

then .

That is, ensuring that the tracking goes both ways does not require us to impose the rather strong assumption A2 of equal relevance of all bits of evidence.

Choosing a particular confirmation measure allows us to extend even further the range over which the bidirectional tracking holds. For example, for or , we can use the bound (4) instead of (3), which gives a ‘relaxed’ version of corollary 1:

Corollary 2. If

then .

Figure 1 illustrates the different ranges of for which uni- and bidirectional tracking between focused correlation and confirmation obtains.

Figure 1

Regimes of for which focused correlation tracks or is tracked by confirmation measures. The relevance of a piece of evidence for a given hypothesis H is , and ‘’ is shorthand for the implication , which represents a tracking of focused correlation by the confirmation measures . Similarly for ‘’. Over a range of values for , bidirectional tracking obtains (bold box); that is, . For specific choices of , for example, for the measures or shown here, the tracking range is extended further.

3.4. Equality of Focused Correlation and Confirmation

For the sake of completeness, let us also explicitly consider the connection between equal values of focused correlation and equal values of confirmation:

Proposition 4. iff

In other words, equal values of focused correlation for the two sets and translate into equal degrees and of confirmation—and vice versa—if and only if the equal-relevance assumption A2 is made. The reason for this result should now be clear from the above discussion of propositions 2 and 3.

4. Discussion

In this article, we have demonstrated that the equal-relevance assumption can be relaxed within certain limits without upsetting the bidirectional tracking between focused correlation and several incremental confirmation measures; if only unidirectional tracking is required, these limits become even less stringent. However, if the degrees of relevance diverge too strongly, the tracking fails. We have shown that the ratio of to is crucial in assessing, both qualitatively and quantitatively, when and where the tracking fails.

However, this observation should not be viewed as reflecting a failure or deficiency of the measure of focused correlation. Rather, it may be considered as illuminating a shortcoming of incremental confirmation measures since they fail to take into account the additional information provided by the relevance of each individual bit of evidence. As we have seen, this information is important in flagging a situation in which a large boost in confirmation caused by adding a piece of evidence has nothing or only little to do with the coincidence of this evidence with the existing body of evidence. One merit of focused correlation is to detect such cases.

In one class of cases that has received a lot of attention, namely, the single-factor common-cause model mentioned in the introduction, any measure linking positive association to positive confirmation breaks down, and comparisons (assuming A2) are never false because they cannot be different. But these facts take nothing away from focused correlation, for two reasons. First, because this class of models is determined nonparametrically, we can learn whether one hypothesis is a common cause (Silva et al. Reference Silva, Glymour, Scheines and Spirtes2006) and then attack the problem by other methods—namely, by looking at the relative strengths of the evidence. We do not need the sign of focused correlation in single-factor common-cause models to indicate the sign of confirmation if we understand clearly when the two do not align. In the case of single-factor common-cause models, the correlation of the evidence is completely determined by the relevant strengths of the evidence (Danks and Glymour Reference Danks, Glymour, Breese and Koller2001). Faulting focused correlation for failing to indicate confirmation in this class of models is akin to faulting screwdrivers for failing to drive nails: it is a true claim that is entirely beside the point. Second, it should be clear that the failure of focused correlation to track differences in confirmation of evidence in common-cause models in proposition 1 is driven by A2 rather than a deficiency in focused correlation. But, again, although our results here allow for some tracking in common-cause models, at bottom a difference in confirmation in this class of models boils down to a difference in the individual strengths, .

Focused correlation is defined generally for n evidence variables, but we only discuss evidence sets of size two, and difficulties loom for attempts to make comparisons of larger evidence sets (Bovens and Hartmann Reference Bovens and Hartmann2006). Even though focused correlation is defined for arbitrary-sized information sets, the relationships of confirmation, covariance, and correlation are binary—or three-place in conditional form. Thus, the expansion of evidence sets beyond size two will require a partition of the evidence set since there are many incremental confirmation questions that are compatible with one focused correlation problem involving an evidence set of a size greater than two. To expect otherwise is a category mistake, and negative results should be no surprise. That said, if there is not an interest in linking focused correlation for large evidence sets—which, technically, is not a measure of correlation but is instead a distance-from-independence measure—to confirmation, then one may explore extending the results presented here to facilitate direct comparisons of large evidence sets.

Finally, two commonly discussed confirmation measures have been omitted from discussion, Carnap's (Reference Carnap1962) relevance measure τ and the old evidence measure oe (Christensen Reference Christensen1999; Joyce Reference Joyce1999), which are defined as

and

Neither measure is tracked by focused correlation. The reason is that both measures are less constrained than incremental confirmation measures. In the case of Carnap's measure of relevance, observe that is the covariance of the binary variables H and E:

Although positive covariance for H and a single evidence variable E is ensured by the positive relevance condition A1, conditions A1 and A2 and equation (9a) do not constrain the sign of for evidence sets of size two. In the case of oe, there is some recognition already of the difference between incremental confirmation measures and measures for novelty of evidence like oe (Joyce Reference Joyce1999). The failure to compare evidence sets by their degree of surprise, particularly under the restrictive conditions of positivity, A1, A2, and equation (9a), is an additional reason to sharply distinguish between these two types of measures. Countermodels for oe are discussed in the appendix.