1. Rival Bayesian Measures of Confirmation

Judgments concerning the support that a piece of information brings to a hypothesis are commonly required in scientific research as well as in other domains (medicine, law), and a major aim of a theory of inductive reasoning is to provide a proper foundation to such judgments.

Within the Bayesian approach to inductive reasoning, an attempt to measure degrees of confirmation, or evidential support, should reflect, and extend, a basic qualitative view of confirmation—labeled the “classificatory concept of confirmation” by Carnap Reference Carnap([1950] 1962, 21–22). This view identifies confirmation with an increase in the probability of the hypothesis (conclusion) h provided by the piece of information (premise) e, neutrality with a lack of impact of e on the probability of h, and disconfirmation with a decrease of such a probability as an effect of e. A common way to convey this distinction is to formalize the confirmation relation by a mathematical function c(e, h), depending on probability values concerning e and h, such that the following condition (BC) is satisfied (BC stands for “Bayesian confirmation”):Footnote ¹

(BC)

Condition (BC) does not put any constraint on the values to be assigned to confirmatory arguments (as long as such values are positive) or to disconfirmatory arguments (as long as such values are negative). Indeed, (BC) does not constrain the choice of one single measure of confirmation as the most adequate (a point emphasized by Fitelson [Reference Fitelson1999]). As a matter of fact, several nonequivalent measures can be, and have been, devised that map relevant probability values onto a number that is positive iff $p(h\vert e) > p(h) $ , amounts to zero iff $p(h\vert e) =p(h) $ , and is negative otherwise. Table 1 provides a catalogue collected from the literature.

Table 1.Alternative Bayesian Measures of Confirmation.

$D(e,\,h) =p(h\vert e) -p(h) $	Carnap (Reference Carnap[1950] 1962)
$S(e,\,h) =p(h\vert e) -p(h\vert \neg e) $	Christensen (Reference Christensen1999)
$M(e,\,h) =p(e\vert h) -p(e) $	Mortimer (Reference Mortimer1988)
$N(e,\,h) =p(e\vert h) -p(e\vert \neg h) $	Nozick (Reference Nozick1981)
$C(e,\,h) =p(e\,\& \,h) -p(e) \cdot p(h) $	Carnap (Reference Carnap[1950] 1962)
$R(e,\,h) =[ p(h\vert e) / p(h) ] -1$	Finch (Reference Finch1960)
$G(e,\,h) =1-[ p(\neg h\vert e) / p(\neg h) ] $	Rips (Reference Rips2001)
$L(e,\,h) =[ p(e\vert h) -p(e\vert \neg h) ] / [ p(e\vert h) +p(e\vert \neg h) ] $	Kemeny and Oppenheim (Reference Kemeny and Oppenheim1952)

Quantitative measures of confirmation such as those in Table 1 have the important property of allowing ordinal judgments concerning inductive strength, such as “hypothesis h receives more empirical support by $e_{1}$ than by $e_{2}$ ” or “e confirms $h_{1}$ to a greater extent than $h_{2}$ .” Two measures of confirmation are ordinally equivalent iff they give the same answer to the following question: Which one, if any, has a higher confirmation value in any given pair of arguments?Footnote ² In this respect, Table 1 provides a representative sample, for most of the Bayesian confirmation measures that we have been able to identify in the literature, but have not included in our list, are ordinally equivalent to some measure that does appear in the list.Footnote ³

2. A First Adequacy Requirement

One way to handle the plurality of Bayesian measures of confirmation is to resort to the long-standing and traditional view of inductive logic as an “extension” of classical deductive logic (Carnap Reference Carnap[1950] 1962). Consider a function v construed on the basis of classical deductive logic and such that, for any argument (e, h), v assigns it the same positive value (e.g., +1) iff $e\vDash h$ (i.e., iff e implies h), an equivalent value with opposite sign (−1) iff $e\vDash \neg h$ (i.e., iff e refutes h), and value 0 otherwise. The relationships between the logical implication or refutation of h by e and the conditional probability of h given e yield that any Bayesian confirmation measure c agrees with v in the minimal sense that if v(e, h) is positive, the same is true of c(e, h); and if v(e, h) is negative, the same is true of c(e, h). Then, consistency with the following principle may be posited as a plausible adequacy requirement for c (here, “Ex” stands for the “extension” from the deductive to the inductive domain):

Fulfillment of principle (Ex₁) guarantees that, by c, any conclusively confirmatory argument (e, h) (i.e., such that $e\vDash h$ ) is assigned a higher value than any argument that is not conclusively confirmatory and any conclusively disconfirmatory argument (e, h) (i.e., such that $e\vDash \neg h$ ) is assigned a lower value than any argument that is not conclusively disconfirmatory.

Remarkably, it can be proved (see below) that only one among the measures of confirmation listed in Table 1 actually fulfills (Ex₁), that is, L.Footnote ⁴ It turns out, however, that there is a rather simple way to obtain a measure of confirmation that does fulfill (Ex₁) from either D, S, M, N, C, R, or G. To see this, let's consider the formulas to which such measures can be reduced when $e\vDash h$ and when $e\vDash \neg h$ , respectively, as reported in Table 2.

Table 2.

Measure	If $e\vDash h$	If $e\vDash \neg h$
$D(e,\,h) $	$p(\neg h) $	$-p(h) $
$S(e,\,h) $	$p(\neg h) / p(\neg e) $	$-p(h) / p(\neg e) $
$M(e,\,h) $	$p(e) \cdot [ p(\neg h) / p(h) ] $	$-p(e) $
$N(e,\,h) $	$p(e) / p(h) $	$-p(e) / p(\neg h) $
$C(e,\,h) $	$p(e) \cdot p(\neg h) $	$-p(e) \cdot p(h) $
$R(e,\,h) $	$p(\neg h) / p(h) $	−1
$G(e,\,h) $	1	$-p(h) / p(\neg h) $

Now, for each measure in Table 2, let's employ the expressions appearing in the second and third columns (taken in absolute value) to “normalize” degrees of positive and negative evidential support, respectively, as follows:

For our present purposes, two facts deserve emphasis. First of all, standard probability calculus yields that

In fact, at this point, we have one single (new) Bayesian measure of confirmation. From now on, we will label it Z.

The second important fact is that Z does fulfill principle (Ex₁) above. The main formal results reported in this section are summarized in the following theorem and demonstrated in its proof (see Appendix A):

3. Z-Measures

By means of Z it is possible to define a whole set (indeed, a continuum) of confirmation measures (we will call them “Z-measures”) that we would like to propose as an account of evidential support within the Bayesian framework. Z-measures are defined as follows:

where α is positive. By being all monotone transformations of the same quantity, all Z-measures are ordinally equivalent. α plays the role of a parameter controlling curvature (see Figure 1).

Figure 1.

Graphical representations of Z-measures with different values of α. On the x-axis lie posterior probability values of h.

As shown by the figure, when $\alpha =1$ , the graphical representation of Z _α is composed of a pair of straight lines; when $\alpha < 1$ , Z _α is an $\textsf{S}$-shaped function, indicating a higher sensitivity of confirmation (disconfirmation) to relatively small positive (negative) departures from the prior probability of h; finally, when $\alpha > 1$ , Z _α is an inverse $\textsf{S}$-shaped function, indicating a higher sensitivity of confirmation (disconfirmation) to relatively wide positive (negative) departures from the prior probability of h.Footnote ⁵

Theorem 1 above immediately implies that all Z-measures fulfill principle (Ex₁). Here are some further desirable properties of Z-measures (which are not, however, sufficient to establish a better adequacy of Z-measures as compared to all their major competitors):

4. Symmetries

Eells and Fitelson (Reference Eells and Fitelson2002) have recently proposed to narrow down the set of adequate Bayesian candidates for quantifying confirmation by considering a series of “symmetries and asymmetries.” We will now present an extended and systematic treatment of this issue.

Consider a set of sentences Γ, closed under negation and conjunction, on which a probability function p is defined.Footnote ⁷ In what follows, by a “symmetry” we will mean a function σ from $\Gamma \otimes \Gamma $ into $\Gamma \otimes \Gamma $ such that σ(e, h) is obtained from (e, h) by applying the negation operator (¬) to either e or h (or both) and/or by inverting them. On the whole, there are seven such symmetry functions:

Here, E stands for “(negation of the) evidence,” H for “(negation of the) hypothesis,” and I for “inversion (of evidence and hypothesis).”

Now let s (for “sign”) be a function from $\Gamma \otimes \Gamma $ into {−1, 0, 1}, defined as follows: $s(e,\,h) =1$ if $p(h\vert e) > p(h) $ ; $s(e,\,h) =0$ if $p(h\vert e) =p(h) $ ; and $s(e,\,h) =-1$ if $p(h\vert e) < p(h) $ .

Function s allows a distinction between convergent and divergent symmetries: a symmetry σ is convergent iff (e, h) and σ(e, h) have the same sign, that is, iff, for any $e,\,h\in \Gamma $ , $s(e,\,h) =s[ \sigma (e,\,h) ] $ ; a symmetry σ is divergent iff (e, h) and σ(e, h) have opposite signs, that is, iff, for any $e,\,h\in \Gamma $ , $s(e,\,h) =-s[ \sigma (e,\,h) ] $ . It can easily be proved that H, E, HI, and EI are divergent symmetries, whereas I, EH, and EHI are convergent.

By means of s, we will also state the following definition:

When a confirmation measure c mirrors a symmetry σ in both cases (confirmation and disconfirmation), then we will simply say that c mirrors σ. (Notice that any measure c trivially mirrors any symmetry σ in case of neutrality, i.e., when $c(e,\,h) =0$ .)

The conditions on which c mirrors any particular symmetry σ can now be stated as follows:

Divergent Symmetries.

Convergent Symmetries.

5. A New Set of Adequacy Requirements

The point of Eells and Fitelson (Reference Eells and Fitelson2002)—whose discussion is in fact limited to E, H, I, and EH Footnote ⁸—is that the appeal of c mirroring any particular symmetry may be assessed by some rather simple and intuitive examples, which different measures of confirmation may or may not match.

To illustrate, take E: Should it be the case that $c(e,\,h) =-c(\neg e,\,h) $ ?

Consider a standard deck of cards. It seems that having drawn a jack card confirms that the card is a face card to a greater extent than having drawn a card that is not a jack disconfirms the same hypothesis. The reason is that in the former case the available evidence conclusively establishes the hypothesis, whereas in the latter case the available evidence does not conclusively refute the hypothesis. Conversely, having drawn an ace card disconfirms that the card is a face card to a greater extent than having drawn a card that is not an ace confirms the same hypothesis. On the basis of similar examples, Eells and Fitelson (Reference Eells and Fitelson2002) argue that an acceptable Bayesian measure of confirmation should not mirror symmetry E.

Is there any general and simple principle to determine how a measure of confirmation should behave as far as all symmetries are concerned? We think there is one, which, just as (Ex₁) above (see Section 2), builds on the traditional view of inductive logic as an “extension” of classical deductive logic. Once again, such a principle may be stated by means of function v, defined on the basis of classical deductive logic (see Section 2):

We shall see shortly why the distinction between the two cases (confirmation vs. disconfirmation) is important. For now, notice that (Ex₂) is a rather strong condition and, as we shall also see, a quite powerful theoretical tool. We don't know of any explicit and general treatment of it. However, it backs up several arguments circulating in inductive logic and in Bayesian confirmation theory in particular. For the moment, we will posit (Ex₂) as a legitimate guideline for the discussion of the symmetries that an adequate Bayesian measure of confirmation should or should not mirror. In what follows, we will try to establish consistency with (Ex₂) as a compelling desideratum through a detailed analysis of its consequences.

Resort to principle (Ex₂) may seem more adequate to identify the symmetries not to be mirrored (see, e.g., the jack and the ace examples above) rather than for the selection of those to be mirrored by c. However, the principle can be usefully applied in the latter sense as well, for instance as far as H is concerned. Indeed, from (Ex₂) it follows that

In the absence of compelling counterexamples (which we were not able to devise), the equality above can be presented as a plausible extension of the fact that e implies h (i.e., $e\vDash h$ ) iff e refutes ¬h (i.e., $e\vDash \neg \neg h$ ).Footnote ⁹

Now consider symmetry I, that is, the following equality:

Should we require that a confirmation measure classify inversely symmetric arguments as equally strong (recall that the inverse symmetry function is convergent), that is, that it mirror I? Eells and Fitelson (Reference Eells and Fitelson2002) argue that we should not, but interestingly in this connection they explicitly resort only to counterexamples involving pairs of confirmatory arguments. This is noteworthy, for consider again the ace example above: in contrast to the jack case, we see as intuitively compelling that having drawn an ace card does disconfirm its being a face card to the same extent that having drawn a face card disconfirms its being an ace. Thus, we concur with Eells and Fitelson in finding unattractive the above equality in case of confirmation but not necessarily in case of disconfirmation as well. Indeed, c mirroring I in case of disconfirmation (but not in case of confirmation) also follows from (Ex₂) and, in the absence of compelling counterexamples (which we were not able to devise), this seems a plausible extension of the theorem of deductive logic according to which $e\vDash \neg h$ iff $h\vDash \neg e$ , that is, a plausible inductive-theoretic counterpart of the commutative (or “inversely symmetric”) nature of logical inconsistency.

Generalizing this line of argument, one can show that principle (Ex₂) yields a definite answer for any symmetry both in case of confirmation and in case of disconfirmation. Table 3 reports the outcomes of such analysis, illustrating in any single case either the relevant parallelism with an analogous state of affairs whithin deductive logic or relevant counterexamples.

Table 3.The Whole Set of Consequences of (Ex₂)

	In Case of Confirmation		In Case of Disconfirmation
	Divergent Symmetries
E	No For some confirmation (e, h) $c(e,\,h) \neq -c(\neg e,\,h) $	e.g., c(jack, face) > −c(not-jack, face)	No For some disconfirmation (e, h) $c(e,\,h) \neq -c(\neg e,\,h) $	e.g., c(ace, face) < −c(not-ace, face)
H	Yes For any confirmation (e, h) $c(e,\,h) =-c(e,\,\neg h) $	e implies h iff e refutes ¬h	Yes For any disconfirmation (e, h) $c(e,\,h) =-c(e,\,\neg h) $	e refutes h iff e implies ¬h
EI	No For some confirmation (e, h) $c(e,\,h) \neq -c(h,\,\neg e) $	e.g., c(jack, face) > −c(face, not-jack)	Yes For any disconfirmation (e, h) $c(e,\,h) =-c(h,\,\neg e) $	e refutes h iff h implies ¬e
HI	Yes For any confirmation (e, h) $c(e,\,h) =-c(\neg h,\,e) $	e implies h iff ¬h refutes e	No For some disconfirmation (e, h) $c(e,\,h) \neq -c(\neg h,\,e) $	e.g., c(ace, face) < −c(not-face, ace)
	Convergent Symmetries
I	No For some confirmation (e, h) $c(e,\,h) \neq c(h,\,e) $	e.g., c(jack, face) > c(face, jack)	Yes For any disconfirmation (e, h) $c(e,\,h) =c(h,\,e) $	e refutes h iff h refutes e
EH	No For some confirmation (e, h) $c(e,\,h) \neq c(\neg e,\,\neg h) $	e.g., c(jack, face) > c(not-jack, not-face)	No For some disconfirmation (e, h) $c(e,\,h) \neq c(\neg e,\,\neg h) $	e.g., c(ace, face) < c(not-ace, not-face)
EHI	Yes For any confirmation (e, h) $c(e,\,h) =c(\neg h,\,\neg e) $	e implies h iff ¬h implies ¬e	No For some disconfirmation (e, h) $c(e,\,h) \neq c(\neg h,\,\neg e) $	e.g., c(ace, face) < c(not-face, not-ace)

6. Why Z-Measures Are Theoretically Appealing

We claim that Z-measures should be of interest for any Bayesian confirmation theorist who finds principle (Ex₂) compelling for the plain fact that such measures are all consistent with (Ex₂) whereas, to our knowledge, no alternative measure proposed so far is. The proof of the following theorem (see Appendix B), which is the main formal result of the present work, illustrates the point.Footnote ¹⁰

The fact that measure L does fulfill principle (Ex₁) but not (Ex₂) shows that the former does not imply the latter. Notably, it is also not the case that (Ex₂) implies (Ex₁).Footnote ¹² Thus both principles are complementary but independent constraints stemming from the same idea of confirmation theory as an extension of deductive logic. Arguably, however, one may still cast doubts on this strict parallelism and question why, after all, we should want a formal account of inductive reasoning to be consistent with (Ex₂).

Let's take symmetry I as an example about which such doubts may typically be raised. Suppose that you must rely on two generally accurate but fallible devices $A_{1}$ and $A_{2}$ in order to obtain information about a playing card. $A_{1}$ works as follows: it classifies a submitted card as a jack or a not-jack and reports the outcome with an accuracy rate of 95% (not depending on whether the card is or is not, in fact, a jack). $A_{2}$ works as follows: it classifies a submitted card as a face or not-face and reports the outcome, again with an accuracy rate of 95% (not depending on whether the card is or is not, in fact, a face). Now suppose that a card is drawn from a standard deck by a genuinely random process and submitted to both $A_{1}$ and $A_{2}$ . Let e be “ $A_{1}$ reports that it is a jack card” and h be “ $A_{2}$ reports that it is a face card.” On reflection, it seems clear to us that e confirms h more than h confirms e, even if none is strictly implied by the other. (If this example sounds unbearably artificial, let $A_{1}$ and $A_{2}$ be two medical tests for two diagnostic hypotheses such that the former implies the latter.)

But now consider a third device $A_{3}$ such that it classifies a card as an ace or a not-ace, again with an accuracy rate of 95% (not depending on whether the card is or is not, in fact, an ace). Suppose that a card is selected by a new random extraction and submitted to both $A_{3}$ and $A_{2}$ . Let e* be “ $A_{3}$ reports that it is an ace” and h, once again, “ $A_{2}$ reports that it is a face card.” Here, we see no reason why the two arguments (e*, h) and (h, e*) should have a different (negative) strength. Sentences e* and h are just “almost” incompatible; and, so it seems, that is all. (If this example sounds unbearably artificial, now let $A_{3}$ and $A_{2}$ be two medical tests for two mutually exclusive diagnostic hypotheses.)

The problem is that, unlike Z-measures, all the competitors violating (Ex₂) also violate intuition here by either ranking (e, h) as an equally strong confirmatory argument as (h, e)Footnote ¹³ or ranking one between (e*, h) and (h, e*) as a significantly stronger disconfirmatory argument than the other (without even agreeing on which one is the strongest!).Footnote ¹⁴

Similar examples may be conceived for any row in Table 2 to underpin the plausibility of (Ex₂) and therefore of Z-measures as a theoretically appropriate basis to assess inductive strength.

7. An Empirical Test of Competing Measures

Formal theories of reasoning may serve various purposes. So far, we have been concerned with the elucidation of a central epistemological concept (confirmation) on the basis of a set of intuitively plausible and explicitly stated constraints. Whether or not a theoretically sound measure of confirmation may also be employed to accurately describe (and predict) human judgment and behavior is a distinguished question—a very intriguing one, to which we will now turn. Our purpose is to extend the assessment of the potentiality of the proposed approach to the empirical domain of the psychology of reasoning and cognition, which has often widely profited from the comparison between actual human performance and compelling formal theories such as probability theory or expected utility theory.

In a previous work (Tentori et al. Reference Tentori, Crupi, Bonini and Osherson2007), an experiment has been presented aiming at an empirical test of some of the alternative Bayesian measures of confirmation. For each of 26 participants, an urn was randomly and covertly selected out of A (30 black balls plus 10 white balls) and B (15 black balls plus 25 white balls). Participants were then asked to judge the quantitative impact of each outcome of a series of 10 random extractions without replacement on the two complementary hypotheses that A versus B had been selected (see Tentori et al. [Reference Tentori, Crupi, Bonini and Osherson2007] for a detailed description of the procedure). The experiment showed that, when predictions about confirmation judgments were computed from the randomness of the initial selection, the composition of the urns, and the outcomes of previous draws, measure L outperformed several competitors (including D, N, and C), yielding a reliably higher average correlation with the judged impacts on both hypotheses A and B provided by the participants. In what follows we will test all the competing confirmation measures listed in Table 1, along with the simplest Z-measure (i.e., with $\alpha =1$ ), against the same data.

As Table 4 shows, Z is the most descriptively accurate measure in this experimental setting. (The table also reports a reliable superiority of the most accurate measures in comparison with posterior probability, taken as a predictor of confirmation judgments. Notice that this indicates that the participants appropriately distinguished posteriors from evidential impact.)

Table 4.Average Correlations for Competing Confirmation Measures.

Measure (Predictor)	Average Correlation with Participants’ Confirmation Judgments for the Hypothesis:
	Urn A Selected	Urn B Selected
Z	.756*	.775*
L	.740*	.754*
N	.730*	.745*
M	.628†	.588
G	.549	.631
R	.619	.557
S	.594	.613
C	.586	.605
D	.573	.589
$p(A[ B] \vert e) $	.488	.508

Note.—Reported values are the average of 26 correlations (one per participant) between the confirmation judgments predicted by each measure (on the basis of the randomness of the initial selection, the composition of the urns, and the outcomes of previous draws) and the confirmation judgments expressed by each participant. Each correlation involved 10 observations. $p(A[ B] \vert e) $ denotes $p(A\vert e) $ or $p(B\vert e) $ as appropriate. Comparisons by paired t-test with the average correlation for $p(A[ B] \vert e) $ : *p < .001; ^†p = .06.

Consistent with these results, Table 5 shows that, by a paired t-test, Z outperforms all competitors in predicting judged impact on both hypotheses A and B, with only one major exception (the lack of a statistically reliable difference from L for hypothesis A). Moreover, a simple nonparametric analysis consisting in counting the subjects (out of 26) for whom Z predicted better than each rival measure shows that Z is generally a better predictor in the majority of cases (see Table 5).

Table 5.Comparison of Z with Other Confirmation Measures.

	L	N	M	G	R	S	C	D
	Hypothesis: Urn A Selected
Z	t = 1,44 n.s.	t = 1,92 p = .07	t = 2,87 p = .008	t = 4,52 p = .0001	t = 3,25 p = .003	t = 3,13 p = .004	t = 3,25 p = .003	T = 3,61 p = .001
	18	18	18	22	21	21	21	21
	Hypothesis: Urn B Selected
Z	t = 2,02 p = .05	t = 2,31 p = .03	t = 3,50 p = .002	t = 3,51 p = .002	t = 4,01 p = .0005	t = 2,84 p = .009	t = 2,96 p = .007	t = 3,28 p = .003
	18	18	22	24	23	19	19	21

Note.—Each higher cell reports a paired t-test between the correlations obtained from the measure in the associated column and from measure Z. Each lower cell shows the number of participants (out of 26) for whom Z predicted better than the rival measure at the top of the column.

These results suggest that the virtues of Z-measures might not be confined to the level of epistemological analysis, but extend to the descriptive dimension of the psychology of confirmation.

We are not claiming that the probabilistic computations involved in the definition of Bayesian confirmation measures should be taken literally as models of the cognitive processes leading to confirmation judgments. We are well aware that, when judging probabilities, subjects often depart from the Bayesian prescriptions (Kahneman, Slovic, and Tversky Reference Kahneman, Slovic and Tversky1982) and conform to them only under specific conditions (Girotto and Gonzalez Reference Girotto and Gonzalez2001). Precisely for this reason, it is noticeable that a quantitative Bayesian account of evidential impact can reach a promising predictive accuracy relative to naïve judgments of confirmation. Whatever the routes by which confirmation judgments are elaborated by nonexpert subjects, the experiment illustrates that some salient features of such judgments may be captured, to a significant degree, by a theoretically sound account of confirmation. In Marr's (Reference Marr1982) terms, such an account might work as an effective, although approximated, computational-level model, compatible with more “realistic” process-level descriptions of the cognitive bases of ordinary inductive reasoning.

In any event, more research is needed to assess the stability and the consequences of the present results. In a touchstone study in the psychology of inductive reasoning with statements involving familiar biological categories and “blank” biological predicates (such as “robins use serotonin as a neurotransmitter”), Osherson et al. (Reference Osherson, Smith, Wilkie, Lopez and Shafir1990) documented that inversely symmetric confirmatory arguments are not generally judged as equally strong, consistent with principle (Ex₂) (see Table 3). Further studies aiming at the empirical test of various symmetries and asymmetries might provide evidence about substantial qualitative phenomena concerning inductive reasoning and play a crucial role in assessing the descriptive reliability of competing measures of confirmation outside the context of our urn-based experiment.

In our opinion, the discussion presented here suggests that the interaction between the epistemological and the cognitive dimension might be fruitful in this field of inquiry as it has been in many other areas of the study of reasoning.