1. Introduction
Leitgeb and Pettigrew (Reference Leitgeb and Pettigrew2010a, Reference Leitgeb and Pettigrew2010b) argue for a number of constraints on rational belief through appeal to the following norm:
Accuracy: An epistemic agent ought to approximate the truth. In other words: she ought to minimize her inaccuracy. (Reference Leitgeb and Pettigrew2010a, 202)
Of course, for Accuracy to be of much use to epistemologists, more must be said, and Leitgeb and Pettigrew are able to make it mathematically precise. In order to show how, I will first streamline the discussion and assume all agents under consideration obey the following synchronic norm:
Probabilism: At any given time, an agent ought to have a probabilistically coherent credence function.
We can treat inaccuracy at a world formally as follows. Let A be a proposition, w a world, and let 
be 1 if A is true at w and be 0 otherwise.Footnote 1 The idea is that 
—the characteristic function of w—represents the best possible credence to have at w since it assigns maximal (minimal) credence to all propositions true (false) there. Leitgeb and Pettigrew argue that we should measure the inaccuracy of a probability function Cr at w by seeing how far it is from χw under the average of the squared Euclidean distance between them.Footnote 2 We end up with the Brier score as our inaccuracy measure:Footnote 3
The Brier score has a natural generalization to a larger class of inaccuracy measures, known as quadratic scoring rules, which are of the following form:
where 
and each 
. Quadratic scoring rules let us thumb the scale and count some propositions as more important than others. If we really care about whether it will rain tomorrow but not so much about whether there are an even number of stars in the universe, we assign a higher weight to the former proposition when measuring our inaccuracy.Footnote 4
In addition to Leitgeb and Pettigrew (Reference Leitgeb and Pettigrew2010a, Reference Leitgeb and Pettigrew2010b), quadratic scoring rules have a number of defenders and admirers; in fact, it is safe to say that they are the clear front-runners in the debate over how best to measure the inaccuracy of probability functions.Footnote 5 Nonetheless, I think that quadratic scoring rules serve as poor measures of inaccuracy because they naturally lead to an extremely unattractive updating procedure, as can be shown through application of a result in Leitgeb and Pettigrew (Reference Leitgeb and Pettigrew2010b). Other measures, such as the logarithmic scoring rule, lead to the more attractive and standard procedure of Jeffrey Conditionalization. Therefore, quadratic rules should be rejected.Footnote 6
2. Inaccuracy and Updating
2.1. Conditionalization
Leitgeb and Pettigrew have four different precisifications of Accuracy. We will be interested in how to update our entire credence function across time, so the version we want is:
Accuracy (Diachronic Expected Global): Suppose an agent has learned evidence between t and t′ that imposes constraints C on her belief function 
at t′, or on the set E of worlds that are epistemically possible for her at t′, or both. Then, at time t′, such an agent ought to have a belief function that satisfies constraints C and is minimal amongst belief functions thus constrained with respect to expected global inaccuracy by the lights of her belief function at time t, relative to a legitimate global inaccuracy measure [i.e., relative to the Brier score for Leitgeb and Pettigrew], and over the set of worlds that are epistemically possible for her at time t′ given the constraints C. (From Reference Leitgeb and Pettigrew2010a, 207, with minor changes)
Here is how ADEG leads to conditionalization: suppose I start with credence function Cr and learn for certain some new information E. I am now epistemically obligated to pick some probability function in set C of probability functions that assign 1 to E. But which should I pick? If I follow ADEG, I ought to pick the member of C that minimizes expected inaccuracy by the lights of Cr. So, if 
is my favored measure of inaccuracy, the best new credence function to adopt upon learning E according to Cr is the credence function 
, such that
is minimal.
It turns out that for nearly any measure of inaccuracy one might think is reasonable (including the quadratic scoring rules), the function that minimizes expected inaccuracy under these constraints is simply 
. In other words, I minimize expected inaccuracy upon learning new information if I update by conditionalization.Footnote 7
2.2. Updating with Uncertain Evidence
I will assume with Leitgeb and Pettigrew that not all updating should be by conditionalization. Sometimes, I get perceptual input that does not raise my credence in any proposition fully to 1. For instance, I might be in a pitch-black room with a piece of paper in front of me that I believe to degree .3 is red. A small bit of light comes in under the door, which allows me a better but not perfect view of the paper. Even though I did not learn any new salient proposition for sure, my credence in red changes to .7, and I am obligated to update the rest of my beliefs. For ease of reference, let's call a situation where we do not update any of our partial beliefs noninferentially to 0 or 1 an Uncertain Evidential Situation (UES).
To set up the primary issue of the paper, we can consider only a special class of UESs, which I will refer to as Jeffrey UESs. Let W be the set of possible worlds (with 
), and let 
be a partition of W with 
. Suppose we get some new uncertain evidence that forces us (noninferentially) to adopt new credence qi in Ei (where 
and 
).
The classic answer to this updating problem is that our new credence 
should be given by:Footnote 8
Jeffrey Conditionalization: In Jeffrey UESs, agents should update their credences through Jeffrey Conditionalization. That is, for any proposition A,
Jeffrey Conditionalization is the obvious way to extend standard conditionalization to cases with uncertain evidence, and it has long been more or less the only game in town when it comes to updating in Jeffrey UESs.
It is surprising, then, that quadratic scoring rules do not lead to Jeffrey Conditionalization. Leitgeb and Pettigrew (Reference Leitgeb and Pettigrew2010b) show that under the Brier score, ADEG requires a different updating procedure entirely, which I will argue has absurd consequences.Footnote 9
As before, we want to update by minimizing expected inaccuracy under constraints. This time, however, we do not restrict the class of worlds under consideration from W to E. Instead, we just consider the same class of worlds W, but we have new constraints imposed, given by the qi's. That is, we choose the credence function with minimal expected inaccuracy that assigns credence qi to Ei for all i.
Here is how this updating procedure works: We first take some element Ej of the partition and then add (!) a constant dj to 
for each world 
. In order to prevent negative credence, our new credence 
is just 
. By requiring that 
, we end up with only one possible choice of dj.Footnote 10 In other words, we end up with
LP Conditionalization: In Jeffrey UESs, agents should update their credences through Leitgeb Pettigrew Conditionalization. That is, let di be the unique real number such that
Then the agent ought to have belief function 
at t′ such that for 
:
(Leitgeb and Pettigrew Reference Leitgeb and Pettigrew2010b, 254)
It is worth emphasizing that choosing this updating procedure is inevitable for adherents of ADEG and the Brier score. Let's review why exactly: an agent starts with some credence function Crt. She then finds herself in a Jeffrey UES, and her new evidence compels her to have credence qi in the propositions Ei. She is still left with continuum-many credence functions to choose from that meet these constraints, but she wants the one she expects at t to be the most accurate of those available. If she uses the Brier score, she will pick the one mandated by LP Conditionalization.Footnote 11 After discussing the problematic cases, I will argue below that the problems with LP Conditionalization should lead to rejection of the Brier score, even as a measure of synchronic accuracy.
3. Problems
3.1. Initial Issues
Some apparent problems with LP Conditionalization are discussed and defended in Leitgeb and Pettigrew (Reference Leitgeb and Pettigrew2010b). These will not be my primary focus here, but it is worth going over them briefly.
The first is that the probability of some worlds can be lowered from a positive prior all the way to 0 in a UES. For instance, consider the case shown in table 1, where 
is the result of raising the probability of 
to .8 and lowering that of 
to .2 under LP Conditionalization. Oddly, this results in assigning w2 credence 0.Footnote 12
| w 1 | w 2 | w 3 | |
|---|---|---|---|
| Crt | .3 | .2 | .5 |
 |
.8 | 0 | .2 |
A related issue is that LP Conditionalization, unlike Jeffrey Conditionalization, does not have standard conditionalization as a limiting case. In other words, raising the probability of one element E of the partition to 1 and LP Conditionalizing is not the same as simply conditionalizing on E. Table 2 shows the different results of raising the posterior of 
to 1 under the alternative procedures.
| w 1 | w 2 | w 3 | w 4 | |
|---|---|---|---|---|
| Crt | .3 | .2 | .25 | .25 |
 |
.6 | .4 | 0 | 0 |
 |
.55 | .45 | 0 | 0 |
Unlike Leitgeb and Pettigrew (Reference Leitgeb and Pettigrew2010b), I do take both of these facts to be problematic, although I do not think they are necessarily fatal. They argue, perhaps rightly, that there is a stark difference between UESs and situations that call for full conditionalization. The latter involve ruling out worlds from the set of epistemic possibilities. The former involve the same set of epistemic possibilities, just with new constraints on the attitudes toward those possibilities. Thus, credence 0 in a proposition, even when there are only finitely many worlds, can represent two very different doxastic attitudes toward it. One of those attitudes rules the proposition out and precludes it from ever being assigned a positive probability in the future, whereas the other maintains its epistemic possibility but assigns it maximally low credence. Although these issues call for further discussion, I will bracket them here.
3.2. Main Problem
Regardless of what one thinks of these results, a more important problem—and a fatal one, in my view—is the potentially dramatic effect LP Conditionalization can have on the likelihood ratios between different propositions. Since LP Conditionalization adds a constant to the prior credence in a world, important evidential relationships reflected in the prior can then be violated. For illustration, we turn to the following two cases.
Case 1
There's a car behind an opaque door, which you are almost sure is blue but which you know might be red. You are almost certain of materialism, but you admit that there's some minute possibility that ghosts exist. For ease of reference, we introduce the following abbreviations for propositions: B = the car is blue; G = there are ghosts. To make the case precise, we stipulate that you have the prior probability function shown in table 3. Now the opaque door is opened, and the lighting is fairly good. You are quite surprised at your sensory input: your new credence that the car is red is .99. Table 4 gives your posterior credence under the two updating procedures.
w 1
 |
w 2
 |
w 3
 |
w 4
 |
|
|---|---|---|---|---|
| Crt | ∼.000476 | .95 | ∼.000025 | .0495 |
| w 1 | w 2 | w 3 | w 4 | |
 |
0 | .01 | ∼.470262 | ∼.519738 |
 |
∼.000005 | ∼.009995 | ∼.000495 | ∼.989505 |
Jeffrey Conditionalization leads to no change in opinion about ghosts. Under LP Conditionalization, however, seeing the car makes you about 47% sure that there are ghosts. Note that the case was originally set up so that you thought the questions of what color the car is and whether there are ghosts were independent, but somehow, merely acquiring information about car color has drastically changed your opinion about materialism. Note that had you come to know the car was red and conditionalized on 
, you would have ended up with credence .0005 that there were ghosts. So, the difference between the near certainty of credence .99 and full knowledge that 
is the difference between credence .47 and credence .0005 that G. I think it is clear that something has gone wrong with LP Conditionalization here. Becoming more confident in one proposition should not alone raise your credence in another, if you initially take them to be independent.
Case 2
Consider the following propositions: U = unemployment will rise before the next election; P = the president will be reelected. Suppose your prior and posterior credences after LP Conditionalization are given in table 5, where 
is the result of becoming .98 sure that unemployment will rise from an initial credence of .2. In this example, we have 
, while 
. However, after becoming more confident that unemployment will rise, LP Conditionalization leads to 
. So, despite the fact that you thought that it was less likely that the president would be reelected given that unemployment rises, becoming nearly sure that unemployment will rise ends up raising your credence that the president will be reelected.
w 1
 |
w 2
 |
w 3
 |
w 4
 |
|
| Crt | .05 | .15 | .38 | .42 |
 |
.44 | .54 | 0 | .02 |
The general problem in both cases is that LP Conditionalization does not respect important evidential relationships reflected in the agent's prior. Furthermore, there is nothing particularly odd or unusual about the structure of either case. Therefore, LP Conditionalizers will often end up with unreasonable posteriors despite perfectly sensible priors.
4. The Brier Score and Synchronic Accuracy
The argument against the Brier score as a measure of inaccuracy comes from its failings in recommending an updating procedure in Jeffrey UESs. Now, a number of authors endorse ADEG or an equivalent norm and use it to argue for constraints on updating procedures.Footnote The only options they have are rejecting the Brier score, accepting LP Conditionalization, or endorsing different synchronic and diachronic accuracy measures. The last option appears ad hoc at best. It amounts to claiming that the kind of accuracy I care about my future beliefs having is different from the kind of accuracy I care about my current beliefs having. Although I will not rule this possibility out, it would require a lot of further motivation. Since we have seen that LP Conditionalization is unattractive, supporters of ADEG ought to take the first option of rejecting the Brier score.
Many defenders of the Brier score, however, have been silent about any accuracy norms for updating beliefs and have considered only its synchronic features. To make the case that the considerations above should lead them to reject the Brier score, I will try to strengthen the argument for a norm like ADEG.
Let's go back to case 1. Instead of jumping into the diachronic case, this time we have three agents with the prior credence functions shown in table 6. That is, Alice's prior is Crt in the original case, Leopold's is 
, and Jeff's is 
. Suppose Alice likes the Brier score as a synchronic measure of accuracy, but she does not endorse ADEG and instead updates by Jeffrey Conditionalization. She is now in an odd situation. She expects Leopold to be more accurate than Jeff overall. However, were she to have the same constraints placed on her credence function that we find in case 1, she would adopt Jeff's current credence function as her own.
w 1
 |
w 2
 |
w 3
 |
w 4
 |
|
|---|---|---|---|---|
| Alice | ∼.000476 | .95 | ∼.000025 | .0495 |
| Leopold | 0 | .01 | ∼.470262 | ∼.519738 |
| Jeff | ∼.000005 | ∼.009995 | ∼.000495 | ∼.989505 |
Here is one way to think about this. Using Accuracy and the Brier score, Alice can create a preference ranking for all possible credence functions in terms of how well she expects them to do.Footnote She of course expects her own credence function to have a higher accuracy than any other, so it is first on her preference list. Still, she does not think all other credence functions are on a par. In particular, she thinks that Jeff's is worse than Leopold's. However, she knows now (or at least can be shown) that were nature to place constraints on her credence function that required her to have credence .99 in 
, she would adopt Jeff's credence over Leopold's. We can dramatize the case as follows: suppose an evil scientist told Alice that he was going to force her to take one of two pills. Pill 1 would change her prior to Leopold's, and pill 2 would change it to Jeff's. Her memory of this event would be wiped clean either way, and she will get no new evidence, uncertain or otherwise, that should affect her credence in either B or G. In this situation, she would opt for pill 1. If, however, instead of an evil scientist, she has a sensory experience that raises her credence in 
to .99, she will opt for Jeff's credence. Even if cases like this one are sometimes permissible, they should not be the standard ones.
Therefore, a norm like ADEG applies at least in normal circumstances since it tells us to pick what we currently take to be the best credence function that meets the constraints. Consequently, we should reject the Brier score as a measure of synchronic accuracy, so long as better alternatives are available. It turns out other less popular measures of accuracy do result in a superior updating procedure, so we should opt for one of them. In the appendix, we show that the logarithmic rule, in particular, leads to Jeffrey Conditionalization.
5. Conclusion
The argument of the article went as follows. First, we should strive to minimize our expected inaccuracy under some reasonable measure, by Accuracy. If we use a quadratic scoring rule and follow ADEG, we end up with LP Conditionalization. We should keep ADEG (at least usually), but we should not update by LP Conditionalization. Therefore, quadratic scoring rules are not reasonable inaccuracy measures.
Appendix
We here sketch a proof of the earlier claim that ADEG combined with the following inaccuracy measure leads to Jeffrey Conditionalization:
Logarithmic Rule (LR): 
.
First, it will be useful to have the following rediscription of Jeffrey Conditionalization. Suppose Alice's prior is Crt, and let 
partition the set of possible worlds W. News comes in, and Alice must update her credence function so that she now has credence qi in each element Ei of the partition. If she Jeffrey Conditionalizes, she updates her credence to 
as follows: for each element Ei, she finds some constant ci such that for all 
, 
. Since 
, exactly one constant will work. Therefore, it will suffice to show that LR together with ADEG require multiplying the prior credence in each member of a given element of the partition by the same constant.
Suppose now that Alice follows ADEG and measures inaccuracy with LR. For the sake of simplicity, we first consider the case in which 
with 
and 
. For readability, we set 
. To follow ADEG, we are now looking for the quadruple 
that minimizes
where 
, 
, and 
for all i. Since E 1 and E 2 are disjoint, minimizing (A1) under these side constraints is just minimizing both of the following equations:
To minimize f, we let
Note that 
and 
for some constants c and c*. Substituting these identities in (A2) gets us 
. So, 
. To minimize g, we follow the same procedure mutatis mutandis. We then have that 
for constants d and d′. In other words, each element of Ei gets multiplied by the same constant ci, as desired. Thus, in this special case in which each 
for all i, we end up with Jeffrey Conditionalization.
For the more general case, suppose 
. We then wish to minimize 
under the side constraints. To do so, pick two distinct βi's to treat as variable, and treat the rest as constant. Without loss of generality, we choose β1 and β2 and set 
for some constant K. Now, we just need to minimize: 
. Running essentially the same argument as above, we find that 
and 
for some constant c. Since the choice of K and the two βi's was arbitrary, we again end up with Jeffrey Conditionalization as desired.
for a world w.
.
is a single-membered element of the partition, we will just look at what happens to the elements of 
. We are given the constraint that 
. LP Conditionalization tells us to meet this constraint by finding some constant 
such that 
. Now, since probabilities cannot be negative, to meet the constraints we need 
, so we have 
. Since 
, we know that 
and 
.
