1. Introduction

Jeffrey’s (Reference Jeffrey1992) epistemological program known as radical probabilism tries to reconcile two goals: it aims at being both mathematically precise and flexible regarding representations of opinions and opinion change. Jeffrey conditioning (also known as probability kinematics) is a prime example (Jeffrey Reference Jeffrey1957, Reference Jeffrey1965, Reference Jeffrey and Lakatos1968). Being less rigid than standard Bayesian conditioning, it nonetheless captures a common type of uncertain observational learning within the framework of probability theory.

Learning from experience is not, however, restricted to Bayesian and Jeffrey conditioning. In the most extreme case the learning event is a black box: we only know an agent’s opinions before and after the learning experience. Despite the lack of structure, there is a family of principles—known as reflection and martingale principles—that are plausible candidates for regulating how to respond to new information in generalized learning situations. The principles say, roughly speaking, that current opinions should cohere with anticipated future opinions.

How can such principles be justified? There are three paradigms (Huttegger Reference Huttegger2017). One is based on diachronic Dutch book arguments (Goldstein Reference Goldstein1983; van Fraassen Reference van Fraassen1984; Skyrms Reference Skyrms1990), the second one proceeds in terms of expected accuracy (Easwaran Reference Easwaran2013; Huttegger Reference Huttegger2013), and the third approach appeals to ideas related to the value of knowledge theorem in decision theory (Huttegger Reference Huttegger2014). The three paradigms are well understood in finite probability spaces. The same is true for the expected accuracy approach and infinite probability spaces. However, a corresponding treatment is missing for the other two approaches. The aim of our article is to fill this gap.

We are going to follow the most common approach to infinite probability spaces, Kolmogorov’s (Reference Kolmogorov1933, Reference Kolmogorov1956) measure-theoretic probability theory. The central concept of Kolmogorov’s theory is the conditional expectation of a random variable given a σ-algebra, which gives rise to a natural formulation of generalized learning. There is an ongoing discussion of the limits of the concept of conditional expectation and the measure-theoretic approach. Kolmogorov’s assumption of countable additivity has not found universal acceptance (de Finetti Reference de Finetti1974; Seidenfeld Reference Seidenfeld and Hendricks2001). Moreover, conditional expectation has some strange consequences when it is taken with respect to certain σ-algebras (Seidenfeld, Schervish, and Kadane Reference Seidenfeld, Schervish and Kadane2001). For the purposes of this article, we take it for granted that conditional expectation is a worthwhile object of study, even if it may not be universally valid.

The article is structured as follows. After introducing Kolmogorov conditional expectation in section 2, we introduce reflection and martingale principles in section 3. Most of the aricle investigates the value-of-knowledge paradigm (sec. 4). In section 5 we show that the diachronic Dutch book argument for Kolmogorov conditional probability presented in Rescorla (Reference Rescorla2018) gives rise to reflection and martingale principles. Finally, in section 6 we conclude by discussing the impact of these arguments on the status of those principles.

2. Conditional Probability and Conditional Expectation

Let (Ω, ℬ, ℙ) be a probability space: Ω is a set of atomic events, ℬ is a σ-algebra of measurable subsets of Ω called “events,” and ℙ is a countably additive probability measure.Footnote ¹ We can think of (Ω, ℬ, ℙ) as an experimental setup, where the elements of Ω represent the most fine-grained experimental outcomes, members of ℬ capture experimental events, and ℙ represents a scientist’s degrees of belief.

Let 𝔊 be a sub-σ-algebra of ℬ. Intuitively, 𝔊 represents the outcomes of an experiment that gives partial information about the space of atomic events. Furthermore, let $X:\Omega \to ℝ$ be a random variable on (Ω, ℬ). Random variables represent quantities that have a determinate value in the experimental setup (i.e., X is ℬ-measurable).

According to Kolmogorov (Reference Kolmogorov1933), the conditional expectation of X given 𝔊, $𝔼\left[X\right|𝔊]$, is itself a random variable. More precisely, $𝔼\left[X\right|𝔊]$ is a 𝔊-measurable real-valued function such that

${\int }_{G}Xd\mathbb{P}={\int }_G𝔼[X\mid 𝔊]d\mathbb{P}$

for all $G\in 𝔊$. The conditional expectation $𝔼\left[X\right|𝔊]$ is almost surely unique and can be thought of as the best estimate of X given the experimental outcomes in 𝔊.Footnote ² For instance, if 𝔊 is the trivial algebra {∅, Ω}, then 𝔊 provides no information at all, and the best estimate of X given 𝔊 is the expectation of X, $𝔼\left[X\right|𝔊]=𝔼[X]$, a.s. (almost surely). Yet, $𝔼\left[X\right|𝔅]=X$ a.s., since X is ℬ-measurable and thus coincides with its best estimate given ℬ.

It is well known that Kolmogorov’s concept of conditional expectation basically coincides with the traditional concept of conditional expectation when Ω is finite. Moreover, conditional probability can be defined in terms of conditional expectation. Let B be an event in ℬ, and let χ_B be the indicator of B. Then the conditional probability of B given 𝔊, $\mathbb{P}\left[B\right|𝔊]$, is equal to $𝔼\left[{\chi }_B\right|𝔊]$ a.s. While conditional probability, thus understood, is a random variable, it effectively coincides with the elementary concept of conditional probability when Ω is finite.

3. Principles for Updating Opinions

Conditioning is the best known Bayesian principle of updating. It is usually stated in terms of partitions: the new probability of an event A is given by the conditional probability of A given the element of the partition that was observed to be true. Conditional probabilities given a σ-field generalize that idea to epistemic situations that cannot be captured by partitions. Conditional expectations given a σ-field generalize Bayesian conditioning to random variables.

In both manifestations, Bayesian conditioning assumes that one learns something with certainty.Footnote ³ Learning with certainty is a limiting case of uncertain learning, as Jeffrey has argued in several places (Jeffrey Reference Jeffrey1957, Reference Jeffrey1965, Reference Jeffrey and Lakatos1968). This led him to develop a broader Bayesian epistemology, radical probabilism, which subsumes Jeffrey conditioning but also includes other forms of updating (Jeffrey Reference Jeffrey1992).

Several authors have argued that there is a family of principles, usually referred to as reflection or martingale principles, that govern how an agent should change opinions in response to new information in all forms of updating that fall under the umbrella of radical probabilism (Goldstein Reference Goldstein1983; van Fraassen Reference van Fraassen1984; Skyrms Reference Skyrms1990; Jeffrey Reference Jeffrey1992). Our first order of business is to formulate them within the conceptual framework sketched in the preceding section.Footnote ⁴

As before, (Ω, ℬ, ℙ) is a probability space. Let $P:\Omega \times 𝔅\to [0,1]$ be a function with the following properties:

1. (i) P(ω, ⋅) is a probability measure on (Ω, ℬ) for each $\omega \in \Omega$;

2. (ii) P(⋅, B) is ℬ-measurable for each $B\in 𝔅$.

We will refer to P as an update policy. An update policy represents an agent’s future probabilities after learning. Let 𝔉 be the σ-algebra generated by the family $\left\{P\right(\cdot ,B):B\in 𝔅\}$. Note that 𝔉 is a sub-σ-algebra of ℬ, since by (ii) P(⋅, B) is ℬ-measurable. Intuitively, 𝔉 represents information about future probabilities after learning. It is the “future algebra.”

The principle of reflection says that present probabilities, conditional on the future, should be equal to future probabilities, or, for all $B\in 𝔅$:

(1) $\begin{array}{}& \mathbb{P}(B\mid 𝔉)=P(\cdot ,B)\text{a.s.}\end{array}$

In other words, for all $B\in 𝔅$, P(⋅, B) is a version of the conditional probability $\mathbb{P}(B\mid 𝔉)$. Notice that reflection implies that $\mathbb{P}(\cdot \mid 𝔉)$ has a regular version. The latter is true, for example, if Ω is a Polish space and ℬ is its Borel σ-algebra. In general, however, Ω’s being Polish is not necessary for the existence of a regular version of $\mathbb{P}(\cdot \mid 𝔉)$. Conversely, if a space has no regular conditional probability given 𝔉, then the principle of reflection does not hold.

Suppose the reflection principle holds. By integrating both sides of (1), we get

(2) $\begin{array}{}& \mathbb{P}\left(B\right)=\int P\left(B\right)d\mathbb{P}\text{.}\end{array}$

We call (2) the weak reflection principle, since it is implied by the reflection principle.

The martingale principle generalizes the reflection principle to ℙ-integrable random variables. In the remainder of the article, we denote the space of ℙ-integrable random variables by ℒ¹. The martingale principle says that, for all $X\in {ℒ}^1$¹ for which $\int X\mathrm{dP}\in {ℒ}^1$:

(3) $\begin{array}{}& 𝔼\left[X\right|𝔉]=\int X\mathrm{dP}\text{a.s.}\end{array}$

If $X={\chi }_B$ is an indicator function, then $\int X\mathrm{dP}=P\left(B\right)$ is bounded and therefore integrable, so the martingale principle implies the reflection principle. In general, it is not the case for an arbitrary random variable in $X\in {ℒ}^1$ that $\int X\mathrm{dP}\in {ℒ}^1$ as well. We will soon show, however, that if the weak reflection principle holds, then $\int X\mathrm{dP}$ is indeed in ℒ¹.

We interpret the martingale principle as saying that best estimates after learning are (almost surely) equal to conditional expectations given the future algebra. This interpretation requires viewing $\int X\mathrm{dP}$ as the best estimate for X after learning. This view seems quite intuitive to us. Since coherent best estimates for random variables are always given by expected values, the view amounts to assuming that learning maintains coherence. To spell this out in more detail, it seems very plausible that an agent’s best estimate for X should be the same as her fair price for a gamble with X. It is the price she would pay (accept) to have the opportunity to collect (pay out) whatever value the random variable X manifests in the actual world. If the agent’s best estimates did not match her fair prices, then she would be making a mistake by her own lights. She would regard her own betting behavior as falling short of the standard set by her best estimates. With best estimates and fair prices thus identified, suppose now that the agent expects to remain coherent after learning. (It is not clear in what sense a change from coherent beliefs to incoherent ones could count as learning, but we can set this issue aside.) By the standard Dutch book argument, coherence demands that fair prices be determined by expected values (e.g., Jeffrey Reference Jeffrey2004, chap. 4). So, since best estimates match fair prices, the agent’s best estimate for X after learning is given by the expected value of X relative to her update policy P. This argument suggests that, if an agent expects to remain coherent after learning, then her best estimate for X after learning should be given by the P-expected value of X. If, moreover, the martingale principle holds, then best estimates after learning are given by conditional expectations.Footnote ⁵

The martingale principle implies

(4) $\begin{array}{}& \int Xd\mathbb{P}=\int \int X\mathrm{dP}d\mathbb{P}\end{array}$

for all $X\in {ℒ}^1$ for which $\int X\mathrm{dP}\in {ℒ}^1$. This says that random variables and their best estimates after learning have the same expected value. We refer to (4) as the weak martingale principle.

Our first result verifies that $\int X\mathrm{dP}$ is integrable in the presence of weak reflection and also shows that the (weak) martingale principle is actually equivalent to the (weak) reflection principle.

Proof. First note that by considering $X={\chi }_B$, it is clear that the (weak) martingale principle implies the (weak) reflection principle, so we only need to verify the converse implications to settle the proposition’s second claim.

Now, suppose that the weak reflection principle holds. Establishing the proposition’s first claim and deriving the weak martingale principle is a straightforward application of what Williams (Reference Williams1991) calls the “standard machine”: we first show that the desired result holds if X is an indicator function, then if X is a simple function, then if X is nonnegative, and finally if X is any integrable random variable. If $X={\chi }_B$ is an indicator function, then clearly $\int X\mathrm{dP}=P\left(B\right)$ is 𝔉-measurable, and (4) follows immediately from (2). If $X={\sum }_{i=1}^n{b}_i{\chi }_{B_i}$ is a simple function, then $\int X\mathrm{dP}$ is a linear combination of 𝔉-measurable functions, and so it is itself 𝔉-measurable; moreover, $\int X\mathrm{dP}$ is clearly bounded and therefore integrable, and

$\int Xd\mathbb{P}=\sum _{i=1}^n{b}_i\mathbb{P}\left(B_i\right)=\sum _{i=1}^n{b}_i\int P\left(B_i\right)d\mathbb{P}=\int \int X\mathrm{dP}d\mathbb{P}\text{,}$

by the linearity of the integral and the weak reflection principle. If X is nonnegative, then it can be approximated from below by an increasing sequence of simple functions Z ₁, Z ₂, … . Then, by the monotone convergence theorem, $\int X\mathrm{dP}$ is the monotone limit of a sequence of 𝔉-measurable functions, and so it is itself 𝔉-measurable, and

$\int Xd\mathbb{P}=\underset{n\to \infty }{\lim }\int Z_{n}d\mathbb{P}=\underset{n\to \infty }{\lim }\int \int Z_n\mathrm{dP}d\mathbb{P}=\int \int X\mathrm{dP}d\mathbb{P}\text{,}$

by the monotone convergence theorem and our previous step. This last equation also reveals that $\int X\mathrm{dP}$ is integrable, because X is. Finally, for arbitrary $X\in {ℒ}^1$, write $X=X^{+}-X^{-}$.Footnote ⁶ Since, X ⁺ and X ⁻ are nonnegative and integrable, by our last step we have that both $\int X^{+}\mathrm{dP}$ and $\int X^{-}\mathrm{dP}$ are integrable. It follows that $\int X\mathrm{dP}=\int X^{+}\mathrm{dP}-\int X^{-}\mathrm{dP}$, being the difference of two 𝔉-measurable integrable functions, is itself 𝔉-measurable and integrable. Moreover,

$\begin{array}{cc}\int Xd\mathbb{P}& =\int X^{+}d\mathbb{P}-\int X^{-}d\mathbb{P}=\int \int X^{+}\mathrm{dP}d\mathbb{P}-\int \int X^{-}\mathrm{dP}d\mathbb{P}\\ & =\int (\int X^{+}\mathrm{dP}-\int X^{-}\mathrm{dP})d\mathbb{P}=\int \int X\mathrm{dP}d\mathbb{P}\end{array}\text{.}$

We have established the proposition’s first claim and shown that the weak reflection principle implies the weak martingale principle. The argument showing that the reflection principle implies the martingale principle is analogous, and we omit it. QED

4. Value of Information

In a seminal paper, Good (Reference Good1967) showed that in certain finite decision problems a Bayesian agent will prefer making a decision after conditioning on new information to making a decision right away. This result, known as the value of knowledge theorem, was anticipated by Savage (Reference Savage1954) and Ramsey (Reference Ramsey1990). Good used it as an argument for the principle of total evidence, which says that in forming opinions one should use all available evidence. Skyrms (Reference Skyrms1990) generalized Good’s theorem to Polish spaces. His argument extends to all probability spaces that admit a regular conditional probability.

The value of knowledge theorem operates under a number of stringent assumptions. One is that there are no costs involved in obtaining new evidence. In addition, the agent is assumed to conform to the principles of Savage’s decision theory before and after the learning event, and the learning event has no effect on the utilities associated with decision problems. There are a number of additional assumptions that we will not mention here (see Skyrms Reference Skyrms1990, for details).

We are not so much interested in defending the plausibility of the assumptions underlying the value of knowledge theorem but in using the value of knowledge idea to study Bayesian learning from experience. For this we tacitly assume that the assumptions hold. Huttegger (Reference Huttegger2014) showed that the value of information can be used to derive the principle of reflection when Ω is finite. Huttegger’s central requirement is that a belief change leads one to expect to make better decisions in a sufficiently large class of decision problems if the belief change is a genuine learning event, in which the agent responds to legitimate new information.

Let’s articulate this idea in the context of a probability space (Ω, ℬ, ℙ) and an update policy P. An act is an integrable real-valued random variable $A:\Omega \to ℝ$. The value A(ω) represents the utility of state ω if act A is chosen. Utilities are assumed to be cardinal utilities. The expected utility of A is thus given by the expectation of A:

$\int A\left(\omega \right)\mathbb{P}\left(d\omega \right)\text{.}$

A decision problem Α is a finite set of acts.

Before a learning event, the expected value of choosing an act from a decision problem ${\rm A}=\left\{A_i\right\}$ is

$\underset{i}{\max }\int A_{i}d\mathbb{P}\mathrm{.}$

We refer to this as the value of the prior Bayes act, since it is the utility a Bayesian agent expects to get in case she makes a choice before new information is available.

After the learning event, the agent adopts the probabilities given by her update policy P. Provided that $\int A_i\mathrm{dP}$ is integrable, the prior expected value of the posterior Bayes act is given by the following integral:

$\int \underset{i}{\max }\int A_i\left(\omega \right)P({\omega }^{\prime },d\omega )\mathbb{P}\left(d{\omega }^{\prime }\right)\text{.}$

We now suppose that the update policy P is in response to a genuine learning event. Then the value of information postulate says that for all decision problems ${\rm A}=\left\{A_i\right\}$, $\int A_i\mathrm{dP}$ is integrable, and

(5) $\begin{array}{}& \int \underset{i}{\max }\int A_i\mathrm{dP}d\mathbb{P}\ge \underset{i}{\max }\int A_{i}d\mathbb{P}\text{.}\end{array}$

This says that the agent does not expect to make worse decisions after acquiring new information.Footnote ⁷

The next result shows that the value of information postulate (5) is equivalent to the weak reflection principle (2).

Proof. By proposition 1, it suffices to establish the equivalence with the weak reflection principle. If the value of information postulate holds, that is,

$\int \underset{i}{\max }\int A_i\mathrm{dP}d\mathbb{P}\ge \underset{i}{\max }\int A_{i}d\mathbb{P}$

for all decision problems ${\rm A}=\left\{A_i\right\}$, then it holds in particular for ${\rm A}=\left\{{\chi }_B\right\}$ with $B\in 𝔅$. Hence,

(6) $\begin{array}{}& \int P\left(B\right)d\mathbb{P}\ge \mathbb{P}\left(B\right)\end{array}$

holds for all $B\in 𝔅$. If the inequality were strict for some B, then putting B^c in (6) would yield a contradiction, so (6) holds with equality for all B, which is to say that the weak reflection principle (2) holds.

Now suppose that the weak reflection principle holds. By proposition 1, the weak martingale principle holds as well, and $\int A_i\mathrm{dP}\in {ℒ}^1$ for all decision problems ${\rm A}=\left\{A_i\right\}$. Let A _max denote a maximizer of $\int A_{i}dP$. Then,

$\underset{i}{\max }\int A_i\mathrm{dP}\ge \int A_{\max }\mathrm{dP}\mathrm{.}$

Integrating both sides of this, we obtain

$\int \underset{i}{\max }\int A_i\mathrm{dP}d\mathbb{P}\ge \int \int A_{\max }dPd\mathbb{P}=\int A_{\max }d\mathbb{P}=\underset{i}{\max }\int A_{i}d\mathbb{P},$

by the weak martingale principle. QED

We now discuss two ways of improving proposition 2. We provide two conditions under which the value of information postulate is equivalent to the reflection principle.

Let us say that P is proper just in case $P(\omega ,F)=1_F\left(\omega \right)$ for all ω in a ℙ-probability 1 set and all $F\in 𝔉$. Elementary conditional probabilities are trivially proper in the sense that $\mathbb{P}(E\mid E)=1$ whenever $\mathbb{P}\left(E\right)>0$. In general, however, regular conditional probabilities are not proper (see Blackwell and Dubins Reference Blackwell and Dubins1975; Seidenfeld et al. Reference Seidenfeld, Schervish and Kadane2001). Properness corresponds to the intuition that conditioning on 𝔉 represents observing which members of 𝔉 are true and which are false. The condition that P is proper is thus one way to generalize the “luminosity” condition that Huttegger (Reference Huttegger2014) used for finite probability spaces, which requires that the agent perfectly recalls how learning affected her probabilities.

We regard the next proposition as one of the two main results of this article. We return to a discussion of its significance in the final section.

Proof. By proposition 1, it suffices to establish the equivalence with the reflection principle. Suppose that P is proper and that the value of information postulate holds. By proposition 2, this implies that the weak reflection principle (2) holds. Moreover, almost surely, for all $F\in 𝔉$, if $\omega \in F$, $P(\omega ,F)=1$, which implies $P(\omega ,B\cap F)=P(\omega ,B)$ for all $B\in 𝔅$. That is, ${\chi }_{F}P(B\cap F)={\chi }_{F}P\left(B\right)$ a.s. Similarly, if $\omega \notin F$, then $P(\omega ,F)=0$, which implies $P(\omega ,B\cap F)=0$. So, ${\chi }_{F^c}P(B\cap F)=0$ a.s. Then, by the weak reflection principle, for all $B\in 𝔅$ and all $F\in 𝔉$,

$\mathbb{P}(B\cap F)=\int P(B\cap F)d\mathbb{P}={\int }_{F}P(B\cap F)d\mathbb{P}+{\int }_{F^c}P(B\cap F)d\mathbb{P}={\int }_{F}P\left(B\right)d\mathbb{P}\text{.}$

This shows that for all $B\in 𝔅$, P(⋅, B) is a version of $\mathbb{P}(B\mid 𝔉)$; that is, the reflection principle holds.

Conversely, whether or not P is proper, the reflection principle implies the weak reflection principle, which in turn implies the value of information postulate, by proposition 2. QED

The connection between the principle of reflection and the value of information postulate is further elucidated by the following property. We say that P is idempotent if for ℙ almost every ω and all $B\in 𝔅$

(7) $\begin{array}{}& P(\omega ,B)=\int P({\omega }^{\prime },B)P(\omega ,d{\omega }^{\prime })\text{.}\end{array}$

The meaning of idempotence becomes clearer if the integral is interpreted as representing the probability of an event. Then the right-hand side of (7) is the probability of first updating states in Ω and then updating the probability of B. Idempotence requires that the resulting probability is the same as updating the probability of B immediately. Thus, iterating P does not change the result of updating beyond the initial application.Footnote ⁸

Regular conditional probabilities are idempotent. So, if the reflection principle holds, then so do weak reflection and idempotence. We establish the converse with the next proposition.

Proof. We establish that the martingale principle holds. In the same way that one can extend the (weak) reflection principle to the (weak) martingale principle, as in the proof of proposition 1, so we can extend idempotence to

(8) $\begin{array}{}& \int X\left({\omega }^{\prime }\right)P(\omega ,d{\omega }^{\prime })=\int \int X\left({\omega }^{″}\right)P({\omega }^{\prime },d{\omega }^{″})P(\omega ,d{\omega }^{\prime })\end{array}$

for all $X\in {ℒ}^1$.

Now, we will show that there exists a sub-σ-algebra 𝔊 of ℬ such that, for all $X\in {ℒ}^1$, $\int X\mathrm{dP}$ is a version of $𝔼\left[X\right|𝔊]$. Let L ¹ be the quotient space of ℙ-integrable random variables that are equal with ℙ-probability 1. By corollary 1 in Douglas (Reference Douglas1965), as presented by Gyenis and Redei (Reference Gyenis and Redei2017, proposition 2.7), it suffices to show that the mapping $𝕐:L^1\to L^1,X\mapsto \int X\mathrm{dP}$ is a linear projection, contractive in the L ¹ norm, such that $𝕐\left({\chi }_{\Omega }\right)={\chi }_{\Omega }$. All of these properties are basically immediate from the definitions of the terms involved.

By proposition 1, $𝕐\left(X\right)\in L^1$ for all $X\in L^1$. The linearity of the integral implies that 𝕐 is linear. It is clear that $𝕐\left({\chi }_{\Omega }\right)={\chi }_{\Omega }$. To say that 𝕐 is a projection means that $𝕐\left(𝕐\right(X\left)\right)=𝕐\left(X\right)$, and this follows from the definition of 𝕐 and (8). To say that 𝕐 is contractive in the L ¹ norm means that $𝔼\left|𝕐\right(X\left)\right|\le 𝔼\left|X\right|$, and this follows from the fact that $\left|𝕐\right(X\left)\right|\le 𝕐\left(\right|X\left|\right)$ and (4), which holds by proposition 1.

Thus,

(9) $\begin{array}{}& 𝔼\left[X\right|𝔊]=𝕐(X)\end{array}$

for some sub-σ-algebra 𝔊 and all $X\in L^1$. Thus, for all $X\in {ℒ}^1$,

(10) $\begin{array}{}& 𝔼\left[X\right|𝔊]=\int XdP\text{a.s.}\end{array}$

We conclude by showing that (10) continues to hold with $𝔊=𝔉$. Since $\int X\mathrm{dP}$ is measurable in both 𝔊 and 𝔉, it follows from (10) that $\int X\mathrm{dP}$ is a version of $𝔼\left[X\right|𝔊\cap 𝔉]$:

$𝔼[X\mid 𝔊\cap 𝔉]=𝔼\left[𝔼\right[X\mid 𝔊]\mid 𝔊\cap 𝔉]=𝔼[\int X\mathrm{dP}\mid 𝔊\cap 𝔉]=\int X\mathrm{dP}\text{a.s.}$

By the minimality of 𝔉, $𝔉\subseteq 𝔊\cap 𝔉$. Hence, $𝔉=𝔊\cap 𝔉$, and we are done. QED

The next proposition is the second main result of the article.

Let $C_{\omega }=\{{\omega }^{\prime }:P({\omega }^{\prime },B)=P(\omega ,B)\text{ for all }B\in 𝔅\}$, where C _ω is the event that updated probabilities are given by P(ω, ⋅). Then, idempotence is connected to the following luminosity condition:

(11) $\begin{array}{}& P(\omega ,C_{\omega })=1\text{for a.s.}\omega \text{.}\end{array}$

This says that the update policy does not assign positive probability to the event that it updates in a different way than it had planned.Footnote ⁹ The update policy, in other words, does not forget any information that was gained from learning.

In order for (11) to be meaningful, C _ω must be measurable. To guarantee measurability, we assume that ℬ is countably generated. In that case, 𝔉 is countably generated, and { $C_{\omega }:\omega \in \Omega$} is a family of 𝔉 atoms in 𝔉.Footnote ¹⁰ To see why, let 𝔄 be a countable algebra generating ℬ. Then $𝔉=\sigma \left(\right\{P(\cdot ,B):B\in 𝔄\left\}\right)$ by a monotone class argument.Footnote ¹¹ Thus, 𝔉 is countably generated: for each $B\in 𝔄$, σ(P(⋅, B)) is countably generated (call the generating set C _B) because P(⋅, B) is a real-valued random variable, and 𝔉 is generated by ${\cup }_{B\in 𝔄}{𝒞}_B$, which is a countable union of countable sets. Moreover, it follows that $C_{\omega }={\cap }_{B\in 𝔄}\{{\omega }^{\prime }:P(\omega ,B)=P({\omega }^{\prime },B)\}$ because 𝔄 is an algebra generating ℬ, and, since this intersection is countable, $C_{\omega }\in 𝔉$. The collection of all C _ω may or may not be countable. In fact, one can see that there are countably many C _ω just in case the range of P—that is, the collection of probability measures $\left\{P\right(\omega ,\cdot ):\omega \in \Omega \}$—is countable.

The following results relate luminosity to the conditions studied above. The first follows from a theorem of Blackwell (Reference Blackwell1942) and was stated independently by Gaifman (Reference Gaifman, Skyrms and Harper1988).

Proof. Suppose the luminosity condition holds. Then, almost surely, $P(\omega ,C_{\omega })=1$, and thus

$\int P({\omega }^{\prime },B)P(\omega ,d{\omega }^{\prime })=\int P({\omega }^{\prime },B)1_{C_{\omega }}\left({\omega }^{\prime }\right)P(\omega ,d{\omega }^{\prime })\text{.}$

Now, for all ${\omega }^{\prime }\in C_{\omega }$, $P(\omega ,\cdot )=P({\omega }^{\prime },\cdot )$, and so the integrand on the right side is equal to P(ω, B). The conclusion follows.

The second statement follows from Blackwell (Reference Blackwell1942, theorem 7), which implies that there is a set N such that $P(\omega ,N)=0$ for (almost) all ω and such that (11) holds for (almost) all $\omega \notin N$. Weak reflection implies that $\mathbb{P}\left(N\right)=0$. QED

By propositions 5 and 6 we have the following corollary.

It remains to clarify the relationship between the luminosity condition and proper update policies.

5. Diachronic Dutch Books

We now turn to reflection and martingale principles in the context of dynamic coherence. Rescorla (Reference Rescorla2018) has shown that P is a conditional probability given a sub-σ-algebra 𝔊 if and only if P is not vulnerable to a diachronic Dutch book when events in 𝔊 are observed. The σ-algebra 𝔊 captures a learning situation in which the agent updates on the outcome of an experiment or an observational setup. This is only superficially different from the learning situations considered in this article. Rescorla’s argument extends straightforwardly to our setting.

For the reflection principle, replace Rescorla’s observational σ-algebra 𝔊 with our future algebra 𝔉. His argument shows that $\mathbb{P}[\cdot |𝔉]$ is the almost surely unique updating strategy that avoids a diachronic Dutch book. More precisely, Rescorla’s main results imply that there is no diachronic Dutch book for the update policy P if and only if P(⋅, B) is a version of $\mathbb{P}\left[B\right|𝔉]$ for every $B\in 𝔅$, that is, if and only if the reflection principle (1) holds and, by proposition 1 if and only if the martingale principle holds.

The Dutch book characterization of reflection, then, simply generalizes the well-known results of Goldstein (Reference Goldstein1983) and van Fraassen (Reference van Fraassen1984) to measure-theoretic conditional probabilities and expectations: in a black box learning situation, diachronic coherence demands conditioning on future probabilities. This point is also discussed in Skyrms (Reference Skyrms1997, 287).

6. Concluding Remarks

The two main results of this article can be summarized as follows:

1. i) In the presence of properness, the value of knowledge postulate is equivalent to the principle of reflection and the martingale principle (proposition 3).

2. ii) The value of knowledge postulate and idempotence are equivalent to the principle of reflection and the martingale principle (proposition 5).

One way to read these results is as justification of the reflection and martingale principles. The value of knowledge postulate is a rather compelling requirement for all genuine learning situations: in the absence of factors that interfere adversely with the acquisition of new information, one should not expect to make worse decisions after adjusting one’s probabilities. Properness is a desirable property for update policies. An improper update policy foresees that it will lose some information that is part of the learning event. While information loss is certainly something that can happen after a learning experience, it does not seem to be entirely appropriate for update policies. If the update policy foresees an information loss, then this should already be factored into the learning event in such a way that the future algebra only captures the actual information gain.

Still, luminosity conditions such as properness have sometimes been criticized as unreasonable assumptions about our introspective access to belief change (Williamson Reference Williamson2002; Weisberg Reference Weisberg2007). Idempotence establishes an alternative route to the principle of reflection. It simply requires that updating be independent of whether information is presented in one or more steps. Unlike properness, idempotence holds for all regular conditional probabilities; it thus constitutes a plausible constraint for all update policies.

Our results can also be read as putting in context so-called counterexamples to the principle of reflection. Any violation of reflection entails a violation of the value of knowledge postulate or a violation of properness and idempotence. This identifies the conceptual space within which we can find counterexamples. They draw on situations in which an agent does not expect to make better decisions based on new beliefs, in which some information that was learned is lost, or in which belief change depends on the way in which information is presented.

Finally, many criticisms of the principle of reflection go hand in hand with criticisms of dynamic Dutch book arguments. Because of our results and Huttegger (Reference Huttegger2017), critics of reflection face a more formidable task, though. Skepticism regarding the principle of reflection does not just entail skepticism regarding diachronic coherence but also accuracy and value of information arguments. Such skepticism might prove difficult to sustain in the face of the general treatment of those arguments given here and, for expected accuracy, in Huttegger (Reference Huttegger2013).