1 Introduction
A statistical model consists of a sample space and a collection of probability distributions on the sample space, one of which is hypothesized to be the data-generating process. The distributions are often indexed by a parameter space, which is an assumption that we will make throughout this article. In a setting where the number of sample observations grows without bound, statisticians consider the question of whether estimates are consistent, where, loosely speaking, an estimate is consistent if it is guaranteed to identify the true parameter.
Consistency is of major importance in classical statistics. For instance, Barnett [Reference Barnett1, p. 135] remarks that “[c]onsistency is generally regarded as an essential property of a reasonable estimator.” As Diaconis and Freedman [Reference Diaconis and Freedman4] point out, it also plays an important role in Bayesian statistics and thus constitutes a point of contact between classical and Bayesian statistics.Footnote 1
Suppose we have a prior probability measure on the parameter space. Doob [Reference Doob and Fréchet5] showed that, under mild conditions, the posteriors are consistent for almost all parameters—that is, with increasing sample size, the posterior concentrates around the true parameter whenever it is in a set of prior probability one. Further, consistency can often be achieved for all parameters, in the finite-dimensional parametric case [Reference Freedman7, Reference Schwartz27].
Things are much more complicated in the nonparametric case. Freedman demonstrated that inconsistency can be topologically generic in this setting [Reference Freedman8]. The literature following Freedman has generalized this result, establishing the pervasiveness of inconsistency for nonparametric models [Reference Diaconis and Freedman4].
In the present article, we investigate statistical consistency from the point of view of computable probability theory and algorithmic randomness (see [Reference Gács9, Reference Hoyrup and Rojas13–Reference Huttegger, Walsh and Blando15, Reference Rute23]). Before stating our main results, we begin by recalling elements of the classical statistical setup in more detail.Footnote 2
1.1 Parameter space and prior
We suppose that the parameter space
$\Omega $
is a Polish space and that the prior p is a probability measure on
$\Omega $
.
1.2 Sample space
We suppose that the sample space
$\mathcal {X}$
is a closed subset of Baire space
$\mathbb {N}^{\mathbb {N}}$
, and hence is representable as the set of paths
$[T]$
through a tree
$T\subseteq \mathbb {N}^{<\mathbb {N}}$
. As Diaconis and Freedman say: “all the paradoxes of inconsistency already appear” in the case of this kind of sample space”.Footnote
3
We use
$x_{n}$
for the finite sequence of natural numbers which results from restricting the element x of the sample space to its first n digits.Footnote
4
Each string of natural numbers
$\sigma $
in T of length n determines a basic clopen subset
$[\sigma ]=\{x\in \mathcal {X}: x_{n}= \sigma \}$
of the sample space
$\mathcal {X}$
.
1.3 Likelihood
We suppose that the likelihood
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is a measurable map from the parameter space
$\Omega $
to the Polish space
$\mathcal {M}^+(\mathcal {X})$
of finite Borel measures on the sample space
$\mathcal {X}$
, and that, for p-a.s. many
$\theta $
, one has that
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is an element of the Polish space
$\mathcal {P}(\mathcal {X})$
of Borel probability measures on
$\mathcal {X}$
. Since any element
$\nu $
of
$\mathcal {M}^+(\mathcal {X})$
is determined by its action on the clopens, we abbreviate
$\nu ([\sigma ])$
as
$\nu (\sigma )$
.
1.4 Product space and joint distribution
The prior and the likelihood induce a probability measure
$\mu $
, called the joint distribution, on the product space
$S=\Omega \times \mathcal {X}$
given by the formula
$\mu (A\times B) = \int _A p(B \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )$
on rectangles. For a
$\mu $
-integrable function
$f:S\rightarrow \mathbb {R}$
, one has the following formula for the integral:
Finally, the joint distribution
$\mu $
induces a pushforward
$\mu _{X}$
on the sample space
$\mathcal {X}$
. Its pushforward on the parameter space
$\Omega $
is just the prior p.
1.5 Posterior
We define the posterior as a map
$\sigma \mapsto p(\cdot \mid \sigma )$
from the tree T to the Polish space
$\mathcal {M}^+(\Omega )$
of finite Borel measures on the parameter space
$\Omega $
by
when the denominator is non-zero, and we assume that the posterior is zero otherwise. For each
$n\geq 0$
, the posterior induces a map
$x\mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
from the sample space
$\mathcal {X}$
to
$\mathcal {M}^+(\Omega )$
. For
$\mu _{X}$
-a.s. many x from
$\mathcal {X}$
, one can show that
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is in
$\mathcal {P}(\Omega )$
, the space of probability measures on the parameter space (cf. Proposition 3.6).
1.6 Relation to discrete forms of Bayes’ theorem
In the case where the parameter space
$\Omega $
is countable and
$\sigma =x_{n}$
, if we momentarily abuse notation by identifying
$\theta $
with its singleton event
$A=\{\theta \}$
, then (1.2) reduces to the familiar discrete form
$ p(\theta \hspace {-1mm}\mid \hspace {-1mm} x_{n}) = \frac {p(x_{n} \mid \theta ) \cdot p(\theta )}{\sum _{\theta \in \Omega } p(x_{n} \mid \theta ) \cdot p(\theta )}$
of Bayes’ Theorem.
1.7 Diagram of the spaces
The situation is then as depicted in Figure 1, where the
$\sigma $
-algebras on all five Polish spaces are the corresponding Borel
$\sigma $
-algebras and
$\mu _{X}$
again refers to the pushforward of
$\mu $
onto the sample space. We name the two projections
$\Theta :S\rightarrow \Omega $
and
$X:S\rightarrow \mathcal {X}$
. This diagram can help keep track of what is meant by the expression
$p(\cdot \mid \cdot )$
. Note that, in our notation,
$p(\cdot \mid \cdot )$
does not mean the ordinary conditional probability with respect to the prior.
The five spaces and maps between them.

Our goal is to study this setup within the framework of computable probability theory and effective analysis. We make the following assumptions.
Definition 1.1 (Effective assumptions).
-
(1) The parameter space $\Omega $
is a computable Polish space, and the prior p is a computable probability measure on
$\Omega $
. -
(2) The sample space $\mathcal {X}$
is the set of paths
$[T]$
through a computable tree
$T\subseteq \mathbb {N}^{<\mathbb {N}}$
which has no dead ends. -
(3) The likelihood $\theta \mapsto p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is
$L_1(p)$
computable, uniformly in
$\sigma $
from T; and
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is in
$\mathcal {P}(\mathcal {X})$
for p-a.s. many
$\theta $
in
$\Omega $
.
For computable Polish spaces and computable probability measures on them, see again the sources [Reference Gács9, Reference Hoyrup and Rojas13–Reference Huttegger, Walsh and Blando15, Reference Rute23]. Readers unfamiliar with computable Polish spaces can still read much of this article, simply by specializing to paradigmatic cases, such as when the parameter space is Cantor space, Baire space,
$[0,1]^n$
, or
$\mathbb {R}^n$
.
Together, (1)–(3) suffice to ensure that the joint distribution
$\mu $
is a computable probability measure on the product space S (cf. Proposition 3.1). In (2), having no dead ends provides a natural countable dense set—namely, the leftmost path above each
$\sigma $
in T—and so ensures that the sample space
$\mathcal {X}=[T]$
is a computable Polish space. This is also natural on the statistical interpretation, since finite samples can at least in principle always be continued.
In (3), if r is a computable real
$\geq 1$
, then, by
$L_r(p)$
computable, we just mean a computable point of the computable Polish space
$L_r(p)$
of Borel measurable functions whose absolute value has r-th power which is integrable (the functions in
$L_r(p)$
being identified when they are p-a.s equal). Its metric is given by the
$L_r(p)$
norm
$\|f\|_{L_r(p)}$
—namely, the r-th root of
$\int _{\Omega } \left |f\right |^r\; dp$
—and its countable dense set is a certain set of rational-valued step functions with events whose p-measure can be computed.Footnote
5
Condition (3) of Definition 1.1 is admittedly a strong condition. For instance, it implies that the
$L_1(p)$
norm
$\|p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )\|_{L_1(p)}$
of the likelihood
$\theta \mapsto p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is computable for any
$\sigma $
, since it is the distance
$\|p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )-0\|_{L_1(p)}$
, and since the distance between any two computable points in a computable Polish space is automatically computable. Further, condition (3) has deeper consequences, such as Proposition 3.5 below, which gives a sufficient condition for parameters
$\theta $
to be such that
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is in
$\mathcal {P}(\mathcal {X})$
.
We now turn to recalling the components of the definition of Schnorr randomness. An
$L_r(p)$
Schnorr test is a function
$g:\Omega \rightarrow [0,\infty ]$
whose
$L_r(p)$
norm
$\|g\|_{L_r(p)}$
is computable and which is lower semi-computable (abbreviated lsc): that is,
$g^{-1}(q,\infty ]$
is c.e. open uniformly in rational
$q\geq 0$
. Since they are lsc functions, the
$L_r(p)$
Schnorr tests are everywhere defined, and one can show that their p-a.s. equivalence class is an
$L_r(p)$
computable point.
Suppose that Y is a computable Polish space and that
$\nu $
is a computable probability measure on Y. Then one defines:Footnote
6
Definition 1.2. A point y of Y is Schnorr
$\nu $
-random, abbreviated
$\mathsf {SR}^{\nu }$
, if for all computable sequences of c.e. opens
$U_n$
with
$\nu (U_n)\leq 2^{-n}$
uniformly computable, one has that y is not in
$\bigcap _n U_n$
.
The sequences
$U_n$
are called sequential Schnorr
$\nu $
-tests. An equivalent characterization of Schnorr randomness, due to Miyabe [Reference Miyabe19, Theorem 3.5], is: y in
$\mathsf {SR}^{\nu }$
iff
$f(y)<\infty $
for all
$L_1(\nu )$
Schnorr tests f.
We need other characterizations of
$\mathsf {SR}^{\nu }$
.
Definition 1.3. A class B is
$\nu $
-effectively
$\Sigma ^0_2$
, abbreviated
$\Sigma ^{0,\nu }_2$
, if it has computable
$\nu $
-measure and it can be written as
$B=\bigcup _i C_i$
where
$C_i$
is a computable sequence of effectively closed sets
$C_i$
which have uniformly computable measure
$\nu (C_i)$
.
We have the following characterizations of
$\mathsf {SR}^{\nu }$
in terms of
$\Sigma ^{0,\nu }_2$
sets.
Proposition 1.4. A point y is in
$\mathsf {SR}^{\nu }$
iff for all computable sequences of
$\Sigma ^{0,\nu }_2$
sets
$B_n$
with
$\nu (B_n)\leq 2^{-n}$
, one has that y is not in
$\bigcap _n B_n$
.
Proposition 1.5. A point y is in
$\mathsf {SR}^{\nu }$
iff y is in all
$\Sigma ^{0,\nu }_2$
classes of
$\nu $
-measure one.
For spaces like Cantor space, Proposition 1.5 was noted by Schnorr in his book [Reference Schnorr26, pp. 68–69], while Proposition 1.4 is new.
Since, in Definition 1.1(3), the likelihood is viewed as an element of the space
$L_1(p)$
, we need to specify how to understand its pointwise behavior.
Definition 1.6 (Convention on versions: Differences of
$L_1(p)$
Schnorr tests).
If f is an
$L_1(p)$
computable function, then we restrict attention to versions of it of the form
$g-h$
, where
$g,h$
are two
$L_1(p)$
Schnorr tests.
By the Miyabe characterization of Schnorr randomness in terms of
$L_1(p)$
Schnorr tests, these versions are always finite on the Schnorr randoms. By further results of Miyabe, all
$L_1(p)$
computable functions have such versions, and any two such versions agree on the Schnorr randoms.Footnote
7
This convention has the virtue of allowing us to use many natural versions. But this convention needs to be paired with some convention on how to treat
$g(\theta )-h(\theta )$
when both values are infinite. So as to avoid having to define notation on partial functions, we treat it as conventionally equal to zero.
The paradigmatic example of a parameter space in nonparametric statistics is as follows.
Definition 1.7 (Infinite-dimensional simplex).
The infinite-dimensional simplex is
$\mathbb {S}_{\infty }= \{\theta \in [0,1]^{\mathbb {N}}: \sum _i \theta (i)=1\}$
.
This is especially natural when the sample space is Baire space
$\mathbb {N}^{\mathbb {N}}$
and when the likelihood is the countable product of
$\theta $
, as a probability measure on the natural numbers. This is what happens in countable versions of stick-breaking and the Dirichlet process in nonparametric statistics (cf. [Reference Ghosal and van der Vaart10, Section 3.3] and [Reference Sethuraman28]).
We then define a classical consistency notion and its effectivization.
Definition 1.8 (Consistency notions, classical and effective).
-
(1) The parameter $\theta $
is classically consistent if
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is in
$\mathcal {P}(\mathcal {X})$
and, for all open U in
$\Omega $
, one has that
$\lim _n p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})=I_U(\theta )$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
-a.s. many x in
$\mathcal {X}$
. -
(2) The parameter $\theta $
is computably consistent if
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is in
$\mathcal {P}(\mathcal {X})$
and, for all c.e. open U in
$\Omega $
with
$p(U)$
computable, one has that
$\lim _n p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})=I_U(\theta )$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
-a.s. many x in
$\mathcal {X}$
.
Since this definition concerns the pointwise behavior of the likelihood, it presupposes a version which accords with Definition 1.6. The classical notion in (1) is close to that used in the traditional statistical literature, although this literature usually restricts it to open U containing the parameter
$\theta $
, as this can be equivalently expressed in terms of weak convergence of measures [Reference Ghosal and van der Vaart10, pp. 123–124].
Our first main theorem is the following effectivization of Doob’s Consistency Theorem (see [Reference Doob and Fréchet5], [Reference Ghosal and van der Vaart10, Theorem 6.9], [Reference Schervish25, Theorem 7.78]).
Theorem 1.9 (Effective Doob Consistency Theorem and Schnorr randomness).
Suppose there is a sequence of uniformly computable continuous functions
${f_n:\mathcal {X}\rightarrow \Omega }$
such that both of the following happen:
-
(1) For all $(\theta ,x)$
in a
$\mu $
-measure one subset of S, we have that
$\lim _n f_n(x)$
exists and is equal to
$\theta $
. -
(2) There is a p-computable basis on $\Omega $
such that, for all
$n\geq 0$
and all elements U of the p-computable basis, the event
$f_n^{-1}(U)$
is uniformly both c.e. open and effectively closed.
Then all pairs
$(\theta , x)$
of parameters and samples in
$\mathsf {SR}^{\mu }$
are such that, for all c.e. open U in
$\Omega $
with
$p(U)$
computable, one has that
$\lim _n p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})=I_U(\theta )$
. Further, all parameters
$\theta $
in
$\mathsf {SR}^{p}$
are computably consistent.
In this, recall that a function between two computable Polish spaces is computable continuous if the inverse image of c.e. opens is uniformly c.e. open. For the notion of a p-computable basis in (2), see Definition 2.1. For the moment, we just note some simple examples. If
$\Omega $
is Cantor space or Baire space, then the basic clopens are a p-computable basis for any computable prior p. If
$\Omega $
is
$\mathbb {R}^n$
or
$[0,1]^n$
then the open boxes with rational-valued corners are a p-computable basis for any atomless computable prior p.
The paradigmatic example of conditions (1) and (2) of Theorem 1.9 is as follows.
Proposition 1.10 (Paradigmatic example: Limiting relative frequencies).
Let the parameter space
$\Omega $
be the unit interval
$[0,1]$
, and let the sample space
$\mathcal {X}$
be Cantor space
$2^{\mathbb {N}}$
. Let the prior p be any computable probability measure on
$\Omega $
. Let C be an effectively closed subset of
$\Omega $
of p-measure one. Let the likelihood
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
, for
$\theta $
in C, be the measure on Cantor space which is the countable product of the measure on
$\{0,1\}$
with
$1$
being assigned probability
$\theta $
. For
$\theta $
not in C, let it be the zero element of
$\mathcal {M}^+(\mathcal {X})$
. Let
$f_n:\mathcal {X}\rightarrow \Omega $
by
$f_n(x)=\frac {1}{n}\sum _{i<n} x(i)$
, so that
$\lim _n f_n$
is the limiting relative frequency of
$1$
’s, if the limit exists. Then conditions (1) and (2) of Theorem 1.9 are satisfied.
We can also effectivize at least some cases of consistency via identifiability.
Proposition 1.11 (Effective identifiability).
Suppose that
$\Omega $
is computably homeomorphic to a computable Polish subspace of
$\mathbb {S}_{\infty }$
via the map
$h:\Omega \rightarrow \mathbb {S}_{\infty }$
.
Suppose that the sample space is Baire space
$\mathbb {N}^{\mathbb {N}}$
.
Suppose that the likelihood
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
from
$\Omega $
to
$\mathcal {P}(\mathcal {X})$
is given by taking
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
to be the countable product of
$h(\theta )$
.
Then all
$\theta $
in
$\mathsf {SR}^{p}$
are computably consistent.
See Section 7 for the definition of computable Polish subspace. Like with the classical proofs of consistency via identifiability (cf. [Reference Ghosal and van der Vaart10, Proposition 6.10]), our effectivization ultimately goes back through limiting relative frequencies.
While the proof of Theorem 1.9 uses lots of facts about Schnorr randomness, it is apriori not clear whether a weaker randomness notion suffices. For instance, there are many randomness notions properly larger than Schnorr randomness: how does one know that Theorem 1.9 does not hold of them?Footnote
8
However, we can show that Theorem 1.9 does not hold of them, at least when we allow the sample space, the likelihood, and the functions
$f_n:\mathcal {X}\rightarrow \Omega $
to vary. To this end, we first define:
Definition 1.12. Suppose that p is a computable probability measure on a computable Polish space
$\Omega $
.
A triple of sample space
$\mathcal {X}$
, likelihood
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
, and uniformly computable continuous functions
$f_n:\mathcal {X}\rightarrow \Omega $
is called an apt triple if both of the following conditions hold:
-
(a) The sample space $\mathcal {X}$
and likelihood
$\theta \mapsto p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
satisfy conditions (2) and (3) of Definition 1.1. -
(b) The uniformly computable continuous functions $f_n:\mathcal {X}\rightarrow \Omega $
satisfy conditions (1) and (2) of Theorem 1.9.
Then we prove the following theorem.
Theorem 1.13. Suppose that p is a computable probability measure on a computable Polish space
$\Omega $
. Suppose that there is at least one apt triple with sample space
$\mathcal {X}$
, likelihood
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
, and uniformly computable continuous functions
$f_n:\mathcal {X}\rightarrow \Omega $
. Then the following are equivalent for a parameter
$\theta _0$
in
$\Omega $
:
-
(1) The parameter $\theta _0$
is in
$\mathsf {SR}^{p}$
. -
(2) For all apt triples with sample space $\mathcal {Y}$
and likelihood
$\theta \mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
and uniformly computable continuous functions
$g_n:\mathcal {Y}\rightarrow \Omega $
, one has that the parameter
$\theta _0$
is computably consistent relative to prior p, sample space
$\mathcal {Y}$
, and likelihood
$\theta \mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
.
This theorem applies in particular to the paradigmatic example of a computable prior on the unit interval (cf. Proposition 1.10). Hence this theorem provides many examples of parameters that are not computably consistent. However, as the proof in Section 6 makes clear, it does so by making
$q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})$
the zero element of
$\mathcal {M}^+(\mathcal {Y})$
, and it is not known to us whether there are other ways of causing the inconsistency of a non-random
$\theta _0$
. Further, as suggested by the statement of (2), and as borne out by its proof in Section 6, Theorem 1.13 produces inconsistency by greatly varying the sample space and likelihood.
By contrast, Freedman produced examples of classical inconsistency by fixing a familiar parameter space and sample space and likelihood and by considering different priors on it. We effectivize Freedman’s Inconsistency Theorem [Reference Ghosal and van der Vaart10, Theorem 6.12] as follows.
Theorem 1.14 (Effective Freedman Inconsistency).
Let the parameter space
$\Omega $
be the infinite-dimensional simplex
$\mathbb {S}_{\infty }$
. Let the sample space
$\mathcal {X}$
be Baire space
$\mathbb {N}^{\mathbb {N}}$
. Let the likelihood
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
from
$\Omega $
to Baire space be given by taking
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
to be the countable product of
$\theta $
, as a probability measure on the natural numbers.
Then the set of points
$( p,\theta )$
in
$\mathcal {P}(\Omega )\times \Omega $
which are not classically consistent is effectively comeager.
In this, effectively comeager means: it contains the countable intersection of a computable sequence of c.e. open sets, each of which is dense.Footnote 9
By a combination of Theorem 1.9 and Proposition 1.10, we can further prove the following corollary.
Corollary 1.15. Let
$\Omega $
and
$\mathcal {X}$
and
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
be as in Theorem 1.14. In any non-empty c.e. open of
$\mathcal {P}(\Omega )\times \Omega $
we can find a computable point
$(p, \theta _0)$
such that:
-
(1) The parameter $\theta _0$
is not computably consistent. -
(2) All parameters $\theta $
in
$\mathsf {SR}^{p}$
are computably consistent.
This corollary provides a computability-theoretic solution to Diaconis and Freedman’s problem of “know[ing] for which [parameters]
$\theta $
the rule [Bayes’ rule] is consistent” [Reference Diaconis and Freedman4, p. 4]. On the one hand, (2) says that a canonical kind of randomness satisfies an effective form of consistency. On the other hand, (1) says that the paradigmatic non-random parameters, namely the computable parameters, do not necessarily have this property. Hence a general solution to Diaconis and Freedman’s problem, which is not given by classical analytic methods, is given by effective methods. It would be well worth sharpening (1) to understand the extent to which other non-randoms do not suffice for consistency, but we leave this to future work.
The topic of this article is closely related to that of Takahashi [Reference Takahashi31]. He was concerned with Martin-Löf randomness in the case where the parameter space and the sample space are both Cantor space.Footnote
10
By contrast, we have focused on Schnorr randomness in the case where the parameter space is an arbitrary computable Polish space and the sample space is a closed subset of Baire space which carries the natural structure of a computable Polish space. Further, whereas we build the joint distribution from the prior and likelihood, Takahashi starts from the assumption of a computable probability measure
$\mu $
on
$2^{\mathbb {N}}\times 2^{\mathbb {N}}$
, defines the prior as the pushforward on the parameter space, and defines the likelihood
${p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )=\lim _n \mu (\Omega \times [\sigma ] \mid [\theta _n]\times \mathcal {X})}$
. This equation is true on the Schnorr randoms in our setting (cf. Proposition 3.10), and so our results apply to Takahashi’s setting. Since the Schnorr randoms are a superset of the Martin-Löf randoms, our Theorem 1.9 thus improves Takahashi’s results, in that we have identified more parameters at which consistency is achieved, albeit at the cost of more constructive constraints on the map from the sample space to the parameter space.Footnote
11
This article is independent of our prior paper [Reference Huttegger, Walsh and Blando15] on Lévy’s Upward Theorem. The basic reason is that the map
$f:\mathcal {X}\rightarrow \Omega $
, given by
$f=\lim _n f_n$
where
$f_n:\mathcal {X}\rightarrow \Omega $
satisfies conditions (1) and (2) of Theorem 1.9, can itself be quite complicated (cf. again the example in Proposition 1.10), and thus complicates any direct application of the results in [Reference Huttegger, Walsh and Blando15]. In more detail: the classical proof of Doob’s Consistency Theorem goes through applying Lévy’s Upward Theorem to the indicator function
$I_{f^{-1}(U)}$
in
$\mathcal {X}$
with respect to the measure
$\mu _{X}$
and the filtration generated by length n-strings, where U is open in
$\Omega $
(cf. the proof of Theorem 1.9 at close of Section 4). Since f is not necessarily computable continuous, the function
$I_{f^{-1}(U)}$
is not necessarily lsc when U is c.e. open. Hence any application of an effective version of Lévy’s Theorem will go through some version of
$I_{f^{-1}(U)}$
, which may behave differently than
$I_{f^{-1}(U)}$
itself does on the relevant random points.Footnote
12
Because of this impediment, we proceed by developing an alternative approach through
$\Sigma ^{0,p}_2$
classes and Propositions 1.4 and 1.5, which in the proof of Theorem 1.9 we apply to the lsc functions
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
(cf. Proposition 3.7, and the proof of Theorem 1.9 in Section 4). The virtue of this approach is that, when U is c.e. open, the posteriors
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
, as quotients of integrals, are less complicated to define pointwise than the function
$I_{f^{-1}(U)}$
is.
This article is organized as follows. In Section 2 we prove Propositions 1.4 and 1.5 and related facts about Schnorr randomness. In Section 3 we verify all the effective properties of the classical statistical setup. In Section 4 we prove Theorem 1.9. In Section 5 we prove Proposition 1.10. In Section 6 we prove Theorem 1.13. In Section 7 we take note of the computable Polish space structure on the infinite-dimensional simplex and prove Proposition 1.11. In Section 8 we prove some elementary results in preparation for Effective Freedman Inconsistency. Finally, in Section 9 we prove Theorem 1.14 and Corollary 1.15.
2
$\Sigma ^0_2$
and Schnorr randomness
In this section, we suppose that Y is a computable Polish space and that
$\nu $
is a computable probability measure on Y. The latter means that
$\nu (U)$
is uniformly left-c.e. as U ranges over c.e. opens U. Equivalently, it means that
$\nu $
is a computable point of the computable Polish space
$\mathcal {P}(Y)$
.
We presuppose familiarity with algorithmic randomness as developed on arbitrary computable Polish spaces.Footnote 13 The only definition from the general theory which bears mentioning, since it can vary slightly between authors, is the following.Footnote 14
Definition 2.1. A
$\nu $
-computable basis is a computable sequence of c.e. opens
$U_0, U_1, \ldots $
such that any c.e. open can be written as an effective union of them and such that
$\nu (U_i)$
is uniformly computable and they are paired with a computable sequence of effectively closed sets
$C_0, C_1, \ldots $
such that
$C_i\supseteq U_i$
and
$\nu (C_i)=\nu (U_i)$
.
The notion of
$\nu $
-computable basis interacts well with another fundamental randomness notion.
Definition 2.2. A point y of Y is Kurtz
$\nu $
-random, abbreviated
$\mathsf {KR}^{\nu }$
, if y is in all c.e. open U with
$\nu (U)=1$
.
Note that if
$U_i, C_i$
are as in the definition of
$\nu $
-computable basis, then
$U_i, C_i$
necessarily agree on
$\mathsf {KR}^{\nu }$
.
By a result of Hoyrup and Rójas which goes through the Effective Baire Category Theorem, such a basis always exists, and indeed can be taken to be among the open balls with centers from the countable dense set with the closed balls being the associated effectively closed sets [Reference Hoyrup and Rojas13, Lemma 5.1.1].
The following proposition collects together some elementary facts about
$\Sigma ^{0,\nu }_2$
classes (cf. Definition 1.3).
Proposition 2.3 (General results on
$\Sigma ^{0,\nu }_2$
classes).
-
(1) If C is effectively closed with $\nu (C)$
computable, then C is
$\Sigma ^{0,\nu }_2$
. -
(2) If U is c.e. open with $\nu (U)$
computable, then U is
$\Sigma ^{0,\nu }_2$
. -
(3) The $\Sigma ^{0,\nu }_2$
classes are closed under finite intersection; they are also closed under finite union and effective union provided the union has computable
$\nu $
-measure. -
(4) If V is a c.e. open set with $\nu (V)$
computable, then for any rational
$\epsilon>0$
we can compute an index for an effectively closed
$D\subseteq V$
such that
$\nu (D)$
is computable and
$\nu (V\setminus D)<\epsilon $
. -
(5) If C is an effectively closed set with $\nu (C)$
computable, then for any rational
$\epsilon>0$
we can compute an index for a c.e. open
$U\supseteq C$
such that
$\nu (U)$
is computable and
$\nu (U\setminus C)<\epsilon $
.
Proof. For (1), this by definition since one can just take
$C_n=C$
for all
$n\geq 0$
.
For (2), let us identify the countable dense set of Y with natural numbers and let us use variables
$i,j$
for this countable dense set, and let d be the metric with respect to which Y is a computable Polish space. Let
$r_n>0$
be a computable sequence of reals such that
$B(i,r_n)$
is a
$\nu $
-computable basis with
$B[i,r_n]$
the associated effectively closed ball with the same
$\nu $
-measure.Footnote
15
It suffices to consider the case of U an open ball
$B(j,\epsilon )$
where
$\epsilon>0$
is rational. In particular it suffices to show that we can compute an index for a c.e. set
$I\subseteq \mathbb {N}\times \mathbb {N}$
such that
$B(j,\epsilon )=\bigcup _{(i,n)\in I} B[i,r_n]$
. We define
$I=\{(i,n)\in \mathbb {N}\times \mathbb {N}: d(i,j)+r_n<\epsilon \}$
. One can then check that one has the identity
$B(j,\epsilon )=\bigcup _{(i,n)\in I} B[i,r_n]$
.
In the remainder of the proofs, we often appeal to the fact that if c.e. opens
$U_0, U_1, \ldots $
have uniformly computable
$\nu $
-measure, then any event in the algebra generated by them also has uniformly computable
$\nu $
-measure(cf. [Reference Huttegger, Walsh and Blando15, Proposition 2.8(2)]).
For (3), it suffices to show it for a single intersection, and in this case just consider the intersection
$C_n\cap C_n^{\prime }$
where
$C_n$
witnesses that B is
$\Sigma ^{0,\nu }_2$
and
$C_n^{\prime }$
witnesses that
$B^{\prime }$
is
$\Sigma ^{0,\nu }_2$
.
For (4), since V is
$\Sigma ^{0,\nu }_2$
, let
$V=\bigcup _n D_n$
, where
$D_n$
is effectively closed with
$\nu (D_n)$
uniformly computable. Without loss of generality
$D_n$
is increasing. Hence, using computability of
$\nu (V)$
, we can compute n such that
$\nu (V\setminus D_n)<\epsilon $
.
For (5) simply apply (4) to the relative complement.
The following is a technical but important strengthening of (5).
Proposition 2.4 (Effectively covering a
$\Sigma ^{0,\nu }_2$
class).
If B is
$\Sigma ^{0,\nu }_2$
, then for any rational
$\epsilon>0$
we can compute an index for a c.e. open
$U\supseteq B$
such that
$\nu (U)$
is computable and
$\nu (U\setminus B)<\epsilon $
.
Proof. Let
$B=\bigcup _n C_n$
where
$C_n$
is uniformly effectively closed and
$\nu (C_n)$
is computable; and where in addition
$\nu (B)$
is computable. Without loss of generality
$C_n$
is increasing. Define uniformly computable reals
$0\leq \eta _n\leq 1$
by
$\eta _0=\nu (C_0)$
and
$\eta _n=\nu (C_n\setminus C_{n-1})$
for
$n>0$
. Since
$\sum _n \eta _n = \nu (B)$
, we have
$\lim _n \eta _n=0$
. Further
$\nu (B\setminus C_m)=\sum _{n>m} \eta _n$
. Let
$\delta _n$
be a uniformly computable sequence of rationals with
$0\leq \delta _n-\eta _n<2^{-(n+1)}$
. Since
$\delta _n<2^{-(n+1)}+\eta _n$
, also
$\lim _n \delta _n=0$
. Further:
By Proposition 2.3(5), uniformly compute c.e. open
$U_n\supseteq C_n$
with
$\nu (U_n)$
computable and
$\nu (U_n\setminus C_n)<\frac {\epsilon }{2}\cdot \delta _n$
. Let
$U=\bigcup _n U_n$
. This gives the required estimate
It remains to show that
$\nu (U)$
is computable. Since
$\nu (B)$
is computable and
$\nu (U)=\nu (B)+\nu (U\setminus B)$
, it suffices to show that
$\nu (U\setminus B)$
is computable. For
$n>0$
:
For
$m>0$
, note that one has the inclusions
where the last inclusion follows from
$C_{n-1}\subseteq U_{n-1}$
. Hence for all
$m> 0$
one has that
$\nu (U\setminus C_m)-\nu ((\bigcup _{n< m} U_n)\setminus C_m)$
is
where in the first line we apply (2.3) and where in the last line we apply (2.2) and then (2.1). Then for all
$m> 0$
we have that
$\left |\nu (U\setminus B)-\nu ((\bigcup _{n<m} U_n)\setminus C_m)\right |$
is
Since the quantity in this last line is uniformly computable and goes to zero as m goes to infinity, we can compute a subsequence which witnesses that
$\nu (U\setminus B)$
is a computable real. We appeal here to the fact that
$(\bigcup _{n<m} U_n)\setminus C_m$
has computable
$\nu $
-measure, since it is in the algebra generated by the uniformly computable sequence
$U_n, Y\setminus C_n$
of c.e. opens with
$\nu $
-computable measure (cf. [Reference Huttegger, Walsh and Blando15, Proposition 2.8(2)]).
From this, Proposition 1.4 easily follows. Proposition 1.5 follows by going back and forth in the natural way between sequential Schnorr
$\nu $
-tests
$U_n$
and the associated
$\Sigma ^{0,\nu }_2$
-class
$\bigcup _n (Y\setminus U_n)$
of
$\nu $
-measure one.
We can then derive the following characterization in terms of lsc functions.
Proposition 2.5. A point y is in
$\mathsf {SR}^{\nu }$
iff for every computable sequence of lsc functions
$f_n:Y\rightarrow [0,\infty ]$
, with
$\sup _{n\geq m} f_n$
an
$L_1(\nu )$
Schnorr test uniformly in
$m\geq 0$
, and for every effectively closed C with
$\nu (C)$
computable and
$f_n\rightarrow 0\, \nu $
-a.s. on C, one has that y is in C implies
$\lim _n f_n(y)=0$
.
Proof. For the forward direction, suppose that
$f_n$
and C are as specified; we want to show that
$\lim _n f_n=0$
on
$\mathsf {SR}^{\nu }\cap C$
. Suppose that
$\epsilon>0$
is rational. For each
$m\geq 0$
, uniformly compute a real
$\epsilon _{m}$
in the interval
$(\frac {\epsilon }{2}, \epsilon )$
such that the c.e. open
$\{y: \sup _{n\geq m} f_n(y)>\epsilon _{m}\}$
has uniformly computable
$\nu $
-measure.Footnote
16
Then its relative complement
$D_{m}: = \{y: \sup _{n\geq m} f_n(y)\leq \epsilon _{m}\}$
is effectively closed and has uniformly computable
$\nu $
-measure as well. Finally, using a
$\nu $
-computable basis, let
$Y\setminus C=\bigcup _m U_m$
, where
$U_m$
is a computable sequence of c.e. opens from the
$\nu $
-computable basis, and let
$C_m$
be effectively closed superset of
$U_m$
with the same
$\nu $
-measure. Consider the class
$V_{\epsilon } = (\bigcup _{m} (D_{m}\cap C)) \cup (\bigcup _m C_m)$
, which is an effective union of effectively closed classes each with uniformly computable measure. Since by hypothesis
$f_n\rightarrow 0\, \nu $
-a.s. on C, there are
$\nu (C)$
-many y in C such that there is
$m\geq 0$
such that for all
$n\geq m$
one has
$f_n(y)<\frac {\epsilon }{2}$
, and so
$\sup _{n\geq m} f_n(y)\leq \frac {\epsilon }{2}<\epsilon _{m}$
and so this y would be in
$D_{m}$
. Hence
$\nu (V_{\epsilon })=1$
, so that
$V_{\epsilon }$
is a
$\Sigma ^{0,\nu }_2$
class of
$\nu $
-measure one. Let y in
$\mathsf {SR}^{\nu }\cap C$
. Then by Proposition 1.5, y is in
$V_{\epsilon }$
. But since y is in
$\mathsf {KR}^{\nu }\cap C$
, we cannot have that y is in
$\bigcup _m C_m$
. (For suppose that y is
$C_m$
for some
$m\geq 0$
. Since
$C_m$
is an effectively closed superset of c.e. open
$U_m$
with the same
$\nu $
-measure, it follows that
$C_m\setminus U_m$
is effectively closed and
$\nu (C_m\setminus U_m)=0$
. Then since y is in
$\mathsf {KR}^{\nu }$
, we have that y is not in
$C_m\setminus U_m$
. Hence y is in
$U_m$
as well. But
$U_m$
is by definition a subset of
$Y \setminus C$
). Thus rather we must have that y is in
$D_{m}\cap C$
for some
$m\geq 0$
, and so
$\sup _{n\geq m} f_n(y)\leq \epsilon _{m}<\epsilon $
.
The backward direction is easy since if
$U_n$
is a decreasing sequential Schnorr
$\nu $
-test and we set
$f_n=I_{U_n}$
, then
$\sup _{n\geq m} f_n =I_{\bigcup _{n\geq m} U_n}$
is an
$L_1(\nu )$
Schnorr test.
Now we turn briefly to
$L_1(\nu )$
computability (cf. [Reference Huttegger, Walsh and Blando15, Section 2.3]).
Proposition 2.6 (
$L_1(\nu )$
computability and indicator functions).
-
(1) If U is c.e. open with $\nu (U)$
computable, then
$I_U$
is
$L_1(\nu )$
computable. -
(2) If C is effectively closed with $\nu (C)$
computable, then
$I_C$
is
$L_1(\nu )$
computable. -
(3) If B is $\Sigma ^{0,\nu }_2$
then
$I_B$
is
$L_1(\nu )$
computable.
Proof. For (1), suppose that
$U=\bigcup _n V_n$
, where
$V_n$
comes from a
$\nu $
-computable basis. Then
$\|I_U-I_{\bigcup _{m<n} V_m}\|_1 =\nu (U)-\nu (\bigcup _{m<n} V_m)$
is computable and goes to zero. Then we can compute sequence
$n_{\ell }$
such that
$\|I_U-I_{\bigcup _{m<n_{\ell }} V_m}\|_1\leq 2^{-\ell }$
. Hence
$I_U$
is
$L_1(\nu )$
computable since
$I_{\bigcup _{m<n_{\ell }} V_m}$
comes from the countable dense set. The proof of (2) follows by taking complements; the proof of (3) is similar to that of (1).
One also has the following, whose proof is very simple.
Proposition 2.7 (
$L_1(\nu )$
-computability and products of bounded functions).
If
$f,g$
are
$L_1(\nu )$
computable and
$\left |f\right |, \left |g\right |\leq c$
then
$fg$
is
$L_1(\nu )$
-computable.
The following result is one of the few pointwise results which we need.
Proposition 2.8. Suppose
$g,h$
are two
$L_1(\nu )$
Schnorr tests. Suppose
$f=g-h$
and suppose
$f\geq 0\, \nu $
-a.s.
Suppose that
$a_n<b_n$
is a computable sequence of non-negative reals. Then:
-
(1) There is a computable sequence of computable reals $r_n$
in
$(a_n, b_n)$
and a computable sequence of
$\kern1.2pt\Sigma ^{0,\nu }_2$
classes
$B_n$
such that
$B_n=f^{-1}(r_n,\infty ]$
on
$\mathsf {SR}^{\nu }$
. -
(2) One has $g\geq h$
on
$\mathsf {SR}^{\nu }$
. -
(3) There is a computable sequence of computable reals $r_n$
in
$(a_n, b_n)$
and a computable sequence of
$\kern1.2pt\Sigma ^{0,\nu }_2$
classes
$B_n$
such that
$B_n=f^{-1}[0,r_n)$
on
$\mathsf {SR}^{\nu }$
.
Moreover, the sequences
$r_n$
in (1) and (3) can be chosen to be the same.
Finally, the result is uniform, in that if
$g_i,h_i$
is a computable sequence of uniformly computable
$L_1(\nu )$
Schnorr tests and we set
$f_i=g_i-h_i$
and suppose
$f_i\geq 0\, \nu $
-a.s., then we can find one and the same sequence
$r_n$
which works for each i.
Proof. For (1), the three
$\nu $
-a.s. non-negative
$L_1(\nu )$
computable functions
$f,g,h$
are such that their pushforwards
$f\#\nu , g\#\nu , h\#\nu $
are computable probability measures on
$\mathbb {R}^{\geq 0}$
. Using the Effective Baire Category Theorem, we can compute a uniformly computable sequence of reals
$r_i$
, dense in
$\mathbb {R}^{>0}$
, such that the events
$f^{-1}(r_i,\infty ]$
and
$g^{-1}(r_i,\infty ]$
and
$h^{-1}(r_i,\infty ]$
have uniformly computable
$\nu $
-measure.Footnote
17
By doing this again for the non-negative
$L_1(\nu )$
computable functions
$f+r_i,g+r_i,h+r_i$
, we can compute a uniformly computable sequence of reals
$s_k$
, dense in
$\mathbb {R}^{>0}$
, such that the events
$(f+r_i)^{-1}(s_k,\infty ]$
and
$(g+r_i)^{-1}(s_k,\infty ]$
and
$(h+r_i)^{-1}(s_k,\infty ]$
have computable
$\nu $
-measure, uniformly in
$i,k\geq 0$
. Define c.e. index set
$I=\{(j,k): s_k<r_j\}$
. For
$i\geq 0$
and
$(j,k)$
in I, define
$\Sigma ^{0,\nu }_2$
class
$B_{i,j,k}=g^{-1}(r_j,\infty ]\cap (h+r_i)^{-1}[0,s_k]$
, and then define
$B_i:=\bigcup _{(j,k)\in I} B_{i,j,k}$
. Then on
$\mathsf {SR}^{\nu }$
we have the identity
$f^{-1}(r_i,\infty ]=B_i$
. Since
$f^{-1}(r_i,\infty ]$
has
$\nu $
-computable measure, so does
$B_i$
, and hence it too is a
$\Sigma ^{0,\nu }_2$
class by Proposition 2.3(3). Then we can finish by using the density of the
$r_i$
sequence on
$\mathbb {R}^{>0}$
to compute a subsequence
$a_n<r_{i_n}<b_n$
.
For (2), fix
$s>0$
to be rational. Let
$b_n=s$
and let
$a_n$
converge up to s. Let
$g_n=g+s, h_n=h+s, f_n=f+s$
. Apply part (1) to
$g_n, h_n, f_n$
and
$(a_n, b_n)$
to get a computable sequence
$r_n$
and
$B_n$
. Then
$f\geq 0\, \nu $
-a.s. implies
$f_n\geq s\, \nu $
-a.s. which implies that
$f_n^{-1}(r_n, \infty ]$
has
$\nu $
-measure one, which implies that
$B_n$
has
$\nu $
-measure one. Then each
$B_n$
has
$\nu $
-measure one, and so
$\mathsf {SR}^{\nu }$
is a subset of each
$B_n$
by Proposition 1.5. Then
$f_n\geq r_n\geq a_n$
on
$\mathsf {SR}^{\nu }$
for all
$n\geq 0$
, and so
$f+s\geq s$
on on
$\mathsf {SR}^{\nu }$
and hence
$f\geq 0$
on
$\mathsf {SR}^{\nu }$
.
For (3), the argument is just like (1) but with the inequalities reversed.
3 Computable Polish space structure on the product
We begin by showing the following proposition.
Proposition 3.1 (The joint distribution is a computable probability measure).
$\mu $
is a computable probability measure on S.
Proof. We must show that for c.e. open
$W\subseteq S$
we have that
$\mu (W)$
is left-c.e. We can write
$W=\bigcup _i (U_i\times [\sigma _i])$
, where
$U_i\subseteq \Omega $
is a computable sequence from a p-computable basis with effectively closed
$C_i\supseteq U_i$
with the same p-measure, and
$\sigma _i$
is a computable sequence from T. It suffices to show that for all n, we have that
$W_n=\bigcup _{i<n} (U_i\times [\sigma _i])$
has uniformly left-c.e.
$\mu $
-measure. For this, in turn, it suffices to show the same of
$W_{n,i}:=\big (U_i\times [\sigma _i]\big )\setminus \bigcup _{j<i} \big ( U_j\times [\sigma _j] \big )$
for each
$i<n$
.
Fix
$i<n$
and for
$j<i$
, define computable set
$I_{i,j}\subseteq T$
so that
$[\sigma _i]\setminus [\sigma _j]=\bigcup _{\eta \in I_{i,j}} [\eta ]$
. Since
$I_{i,j}$
is computable let
$I_{i,j,s}$
be a computable non-decreasing sequence such that
$I_{i,j}=\bigcup _s I_{i,j,s}$
and
$J_{i,j,s}:=I_{i,j,s+1}\setminus I_{i,j,s}$
has at most one element.
For each
$\tau $
in
$2^{<\mathbb {N}}$
with length
$i>0$
and each
$\varsigma $
in
$\mathbb {N}^{<\mathbb {N}}$
of length i, define the following c.e. open
$U_{\tau }\subseteq \Omega $
and c.e. open
$V_{\tau ,\varsigma }\subseteq \mathcal {X}$
, wherein j ranges over
$<i$
:
Let
$I_i$
be the computable index set
${\{(\tau , \varsigma ){\kern-1pt}\in{\kern-1pt} 2^i{\kern-1pt}\times{\kern-1pt} \mathbb {N}^i: \forall \; j<i \; (\tau (j){\kern-1pt}={\kern-1pt}1\rightarrow \varsigma (j){\kern-1pt}={\kern-1pt}0)\}}$
. Then on
$\mathsf {KR}^{p}\times \mathcal {X}$
one has
$W_{n,i}=\bigsqcup _{(\tau ,\varsigma )\in I_i} (U_{\tau }\times V_{\tau ,\varsigma })$
, where the disjoint union is disjoint only on
$\mathsf {KR}^{p}\times \mathcal {X}$
. Then
$\mu (W_{n,i})=\sum _{(\tau ,\varsigma )\in I_i} \mu (U_{\tau }\times V_{\tau ,\varsigma })$
.
Hence, we have reduced the problem to showing that
$\mu (U\times [\sigma ])$
is left-c.e. when
$U\subseteq \Omega $
is c.e. open. Let
$U=\bigcup _i U_i$
, where
$U_i$
is an increasing sequence of c.e. opens with uniformly p-computable measure. By Definition 1.1(3), one has that
$\theta \mapsto p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is uniformly
$L_1(p)$
computable, and hence so is
$\theta \mapsto I_{U_i}(\theta )\cdot p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
by Propositions 2.6(1) and 2.7. Then
$\mu (U\times [\sigma ])>q$
iff there is
$i\geq 0$
with
$\mu (U_i\times [\sigma ])>q$
, which by definition happens iff
$\int _{U_i} p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )>q$
. Since
$\int _{U_i} p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )$
is a computable real uniformly in
$i\geq 0$
, we are done and indeed
$\mu (U\times [\sigma ])$
is left-c.e.
As the last paragraph makes clear,
$\mu $
is computable if
$\int _{U_i} p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )$
is merely left-c.e. rather than computable. We do not know of an interesting condition, for which this holds, which is broader than the
$L_1(p)$
computability of the likelihood. But
$L_1(p)$
computability does get us the following natural
$\mu $
-computable basis.
Proposition 3.2 (The
$\mu $
-computable basis).
If
$U\subseteq \Omega $
is c.e. open with
$p(U)$
computable, then
$\mu (U\times [\sigma ])$
is computable, uniformly in
$\sigma $
from T.
This implies that
$\{U_i\times [\sigma ]: i\in I, \sigma \in T\}$
is a
$\mu $
-computable basis for any p-computable basis
$\{U_i: i\in I\}$
.
Further, if
$C_i\supseteq U_i$
is the associated effectively closed superset with
$C_i,U_i$
of same p-measure, then
$C_i\times [\sigma ]\supseteq U_i\times [\sigma ]$
is effectively closed superset with
$C_i\times [\sigma ], U_i\times [\sigma ]$
of same
$\mu $
-measure.
Proof. By the same considerations as at the end of the last proof.
Since projection maps are computable continuous, we have the following remark.
Remark 3.3. The pushforward
$\mu _{X}$
is a computable probability measure on the sample space
$\mathcal {X}$
. The
$\mu _{X}$
-computable basis is just the basic clopens
$[\sigma ]$
for
$\sigma $
in T.
Proposition 3.4.
$\mathsf {KR}^{\mu }\subseteq \mathsf {KR}^p\times \mathcal {X}$
and
$\mathsf {KR}^{\mu }\subseteq \Omega \times \mathsf {KR}^{\mu _{X}}$
, and likewise for Schnorr.
Note that since
$\mathsf {KR}^{\mu }$
has
$\mu $
-measure one, we have that
$\mathsf {KR}^p\times \mathcal {X}$
has
$\mu $
-measure one. Hence in proving Theorem 1.9, we can assume that the
$\mu $
-measure one set in question in condition (1) of Theorem 1.9 is a subset of
$\mathsf {KR}^p\times \mathcal {X}$
.
We now discuss when the likelihoods are probability measures. This is a pointwise result and so we use the conventions on versions (cf. Definition 1.6). It can be seen as an effective variant of classical results about regularity of conditional probability measures, which traditionally go through being a Borel subset of a Polish space (cf. [Reference Schervish25, Theorem B.32]).
Proposition 3.5 (The likelihoods and probability measures).
For all
$\theta _0$
in
$\mathsf {SR}^{p}$
one has that
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})$
is in
$\mathcal {P}(\mathcal {X})$
.
Proof. We suppose that there is a uniformly computable sequence of
$L_1(p)$
Schnorr tests
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot ), h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )$
such that
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )=g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )-h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )$
pointwise. By Proposition 2.8(2) we have
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )\geq h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )$
on
$\mathsf {SR}^{p}$
.
Where
$\emptyset $
is the root of the tree T, we show that
$p(\emptyset \hspace {-1mm}\mid \hspace {-1mm} \theta )=1$
for
$\theta $
in
$\mathsf {SR}^{p}$
:
-
– Suppose $0<s<1$
is rational. By Proposition 2.8, choose a computable r in
$(s,1)$
, and a
$\Sigma ^{0,p}_2$
set B which is equal to
$\{\theta : p(\emptyset \hspace {-1mm}\mid \hspace {-1mm} \theta )>r\}$
on
$\mathsf {SR}^{p}$
. Then
$p(\emptyset \hspace {-1mm}\mid \hspace {-1mm} \theta )> r>s$
for
$\theta $
in
$\mathsf {SR}^{p}$
by Proposition 1.5. Since this holds for all rational s in
$(0,1)$
we have
$p(\emptyset \hspace {-1mm}\mid \hspace {-1mm} \theta )\geq 1$
for
$\theta $
in
$\mathsf {SR}^{p}$
. -
– Suppose $s>1$
is rational. By Proposition 2.8, choose a computable r in
$(1,s)$
and a
$\Sigma ^{0,p}_2$
set
$B^{\prime }$
which is equal to
$\{\theta : p(\emptyset \hspace {-1mm}\mid \hspace {-1mm} \theta )>r\}$
on
$\mathsf {SR}^{p}$
. But then
$B^{\prime }$
is p-null, and no element of
$\mathsf {KR}^{p}$
can be in a
$\Sigma ^{0,p}_2\, p$
-null set. Since this holds for all rational
$s>1$
, we have
$p(\emptyset \hspace {-1mm}\mid \hspace {-1mm} \theta )\leq 1$
for
$\theta $
in
$\mathsf {SR}^{p}$
.
Suppose that
$\sigma $
in T is fixed; enumerate its immediate successors in T as
$\tau _0, \tau _1, \ldots $
. We may suppose by induction that we know
$0\leq p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\leq 1$
on
$\theta $
in
$\mathsf {SR}^{p}$
. We want to show that
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )=\sum _i p(\tau _i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
for all
$\theta $
in
$\mathsf {SR}^{p}$
:
-
– First we show that for all $\theta $
in
$\mathsf {SR}^{p}$
, we have
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\leq \sum _i p(\tau _i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
. By Proposition 2.8, choose dense computable
$r_n$
sequence in
$(0,1)$
and a computable sequence of
$\Sigma ^{0,p}_2$
sets
$B_{n,j}$
which are equal to
$\{\theta : p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )> r_n> \sum _{i\leq j} p(\tau _i \hspace {-1mm}\mid \hspace {-1mm} \theta )\}$
on
$\mathsf {SR}^{p}$
. Fix
$n\geq 0$
and set
$B_n:=\bigcap _j B_{n,j}$
. Since
$B_n$
is p-null by our standing effective assumptions (in particular by the second conjunct of Definition 1.1(3)), we can compute a subsequence
$B_{n,j_m}$
with
$\nu (B_{n,j_m})\leq 2^{-m}$
for all
$m\geq 0$
. By Proposition 1.4, no element of
$\mathsf {SR}^{p}$
is in
$B_n$
. Since this holds for all
$n\geq 0$
and since
$r_n$
is dense in
$(0,1)$
, we are done. -
– Second we show that for all $\theta $
in
$\mathsf {SR}^{p}$
, we have
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\geq \sum _i p(\tau _i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
. Fix
${j>0}$
. By Proposition 2.8, choose dense computable
$r_n$
sequence in
$(0,1)$
and a computable sequence of
$\Sigma ^{0,p}_2$
sets
$B_n$
which are equal to
$\{\theta : p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )< r_n< \sum _{i\leq j} p(\tau _i \hspace {-1mm}\mid \hspace {-1mm} \theta )\}$
on
$\mathsf {SR}^{p}$
. But
$B:=\bigcup _n B_n$
is p-null by our standing effective assumptions (in particular by the second conjunct of Definition 1.1(3)), and hence by Proposition 2.3(3), B is a
$\Sigma ^{0,p}_2\, p$
-null class. No element of
$\mathsf {KR}^{p}$
can be in a
$\Sigma ^{0,p}_2\, p$
-null set. Since this holds for all
$j>0$
, we have that
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\geq \sum _i p(\tau _i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
for all
$\theta $
in
$\mathsf {SR}^{p}$
.
There is a similar but much easier result for posteriors, whose proof we omit.
Proposition 3.6 (The posterior and probability measures).
One has that
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is in
$\mathcal {P}(\Omega )$
for all x in
$\mathsf {KR}^{\mu _{X}}$
and all
$n\geq 0$
.
We note that the posterior is an integral test.
Proposition 3.7 (The posterior and integral tests).
If
$U\subseteq \Omega $
is c.e. open, then
$x\mapsto p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is lsc and
$\leq 1$
, uniformly in
$n\geq 0$
.
Proof. For rational
$q>0$
, one has
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})>q$
iff there is
$\sigma $
in T of length n such that x is in
$[\sigma ]$
and there is rational
$r>0$
with
$\mu (U\times [\sigma ])>r$
and
${0<\mu (\Omega \times [\sigma ])\leq \frac {r}{q}}$
. This is a c.e. open condition in variable x, since
$U\times [\sigma ]$
is c.e. open in S and since
$\Omega \times [\sigma ]$
is both c.e. open and effectively closed in S.
Recall the notation from, for example, the Fubini–Tonelli Theorems: if
$B\subseteq S$
and
$\theta $
in
$\Omega $
, then the
$\theta $
-section
$B^{\theta }$
of B is defined as
$B^{\theta }:=\{x\in \mathcal {X} : (\theta , x)\in B\}$
. We have the following proposition.
Proposition 3.8 (Parameter space sections of the likelihood, evaluated at c.e. opens with computable measure, are
$L_1(p)$
computable).
-
(1) The map $\theta \mapsto p(U^{\theta } \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is
$L_1(p)$
computable, for c.e. open
$U\subseteq S$
with
$\mu (U)$
computable. -
(2) Any version of it as a difference of $L_1(p)$
Schnorr tests agrees on points
$\theta $
in
$\mathsf {SR}^{p}$
with the value which the element
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
of
$\mathcal {P}(\mathcal {X})$
assigns to the event
$U^{\theta }$
.
In (2), we are referring to the probability measure
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
from Proposition 3.5, which as established by that proposition is a probability measure when
$\theta $
is in
$\mathsf {SR}^{p}$
.
Proof. We first prove that (1) and (2) hold for c.e. opens V in S which are finite unions
$\bigcup _{i\leq n} (W_i\times [\sigma _i])$
of elements from the
$\mu $
-computable basis. For each
$i\leq n$
, choose
$C_i\supseteq W_i$
effectively closed with the same p-measure as
$W_i$
. For each
$J\subseteq [0,n]$
let
$J^{\prime }=\{i\in J: \forall \; j\in J\; \sigma _j\nprec \sigma _i\}$
and define the following events where i in the subscript ranges over
$\leq n$
:
Then
$p(V^{\theta } \hspace {-1mm}\mid \hspace {-1mm} \theta )=\sum _{J\subseteq [0,n]} \sum _{i\in J^{\prime }} I_{A_J}(\theta ) \cdot p(\sigma _i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
. By Propositions 2.6 and 2.7 this is a finite sum of
$L_1(p)$
computable functions, since
$A_J$
comes from the algebra generated by the p-computable basis and since
$\theta \mapsto p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is
$L_1(p)$
computable uniformly in
$\sigma $
from T. This takes care of (1). For (2), it suffices to find one such version, since all such versions agree on the Schnorr randoms. Suppose that
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )$
uses the version which is the difference of
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )$
and
$ h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )$
, where these are
$L_1(p)$
Schnorr tests, uniformly in
$\sigma $
from T. Then to finish (2), we note that
$p(V^{\theta } \hspace {-1mm}\mid \hspace {-1mm} \theta )$
has a version
$f(V \hspace {-1mm}\mid \hspace {-1mm} \theta )$
which is the difference of
Second, we treat the general case. Let
$U=\bigcup _m U_m$
, where
$U_m$
comes from the
$\mu $
-computable basis. Let
$V_{\ell }=\bigcup _{m\leq \ell } U_{m}$
. For each
$n\geq 0$
, compute
$\ell _n\geq 0$
such that
$\mu (U\setminus V_{\ell _n})<2^{-n}$
. One has that
$\| p(U^{\cdot } \hspace {-1mm}\mid \hspace {-1mm} \cdot ) - p(V_{\ell _n}^{\cdot } \hspace {-1mm}\mid \hspace {-1mm} \cdot ) \|_{L_1(p)}$
is equal to
By (1) applied to
$V_{\ell _n}$
, we have that
$\theta \mapsto p(V_{\ell _n} \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is
$L_1(p)$
computable; and we just showed that these converge fast to
$\theta \mapsto p(U \hspace {-1mm}\mid \hspace {-1mm} \theta )$
in
$L_1(p)$
; and so we are done with (1). For (2), let
$\theta \mapsto p(U^{\theta } \hspace {-1mm}\mid \hspace {-1mm} \theta )$
have a version f which is a difference of
$L_1(p)$
Schnorr tests
$g,h$
. By (2) applied to
$V_i$
, we have that
$f(V_i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is increasing and its limit exists in
$[0,1]$
, where
$f(V_i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is the difference of
$g(V_i \hspace {-1mm}\mid \hspace {-1mm} \theta ), h(V_i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
from (3.1). It suffices to show the identity
$f(\theta )=\lim _i f(V_i \hspace {-1mm}\mid \hspace {-1mm} \theta )$
for
$\theta $
in
$\mathsf {SR}^{p}$
. The proof of this identity, via the two inequalities, follows as in the proof of Proposition 3.5.
We can use this to prove the next proposition. See Takahashi [Reference Takahashi31, Theorems 3.1–3.2] for the analogue in his setting, and for precedents in Vovk and V’Yugin. As Takahashi notes, these results are reminiscent of van Lambalgen’s Theorem. So as to avoid further notation, we have not stated them in terms of oracle computation. Rute [Reference Rute24, Theorem 8.2] proves a similar result for Schnorr randomness when, in our terminology and setting, the likelihood map is computable continuous from
$\Omega $
to
$\mathcal {M}^+(\mathcal {X})$
.
Proposition 3.9 (Relation between randomness on prior and joint distribution).
If
$\theta $
in
$\mathsf {SR}^{p}$
, then
$(\theta ,x)$
is in
$\mathsf {SR}^{\mu }$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
-a.s. many x in
$\mathcal {X}$
.
Proof. Fix
$\theta $
in
$\mathsf {SR}^{p}$
. Suppose
$(\theta ,x)$
is not in
$\mathsf {SR}^{\mu }$
. Choose a sequential Schnorr
$\mu $
-test
$U_n$
with
$(\theta ,x)$
in
$\bigcap _n U_n$
. By Proposition 3.8 the map
$\theta ^{\prime }\mapsto p(U_n^{\theta ^{\prime }} \hspace {-1mm}\mid \hspace {-1mm} \theta ^{\prime })$
is
$L_1(p)$
computable, uniformly in
$n\geq 0$
. By Proposition 2.8, there is a computable sequence of computable reals
$\eta _n$
in the interval
$(2^{-n}, 2^{-n+1})$
and a computable sequence of
$\Sigma ^{0,p}_2$
classes
$B_n$
such that
$B_n=\{\theta ^{\prime }: p(U_{2n}^{\theta ^{\prime }} \hspace {-1mm}\mid \hspace {-1mm} \theta ^{\prime })>\eta _n\}$
on
$\mathsf {SR}^{p}$
. Then:
By Proposition 1.4 and by
$\theta $
being in
$\mathsf {SR}^{p}$
, there is
$n_0> 0$
such that for all
$n\geq n_0$
one has
$p(U_{2n}^{\theta } \hspace {-1mm}\mid \hspace {-1mm} \theta ) \leq 2^{-n+1}$
. Then by Proposition 3.8(2), the sample x is in the
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
-null set
$\bigcap _{n\geq n_0} U_{2n}^{\theta }$
. With
$\theta $
fixed, there are only countably many such null sets for each Schnorr
$\mu $
-test
$U_n$
. And since there are only countably many Schnorr
$\mu $
-tests, there are only
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
-null many x such that
$(\theta ,x)$
is not in
$\mathsf {SR}^{\mu }$
.
Finally, as mentioned in Section 1, in the case where the parameter space is Cantor space, one has that our version of the likelihood (Definition 1.6) agrees with Takahashi’s preferred version of the likelihood [Reference Takahashi31, Definition 2.2].
Proposition 3.10. If
$\Omega =2^{\mathbb {N}}$
and
$\theta $
in
$\mathsf {SR}^{p}$
, one has
$\lim _n \mu (\Omega \times [\sigma ] \mid [\theta _n]\times \mathcal {X})=p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
for all
$\sigma $
in T.
Proof. This follows from the identity
$\mu (\Omega \times [\sigma ] \mid [\theta _n]\times \mathcal {X})=\frac {1}{p([\theta _n])} \int _{[\theta _n]} p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ^{\prime }) dp(\theta ^{\prime })$
and the effective version of Lévy’s Upward Theorem ([Reference Huttegger, Walsh and Blando15]; see discussion therein of [Reference Pathak, Rojas and Simpson20, Reference Rute22]).
4 Proof of Theorem 1.9
We first note the following elementary result.
Proposition 4.1. Suppose
$f,f_n:\mathcal {X}\rightarrow \Omega $
are Borel measurable. Suppose that
$f:\mathcal {X}\rightarrow \Omega $
is such that
$f(x)=\theta $
for all
$(\theta ,x)$
in a
$\mu $
-measure one set. Then:
-
(1) For all Borel $A\subseteq \Omega $
and
$B\subseteq \mathcal {X}$
, one has
$A\times B=\Omega \times (f^{-1}(A)\cap B)\, \mu $
-a.s. -
(2) For all Borel $A\subseteq \Omega $
one has
$p(A \hspace {-1mm}\mid \hspace {-1mm} \sigma ) = \mu _{X}(f^{-1}(A)\mid \sigma )$
if
$\mu _{X}(\sigma )>0$
. -
(3) If $\kern1.2pt\lim _n (f_n\circ X)=(f\circ X)\, \mu $
-a.s., then
$\lim _n f_n=f\, \mu _{X}$
-a.s.
In (3), the function
$X:S\rightarrow \mathcal {X}$
is again the projection onto the sample space (cf. diagram in Section 1). Further, in (3), if the
$\mu $
-measure one event in the antecedent is Borel then the
$\mu $
-measure one event in the consequent can be chosen to be Borel using the proof of the universal measurability of analytic sets [Reference Kechris16, Theorem 21.10].
The following is the key proposition used in the proof of Theorem 1.9.
Proposition 4.2. Suppose that the prior p is computable.
Suppose there is a sequence of uniformly computable continuous functions
${f_n:\mathcal {X}\rightarrow \Omega }$
satisfying conditions (1) and (2) of Theorem 1.9. Let
$f=\lim _n f_n$
when it exists; and zero otherwise.
Let
$U\subseteq \Omega $
be c.e. open with
$p(U)$
computable.
Then
$I_{f^{-1}(U)}$
is a computable point of
$L_1(\mu _{X})$
.
Further, for all rational
$\epsilon>0$
, one can compute an index for an element
$V\subseteq U$
of the p-computable basis and an
$n\geq 0$
such that
$\|I_{f^{-1}(U)}-I_{f_n^{-1}(V)}\|_{L_1(\mu _{X})}<\epsilon $
, which is the same as
$\mu _{X}(f^{-1}(U)\triangle f^{-1}_n(V))<\epsilon $
.
Proof. First suppose that
$V\subseteq \Omega $
is an element of the p-computable basis, with effectively closed
$C\supseteq V$
such that
$V,C$
have same p measure. We first show that
$I_{f^{-1}(V)}$
is a computable point of
$L_1(\mu _{X})$
.
Let
$D\subseteq \Omega \times \mathcal {X}$
be a
$\mu $
-measure one subset of
$\mathsf {KR}^{p}\times \mathcal {X}$
such that for
$(\theta , x)$
in D we have
$\lim _n f_n(x)$
exists and is equal to
$\theta $
(cf. remarks immediately after Proposition 3.4).
Let us show that
$\lim _n (I_{f^{-1}_n(V)}\circ X)=I_{f^{-1}(V)}\circ X\, \mu $
-a.s. It suffices to show that it happens on D. Suppose that
$(\theta , x)$
is in D.
-
– First suppose that $I_{f^{-1}(V)}(x)=1$
. Then
$f(x)$
in V. Since V is open and we are on D, eventually
$f_n(x)$
in V, which implies that eventually
$I_{f^{-1}_n(V)}(x)=1$
. -
– Second suppose that $I_{f^{-1}(V)}(x)=0$
. Then
$f(x)$
is in
$\Omega \setminus V$
. Since we are on
$D\subseteq \mathsf {KR}^{p}\times \mathcal {X}$
, we have that
$\theta =f(x)$
is in the c.e. open
$\Omega \setminus C$
. Suppose that there were infinitely many
$n\geq 0$
with
$I_{f_n^{-1}(V)}(x)=1$
, so that there were infinitely many
$n\geq 0$
with
$f_n(x)$
in V. Since we are on D, we then have that
$f(x)$
is in the topological closure
$\overline {V}$
of V, and hence
$f(x)$
in C, since C is an effectively closed superset of V; a contradiction. Hence, rather there are only finitely many
$n\geq 0$
with
$I_{f_n^{-1}(V)}(x)=1$
, and hence
$\lim _n f_n(x)=0$
, which is what we wanted to show.
Since
$\lim _n (I_{f^{-1}_n(V)}\circ X)=I_{f^{-1}(V)}\circ X\, \mu $
-a.s., by Proposition 4.1(3) we have that
$\lim _n I_{f^{-1}_n(V)}=I_{f^{-1}(V)}\, \mu _{X}$
-a.s. By the Dominated Convergence Theorem,
$I_{f^{-1}_n(V)}\rightarrow I_{f^{-1}(V)}$
in
$L_1(\mu _{X})$
.
We argue that
$I_{f^{-1}_n(V)}$
is uniformly a computable point of
$L_1(\mu _{X})$
. Since
$f_n^{-1}(V)$
is c.e. open we have that
$I_{f^{-1}_n(V)}$
is lsc. It further has computable
$\mu _{X}$
expectation. For: by our hypothesis on the p-computable basis, we have that
$f_n^{-1}(V)$
is both c.e. open and effectively closed, so that
$\mathbb {E}_{\mu _{X}} I_{f_n^{-1}(V)} = \mu _{X}(f_n^{-1}(V))$
is both left-c.e. and right-c.e. and hence computable. Hence
$I_{f^{-1}_n(V)}$
is an
$L_1(\mu _{X})$
Schnorr test and so a computable point of
$L_1(\mu _{X})$
.
Further, the
$L_1(\mu _{X})$
distance between
$I_{f^{-1}_n(V)}, I_{f^{-1}(V)}$
is uniformly computable:
To get the last identity, we use Proposition 4.1(1). Further,
$\mu (V\times \mathcal {X})=p(V)$
, which is computable by hypothesis. And
$\mu (\Omega \times f_n^{-1}(V))=\mu _{X}(f_n^{-1}(V))$
, which is computable by the previous paragraph. Hence, since (4.1) is the measure of a Boolean combination of c.e. open events in S with computable
$\mu $
-measure, it too has
$\mu $
-computable measure, and uniformly so.
Hence indeed
$I_{f^{-1}(V)}$
is a computable point of
$L_1(\mu _{X})$
, when
$V\subseteq \Omega $
is an element of the p-computable basis.
Now suppose that U is a c.e. open with
$p(U)$
computable. Then choose a computable sequence of elements
$V_n$
from the p-computable basis such that
$U=\bigcup _n V_n$
. By the previous argument, one has that
$I_{f^{-1}(V_n)}$
is uniformly a computable point of
$L_1(\mu _{X})$
. One has that:
The second to last identity follows again from Proposition 4.1(1). Hence the
$L_1(\mu _{X})$
distance between
$I_{f^{-1}(V_n)}, I_{f^{-1}(U)}$
is uniformly computable and goes to zero. Hence we can compute a subsequence such that
$I_{f^{-1}(V_n)}\rightarrow I_{f^{-1}(U)}$
fast, so that
$I_{f^{-1}(U)}$
is likewise a computable point of
$L_1(\mu _{X})$
.
Finally, we need one classical estimate from Doob’s Maximal Inequality [Reference Gut11, Theorem 9.4].
Proposition 4.3 (Classical estimate from Doob’s Maximal Inequality).
Let
$\mathscr {F}_{n}$
be the
$\sigma $
-algebra of events on
$\mathcal {X}$
generated by the basic clopens
$[\sigma ]$
, where
$\sigma $
ranges over length n strings from T.
Suppose that
$B\subseteq \mathcal {X}$
is a Borel event. Then, as
$n\geq 0$
, the sequence of functions
$M_n(x):=\mu _{X}(B\mid x_{n})$
is a classical martingale in
$L_2(\mu _{X})$
with respect to
$\mathscr {F}_{n}$
. Further, for all
$m\geq 0$
, one has
$\|\sup _{n\geq m} \mu _{X}(B\mid \cdot _n)\|_{L_2(\mu _{X})}\leq 2\cdot \sqrt {\mu _{X}(B)}$
.
Proof. As a version of the conditional expectation, we use:
$\mathbb {E}_{\mu _{X}}[f\mid \mathscr {F}_{n}](x):=\frac {1}{\mu _{X}([x_{n}])} \int _{[x_{n}]} f(x^{\prime }) \; d\mu _{X}(x^{\prime })$
when
$\mu _{X}([x_{n}])>0$
, and zero otherwise.
Then
$M_n(x)=\mu _{X}(B\mid x_{n})$
is a classical martingale since it is the conditional expectation of the random variable
$I_B$
.
For the estimate, fix
$m\geq 0$
. By Doob’s Maximal Inequality followed by Conditional Jensen one has
Then we are done by applying the Monotone Convergence Theorem on right-hand side to take
$k\rightarrow \infty $
.
Now we prove Theorem 1.9.
Proof. By Propositions 3.5 and 3.9 it suffices to show the part about pairs. Let
$U\subseteq \Omega $
c.e. open with
$p(U)$
computable.
We show
$\sup _{n\geq m} p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is a Schnorr
$L_2(\mu _{X})$
-test, uniformly in
$m\geq 0$
. By Proposition 3.7 and
$0\leq p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})\leq 1$
, one has that
$\sup _{n\geq m} p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is lsc and
$\leq 1$
, uniformly in
$m\geq 0$
. Hence, it remains to show that
$\sup _{n\geq m} p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
has computable
$L_2(\mu _{X})$
-norm, uniformly in
$m\geq 0$
. To this end, it suffices to show that given rational
$\epsilon>0$
we can compute an index for a computable point of
$L_2(\mu _{X})$
within
$\epsilon $
of
$\sup _{n\geq m} p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
.
Let
$\epsilon>0$
be rational. By Proposition 4.2, compute an index for an element
$V\subseteq U$
of the p-computable basis and compute
$i\geq 0$
such that
$\mu _{X}(f^{-1}(U)\triangle f^{-1}_i(V))<\frac {\epsilon ^2}{8}$
. Since
$f_i^{-1}(V)$
is c.e. open, we can write
$f_i^{-1}(V)=\bigsqcup _k [\sigma _k]$
, for a computable sequence of strings
$\sigma _k$
in T. Since
$f_i^{-1}(V)$
is also effectively closed, we have
$\mu _{X}(f_i^{-1}(V))$
is computable. Hence, we can compute j such that
$\mu _{X}(f_i^{-1}(V)\setminus \bigsqcup _{k\leq j} [\sigma _k])<\frac {\epsilon ^2}{8}$
. Putting these together, we have
$\mu _{X}(f^{-1}(U)\triangle \bigsqcup _{k\leq j} [\sigma _k])<\frac {\epsilon ^2}{4}$
. Compute
$\ell $
which is strictly greater than the maximum length of
$\sigma _k$
for
$k\leq j$
. Then one has the following, where the norm is with respect to
$L_2(\mu _{X})$
and where the first inequality uses Proposition 4.1(2):
The last two inequalities use Proposition 4.3 and our earlier estimate. Since the function
$x\mapsto \sup _{m\leq n \leq \ell } \mu _{X}(\bigsqcup _{k\leq j} [\sigma _k]\mid x_{n})$
is a computable element of
$L_2(\mu _{X})$
, we are done. Namely, we have shown that the function
$x\mapsto \sup _{n\geq m} p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is indeed a Schnorr
$L_2(\mu _{X})$
-test, uniformly in
$m\geq 0$
.
By change of variables, the function
$(\theta ,x)\mapsto \sup _{n\geq m} p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is a Schnorr
$L_2(\mu )$
-test.
By the classical version of Lévy’s Upward Theorem, one has that
$ \mu _{X}(f^{-1}(U)\mid x_{n})\rightarrow I_{f^{-1}(U)}(x)$
for
$\mu _{X}$
-a.s. many x from
$\mathcal {X}$
. Hence by Proposition 4.1(2) we have
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})\rightarrow I_{U\times \mathcal {X}}(\theta , x)$
for
$\mu $
-a.s. many
$(\theta ,x)$
from S.
Then putting the two previous paragraphs together with Proposition 2.5, we have: for all
$(\theta ,x)$
in
$\mathsf {SR}^{\mu }\setminus (U\times \mathcal {X})$
that
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n}) \rightarrow 0$
.
Now it remains to show for all
$(\theta ,x)$
in
$\mathsf {SR}^{\mu }\cap (U\times \mathcal {X})$
that
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n}) \rightarrow 1$
.
Let
$U=\bigcup _m U_m$
, where
$U_m$
is a computable sequence of c.e. opens from the p-computable basis. Choose computable sequence
$C_m\supseteq U_m$
of effectively closed sets such that
$U_m, C_m$
have same p-measure.
It suffices to show that for each
$m\geq 0$
and each
$(\theta ,x)$
in
$\mathsf {SR}^{\mu }\cap (U_m\times \mathcal {X})$
that
$p(U_m \hspace {-1mm}\mid \hspace {-1mm} x_{n}) \rightarrow 1$
. For, given
$(\theta ,x)$
in
$\mathsf {KR}^{\mu }$
one has that
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is in
$\mathcal {P}(\Omega )$
by Propositions 3.4 and 3.6 and so this and
$U_m\subseteq U$
also makes it that for each
$m\geq 0$
and
$(\theta ,x)$
in
$\mathsf {SR}^{\mu }\cap (U_m\times \mathcal {X})$
one has
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n}) \rightarrow 1$
.
Fix
$m\geq 0$
for the remainder of the proof. Let
$V_m=\Omega \setminus C_m$
, which is c.e. open with computable p-measure. Then we can run the argument above to conclude that for all
$(\theta ,x)$
in
$\mathsf {SR}^{\mu }\setminus (V_m\times \mathcal {X})$
one has that
$p(V_m \hspace {-1mm}\mid \hspace {-1mm} x_{n})\rightarrow 0$
. Let
$(\theta ,x)$
in
$\mathsf {SR}^{\mu }\cap (U_m\times \mathcal {X})$
. By Proposition 3.4, we have
$\mathsf {SR}^{\mu }\subseteq \mathsf {KR}^{p}\times \mathcal {X}$
, and so
$(\theta ,x)$
is in
$\mathsf {SR}^{\mu }\setminus (V_m\times \mathcal {X})$
. Hence
$p(V_m \hspace {-1mm}\mid \hspace {-1mm} x_{n})\rightarrow 0$
. Due to
$(\theta ,x)$
in
$\mathsf {KR}^{\mu }$
, one has
$p(\cdot \mid x_{n})$
is in
$\mathcal {P}(\Omega )$
by Propositions 3.4 and 3.6, so that
$p(U_m \hspace {-1mm}\mid \hspace {-1mm} x_{n})=p(C_m \hspace {-1mm}\mid \hspace {-1mm} x_{n}) \rightarrow 1$
. Note the identity follows from the fact that for
$(\theta ,x)$
in
$\mathsf {KR}^{\mu }$
we have
$p(\mathsf {KR}^{p} \hspace {-1mm}\mid \hspace {-1mm} x_{n})=1$
, as one can see from the formula for the posterior in (1.2).
5 Limiting relative frequencies
In this section, we prove Proposition 1.10, largely following traditional proofs of the Strong Law of Large Numbers for Bernoulli trials (e.g., [Reference McKean18, pp. 43–44]). Since we are proving Proposition 1.10, everything in this section (the parameter space, sample space, etc.) are as in the statement of the proposition.
Before we begin, since this proposition provides a sufficient condition for the application of Theorem 1.9, let us say a brief word about this application. Let
$C\subseteq \Omega $
of p-measure one be the set mentioned in the statement of Proposition 1.10.
First, note that since
$p(C)=1$
, we have that
$\theta $
is in C whenever
$(\theta , x)$
is in
$\mathsf {KR}^{\mu }$
, by Proposition 3.4.
Second, for a binary string
$\sigma $
of length n which possesses i ones, the likelihood in Proposition 1.10 is defined as
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )=\theta ^i \cdot (1-\theta )^{n-i} \cdot I_{C}(\theta )$
. Then we can use the version in the sense of Definition 1.6 given by
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) = \theta ^i \cdot (1-\theta )^{n-i}$
and
$h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) = \theta ^i \cdot (1-\theta )^{n-i}\cdot I_{\Omega \setminus C}(\theta )$
. Further, by the previous paragraph, for
$\theta $
in the
$\mu $
-measure one set
$\mathsf {KR}^{\mu }$
, we have
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )-h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) = \theta ^i \cdot (1-\theta )^{n-i}$
.
Now we turn to the proof of Proposition 1.10. Where
$\Theta :S\rightarrow \Omega $
is the projection onto the first component (cf. diagram in §1), we define:
Definition 5.1.
$\alpha = \mathbb {E}_{\mu } \Theta =\int _{\Omega } \theta \; dp(\theta )$
and
$\beta =\mathbb {E}_{\mu } \Theta ^2=\int _{\Omega } \theta ^2 \; dp(\theta )$
.
Then
$\alpha \geq \beta $
are computable reals since p is computable and the identity map from
$\Omega $
to
$\mathbb {R}$
and the square map are computable continuous. We can then calculate the expectations of the following random variables in terms of
$\alpha ,\beta $
, where
$g_i(\theta ,x)=x(i)$
and where
$i,j$
distinct:
Then we show:
Proposition 5.2. There is a p-computable basis on
$\Omega $
such that for all
$n\geq 0$
and all elements U of the p-computable basis, the event
$f_n^{-1}(U)$
is uniformly both c.e. open and effectively closed.
Further, for
$\epsilon>0$
and
$n\geq 0$
one has
$\mu (\left |f_n\circ X -\Theta \right |>\epsilon )\leq \epsilon ^{-2} \cdot n^{-1} \cdot (\alpha -\beta )$
Proof. Using the Effective Baire Category Theorem ([Reference Hoyrup and Rojas13, Theorem 5.1.2]), we can choose a p-computable basis consisting of intervals
$(p,q)$
and
$[0,p)$
and
$(q,1]$
and
$[0,1]$
such that
$0\leq p<q\leq 1$
is a uniformly computable sequence of reals such that these
$p,q$
are not equal to any element of the countable set
${\{\frac {1}{n}\sum _{i<n} \sigma (i) : \sigma \in 2^n, n>0\}}$
. This assumption makes both
$I_{n,p,q} =\{\sigma \in 2^n: p<\frac {1}{n}\sum _{i<n} \sigma (i)<q\}$
and its relative complement
$J_{n,p,q} =2^n\setminus I_{n,p,q}$
uniformly c.e. in
$n,p,q$
. Further, one has
$f_n^{-1}(p,q)=\bigsqcup _{\sigma \in I_{n,p,q}} [\sigma ]$
and
$\mathcal {X} \setminus f_n^{-1}(p,q)=\bigsqcup _{\sigma \in J_{n,p,q}} [\sigma ]$
. Hence,
$f_n^{-1}(p,q)$
is both c.e. open and effectively closed. The cases of
$[0,p)$
and
$(q,1]$
are similar. For
$\epsilon>0$
and
$n\geq 0$
one has:
Proposition 5.3. For
$(\theta ,x)$
in
$\mathsf {SR}^{\mu }$
one has
$\lim _n f_n(x)= \theta $
.
Proof. Since
$f_n, \Theta $
are computable continuous, so is
$\left |f_n\circ X -\Theta \right |$
. By Proposition 2.8, choose a computable sequence of reals
$\eta _n$
for
$n\geq 1$
such that
$\eta _n$
is in the interval
$((n+1)^{-\frac {1}{3}}, n^{-\frac {1}{3}})$
and such that the event
$\left |f_{n^2}\circ X -\Theta \right |>\eta _n$
has computable
$\mu $
-measure. Then let
$U_n$
be this event, and set
$g=\sum _{n\geq 1} I_{U_n}$
. Then one has
$\mu (U_n)\leq \mu \big (\left |f_{n^2}\circ X -\Theta \right |>(n+1)^{-\frac {1}{3}}\big )\leq (n+1)^{\frac {2}{3}}\cdot n^{-2} \cdot (\alpha -\beta ) \leq \sqrt [3]{4} \cdot n^{-\frac {4}{3}} \cdot (\alpha -\beta )$
. Then one has
$\mathbb {E}_{\mu } g \leq \sum _{n=1}^{\infty } \sqrt [3]{4} \cdot n^{-\frac {4}{3}} \cdot (\alpha -\beta )$
, and since the latter is finite and computable so is
$\mathbb {E}_{\mu } g$
by the Comparison Test. Hence g is an
$L_1(\mu )$
Schnorr test.
Suppose that
$(\theta ,x)$
is in
$\mathsf {SR}^{\mu }$
. Choose
$n_0> 0$
such that for all
$n\geq n_0$
one has
$(\theta ,x)$
is not in
$U_n$
. Then for all
$n\geq n_0$
we have
$\left |\sum _{i<n^2} (x(i)-\theta )\right |\leq n^{\frac {5}{3}}$
.
Let
$m> n_0^2$
. Choose least
$n> n_0$
with
$n^2<m\leq (n+1)^2$
. Then one has
Hence
$\left |(\frac {1}{m} \sum _{i<m} x(i))-\theta \right |=\left |\frac {1}{m} \sum _{i<m} (x(i)-\theta )\right |\leq m^{-\frac {1}{6}}+2\cdot m^{-\frac {1}{2}}+m^{-1}$
.
6 Proof of Theorem 1.13
Proof. The direction from (1) to (2) follows directly from Theorem 1.9.
We now show that “not (1)” implies “not (2).” Suppose that
$\theta _0$
is in
$\bigcap _n U_n$
, where
$U_n$
is a sequential Schnorr p-test. Without loss of generality
$U_0=\Omega $
and
$U_n$
is decreasing. Note that since
$U_0=\Omega $
, we have
$\sum _n p(U_n\setminus U_{n+1})=1$
and hence
$\sum _n \mu ((U_n\setminus U_{n+1})\times \mathcal {X})=1$
.
By the hypothesis of the Theorem, choose an apt triple
$\mathcal {X}, \theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta }), f_n:\mathcal {X}\rightarrow \Omega $
(cf. Definition 1.12). We will construct another apt triple
$\mathcal {Y}, \theta \mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta }), g_n:\mathcal {Y}\rightarrow \Omega $
such that
$\theta _0$
is not computably consistent relative to
$p, \mathcal {Y}, \theta \mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
.
Let
$\mathcal {Y}=[T^{\infty }]$
, where
$T^{\infty }=\{\emptyset \}\cup \{(m)^{\frown }\sigma : m\geq 0, \sigma \in T\}$
, where
$\emptyset $
denotes the root (the unique length zero string). That is,
$T^{\infty }$
copies T countably many times after the root
$\emptyset $
. Clearly since T is a computable subtree of
$\mathbb {N}^{<\mathbb {N}}$
with no dead ends, so is
$T^{\infty }$
.
Define likelihood as follows, for
$\tau $
in
$T^{\infty }$
:
The idea is: if
$\theta $
is in
$U_m\setminus U_{m+1}$
, then
$q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
puts all the probability on the m-th copy of T in
$T^{\infty }$
, where it copies the original likelihood; whereas if
$\theta $
is in the p-null set
$\bigcap _n U_n$
, then
$q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is the zero element of
$\mathcal {M}^+(\mathcal {Y})$
.
Regarding the effective properties of the likelihood function (cf. Definition 1.1(3), note that for p-a.s. many
$\theta $
in
$\Omega $
, one has that
$\theta $
is not in
$\bigcap _n U_n$
and hence is in some
$U_m\setminus U_{m+1}$
, and thus:
-
– The likelihood function $\theta \mapsto q(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is
$L_1(p)$
computable, uniformly in
$\tau $
in
$T^{\infty }$
by Proposition 2.6(3) and Proposition 2.7. -
– $q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is in
$\mathcal {P}(\mathcal {Y})$
for p-a.s. many
$\theta $
in
$\Omega $
since
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is in
$\mathcal {P}(\mathcal {X})$
for p-a.s. many
$\theta $
in
$\Omega $
.
Suppose that, per Definition 1.6,
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is understood pointwise as its version
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )-h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
, where these are
$L_1(p)$
Schnorr tests, uniformly in
$\sigma $
from T. We define a pair
$G(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta ), H(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )$
of
$L_1(p)$
Schnorr tests, uniformly in
$\tau $
in
$T^{(\infty )}$
, such that
$G(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )-H(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is a version of
$q(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )$
:
-
– If $\tau $
is the root, define
$G(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta ) = \sum _m I_{U_{m}}(\theta )$
and
$H(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta ) = \sum _m I_{U_{m+1}}(\theta )$
. -
– If $\tau = (m)^{\frown }\sigma $
, define
$G((m)^{\frown }\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) =g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\cdot I_{U_m}(\theta )+h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\cdot I_{U_{m+1}}(\theta )$
and
$H((m)^{\frown }\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) =g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\cdot I_{U_{m+1}}(\theta )+h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\cdot I_{U_m}(\theta )$
.
Then
$G(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta ), H(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )$
are finite for
$\theta $
in
$\mathsf {SR}^{p}$
, and on these points we have that
$G(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )-H(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )$
agrees with (6.1), when
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is understood pointwise as its version
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )-h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
. When
$\theta $
is in
$\bigcap _n U_n$
, by our convention that
$\infty -\infty =0$
, we have that
$G(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )-H(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )=0$
for all
$\tau $
. Hence when
$\theta $
is in
$\bigcap _n U_n$
, this version of
$q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is the zero element of
$\mathcal {M}^+(\mathcal {Y})$
, and hence not an element of
$\mathcal {P}(\mathcal {Y})$
.Footnote
18
By the first conjunct of Definition 1.8(2), any
$\theta $
in
$\bigcap _n U_n$
is not computably consistent.
Define uniformly computable continuous
$g_n:\mathcal {Y}\rightarrow \Omega $
by
$g_n((m)^{\frown } x) = f_n(x)$
. It remains to show that
$\mathcal {Y}, \theta \mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta }), g_n$
satisfies conditions (1)-(2) of Theorem 1.9.
To this end, define uniformly computable continuous
$\iota _m:S\rightarrow \Omega \times \mathcal {Y}$
by
$\iota _m(\theta , x)=(\theta , (m)^{\frown } x)$
. Further, let
$\nu $
be the joint distribution on the product space
$\Omega \times \mathcal {Y}$
formed from the prior p and the new likelihood
$\theta \mapsto q(\tau \hspace {-1mm}\mid \hspace {-1mm} \theta )$
. Then for each Borel event
$B\subseteq \Omega \times \mathcal {Y}$
, we have
This follows by a simple induction on B, which we omit for reasons of space.
Finally, here is the verification that
$\mathcal {Y}, \theta \mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta }), g_n$
satisfies conditions (1)-(2) of Theorem 1.9:
-
– Suppose that A is a $\mu $
-measure one subset of
$\mathsf {KR}^{p}\times \mathcal {X}$
on which
$\lim _n f_n(x)$
exists and is equal to
$\theta $
. Let
$B=\bigsqcup _m \iota _m(A)$
, so that
$\iota _m^{-1}(B)=A$
for all
$m\geq 0$
and so that
$B\subseteq \mathsf {KR}^{p}\times \mathcal {Y}$
. Then by (6.2) one has $$ \begin{align*} \nu(B) = \sum_m \mu(A\cap \big((U_m\setminus U_{m+1})\times \mathcal{X}\big)) = \sum_m \mu((U_m\setminus U_{m+1})\times \mathcal{X}) =1 \end{align*} $$Finally note that for all $(\theta , (m)^{\frown } x)$
in B one has that
$(\theta , x)$
is in A, and hence by condition (1) of Theorem 1.9, we have that the limit
$\lim _n g_n((m)^{\frown } x))=\lim _n f_n(x)$
exists and is equal to
$\theta $
.
-
– Suppose that $U\subseteq \Omega $
is from a p-computable basis. Then, by condition (2) of Theorem 1.9 we can write
$f_n^{-1}(U)=\bigsqcup _i [\sigma _i]$
and
$\mathcal {X}\setminus f_n^{-1}(U)=\bigsqcup _i [\tau _i]$
for computable sequences of
$\tau _i, \sigma _i$
from T. Then one has
$g_n^{-1}(U)=\bigsqcup _m \bigsqcup _i [(m)^{\frown } \sigma _i]$
and
$\mathcal {Y}\setminus g_n^{-1}(U)=\bigsqcup _m \bigsqcup _i [(m)^{\frown } \tau _i]$
.
7 The infinite simplex, subspaces, and identifiability
The infinite dimensional simplex
$\mathbb {S}_{\infty }$
(Definition 1.7) is a subspace of the Hilbert cube
$[0,1]^{\mathbb {N}}$
. But since
$\mathbb {S}_{\infty }$
is not closed in
$[0,1]^{\mathbb {N}}$
, some care has to be devoted to describing the computable Polish space structure on
$\mathbb {S}_{\infty }$
.
We define the natural effectivization of the subspace relation:
Definition 7.1. We say
$Y_0$
is a computable Polish subspace of Y if
-
(1) $Y,Y_0$
are computable Polish spaces and
$Y_0\subseteq Y$
-
(2) There is a computable procedure which when given an index for a c.e. open U in Y returns an index for a c.e. open V in $Y_0$
such that
$V=U\cap Y_0$
. -
(3) There is a computable procedure which when given an index for a c.e. open V in $Y_0$
returns an index for a c.e. open U in Y such that
$V=U\cap Y_0$
.
Remark 7.2. To illustrate how this concept is used, note that when
$Y_0$
is a computable Polish subspace of Y, we have:
-
(1) If $f:Y\rightarrow Z$
is a computable continuous function, then
$f\upharpoonright Y_0:Y_0\rightarrow Z$
is computable continuous. -
(2) If $U_0, U_1, \ldots $
is a computable basis in Y, then
$U_0\cap Y_0, U_1\cap Y_0, \ldots $
is a computable basis in
$Y_0$
.
In the last point, a computable basis in a computable Polish space is just a uniformly computable sequence of c.e. opens such that every c.e. open can be written as an effective union of sets from the sequence.Footnote 19
We build up to proving:
Proposition 7.3.
$\mathbb {S}_{\infty }$
is a computable Polish subspace of
$[0,1]^{\mathbb {N}}$
.
But the simplest example of Definition 7.1 is:
Example 7.4. If
$Y_0$
is effectively closed in Y and inherits the metric from Y and has as a countable dense set a uniformly computable sequence of points from Y, then
$Y_0$
is a computable Polish subspace of Y.
However,
$\mathbb {S}_{\infty }$
is properly
$\Pi ^0_2$
in
$[0,1]^{\mathbb {N}}$
, and so one needs an effective version of the classical result that
$\Pi ^0_2$
subspaces of Polish spaces are Polish [Reference Kechris16, p. 17]:
Proposition 7.5. Suppose that
$U_n$
is a uniformly computable sequence of c.e. opens in a computable Polish space Y, with
$C_n=Y\setminus U_n$
the corresponding effectively closed sets, which we further suppose to be non-empty.
Let d be the metric with respect to which Y is a computable Polish space. Suppose that d is bounded above by some rational constant
$c>1$
. Suppose further that the map
$y\mapsto d(y, C_n):=\inf _{z\in C_n} d(y,z)$
is computable continuous, uniformly in
$n\geq 0$
.
Let
$Y_0=\bigcap _n U_n$
, and suppose that
$Y_0$
contains a computable subset
$D_0$
of the countable dense set D of Y, such that
$D_0$
is dense in
$Y_0$
.
Then
$Y_0$
is a computable Polish subspace of Y space with metric:
Proof. We identify the countable dense set D of Y with natural numbers and use variables
$i,j$
for them in this proof.
Since classically
$d_0$
provides a complete compatible metric for
$Y_0$
(cf. [Reference Kechris16, p. 17]), to show that
$Y_0$
is a computable Polish space it remains to show that if
$i,j$
come from
$D_0$
, then
$d_0(i,j)$
is uniformly computable. To see this, note that since
$i,j$
are computable points of
$Y_0\subseteq U_n$
, we have that
$d(i,C_n), d(j,C_n)>0$
are uniformly computable. We can search for, and find, a non-zero rational, uniformly in
$i,j,n$
, which is below
$d(i,C_n), d(j,C_n)$
. Using this, we can compute
$\min \big (2^{-(n+1)}, \left |\frac {1}{d(i,C_n)}-\frac {1}{d(j,C_n)}\right |\big )$
. Then by the Comparison Test,
$d_0(i,j)$
is computable, and uniformly so. Hence indeed
$Y_0$
is a computable Polish space.
Now we show that
$Y_0$
is a computable Polish subspace of Y.
We focus on condition (3) of Definition 7.1 since the other one is similar but easier. It suffices to consider the case where V is an open ball
$B_{d_0}(j,\epsilon ):=\{y_0\in Y_0: d_0(j,y_0)<\epsilon \}$
in
$Y_0$
, where j in
$D_0$
and
$\epsilon>0$
is rational; we must show that
$B_{d_0}(j, \epsilon )=U\cap Y_0$
for some c.e. open U in Y. We proceed as follows:
-
– For each rational q in $(0,\frac {1}{c})$
compute
$n_q\geq 0$
such that
$2^{-(n_q+1)}<\frac {q}{3}$
. -
– For each $(i,q)$
in
$D_0\times (\mathbb {Q}\cap (0,\frac {1}{c}))$
, compute real numbers $$ \begin{align*} \delta_{i,q,n}:=\frac{1}{d(i,C_n)}+ \frac{q}{3}\cdot 2^{-(n+1)}, \hspace{10mm} \eta_{i,q,n}:=\frac{1}{d(i,C_n)}- \frac{q}{3}\cdot 2^{-(n+1)} \end{align*} $$Since the metric d is bounded $<c$
and since q is from
$(0,\frac {1}{c})$
, note that we have that
$\eta _{i,q,n}>0$
; and trivially
$\delta _{i,q,n}>0$
.
-
– Define the c.e. index set $I=\{(i,q)\in D_0\times (\mathbb {Q}\cap (0,\frac {1}{c})): d_0(i,j)+\frac {q}{3}<\epsilon \}$
. -
– For each $(i,q)$
in I let
$U_{i,q,n}$
be the c.e. open (7.2) $$ \begin{align} U_{i,q,n}=(d(\cdot, C_n))^{-1}(-\infty,\frac{1}{\eta_{i,q,n}})\cap (d(\cdot, C_n))^{-1}(\frac{1}{\delta_{i,q,n}},\infty) \end{align} $$
Then we claim that
$B_{d_0}(j,\epsilon )=Y_0\cap \bigcup _{(i,q)\in I} B_d(i,\frac {q}{3})\cap \bigcap _{n\leq n(q)} U_{i,q,n}$
:
-
• First suppose that x in $Y_0\cap B_d(i,\frac {q}{3})\cap \bigcap _{n\leq n(q)} U_{i,q,n}$
for some
$(i,q)$
in I. Then for
$n\leq n(q)$
, membership in
$U_{i,q,n}$
implies that
$\left |\frac {1}{d(i,C_n)}-\frac {1}{d(x,C_n)}\right |<\frac {q}{3}\cdot 2^{-(n+1)}$
. Together with membership in
$ B_d(i,\frac {q}{3})$
and the definition of
$n_q$
we then have that
$d_0(x,i)<q$
. Then
$d_0(x,j)\leq d_0(x,i)+d_0(i,j)<q+d_0(i,j)<\epsilon $
. -
• Second suppose that x is in $B_{d_0}(j,\epsilon )$
. Choose non-zero rational q such that both
$q<\frac {3}{2}(\epsilon -d_0(x,j))$
and
$q<\frac {1}{c}$
. Since for each
$n\leq n_q$
, the function
$y\mapsto \frac {1}{d(y,C_n)}$
is continuous on
$U_n$
and x is in
$U_n$
, we can find i in
$D_0$
such that x is in
$B_{d_0}(i,\frac {q}{3})$
and x is in
$U_{i,q,n}$
for all
$n\leq n_q$
. Further
$(i,q)$
is in I: for we have
$d_0(i,j)+\frac {q}{3}\leq d_0(i,x)+d_0(x,j)+\frac {q}{3}<\frac {2}{3}q+d_0(x,j)<\epsilon $
.
In order to apply this proposition to
$\mathbb {S}_{\infty }$
and
$[0,1]^{\mathbb {N}}$
, we use:
Definition 7.6 (Computably Compact).
A computable Polish space Y is computably compact if there is a partial computable function which when given an index for a computable sequence of c.e. opens
$U_0, U_1, \ldots $
, if the sequence covers Y then the computable function halts and returns an
$n\geq 0$
such that
$U_0, \ldots , U_n$
covers Y.
(This definition is implied by that given in [Reference Brattka, Gheradi, Pauly, Brattka and Hertling3, Definition 11.2.10(2)]).
The finite product
$[0,1]^n$
is computably compact, using a basis of rectangles with rational-valued corners, and this extends to
$[0,1]^{\mathbb {N}}$
. Likewise, Cantor space
$2^{\mathbb {N}}$
is computably compact since one can tell when a finite set of basic clopens covers the space. An important consequence of computable compactness is the following (cf. [Reference Porter, Day and Downey21, Theorem 3.14] and references therein).
Proposition 7.7. If Y is computably compact and
$f:Y\rightarrow \mathbb {R}$
is computable continuous, then both
$\sup _{y\in Y} f(y)$
and
$\inf _{y\in Y} f(y)$
are computable real numbers.
We now prove Proposition 7.3:
Proof. The Hilbert cube
$[0,1]^{\mathbb {N}}$
with its metric
$d(x,y)=\sum _n 2^{-(n+1)} \left |x(n)-y(n)\right |$
is a computable Polish space with the dense set being the points
$\theta $
in
$[0,1]^{\mathbb {N}}$
which are zero after some point. One has that
$\mathbb {S}_{\infty }=\bigcap _n U_n\cap \bigcap _n V_n$
, where
$U_n,V_n$
are the below c.e. opens with respective relative non-empty complements
$C_n,D_n$
:
Further,
$C_n, D_n$
are computable Polish spaces which are computably compact:
-
– For $C_n$
: it is effectively closed in
$[0,1]^{\mathbb {N}}$
since it is
$f^{-1}[0, 1-\frac {1}{n+1}]$
, where
$f:[0,1]^{\mathbb {N}}\rightarrow [0,\infty ]$
is the lsc function
$f(x)=\sum _i x(i)$
. Further, the intersection of the countable dense set of
$[0,1]^{\mathbb {N}}$
with
$C_n$
is computable and dense in
$C_n$
. Finally, the computable compactness of
$C_n$
follows from that
$[0,1]^{\mathbb {N}}$
, since
$C_n$
is effectively closed in
$[0,1]^{\mathbb {N}}$
. -
– For $D_n$
: it is effectively closed in
$[0,1]^{\mathbb {N}}$
since it is
$g_n^{-1}[1+\frac {1}{n+1},\infty )$
, where
$g_n:[0,1]^{\mathbb {N}}\rightarrow [0,\infty )$
is the computable continuous function
$g_n(x)=\sum _{i\leq n+1} x(i)$
. The other parts follows just as for
$C_n$
.
For each element x of the countable dense set of
$[0,1]^{\mathbb {N}}$
and each
$n\geq 0$
, one can consider the computable continuous map
$f_{x,n}:C_n\rightarrow [0,\infty )$
given by
$f_{x,n}(y)=d(x,y)$
. By Proposition 7.7, one has that
$\inf _{y\in C_n} d(x,y)=d(x,C_n)$
is a computable real, uniformly in a point x of the countable dense set and in
$n\geq 0$
. Since
${y\mapsto d(y,C_n)}$
is 1-Lipschitz, one then has that
$y\mapsto d(y,C_n)$
is computable continuous, uniformly in
$n\geq 0$
(cf. [Reference Huttegger, Walsh and Blando15, Proposition 2.5]). The same is true of
$y\mapsto d(y,D_n)$
, and hence the hypotheses of Proposition 7.5 are satisfied.
From this it follows:
Proposition 7.8. For each
$i\geq 0$
, the projection maps
$\pi _i:\mathbb {S}_{\infty }\rightarrow [0,1]$
given by
$\pi _i(\theta )=\theta (i)$
are computable continuous and computable open.
In this computable open means: images of c.e. opens are uniformly c.e. open.
Proof. Computable continuity follows automatically from Remark 7.2(1). For computable open, suppose that
$V\subseteq \mathbb {S}_{\infty }$
is c.e. open; we want to show that its image
$\pi _i(V)$
is c.e. open in
$[0,1]$
, uniformly in an index for V. Since images commute with unions, by Remark 7.2(2), it suffices to show it for a c.e. open of the form
$V=\{\theta \in \mathbb {S}_{\infty }: \theta \in \prod _{j<n} V_j\times \prod _{j\geq n} [0,1]\}$
where
$V_j$
is
$(p_j,q_j)$
or
$[0,q_j)$
or
$(p_j,1]$
or
$[0,1]$
for rational
$0<p_j<q_j<1$
. There are then three cases.
Case 1:
$i<n$
and
$1\notin V_i$
. Then define
$U=\{\theta \in \prod _{j<n} V_j: \sum _{j<n} \theta (j)< 1\}$
; and note
$\pi _i(V)=\pi _i(U)$
. Since U is c.e. open in
$[0,1]^n$
,
$\pi _i(U)$
is c.e. open in
$[0,1]$
.
Case 2:
$i<n$
and
$1\in V_i$
. Then we first check whether for all
$j<n$
with
$j\neq i$
we have
$V_j=[0,q_j)$
for a rational
$0<q_j<1$
or
$V_j=[0,1]$
. If no, V is the same as what one gets by changing
$V_i$
to
$V_i\setminus \{1\}$
, and we are done by the previous paragraph. If yes, then we have
$\pi _i(V)=V_i$
.
Case 3:
$i\geq n$
. This follows Case 1-2 by setting
$V_j=[0,1]$
for all
$j\in [n,i]$
.
We show how to extend Proposition 1.10 to
$\mathbb {S}_{\infty }$
:
Proposition 7.9. Suppose that the parameter space
$\Omega $
is
$\mathbb {S}_{\infty }$
and that the sample space
$\mathcal {X}$
is Baire space
$\mathbb {N}^{\mathbb {N}}$
.
Suppose that C is a p-measure one effectively closed subset of
$\mathbb {S}_{\infty }$
.
Suppose that the likelihood
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is given by the following for
$\sigma $
in
$\mathbb {N}^{<\mathbb {N}}$
:
Then conditions (1) and (2) of Theorem 1.9 are satisfied.
Note that the likelihood
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
is defined on C by taking
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
to be the countable product of
$\theta $
, as a probability measure on the natural numbers; and that off of C it is the zero element of
$\mathcal {M}^+(\mathcal {X})$
.
Proof. We use these four computable continuous maps, which commute:

First we verify condition (2) of Theorem 1.9. For each
$j\geq 0$
, consider the pushforward probability measure
$p_j:=\pi _j \# p$
on
$[0,1]$
. By Proposition 5.2, uniformly in
$j\geq 0$
we can compute a
$p_j$
-computable basis on
$[0,1]$
such that for all
$n\geq 0$
and all elements U of the
$p_j$
-computable basis on
$[0,1]$
one has that
$g_n^{-1}(U)$
is uniformly both c.e. open and effectively closed in
$2^{\mathbb {N}}$
. Since
$c_j$
is computable continuous, this means that
$(g_n\circ c_j)^{-1}(U)=(\pi _j \circ f_n)^{-1}(U)$
is uniformly both c.e. open and effectively closed in
$\mathbb {N}^{\mathbb {N}}$
. Then a p-computable basis on
$\mathbb {S}_{\infty }$
is given by elements of the form
$V=\bigcap _{j<m} \pi _j^{-1}(U_j)$
, where
$U_j$
for
$j<m$
comes from the
$p_j$
-computable basis on
$[0,1]$
. For this V, we have that
$f_n^{-1}(V)=\bigcap _{j<m} (\pi _j\circ f_n)^{-1}(U_j)$
; thus
$f_n^{-1}(V)$
is both c.e. open and effectively closed in
$\mathcal {X}$
.
Second we verify condition (1) of Theorem 1.9. For each
$j\geq 0$
, consider the pushforward probability measure
$\mu _j:=(\pi _j\otimes c_j)\# \mu $
on
$[0,1]\times 2^{\mathbb {N}}$
, and for each
$j\geq 0$
and
$\theta $
in
$\mathbb {S}_{\infty }$
consider the pushforward measure
$p_{j}(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta }):=c_j\# p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
on Cantor space
$2^{\mathbb {N}}$
. On the p-measure one set C, this is the same as the probability measure on Cantor space which gives heads probability
$\theta (j)$
on each flip. Using the Greek letter
$\rho $
(rho), let
$\rho (\cdot \hspace {-.5mm}\mid \hspace {-.5mm}\upsilon )$
be the probability on Cantor space parameterized by
$\upsilon $
from
$[0,1]$
which gives heads probability
$\upsilon $
on each flip; then we have
$p_{j}(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })=\rho (\cdot \hspace {-.5mm}\mid \hspace {-.5mm} \theta (j))\cdot I_C(\theta )$
. Further one has the equation
$\mu _j((a,b)\times [\sigma ]) = \int _a^b \rho ([\sigma ]\hspace {-.5mm}\mid \hspace {-.5mm} \upsilon ) \; d p_j(\upsilon )$
, which says that the pushforward
$\mu _j$
is the same as the joint distribution given by prior
$p_j$
on
$[0,1]$
and likelihood
$\upsilon \mapsto \rho (\cdot \hspace {-.5mm}\mid \hspace {-.5mm}\upsilon )$
. We now finish the verification of condition (1) of Theorem 1.9. It suffices to show that for all
$(\theta , x)$
in
$\mathsf {SR}^{\mu }$
, one has that
$\lim _n f_n(x) = \theta $
. To show this is to show that, for all
$j\geq 0$
, we have
$\lim _n \pi _j(f_n(x)) = \pi _j(\theta )$
, which by the diagram is the same as
$\lim _n g_n(c_j(x))=\theta (j)$
. Let
$j\geq 0$
. Since
$(\theta , x)$
is in
$\mathsf {SR}^{\mu }$
and since
$\pi _j\otimes c_j:S\rightarrow [0,1]\times 2^{\mathbb {N}}$
is computable continuous, we have that
$(\theta (j), c_j(x))$
is in
$\mathsf {SR}^{\mu _j}$
. By Proposition 5.3 one has that
$\lim _n g_n(c_j(x))=\theta (j)$
, which is what we wanted to show.
The following proposition is a preliminary to our result on identifiability.
Proposition 7.10. Suppose that the parameter space
$\Omega $
is an effectively closed subset of a computable Polish space
$\widetilde {\Omega }$
, and that
$\Omega $
is a computable Polish subspace of
$\kern1.2pt\widetilde {\Omega }$
.
Define extended priors and likelihoods on the superspace
$\widetilde {\Omega }$
as follows:
-
(1) Define prior q on $\kern1.2pt\widetilde {\Omega }$
by pushforward
$q=\iota \# p$
, where
$\iota :\Omega \rightarrow \widetilde {\Omega }$
is the identity. -
(2) Define likelihood $\widetilde {\theta }\mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\widetilde {\theta }})$
on
$\widetilde {\Omega }$
by
$q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\widetilde {\theta }})=p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\widetilde {\theta }})\cdot I_{\Omega }(\widetilde {\theta })$
.
Suppose that conditions (1) and (2) of Theorem 1.9 hold with respect to
$\widetilde {\Omega }$
, q, and
$\widetilde {\theta }\mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\widetilde {\theta }})$
.
Then all
$\theta $
in
$\mathsf {SR}^{p}$
are computably consistent relative to
$\Omega $
, p, and
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
.
Proof. Note that
$\iota ^{-1}(A)=A\cap \Omega $
for all events
$A\subseteq \widetilde {\Omega }$
. Further note that
$\widetilde {\Omega }\setminus \Omega $
is a q-null c.e. open in
$\widetilde {\Omega }$
. If
$\nu $
denotes the new joint distribution on
$\widetilde {\Omega }\times \mathcal {X}$
with respect to the extended prior and likelihood, then for Borel events
$A\subseteq \widetilde {\Omega }$
and
$B\subseteq \mathcal {X}$
, one has
$\nu (A\times B)=\mu ((A\cap \Omega )\times B)$
. Further for Borel events
$A\subseteq \widetilde {\Omega }$
we have
$q(A \hspace {-1mm}\mid \hspace {-1mm} x_{n}) =p(A\cap \Omega \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
when the denominator is non-zero.
We show that for any version of the original likelihood
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )$
in the sense of Definition 1.6, there is a version of extended likelihood
$q(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot )$
in the sense of Definition 1.6 which agree on
$\mathsf {SR}^{q}$
. For, suppose that for a version of
$\theta \mapsto p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
we use the difference
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )-h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
, where
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot ), h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \cdot ):\Omega \rightarrow [0,\infty ]$
are
$L_1(p)$
Schnorr tests. Since
$\widetilde {\Omega }\setminus \Omega $
is c.e. open in
$\widetilde {\Omega }$
, we have that
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \widetilde {\theta }) + h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \widetilde {\theta }) \cdot I_{\widetilde {\Omega }\setminus \Omega }(\widetilde {\theta })$
and
$g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \widetilde {\theta })\cdot I_{\widetilde {\Omega }\setminus \Omega }(\widetilde {\theta }) + h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \widetilde {\theta })$
are
$L_1(q)$
Schnorr tests. Further, on
$\mathsf {SR}^{q}$
, their difference is equal to
$\big (g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \widetilde {\theta })-h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )\big )\big (1-I_{\widetilde {\Omega }\setminus \Omega }(\widetilde {\theta })\big ) =\big (g(\sigma \hspace {-1mm}\mid \hspace {-1mm} \widetilde {\theta })-h(\sigma \hspace {-1mm}\mid \hspace {-1mm} \widetilde {\theta })\big )(I_{\Omega }(\widetilde {\theta }))$
.
Suppose that
$\theta $
is in
$\mathsf {SR}^{p}$
. Since
$\iota :\Omega \rightarrow \widetilde {\Omega }$
is computable continuous, we have that
$\theta $
is in
$\mathsf {SR}^{q}$
. Then by Theorem 1.9 we have that
$\theta $
is computable consistent relative to
$\widetilde {\Omega }$
, q, and
$\widetilde {\theta }\mapsto q(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\widetilde {\theta }})$
. Let
$U\subseteq \Omega $
be c.e. open with
$p(U)$
computable. Since
$\Omega $
is a computable Polish subspace of
$\widetilde {\Omega }$
we have that
$U=\Omega \cap V$
, where V is a c.e. open in
$\widetilde {\Omega }$
. Then
$q(V)=p(\Omega \cap V)=p(U)$
is computable. Then
$\lim _n q(V \hspace {-1mm}\mid \hspace {-1mm} x_{n})=I_{V}(\theta )$
for
$q(\cdot \hspace {-1mm}\mid \hspace {-1mm} \theta )$
-many x in
$\mathcal {X}$
. Since
$\theta $
is in
$\Omega $
, we have
$\lim _n q(V \hspace {-1mm}\mid \hspace {-1mm} x_{n})=I_{V}(\theta )$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} \theta )$
-many x in
$\mathcal {X}$
. By the first paragraph, we have
$\lim _n p(V\cap \Omega \hspace {-1mm}\mid \hspace {-1mm} x_{n})=I_{V\cap \Omega }(\theta )$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} \theta )$
-many x in
$\mathcal {X}$
. By
$U=\Omega \cap V$
, we have that
$\lim _n p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})=I_{U}(\theta )$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} \theta )$
-many x in
$\mathcal {X}$
.
Here is then the proof of Proposition 1.11.
Proof. By passing to the image, we may assume that
$\Omega $
is an effectively closed subset of
$\widetilde {\Omega }:=\mathbb {S}_{\infty }$
, and that
$\Omega $
is a computable Polish subspace of
$\widetilde {\Omega }$
. Define the extended prior and likelihood relative to
$\widetilde {\Omega }$
as in the statement of Proposition 7.10. Then by Proposition 7.9 conditions (1) and (2) of Theorem 1.9 hold with respect to
$\widetilde {\Omega }$
and the extended prior and likelihood. Then we are done by Proposition 7.10.
8 Sufficient conditions for inconsistency
Up to this point in the article, we have been assuming that the prior p is a fixed computable probability measure on the parameter space
$\Omega $
, as in the effective assumptions (Definition 1.1(1)). To handle Freedman’s Inconsistency Theorem, we need to shift to considering the prior to be an arbitrary probability measure on the parameter space
$\Omega $
, that is, an arbitrary element of
$\mathcal {P}(\Omega )$
.
We note an elementary proposition, which makes salient a double integral.
Proposition 8.1. If
$(p, \theta )$
is classically consistent, then for all closed C not containing
$\theta $
one has
The same holds true if p is computable and
$(p,\theta )$
is computably consistent.
Proof. One uses the hypothesis and the “open set” form of Portmanteau to get that
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})\rightarrow \delta _{\theta }$
in
$\mathcal {P}(\Omega )$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
-a.s. many x in
$\mathcal {X}$
. Then one uses the “closed set” form of Portmanteau and the Dominated Convergence Theorem to get the conclusion.
The first step (the “open set” step) of the proof also works under the effective hypothesis since the c.e. opens with computable p-measure form a topological basis.
The following proposition provides a sufficient condition for inconsistency.
Proposition 8.2. Suppose that
$\Omega $
has at least two elements. Suppose that
$(p, \theta )$
is classically consistent. Then there is a c.e. open V such that
The same holds if p is computable and
$(p,\theta )$
is computably consistent.
Proof. Since
$\Omega $
has at least two elements, choose disjoint c.e. opens
$U,V$
such that
$\theta $
is in U. Let
$C=\Omega \setminus U$
be the relative complement, so that
$V\subseteq C$
. Since
$(p, \theta )$
is classically consistent, by Proposition 8.1 we have
$\lim _n \int _{\mathcal {X}} p(C \hspace {-1mm}\mid \hspace {-1mm} x_{n}) \; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })(x)=0$
. Since
$V\subseteq C$
we have
$\lim _n \int _{\mathcal {X}} p(V \hspace {-1mm}\mid \hspace {-1mm} x_{n}) \; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })(x)=0$
. From this the conclusion follows easily.
Now we discuss how to ensure that this double integral is lsc.
Proposition 8.3 (Integrals lsc as function of prior, when likelihood comp. cont.).
Suppose the likelihood
$\theta \mapsto p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })$
maps from
$\Omega $
into
$\mathcal {P}(\mathcal {X})$
. Suppose further the likelihood
$\theta \mapsto p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is computable continuous, uniformly in
$\sigma $
from T. Then:
-
(1) The map $(p,x)\mapsto p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
is lsc from
$\mathcal {P}(\Omega )\times \mathcal {X}$
to
$[0,1]$
, uniformly in c.e. open
$U\subseteq \Omega $
and
$n\geq 0$
. -
(2) The map $(p,x)\mapsto \int _{\Omega } f(\theta ^{\prime }) \; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})(\theta ^{\prime })$
is lsc from
$\mathcal {P}(\Omega )\times \mathcal {X}$
to
$[0,\infty ]$
, uniformly in
$n\geq 0$
and lsc
$f:\Omega \rightarrow [0,\infty ]$
. -
(3) The map $(p, \theta )\mapsto \int _{\mathcal {X}} g(p,x^{\prime }) \; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })(x^{\prime })$
is lsc from
$\mathcal {P}(\Omega )\times \Omega $
to
$[0,\infty ]$
, uniformly in lsc
$g:\mathcal {P}(\Omega )\times \mathcal {X}\rightarrow [0,\infty ]$
. -
(4) The map $(p, \theta )\mapsto \int _{\mathcal {X}} \int _{\Omega } f(\theta ^{\prime }) \; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})(\theta ^{\prime }) \;dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })(x)$
is lsc from
$\mathcal {P}(\Omega )\times \Omega $
to
$[0,\infty ]$
, uniformly in
$n\geq 0$
and lsc
$f:\Omega \rightarrow [0,\infty ]$
.
Note that (1) improves Proposition 3.7, since the latter just concerned a fixed computable prior.
Proof. For (1), the hypothesis of the proposition implies that uniformly in c.e. open
$U\subseteq \Omega $
and
$\sigma $
in T, one has that
$\theta \mapsto I_U(\theta ) \cdot p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is lsc from
$\Omega $
to
$[0,1]$
. Then uniformly in c.e. open
$U\subseteq \Omega $
and
$\sigma $
in T one has that
$p\mapsto \int _U p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )$
is lsc from
$\mathcal {P}(\Omega )$
to
$[0,1]$
, by Hoyrup and Rojas [Reference Hoyrup and Rojas13, Proposition 4.3.1].
Further, the hypothesis of the proposition implies that
$\theta \mapsto (1- p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ))$
is also lsc from
$\Omega $
to
$[0,1]$
, uniformly in
$\sigma $
from T. Then uniformly in
$\sigma $
from T one has that
$p\mapsto \int _{\Omega } (1-p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )) \; dp(\theta )$
is lsc from
$\mathcal {P}(\Omega )$
to
$[0,1]$
by [Reference Hoyrup and Rojas13, Proposition 4.3.1]. Hence, uniformly in
$\sigma $
from T one has that
$p\mapsto \int _{\Omega } p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )$
is usc (i.e., upper semi-computable) from
$\mathcal {P}(\Omega )$
to
$[0,1]$
. By the previous paragraph with
$U=\Omega $
, one has that uniformly in
$\sigma $
from T the map
$p\mapsto \int _{\Omega } p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )$
is computable continuous from
$\mathcal {P}(\Omega )$
to
$[0,1]$
.
To finish (1), it suffices to note by the definition of the posterior
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})$
in (1.2) that for rational
$q>0$
we have
$p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})>q$
iff there is
$\sigma $
in T of length n with x in
$[\sigma ]$
and there is rational
$r>0$
such that
$\int _U p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )>r$
and
${0< \int _{\Omega } p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) \; dp(\theta )<\frac {r}{q}}$
. These inequalities are c.e. open conditions in
$(p,x)$
by the previous two paragraphs.
For (2), any lsc function
$f:\Omega \rightarrow [0,\infty ]$
can be written as
$f=\sup _s f_s$
, where
$f_s$
is a computable sequence of simple functions of the form
$\sum _{i=1}^{n_s} q_{s,i}\cdot I_{U_{s,i}}$
, where
$n_s\geq 0$
is computable and
$q_{s,i}$
is a computable sequence of non-negative rationals and
$U_{s,i}$
is a computable sequence of c.e. opens in
$\Omega $
(cf. proof of Hoyrup and Rojas [Reference Hoyrup and Rojas13, Proposition 4.3.1]). Hence one has
$\int _{\Omega } f(\theta ^{\prime }) \; dp(\cdot \mid x_{n})(\theta ^{\prime })>r $
iff there is
$s\geq 0$
such that
$\sum _{i=1}^{n_s} q_{s,i} \cdot p(U_{s,i} \hspace {-1mm}\mid \hspace {-1mm} x_{n})>r$
, which is a c.e. open condition in
$(p,x)$
by (1).
For (3), any lsc
$g:\mathcal {P}(\Omega )\times \mathcal {X}\rightarrow [0,\infty ]$
can be written as
$g=\sup _s g_s$
, where
$g_s$
is a computable sequence of simple functions of the form
$\sum _{i=1}^{n_s} q_{s,i}\cdot I_{W_{s,i}}$
, where
$n_s\geq 0$
is computable and
$q_{s,i}$
is a computable sequence of non-negative rationals and
$W_{s,i}$
is a computable sequence of c.e. opens in
$\mathcal {P}(\Omega )\times \mathcal {X}$
(cf. again proof of Hoyrup and Rojas [Reference Hoyrup and Rojas13, Proposition 4.3.1])). Let
$W_{s,i}=\bigcup _{j} (U_{s,i,j}\times V_{s,i,j})$
for some computable sequences
$U_{s,i,j}$
of c.e. opens in
$\mathcal {P}(\Omega )$
and
$V_{s,i,j}$
of c.e. opens in
$\mathcal {X}$
. Then one has that
$\int _{\mathcal {X}} g(p,x^{\prime }) \; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta })(x^{\prime })>r$
iff there is
$s\geq 0$
and there are finite sets of natural numbers
$\{J_{s,i}: i<n_s\}$
such that p is in
$\bigcap _{i<n_s} \bigcap _{j\in J_{s,i}} U_{s,i,j}$
and
$\sum _{i=1}^{n_s} q_{s,i} \cdot p(\bigcup _{j\in J_{s,i}} V_{s,i,j} \hspace {-1mm}\mid \hspace {-1mm} \theta )>r$
, which is a c.e. open condition in
$(p,\theta )$
.
Finally, (4) follows directly from (2) and (3).
9 Effective Freedman Inconsistency
We begin with the proof of Theorem 1.14, which effectivizes the proof in [Reference Ghosal and van der Vaart10, Theorem 6.12]. In this theorem, recall that the parameter space
$\Omega $
is
$\mathbb {S}_{\infty }$
, and so we just write
$\mathbb {S}_{\infty }$
instead of
$\Omega $
in this section.
Note that we have the formula
$p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta ) = \prod _{i<\left |\sigma \right |} \theta (\sigma (i))$
for the likelihood, and that
$\theta \mapsto \prod _{i<\left |\sigma \right |} \theta (\sigma (i))$
is computable continuous from
$[0,1]^{\mathbb {N}}$
to
$[0,1]$
. By Proposition 7.3 and Remark 7.2(1), we have that
$\theta \mapsto p(\sigma \hspace {-1mm}\mid \hspace {-1mm} \theta )$
is computable continuous from
$\mathbb {S}_{\infty }$
to
$[0,1]$
, which puts us in a position to later apply Proposition 8.3.
Further,
$\mathbb {S}_{\infty }^+:=\{\theta \in \mathbb {S}_{\infty } : \forall \; i\geq 0\; \theta (i)>0\}$
is effectively comeager in
$\mathbb {S}_{\infty }$
. For, the set
$\{\theta \in \mathbb {S}_{\infty } : \theta (i)>0\}$
is c.e. open and dense for each
$i\geq 0$
: for, it is
$\pi _i^{-1}((0,1])$
, where
$\pi _i:\mathbb {S}_{\infty }\rightarrow [0,1]$
is the projection map, which is computable continuous and computable open by Proposition 7.8. By the Baire Category Theorem,
$\mathbb {S}_{\infty }^+$
is dense in
$\mathbb {S}_{\infty }$
. By the Effective Baire Category Theorem [Reference Hoyrup and Rojas13, Theorem 5.1.2] we can find a uniformly computable sequence of points
$\theta _0^+, \theta _1^+, \ldots $
in
$\mathbb {S}_{\infty }^+$
which is dense in
$\mathbb {S}_{\infty }$
.
Further,
$\mathbb {S}_{\infty }^-=\{\theta \in \mathbb {S}_{\infty } : \exists \; i\geq 0\; \theta (i)=0\}$
is dense since it is a superset of the countable dense set.
Finally, let
$D_i$
be the set of
$\sum _{j=1}^m q_j \delta _{\theta _j}$
, where
$0<q_j<1$
is rational and
${\sum _{j=1}^m q_j =1}$
, and
$\theta _1=\theta ^+_i$
and for
$1<j\leq m$
one has that
$\theta _j$
is from
$\mathbb {S}_{\infty }^-$
. It is not hard to see, by a simple classical argument, that this is dense in
$\mathcal {P}(\mathbb {S}_{\infty })$
.
Then one has the following proposition.
Proposition 9.1. Let
$\theta _0$
be from
$\mathbb {S}_{\infty }^+$
. Let p be from
$D_i$
. Then for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})$
-a.s. many x in
$\mathcal {X}$
one has
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})\rightarrow \delta _{\theta _i^+}$
in
$\mathcal {P}(\mathbb {S}_{\infty })$
.
Proof. Let
$p = \sum _{j=1}^m q_j \delta _{\theta _j}$
, where this is as in the definition of
$D_i$
in the previous paragraph. The non-trivial case is
$m>1$
. For each
$1<j\leq m$
, choose
$k_j\geq 0$
with
$\theta _j(k_j)=0$
. Let
$\epsilon = \prod _{j=2}^m \theta _0(k_j)$
, which since
$\theta _0$
is in
$\mathbb {S}_{\infty }^+$
is necessarily
$>0$
. Let
$A=\bigcap _{\ell } A_{\ell }$
, where
$A_{\ell } = \{ x\in \mathcal {X}: \forall \; 1<j\leq m \; \exists \; n\geq \ell \; x(n)=k_j\}$
. Then A is a tailset, that is, if
$x, x^{\prime }$
differ by finitely much and one is in A, then the other is as well. Since
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})$
is a product measure, by the zero-one law, A either has measure zero or one. We claim it has measure one. If not, then it has measure zero. Then choose sufficiently large
$\ell \geq 0$
such that
$p(A_{\ell } \hspace {-1mm}\mid \hspace {-1mm} \theta _0)<\epsilon $
. Consider the event
$B=\{x: x(\ell +2)=k_2, x(\ell +3)=k_3, \ldots , x(\ell +m)=k_m\}$
. Then by independence we have
$p(B \hspace {-1mm}\mid \hspace {-1mm} \theta _0)= \prod _{j=2}^m \theta _0(k_j)$
, which is
$\epsilon $
, a contradiction to B being a subset of
$A_{\ell }$
. Hence rather we have that
$p(A \hspace {-1mm}\mid \hspace {-1mm} \theta _0)=1$
.
By Portmanteau it suffices to check that for all x in A and all open U containing
$\theta _1$
one has
$\liminf _n p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})=1$
. Suppose not. Then there is x in A and open U containing
$\theta _1$
such that
$\liminf _n p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})< 1$
. Then there is
$\ell _0\geq 0$
such that for all
$\ell \geq \ell _0$
one has
$\inf _{n\geq \ell } p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})< 1$
. Since x is in
$A_{\ell _0}$
, choose
$\ell>\ell _0$
such that for all
$1<j\leq m$
one has that
$x(\ell ^{\prime })=k_j$
for some
$\ell ^{\prime }$
in the interval
$[\ell _0, \ell )$
. This has the consequence that for all
$n\geq \ell $
one has that
$p(x_{n} \hspace {-1mm}\mid \hspace {-1mm} \theta _j)=0$
for all
$1<j\leq m$
. Hence for any
$n\geq \ell $
and any event C, we have that
When we set
$C=\mathbb {S}_{\infty }$
, then we get a value
$>0$
since
$\theta _i^+$
comes from
$\mathbb {S}_{\infty }^+$
. For
$n\geq \ell $
:
and this contradicts that
$\inf _{n\geq \ell } p(U \hspace {-1mm}\mid \hspace {-1mm} x_{n})< 1$
.
For each
$i\geq 0$
and each
$\theta _i^+$
on our list, we construct a uniformly computable continuous sequence
$\chi _{i,k}: \mathbb {S}_{\infty } \rightarrow [0,1]$
such that (i)
$\chi _{i,k}(\theta _i^+)=1$
for all
$k\geq 0$
and (ii) if
$\theta ^+_i$
is in the open V, then eventually
$\chi _{i,k}\leq I_V$
. Let
$\epsilon _k\rightarrow 0$
be a computable sequence of positive rationals which is strictly decreasing. Let
$V_{i,k} = B(\theta ^+_i, \epsilon _k)$
, noting that
$\overline {V_{i,k+1}}\subseteq B[\theta ^+_i, \epsilon _{k+1}]\subseteq B(\theta ^+_i, \epsilon _k)=V_{i,k}$
. Since
$B[\theta ^+_i, \epsilon _{k+1}]$
and
$\mathbb {S}_{\infty } \setminus B(\theta ^+_i, \epsilon _k)$
are effectively closed and disjoint, by the effective Urysohn Lemma [Reference Simpson29, Lemmas II.7(1) and II.7.3, p. 90] there are uniformly computable continuous
$\chi _{i,k}: \mathbb {S}_{\infty } \rightarrow [0,1]$
such that
$\chi _{i,k}\upharpoonright B[\theta ^+_i, \epsilon _{k+1}]=1$
and
$\chi _{i,k}\upharpoonright (\mathbb {S}_{\infty } \setminus B(\theta ^+_i, \epsilon _k))=0$
. Then
$\chi _{i,k}\upharpoonright V_{i,k+1}=1$
and
$\chi _{i,k}\upharpoonright (\mathbb {S}_{\infty } \setminus V_{i,k})=0$
.
By Proposition 8.3(4), the following are c.e. opens, where
$0<\epsilon <1$
is rational:
We show that each of these c.e. opens is dense in
$\mathcal {P}(\mathbb {S}_{\infty })\times \mathbb {S}_{\infty }$
. Suppose that a pair in
$\mathcal {P}(\mathbb {S}_{\infty })\times \mathbb {S}_{\infty }$
is given. Find
$(p, \theta _0)$
in
$D_i\times \mathbb {S}_{\infty }^+$
arbitrarily close to the given pair. We argue that the pair
$(p, \theta _0)$
is in
$U_{i,\epsilon ,m,k}$
. By Proposition 9.1, one has that
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})\rightarrow \delta _{\theta _i^+}$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})$
-a.s. many x in
$\mathcal {X}$
. By the Portmanteau Theorem, we have
$\int _{\mathbb {S}_{\infty }} \chi _{i,k}(\theta ^{\prime })\; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})(\theta ^{\prime })\rightarrow \int _{\mathbb {S}_{\infty }} \chi _{i,k}(\theta ^{\prime })\;d\delta _{\theta _i^+}(\theta ^{\prime })$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})$
-a.s. many x in
$\mathcal {X}$
. By property (i) of
$\chi _{i,k}$
above, we have
$\int _{\mathbb {S}_{\infty }} \chi _{i,k}(\theta ^{\prime })\;d\delta _{\theta _i^+}(\theta ^{\prime })= \chi _{i,k}(\theta _i^+)=1$
. Hence
$\int _{\mathbb {S}_{\infty }} \chi _{i,k}(\theta ^{\prime })\; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})(\theta ^{\prime })\rightarrow 1$
for
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})$
-a.s. many x in
$\mathcal {X}$
. By applying the Dominated Convergence Theorem with respect to
$p(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})$
, we have
$\lim _n \int _{\mathcal {X}} \int _{\mathbb {S}_{\infty }} \chi _{i,k}(\theta ^{\prime })\; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} x_{n})(\theta ^{\prime }) \; dp(\cdot \hspace {-1mm}\mid \hspace {-1mm} {\theta _0})(x)=1$
. This then puts the pair
$(p, \theta _0)$
in
$U_{i,\epsilon ,m,k}$
. Thus indeed
$U_{i,\epsilon ,m,k}$
is dense.
Let
$V\subseteq \mathbb {S}_{\infty }$
be non-empty c.e. open, let
$0<\epsilon <1$
be rational and consider
Suppose
$(p,\theta )$
is in
$\bigcap _{i,\epsilon , m,k} U_{i,\epsilon , m,k}$
. We argue that
$(p,\theta )$
is in the set in (9.2). Since V is non-empty, there is
$\theta ^+_i$
in V. Then by property (ii) of
$\chi _{i,k}$
above we have that there is
$k_0\geq 0$
such that for all
$k\geq k_0$
we have
$\chi _{i,k}\leq I_V$
. Then we are done since
$(p, \theta )$
is in
$U_{i,\epsilon , m,k}$
for all
$m\geq 0$
and all
$k\geq k_0$
.
Then by Proposition 8.2, we have finished proving Theorem 1.14.
It remains to prove Corollary 1.15. By the Effective Baire Category Theorem [Reference Hoyrup and Rojas13, Theorem 5.1.2] and Theorem 1.14, in any non-empty c.e. open in
$\mathcal {P}(\mathbb {S}_{\infty })\times \mathbb {S}_{\infty }$
we can find a computable point
$(p, \theta _0)$
in
$\bigcap _{i,\epsilon , m,k} U_{i,\epsilon , m,k}$
. Using the end of the proof of Theorem 1.14, starting at (9.2), together with Proposition 8.2 specialized to the computable prior p, we further get that
$(p, \theta _0)$
is not computably consistent. Hence we have finished part (1). And part (2) follows from Proposition 7.9 and Theorem 1.9.
Acknowledgments
Thanks to the referees and editors for the helpful comments and feedback. Thanks also to the members of the Department of Philosophy and the Department of Statistics at the University of Toronto for the stimulating questions and discussion.
Funding
The authors received no external funding for this work.


