1. Introduction

The so-called principle of regularity is the doctrine that only necessary truths and falsehoods should have the extreme probabilities 1 and 0, respectively. Within the field of the standard real numbers, outcome spaces of the size of the continuum like the closed unit interval pose an obvious problem for the principle because there can be at most countably many disjoint events of positive (standard) probability. But the hyperreal numbers, a field extension of the real numbers, seem to hold out the promise of overcoming this objection by allowing all those events to take positive infinitesimal probabilities.Footnote ¹ Indeed a well-known result of Bernstein and Wattenberg (Reference Bernstein, Wattenberg and Luxemburg1969) shows that there is a measure that assigns positive infinitesimals over the points of the standard unit interval and differs by an infinitesimal from Lebesgue measure on the Lebesgue-measurable sets. The unit interval can be represented by the set of all infinite sequences of zeros and ones, the usual mathematical model of the ‘experiment’ of tossing a coin infinitely many times.Footnote ² However, in a well-known paper Timothy Williamson (Reference Williamson2007) claimed to show, using just that example, that the appeal to hyperreal probabilities cannot save regularity.Footnote ³

Given that a familiar way of defining Lebesgue measure on the measurable subsets of [0, 1] employs the model of infinite independent sequences of tosses of a fair coin, Bernstein and Wattenberg’s result described earlier might seem to refute Williamson avant la lettre, while in a paper published coincidentally in the same year as Williamson’s, Frederik Herzberg (Reference Herzberg2007) showed how to assign positive infinitesimal probabilities to independent infinite sequences of tosses of a coin with an arbitrary fixed probability of landing heads.Footnote ⁴ But Williamson’s argument has garnered some support in the literature (e.g., Hájek Reference Hájek2017), and in what follows I present the argument, argue that what appears to be a crucial premise that two relevant events are isomorphic fails, and show that nevertheless the events must have the same probability. Indeed, this is actually a theorem about a type of Bernoulli process. However, the theorem holds for standard-valued probabilities only, and it is not correct if they can take infinitesimal values. I will conclude by observing that what Williamson’s coin-tossing example genuinely does show is that the hyperreals are not merely a device for simplifying and thereby solving standard proofs in mathematics,Footnote ⁵ as is often claimed even by their advocates, but that they can make real discriminations that are invisible when viewed through the lens of the real numbers alone.

2. Williamson’s Argument

A coin is flipped infinitely many times. Suppose H(1…) represents the event ‘all the outcomes are heads’, and H(2…) ‘all the outcomes after the first are heads’ (Williamson Reference Williamson2007, 4; his notation). Then according to Williamson, “H(1…) and H(2…) are isomorphic events. More precisely, we can map the constituent single-toss events of H(1…) one-one onto the constituent single-toss events of H(2…) in a natural way that preserves the physical structure of the set-up just by mapping each toss to its successor. H(1…) and H(2…) are events of exactly the same qualitative type; they differ only in the inconsequential respect that H(2…) starts one second after H(1…). That H(2…) is preceded by another toss is irrelevant, given the independence of the tosses. Thus H(1…) and H(2…) should have the same probability” (5).

From the last quoted sentence we infer that the common probability of H(1…) and H(2…) must be 0. For

(1) $\begin{array}{}& P\left(H\left(1\dots \right)\right)=P\left(H\left(2\dots \right)|H\left(1\right)\right)P\left(H\left(1\right)\right),\end{array}$

where H(1) is the event ‘the first toss lands heads’. So by independence

$P\left(H\left(1\dots \right)\right)=P\left(H\left(2\dots \right)\right)P\left(H\left(1\right)\right)\mathrm{.}$

That is,

$P\left(H\left(1\dots \right)\right)=(1/2)P\left(H\left(2\dots \right)\right)\mathrm{.}$

But because of the presumed isomorphism of H(1…) and H(2…), we have

$P\left(H\left(1\dots \right)\right)=P\left(H\left(2\dots \right)\right)\text{,}$

hence

$2P\left(H\left(1\dots \right)\right)=P\left(H\left(1\dots \right)\right)\text{,}$

and so, noting that infinitesimals obey the same field rules as the real numbers themselves, we must have $P\left(H\left(1\dots \right)\right)=0$, contradicting regularity.

3. Why the Argument Fails

Williamson’s argument raises several questions. Most prominently, it is not clear what his claim that the ‘events’ H(1…) and H(2 … .) are isomorphic can mean. Events, according to Kolmogorovian orthodoxy, are (measurable) subsets of an outcome space. Since Williamson’s experiment consists in observing the outcomes of a coin flipped infinitely many times, the outcome space can be represented by the uncountably infinite set of all infinite sequences of ones and zeroes (1 for heads, 0 for tails). The event H(1…), ‘all the outcomes are heads’, defined in that space is clearly the singleton {x}, where x is the sequence all of which are ones, and the event H(2…), ‘all the outcomes after the first are heads’, is equally clearly the pair {x, y}, where y is like x except that its first member is zero. But these two events H(1…) and H(2…) are two sets without structure, one of which has two members and the other just one, and for unstructured sets isomorphism reduces to merely having the same cardinality. Howson (Reference Howson2017) pointed this out and concluded that no appeal to isomorphism can justify inferring the equiprobability of H(1…) and H(2…).

It seems that Williamson is simply overlooking his own description of H(2…) as the event ‘all tosses after the first land heads’, defined in the sample space just described, and hence as a disjunctive event in that space and taking it, incorrectly, to be a (singleton) sequence of events in its own right.Footnote ⁶ Such a view of H(2…) does seem implicit in his claim that “we can map the constituent single-toss events of H(1…) one-one onto the constituent single-toss events of H(2…) in a natural way that preserves the physical structure of the set-up just by mapping each toss to its successor” (Williamson Reference Williamson2007, 5, my emphasis). But in that case how does Williamson reconcile a view of H(2…) as a singleton sequence with the step $P\left(H\left(1\dots \right)\right)=\left(1/2\right)P\left(H\left(2\dots \right)\right)$? According to Benci, Horsten, and Wenmackers (Reference Benci, Horsten and Wenmackers2018, 528–30), Williamson is simply conflating two distinct experiments, or ‘models’, as they call them, one of which (their model A) records the outcomes of an infinite sequence of coin tosses starting at some given moment, and the other (model B) records as its nth outcome the n + 1th outcome as recorded by A. In model B, H(2…) is indeed now a singleton sequence containing ones only, thereby identical to H(1…) in model A and so justifying the identity $P_A\left(H\left(1\dots \right)\right)=P_B\left(H\left(2\dots \right)\right)$. But, Benci et al. continue, “[Williamson] glosses this as $\mathrm{Prob}\left(H\left(1\dots \right)\right)=\mathrm{Prob}\left(H\left(2\dots \right)\right)$, thus turning the probabilities involved into evaluations within the same model” (529).

The problem with Benci et al.’s account is that Williamson actually does distinguish their model A and model B, even if not in those precise terms, and still insists that $P\left(H\left(1\dots \right)\right)=P\left(H\left(2\dots \right)\right)$. For in an alternative version of his argument, he considers two spatiotemporally distinct sequences of tosses of two ‘qualitatively identical’ coins, both yielding the outcomes ‘all heads’. One of those sequences (clearly corresponding to Benci et al.’s model B) starts at the second toss of the other coin (corresponding to their model A): “suppose that another fair coin, qualitatively identical with the first, will also be tossed infinitely many times at one second intervals, starting at the same time as the second toss of the first coin, all tosses being independent. Let H*(1…) be the event that every toss of the second coin comes up heads, and H*(2…) the event that every toss after the first of the second coin comes up heads” (Williamson Reference Williamson2007, 6).

Thus, in this version of Williamson’s argument Benci et al.’s P_B(H(2…)) becomes Williamson’s P(H*(1…)). And while Williamson does indeed ‘exploit the intuition that ${\mathrm{Prob}}_A\left(H\left(1\dots \right)\right)={\mathrm{Prob}}_B\left(H\left(2\dots \right)\right)$’, he does not ‘gloss’ it as $\mathrm{Prob}\left(H\left(1\dots \right)\right)=\mathrm{Prob}\left(H\left(2\dots \right)\right)$. Instead, he renders it in his own notation as “H(1…) and H*(1…) should be equiprobable, because the probability that a coin comes up heads on every toss does not depend on when one starts tossing, and there is no qualitative difference between the coins” (Williamson Reference Williamson2007, 5).Footnote ⁷ Williamson’s equating Prob(H(1…)) and Prob(H(2…)) arises not from any unconscious conflation of two models, therefore, but from a belief, as the following quotation makes quite clear, that H(2…) is an infinite sequence in the same sample space as H(1…), structurally identical to H*(1…) and hence—given his immediately preceding observation—to H(1…): “But for the same reason H*(1…) and H(2…) should also be equiprobable. These two infinite sequences of tosses proceed in parallel, synchronically, and there is no qualitative difference between the coins; in particular, that the first coin will be tossed once before the H(2…) sequence begins is irrelevant.Footnote ⁸ By transitivity, H(1…) and H(2…) should be equiprobable: [hence] Prob(H(1…)) = Prob(H(2…))” (Reference Williamson2007, 5, my emphasis).

But surely, it might be said in defense of Williamson, H(2…) does have a straightforward representation as a single sequence of events, one sanctioned by formal probability theory itself. For where X_i is a two-valued random variable defined on $X={\left\{0,1\right\}}^N$ (usually written 2^N), and $N=\left\{1,2,3,\dots \right\}$, with $X_i\left(x\right)=x_i$, $i=1$, 2, …, H(2…) is the sequence 〈 $X_2=1$, $X_3=1$, …, $X_{n+1}=1$, …〉, a sequence moreover that clearly is a subsequence of 〈 $X_1=1$, $X_2=1$, …, $X_n=1$, …〉, that is, of H(1…). Thus, the successor function $f:N\to N$, $f\left(i\right)=i+1$, which sends ‘ $X_i=1$’ to ‘ $X_{f\left(i\right)}=1$’, really does appear to induce an isomorphic embedding of H(2…) into H(1…). So is Williamson right after all?

No. 〈 $X_1=1$, $X_2=1$, …, $X_n=1$, …〉 is not a sequence but a conjunction of formulas ‘ $X_i=1$’, signified by the fact that in the set-theoretical language of measure theory it is represented as the set-theoretical intersection of all sets of the form ${\{x:X}_i\left(x\right)=1\}$.Footnote ⁹ Correspondingly, the formal-logical representation of H(2…) is a disjunction of conjunctions of atomic sentences ‘ $X_i=1$’, $i=1$, 2, …, in the minimal language of the experiment: it is ‘ ${(X}_1=1{\&Y)\vee (X}_1=0\&Y)$’, where Y is ‘ $X_2=1$ & $X_3=1$ & … & $X_{n+1}=1$ & …’. By contrast, H(1…) is a simple conjunction of atomic items. “But there are no such things as infinite conjunctions” I hear someone say. In fact there are, and they are well-behaved items of so-called infinitary languages extending first-order logic, obeying the de Morgan’s laws, and so on.Footnote ¹⁰ The ordinary language of set theory makes these languages redundant, at least for measure-theoretic applications, which is why they are rarely if ever employed in those contexts; instead, countably infinite intersections and unions do routine service in defining infinitary events.

4. The Argument Redux

At the end of their discussion of Williamson’s argument, Benci et al. remark that it “crucially turns on translation symmetry” (Reference Benci, Horsten and Wenmackers2018, 530). I assume (they do not elaborate) that they are referring to Williamson’s seeming beliefs that H(2…) is just the sequence H(1…) shifted, or translated, forward one place and that such a shift must therefore preserve probabilities. Williamson’s argument fails because H(2…) is not the sequence H(1…) translated, but there is something to it nonetheless. As we will now see there is a translation relating the two events, which can be shown to preserve probabilities without appealing to any spurious assumption of isomorphism and without conflating distinct probability models. On the contrary, only one is involved, called a one-sided Bernoulli process with probability p, typically seen as modeling an infinite sequence of coin tosses under the assumptions that (i) the tosses are independent and (ii), one might say in Williamsonese, ‘qualitatively identical’ in that there is exactly the same probability p of a head at each toss however far along the sequence one goes (the conditions i and ii are often abbreviated to ‘iid’, for ‘independent, identically distributed’, and they are exactly the formal conditions appealed to by Williamson in his argument). In the current case the parameter p is of course equal to 1/2.

Formally, we are talking about a single Kolmogorovian probability space (X, F, P), where $X=2^N={\left\{0,1\right\}}^N$, the set of all denumerably infinite sequences of zeroes and ones; F is the class of events defined in X, a σ-algebra of subsets of X generated by the cylinder sets (so all singleton sequences are events); and P is a product measure.Footnote ¹¹ The distinctive additional feature of such a Bernoulli process is a map T from X to X, the shift, or left shift, which moves each individual outcome in any sequence $x\in X$ one place to the left.Footnote ¹² It is easily proved that the shift operation is probability preserving: that is, for any event C in F, $P\left(C\right)=P\left(T^{-1}C\right)$. But this implies that $P\left(H\left(1\dots \right)\right)=P\left(H\left(2\dots \right)\right)$. For T(0, 1, 1, …), that is, the shift of the sequence zero followed by all ones, is the sequence (1, 1, 1, …, 1, …), that is, the singleton sequence in the extension of H(1…), and the shift of that sequence is itself. Hence, T⁻¹(1, 1, …, 1, …) is the pair {(0, 1, 1, …, 1, …), (1, 1, 1, …, 1, …)}, that is, H(2…). Hence, by the measure-preservation property of T we have $P\left(H\left(2\dots \right)\right)=P\left(H\left(1\dots \right)\right)$.Footnote ¹³ So we seem to have secured Williamson’s premise that $P\left(H\left(1\dots \right)\right)=P\left(H\left(2\dots \right)\right)$ as a theorem about a single probability space, appealing only to the iid properties of Bernoulli processes, with no appeal to any spurious assumption that H(1…) and H(2…) are isomorphic.

QED for Williamson? Not at all. What his argument reveals is that measure invariance is an artifact of standard probabilities where infinitary events like H(1…) and H(2…) are concerned, since all infinite sequences with probability 1/2 of heads have limiting standard probability 0, with that of H(2…) being the sum of two zeros. By contrast, if P can take values in the hyperreal unit interval, as finer measures of possibility than the reals, then those probabilities can quite consistently be infinitesimal and positive. Indeed, the sequence ${\left(2^{-n}\right)}_{0\le n}$ itself, factored by some nonprincipal ultrafilter on N, would be as good as any other infinitesimal value of the probability of H(1…). But while there may be indeterminacy of specific values for P(H(1…)) and P(H(2…), the relation between them is not at all indeterminate.Footnote ¹⁴ For Williamson’s own proof tells us that whatever infinitesimal probability value H(2…) is assigned, it has to be twice that of H(1…), and if we like we can even construe this as actually measuring the infinitesimal difference between the extensions of H(1…) and H(2…), that is, between the singleton sequence in X consisting of ones only and the pair of sequences one of which is the sequence consisting of ones only and the other is the same sequence but starting with a zero instead of a one.

But allowing infinitesimal-valued probabilities does seem to entail violations of what have hitherto been thought of as fundamental symmetry principles,Footnote ¹⁵ not just measure invariance under shifts in Bernoulli processes (although of course all finite sequences remain invariant) but also translation invariance of Lebesgue measure on the measurable subsets of [0, 1], rotational invariance on the circle, and so on. If one accepts hyperreals as legitimate entities in their own right, however, and not merely Leibnizian ‘ideal elements’ simplifying standard definitions and proofs,Footnote ¹⁶ then those classical results do become an artifact of restricting values to the real number system. But, to the extent to which the real and complex numbers remain the working tool of mathematical physics, those principles are unlikely to be dethroned.

5. Conclusion

Not only can regularity be consistently maintained in the context of Williamson’s example, but we see that the hyperreals can provide a finer analysis of what happens at standard limit points, where the standard lens fails to reveal fine enough differences of structure. That H(2…) is a composite event in the domain of P while H(1…) is a singleton sequence is a real, if infinitesimal, difference between them, which the hyperreals capture and which the standard reals cannot, blurring as they do the probabilities of all the singleton sequences as the zero limit of (2⁻ⁿ).Footnote ¹⁷