When participants are asked to reason about statistical data, they tend to ignore base rates. But there is a problem with the experiments that the target authors do not address. A probability is a mathematical abstraction and cannot be presented as a stimulus (though it can be realised as a property of an otherwise random sequence of stimuli). The research reviewed in the target article substitutes values for probabilities and presents participants with exercises in mental arithmetic. ”Base-rate neglect” might therefore be either the result of a failure to understand Bayes' Theorem, or due to insufficient ability in mental arithmetic. The authors do not enquire which.
The same question can be presented at different levels of difficulty. The mammography example from Gigerenzer and Hoffrage (Reference Gigerenzer and Hoffrage1995), in probability format, requires participants to multiply 0.01 by 0.8 and (1-0.01) by 0.096 and then compare the two. In the frequency version, participants merely have to compare 8 with the sum of 8 and 95. It is not surprising that the latter version elicited a greater number of correct (Bayesian) answers. The results summarised in Table 3 of the target article suggest that a substantial proportion of incorrect answers are consequent on difficulties in mental arithmetic. Performance needs to be related to ability in mental calculation. But participants have been uniformly drawn from university populations and the results lack generality. Except for Brase et al.'s (Reference Brase, Fiddick and Harries2006) study, the matter of participants' prior education has been ignored.
The “probability” problem can be circumvented in two different ways. First, as in gambling: Gamblers – not gamblers doing mental arithmetic, not even in a casino (Lichtenstein & Slovic Reference Lichtenstein and Slovic1973), but real gamblers chancing their own real money – are sensitive to “base rates” that do not even exist! ”Roulette players believe that certain numbers are due, when they have not come up for a long time” (Wagenaar Reference Wagenaar1988, p. 112). Gamblers do not reason rationally, else there would be no bookmakers or casinos in business. Moreover, the assessment of probability divides into at most five categories (Laming Reference Laming2004, Ch. 16).
The alternative is to realise probabilities as the relative frequencies of different stimuli in an otherwise random sequence. Figure 1 reproduces signal-detection data from Tanner et al. (Reference Tanner, Swets and Green1956). Different proportions of signal (SN) trials lead to different probabilities of detections and false-positives. Two-choice reaction times exhibit a similar phenomenon. The mean reaction times to two signals are different; the more frequent signal elicits a systematically faster response (see Laming Reference Laming1968, Fig. 5.2). In 1994 the Public Health Laboratory Service (PHLS) in England introduced a saliva test for rubella. When a doctor notified the PHLS of a diagnosis, a kit was sent, and the swab, when returned, was tested for the disease. Figure 2 (right hand scale) shows the proportions of returned swabs that tested positive – this reflects the incidence of the disease – and (left hand scale) the numbers of notifications. Diagnoses of rubella followed the rise in incidence after a lag of eight weeks.
Signal detection data from Tanner et al. (Reference Tanner, Swets and Green1956). (From Signal Detection Theory and Psychophysics by D. M. Green and J. A Swets, p. 88. © 1966, J. A Swets. Adapted with permission.)
Numbers of reported diagnoses of rubella and percentage confirmed. (Data from Communicable Disease Report, PHLS Communicable Disease Surveillance Centre, 1995–96.)
So, people do not ignore base rates. But do they update their expectations in line with Bayes' Theorem? Green (Reference Green1960) calculated an optimum (Bayesian) placement of the criteria for the experiment in Figure 1, and found that the actual placements of the criteria did not vary so widely as Bayes' theorem prescribes. A much better prescription of criterion placement is provided by a scheme of probability matching suggested by Thomas and Legge (Reference Thomas and Legge1970). Under this scheme, the numbers of different responses are adjusted to match the numbers of different stimuli, or, what is equivalent, there are equal numbers of errors of each kind, irrespective of the proportions of the different stimuli. The two schemes are compared in Laming (Reference Laming, Benferhat and Besnard2001, Fig. 2).
An experiment by Tanner et al. (Reference Tanner, Rauk and Atkinson1970) reveals how this scheme is effected. These authors partitioned their data according to the event (detection, miss, correct rejection, false positive) on the preceding trial. This analysis showed that the effective operating point fluctuates; when there is no feedback errors can be seen to occur chiefly at extreme swings of the criterion (see Laming Reference Laming2004, Fig. 12.4). But when feedback is supplied, performance on the trial following an error is different; that is, knowing one has just made an error generates a correction to the criterion. Given feedback (as in Fig. 1), participants adjust to the prevailing proportions of signal and noise trials by means of a substantial correction to criterion following each error. The effective criterion oscillates around a value at which the numbers (not proportions) of errors of each kind are equal. A second observer in the experiment by Tanner et al. (Reference Tanner, Swets and Green1956; see Green & Swets Reference Green and Swets1966, p. 95) displayed a highly asymmetric operating characteristic. Green's (Reference Green1960) calculations no longer apply. But this second observer's data still showed approximate equality between the numbers of each kind of error, irrespective of the proportions of signal and noise trials (Laming Reference Laming, Benferhat and Besnard2001, Table 2). A similar relationship is found in two-choice reaction experiments; the variation of reaction time with signal probability can be reduced to a common scheme of sequential interactions operating on signal sequences of different composition (Laming Reference Laming1968, Ch. 8).
To sum up: In real life people do not ignore base rates. “Base-rate neglect” is specific to exercises in mental arithmetic. People do not use Bayes' Theorem either. The nature of sensitivity to frequencies of events means that people adjust automatically to changes in base rate (e.g., Fig. 2), and automatic adjustment to changes in base rate is incompatible with the use of Bayes' Theorem itself. It follows that there is no reason why people should learn anything about Bayes' Theorem from natural experience – they learn only if they have (informal) lessons in probability theory. The research summarised in the target article tells us only about the prior education of the participants. It leads us astray in the matter of how people update their prior expectations.
