Rahnev & Denison (R&D) rightly point out a fact endorsed by many prominent Bayesian theorists, that human behavior often deviates from the predictions of statistically optimal models (Acerbi et al. Reference Acerbi, Vijayakumar and Wolpert2014b; Beck et al. Reference Beck, Ma, Pitkow, Latham and Pouget2012; Summerfield & Tsetsos Reference Summerfield and Tsetsos2015). Being data scientists, we want to first trace the way the literature handles the concepts discussed by the authors. Specifically, we want to establish whether there is a “current narrow focus on optimality,” as argued by R&D (sect. 1, para. 5). We searched PubMed for journal articles published between January 1, 1995, and April 1, 2018, that addressed optimality in human behavior. We found that although the number of articles that apply Bayesian modeling to human behavior has increased (Fig. 1), only a small proportion of the articles that we analyzed make optimality or near-optimality claims (0.30), and that claims of suboptimality are in roughly equal proportion (0.25). Nor have optimality claims drastically increased relative to the overall number of articles that use Bayesian modeling over the last two decades: 2000s (0.32) and 2010s (0.30). The majority of publications that we analyzed use Bayesian modeling as a tool to understand behavior at the computational level and/or emphasize the suboptimality of behavior. Therefore, optimality claims may not be as prevalent as the authors contend.

Figure 1.

Number of journal articles as a function of year of publication that use Bayesian models to describe behavior, that make (near) optimality claims, and that make suboptimality claims. We used the following search terms: “(sub)optimality perception,” “(sub)optimal perceptual decision making,” “Bayesian psychophysics,” and “Bayesian observer model behavior/perception.” We included in our analysis the 185 articles that reported human psychophysics data and modeling. We manually coded the type of the claim made by each paper.

We endorse the view of R&D that humans deviate from statistically optimal behavior, and some of our recent perceptual work has added new ways in which behavior is suboptimal. For example, prior knowledge acquired in a sensorimotor task does not fully generalize to a perceptual decision-making task that requires the same probabilistic treatment of sensory and prior information (Chambers et al. Reference Chambers, Fernandes and Kording2017a). Also, we show that young children fail to incorporate experimentally imposed priors into their sensorimotor estimates and learn to approximate efficient use of statistical information during development (Chambers et al. Reference Chambers, Sokhey, Gaebler-Spira and Kording2017b). If the underlying neural processes are innately “Bayes optimal,” then behavior should reflect this under different conditions and without learning. We have observed this not to be the case when the decision modality changes in decision making in adults and during child development. These findings underline the importance of assessing the generality of claims concerning the efficient use of statistical information in human decision making and add to the literature on suboptimal perceptual decision making discussed by R&D.

R&D advocate for a change in the culture of how behavioral data are modeled. They argue that much existing work is limited because it includes one ad hoc model for one set of experiments. We agree that this is problematic because based on this ad hoc approach we cannot know how well optimal models generalize or “transfer” to other situations (Maloney & Mamassian Reference Maloney and Mamassian2009). The authors recommend that in cases of findings of suboptimality, the assumptions of each model component (likelihood, prior, cost function, decision rule) should be examined, and that in cases of good agreement with optimal models, model performance should be examined under different conditions and tasks. We agree with the authors’ prescriptions on a more rigorous approach to following up on previous claims and believe that this practice will help the field to develop models that provide more complete accounts of human behavior.

However, it should be emphasized that Bayesian optimal models remain an important tool for building computational models of behavior. Uncertainty exists in the outside world, and we do well in everyday tasks by taking uncertainty into account (Kersten et al. Reference Kersten, Mamassian and Yuille2004; Körding & Wolpert Reference Körding and Wolpert2006; Vilares & Kording Reference Vilares and Kording2011). Alternative non-optimal frameworks, like the “bag of tricks” (Ramachandran Reference Ramachandran and Blakemore1990) mentioned by the authors and heuristic models (Gigerenzer & Gaissmaier Reference Gigerenzer and Gaissmaier2011; Kahneman et al. Reference Kahneman, Slovic and Tversky1982a) do not contain explicit formulations of how probabilistic information should be combined and do not as easily capture how we deal with uncertainty as Bayesian models do. The Bayesian framework has other advantages as a computational framework. It provides an important benchmark for human performance that can constrain non-optimal models. It is a compact mathematical framework that avoids assumptions about implementational details. It is transparent: The components of the model (prior, likelihood, cost function, decision) must be declared (Griffiths et al. Reference Griffiths, Chater, Norris and Pouget2012). Although we agree that the usefulness of labeling of behavior as “optimal” is questionable, we maintain that Bayesian modeling is still an important tool for the computational understanding of behavior.