There is an important point in this target article by Jones & Love (J&L); unfortunately it is hard to see as it is swimming below some very red herrings. Bayesians do not reject mechanism, mental representation, or psychological or neural process. Comparisons to behaviorism are thus misleading. Moreover, the utility of particular Bayesian models does not rest on whether they engage with mechanistic issues, nor even whether they are concerned with issues of mechanism at all; it rests on what new light the Bayesian analysis sheds on the problem, and this depends on the same principles of evaluation that should be applied to any theory. There is a meaningful debate to be had about the value of Bayesian modeling, but ironically, given the title of the article, we fear that J&L's wholesale rejection of “Bayesian Fundamentalism” will lead to arguments about whose religion is better. In geopolitics, these kinds of disputes often produce “intractable conflicts” and do not lead anywhere productive (Bar-Tal Reference Bar-Tal2002). In this commentary we first highlight one way in which J&L's analysis goes awry and clarify what we believe the debate should not be about. We then turn to what the debate should be about and suggest some prescriptions for good Bayesian practice.
A central pillar of J&L's argument is the claim that Bayesian modeling at the computational level represents a rejection of mechanism and psychological process in favor of a purely mathematical analysis. This is a straw man that conflates the rejection of psychological representations and processes with what Bayesian models really do, which is abstract over cognitive processes. Rejecting mechanism would indeed imply that there is nothing to a Bayesian model except for the “conceptually trivial” (according to J&L; see target article, Abstract) mathematical relation between inputs and outputs. This idea is reflected in the puzzling language that the authors use to explain the nature of the hypothesis space. They write, “In general, a hypothesis is nothing more than a probability distribution” (sect. 3, para. 2, repeated in para. 9). It is true that any Bayesian model is committed to the assumption that it is coherent to represent a probability distribution over the hypothesis space, but this does not mean that a hypothesis is a probability distribution. Rather, in the Bayesian models we are familiar with, hypotheses are interpretable as psychological constructs, such as partitions of conceptual space (e.g., Tenenbaum & Griffiths Reference Tenenbaum and Griffiths2001) or causal models (e.g., Steyvers et al. Reference Steyvers, Tenenbaum, Wagenmakers and Blum2003). For that reason, a Bayesian model for a particular task is not conceptually trivial, but rather embodies an analysis specific to the mental models presumed to be relevant to that task. It is true that modelers are not always as clear as they should be about whether these hypotheses represent psychological entities or merely a conceptual analysis of the task (or both), and the import of the model does depend critically on that. But either way, such a model can be extremely useful. In the former case, it provides a descriptive hypothesis. In the latter, it may shed new light on how we should think about the task. Critically, none of this is contingent on whether the model engages with questions about reaction time, neurophysiological information, or other “mechanistic” data.
An important criticism that we believe should be taken away from the target article is that some Bayesian modelers fail to clearly distinguish normative from descriptive claims. Vagueness about this issue has led to confusion between two very different ideas: “rational” and “computational” (Sloman & Fernbach Reference Sloman, Fernbach, Chater and Oaksford2008). A rational model is one that explains data by showing how it reflects optimal behavior, given the task at hand (Anderson Reference Anderson1990). A computational model is one that describes the function that people are engaged in, whether or not that function is what they should be engaged in. We agree completely with J&L on this issue: The fact that a model is at the computational level is not an excuse for its failure to fit data. Violations of a model's predictions should be taken seriously and not explained away as due to the approximate way the optimal computation is implemented. And a rational analysis does not demonstrate rationality if people do not abide by it. The beauty of rational analysis is its ability to explain behavior by appealing to the structure of the world. When a model fails to be crystal clear about which of its parts reflect the world and which reflect mental or neural facts, its contribution becomes obscure.
This issue is not a problem with Bayesianism per se, but it does suggest principles for good Bayesian practice. For one, model comparisons and sensitivity analyses should be de rigueur to show whether the phenomenon falls uniquely out of the Bayesian model or whether the model is merely consistent with it. It is often helpful to a reader to explain precisely what in the model is generating the phenomenon of interest, and whether that aspect is inherent to the model's structure and would emerge from any parameterization, or whether it is due to an assumption. Modelers should be clearer about whether their model is a psychological theory or a normative analysis, and they should consider contradictory data conscientiously as opposed to explaining it away. Of course, these are just recommendations for good scientific practice and thus they hardly seem controversial. J&L are right to point out that Bayesian modelers sometimes ignore them.
Were these practices adopted more systematically, Bayesian models would become more falsifiable. This is a double-edged sword, of course: Given the reams of evidence that cognition is fallible, Bayesians are fighting an uphill battle (at least in high-order cognition). But only falsifiable theories can be truly revolutionary.
There is an important point in this target article by Jones & Love (J&L); unfortunately it is hard to see as it is swimming below some very red herrings. Bayesians do not reject mechanism, mental representation, or psychological or neural process. Comparisons to behaviorism are thus misleading. Moreover, the utility of particular Bayesian models does not rest on whether they engage with mechanistic issues, nor even whether they are concerned with issues of mechanism at all; it rests on what new light the Bayesian analysis sheds on the problem, and this depends on the same principles of evaluation that should be applied to any theory. There is a meaningful debate to be had about the value of Bayesian modeling, but ironically, given the title of the article, we fear that J&L's wholesale rejection of “Bayesian Fundamentalism” will lead to arguments about whose religion is better. In geopolitics, these kinds of disputes often produce “intractable conflicts” and do not lead anywhere productive (Bar-Tal Reference Bar-Tal2002). In this commentary we first highlight one way in which J&L's analysis goes awry and clarify what we believe the debate should not be about. We then turn to what the debate should be about and suggest some prescriptions for good Bayesian practice.
A central pillar of J&L's argument is the claim that Bayesian modeling at the computational level represents a rejection of mechanism and psychological process in favor of a purely mathematical analysis. This is a straw man that conflates the rejection of psychological representations and processes with what Bayesian models really do, which is abstract over cognitive processes. Rejecting mechanism would indeed imply that there is nothing to a Bayesian model except for the “conceptually trivial” (according to J&L; see target article, Abstract) mathematical relation between inputs and outputs. This idea is reflected in the puzzling language that the authors use to explain the nature of the hypothesis space. They write, “In general, a hypothesis is nothing more than a probability distribution” (sect. 3, para. 2, repeated in para. 9). It is true that any Bayesian model is committed to the assumption that it is coherent to represent a probability distribution over the hypothesis space, but this does not mean that a hypothesis is a probability distribution. Rather, in the Bayesian models we are familiar with, hypotheses are interpretable as psychological constructs, such as partitions of conceptual space (e.g., Tenenbaum & Griffiths Reference Tenenbaum and Griffiths2001) or causal models (e.g., Steyvers et al. Reference Steyvers, Tenenbaum, Wagenmakers and Blum2003). For that reason, a Bayesian model for a particular task is not conceptually trivial, but rather embodies an analysis specific to the mental models presumed to be relevant to that task. It is true that modelers are not always as clear as they should be about whether these hypotheses represent psychological entities or merely a conceptual analysis of the task (or both), and the import of the model does depend critically on that. But either way, such a model can be extremely useful. In the former case, it provides a descriptive hypothesis. In the latter, it may shed new light on how we should think about the task. Critically, none of this is contingent on whether the model engages with questions about reaction time, neurophysiological information, or other “mechanistic” data.
An important criticism that we believe should be taken away from the target article is that some Bayesian modelers fail to clearly distinguish normative from descriptive claims. Vagueness about this issue has led to confusion between two very different ideas: “rational” and “computational” (Sloman & Fernbach Reference Sloman, Fernbach, Chater and Oaksford2008). A rational model is one that explains data by showing how it reflects optimal behavior, given the task at hand (Anderson Reference Anderson1990). A computational model is one that describes the function that people are engaged in, whether or not that function is what they should be engaged in. We agree completely with J&L on this issue: The fact that a model is at the computational level is not an excuse for its failure to fit data. Violations of a model's predictions should be taken seriously and not explained away as due to the approximate way the optimal computation is implemented. And a rational analysis does not demonstrate rationality if people do not abide by it. The beauty of rational analysis is its ability to explain behavior by appealing to the structure of the world. When a model fails to be crystal clear about which of its parts reflect the world and which reflect mental or neural facts, its contribution becomes obscure.
This issue is not a problem with Bayesianism per se, but it does suggest principles for good Bayesian practice. For one, model comparisons and sensitivity analyses should be de rigueur to show whether the phenomenon falls uniquely out of the Bayesian model or whether the model is merely consistent with it. It is often helpful to a reader to explain precisely what in the model is generating the phenomenon of interest, and whether that aspect is inherent to the model's structure and would emerge from any parameterization, or whether it is due to an assumption. Modelers should be clearer about whether their model is a psychological theory or a normative analysis, and they should consider contradictory data conscientiously as opposed to explaining it away. Of course, these are just recommendations for good scientific practice and thus they hardly seem controversial. J&L are right to point out that Bayesian modelers sometimes ignore them.
Were these practices adopted more systematically, Bayesian models would become more falsifiable. This is a double-edged sword, of course: Given the reams of evidence that cognition is fallible, Bayesians are fighting an uphill battle (at least in high-order cognition). But only falsifiable theories can be truly revolutionary.