Should Bayesian researchers focus on “enlightened” modelling that seriously considers the interplay between rational and mechanistic accounts of cognition, rather than a “fundamentalist” approach that restricts itself to rational accounts only? Like many scientists, we see great promise in the “enlightened” research program. We argue, however, that enlightened Bayesianism is deeply reliant on research into Bayesian fundamentals, and the fundamentals cannot be abandoned without greatly affecting more enlightened work. Without solid fundamental work to extend, enlightened research will be far more difficult.
To illustrate this, consider the paper by Sanborn et al. (Reference Sanborn, Griffiths and Navarro2010a), which Jones & Love (J&L) consider to be “enlightened” as it seeks to adapt an ideal Bayesian model to incorporate insights about psychological process. To achieve this, however, it relies heavily upon work that itself would not have counted as “enlightened.” The comparison between Gibbs sampling and particle filtering as rival process models grew from “unenlightened” research that used these algorithms purely as methodological tools. As such, without this “fundamentalist” work the enlightened paper simply would not have been written.
Enlightened research can depend on fundamentals in other ways. Rather than adapt an existing Bayesian model to incorporate process constraints, Navarro and Perfors (Reference Navarro and Perfors2011) used both Bayesian fundamentals (an abstract hypothesis space) and process fundamentals (capacity limitations on working memory) as the foundations of an analysis of human hypothesis testing. Identifying a conditionally optimal learning strategy, given the process constraint, turned out to reproduce the “positive test strategy” that people typically employ (Wason Reference Wason1960), but only under certain assumptions about what kinds of hypotheses are allowed to form the abstract hypothesis space. This analysis, which extended existing work (Klayman & Ha Reference Klayman and Ha1987; Oaksford & Chater Reference Oaksford and Chater1994) and led us to new insights about what kinds of hypotheses human learners “should” entertain, could not have been done without “fundamentalist” research into both the statistical and the mechanistic basis of human learning.
Not only do “enlightened” papers depend on fundamental ones, we suggest that they are a natural outgrowth of those papers. Consider the early work on Bayesian concept learning, which contained a tension between the “weak sampling” assumption of Shepard (Reference Shepard1987) and the “strong sampling” assumption of Tenenbaum and Griffiths (Reference Tenenbaum and Griffiths2001). When strong sampling was introduced, it would presumably have counted as “fundamentalism,” since the 2001 paper contains very little by way of empirical data or consideration of the sampling structure of natural environments. Nevertheless, it served as a foundation for later papers that discussed exactly those issues. For instance, Xu and Tenenbaum (Reference Xu and Tenenbaum2007a) looked at how human learning is shaped by explicit changes to the sampling model. This in turn led Navarro et al. (in press) to propose a more general class of sampling models, and to pit them all against one another in an empirical test. (It turned out that there are quite strong individual differences in what people use as their “default” sampling assumption.) The change over time is instructive: What we observe is a gradual shift from simpler “fundamentalist” papers that develop the theory in a reduced form, towards a richer framework that begins to capture the subtleties of the psychology in play.
Even J&L's own chosen examples show the same pattern. Consider the Kemp et al. (Reference Kemp, Perfors and Tenenbaum2007) article, which J&L cite as a prime example of “fundamentalist” Bayesianism, since it introduces no new data and covers similar ground to previous connectionist models (Colunga & Smith Reference Colunga and Smith2005). Viewing the paper in isolation, we might agree that the value added is minor. But the framework it introduced has been a valuable tool for subsequent research. An extension of the model has been used to investigate how adults learn to perform abstract “second order” generalizations (Perfors & Tenenbaum Reference Perfors, Tenenbaum, Taatgen, van Rijn, Schomaker and Nerbonne2009) and to address long-debated issues in verb learning (Perfors et al. Reference Perfors, Tenenbaum and Wonnacott2010). A related model has even been used to investigate process-level constraints; Perfors (in press) uses it to investigate whether or not memory limitations can produce a “less is more” effect in language acquisition. It is from the basic, fundamental research performed by Kemp et al. (Reference Kemp, Perfors and Tenenbaum2007) that these richer, more enlightened projects have grown.
Viewed more broadly, the principle of “enlightenment growing from fundamentals” is applicable beyond Bayesian modelling; our last example is therefore an inversion. We suggest that J&L understate the importance of computational considerations in good process modelling. For instance, one of their key examples comes from Sakamoto et al. (Reference Sakamoto, Jones and Love2008), who consider mechanistic models of category learning. That paper might be characterized as a “fundamentalist” work in process modelling, insofar as it gives no consideration to the computational level issues that pertain to their choice of learning problem. As consequence of this “process fundamentalism,” the “rational” model that paper employs is not actually a rational model. It is highly mis-specified for the problem of learning time-inhomogeneous categories. In recent work (Navarro & Perfors Reference Navarro, Perfors, Taatgen, van Rijn, Schomaker and Nerbonne2009), we discuss this concern and introduce extensions to the experimental framework aimed at highlighting the computational considerations involved; at present, we are working on model development to build on this. However, the goal in our work is not to deny the importance of process, but to learn which aspects of human behaviour are attributable to computational level issues and which aspects reflect process limitations. In this case, that goal is met by building on fundamental work on the process level (i.e., Sakamoto et al.'s 2008 paper) and adding computational considerations. In general, attaining the goal of “enlightened” research is possible only if fundamentals on both levels are taken seriously – if researchers deny neither psychological mechanism nor ideal computation.
Like J&L, we believe that it is the interaction between the twin considerations of computation and process that leads us to learn about the mind. However, this should not lead us to abandon work that focuses on only one of these two components. Enlightened research is constructed from the building blocks that fundamental work provides.
Should Bayesian researchers focus on “enlightened” modelling that seriously considers the interplay between rational and mechanistic accounts of cognition, rather than a “fundamentalist” approach that restricts itself to rational accounts only? Like many scientists, we see great promise in the “enlightened” research program. We argue, however, that enlightened Bayesianism is deeply reliant on research into Bayesian fundamentals, and the fundamentals cannot be abandoned without greatly affecting more enlightened work. Without solid fundamental work to extend, enlightened research will be far more difficult.
To illustrate this, consider the paper by Sanborn et al. (Reference Sanborn, Griffiths and Navarro2010a), which Jones & Love (J&L) consider to be “enlightened” as it seeks to adapt an ideal Bayesian model to incorporate insights about psychological process. To achieve this, however, it relies heavily upon work that itself would not have counted as “enlightened.” The comparison between Gibbs sampling and particle filtering as rival process models grew from “unenlightened” research that used these algorithms purely as methodological tools. As such, without this “fundamentalist” work the enlightened paper simply would not have been written.
Enlightened research can depend on fundamentals in other ways. Rather than adapt an existing Bayesian model to incorporate process constraints, Navarro and Perfors (Reference Navarro and Perfors2011) used both Bayesian fundamentals (an abstract hypothesis space) and process fundamentals (capacity limitations on working memory) as the foundations of an analysis of human hypothesis testing. Identifying a conditionally optimal learning strategy, given the process constraint, turned out to reproduce the “positive test strategy” that people typically employ (Wason Reference Wason1960), but only under certain assumptions about what kinds of hypotheses are allowed to form the abstract hypothesis space. This analysis, which extended existing work (Klayman & Ha Reference Klayman and Ha1987; Oaksford & Chater Reference Oaksford and Chater1994) and led us to new insights about what kinds of hypotheses human learners “should” entertain, could not have been done without “fundamentalist” research into both the statistical and the mechanistic basis of human learning.
Not only do “enlightened” papers depend on fundamental ones, we suggest that they are a natural outgrowth of those papers. Consider the early work on Bayesian concept learning, which contained a tension between the “weak sampling” assumption of Shepard (Reference Shepard1987) and the “strong sampling” assumption of Tenenbaum and Griffiths (Reference Tenenbaum and Griffiths2001). When strong sampling was introduced, it would presumably have counted as “fundamentalism,” since the 2001 paper contains very little by way of empirical data or consideration of the sampling structure of natural environments. Nevertheless, it served as a foundation for later papers that discussed exactly those issues. For instance, Xu and Tenenbaum (Reference Xu and Tenenbaum2007a) looked at how human learning is shaped by explicit changes to the sampling model. This in turn led Navarro et al. (in press) to propose a more general class of sampling models, and to pit them all against one another in an empirical test. (It turned out that there are quite strong individual differences in what people use as their “default” sampling assumption.) The change over time is instructive: What we observe is a gradual shift from simpler “fundamentalist” papers that develop the theory in a reduced form, towards a richer framework that begins to capture the subtleties of the psychology in play.
Even J&L's own chosen examples show the same pattern. Consider the Kemp et al. (Reference Kemp, Perfors and Tenenbaum2007) article, which J&L cite as a prime example of “fundamentalist” Bayesianism, since it introduces no new data and covers similar ground to previous connectionist models (Colunga & Smith Reference Colunga and Smith2005). Viewing the paper in isolation, we might agree that the value added is minor. But the framework it introduced has been a valuable tool for subsequent research. An extension of the model has been used to investigate how adults learn to perform abstract “second order” generalizations (Perfors & Tenenbaum Reference Perfors, Tenenbaum, Taatgen, van Rijn, Schomaker and Nerbonne2009) and to address long-debated issues in verb learning (Perfors et al. Reference Perfors, Tenenbaum and Wonnacott2010). A related model has even been used to investigate process-level constraints; Perfors (in press) uses it to investigate whether or not memory limitations can produce a “less is more” effect in language acquisition. It is from the basic, fundamental research performed by Kemp et al. (Reference Kemp, Perfors and Tenenbaum2007) that these richer, more enlightened projects have grown.
Viewed more broadly, the principle of “enlightenment growing from fundamentals” is applicable beyond Bayesian modelling; our last example is therefore an inversion. We suggest that J&L understate the importance of computational considerations in good process modelling. For instance, one of their key examples comes from Sakamoto et al. (Reference Sakamoto, Jones and Love2008), who consider mechanistic models of category learning. That paper might be characterized as a “fundamentalist” work in process modelling, insofar as it gives no consideration to the computational level issues that pertain to their choice of learning problem. As consequence of this “process fundamentalism,” the “rational” model that paper employs is not actually a rational model. It is highly mis-specified for the problem of learning time-inhomogeneous categories. In recent work (Navarro & Perfors Reference Navarro, Perfors, Taatgen, van Rijn, Schomaker and Nerbonne2009), we discuss this concern and introduce extensions to the experimental framework aimed at highlighting the computational considerations involved; at present, we are working on model development to build on this. However, the goal in our work is not to deny the importance of process, but to learn which aspects of human behaviour are attributable to computational level issues and which aspects reflect process limitations. In this case, that goal is met by building on fundamental work on the process level (i.e., Sakamoto et al.'s 2008 paper) and adding computational considerations. In general, attaining the goal of “enlightened” research is possible only if fundamentals on both levels are taken seriously – if researchers deny neither psychological mechanism nor ideal computation.
Like J&L, we believe that it is the interaction between the twin considerations of computation and process that leads us to learn about the mind. However, this should not lead us to abandon work that focuses on only one of these two components. Enlightened research is constructed from the building blocks that fundamental work provides.