Jones & Love (J&L) reveal the shortcomings of current Bayesian approaches to psychological processes. We agree with the authors on this but believe they still show too much faith in the potential of the Bayesian approach as an alternative to connectionist modeling. We illustrate the problems with a discussion of research on human causal induction. Bayesian fundamentalism has taken firm hold in this field; however, there is also a tradition of associative analysis, allowing us to consider the added value of Bayesianism.
Causal reasoning in naturalistic environments quickly becomes a mathematically complex problem requiring important auxiliary processes beyond the math. According to a Bayesian analysis, people start by representing and selecting from multiple causal structures. For example, three events A, B, and C can be described by several classes of causal model, including a causal chain in which A causes B which then causes C (A→B→C) and a common cause model in which A causes B and C (C←A→B). Observations are used to decide which is most likely. A simple Bayesian analysis has problems because some of the models can be Bayesian equivalents. For example, a deterministic chain model (A→B→C) and a deterministic common cause model (C←A→B) produce the same empirical probabilities. Either all three events are present on a trial or none are.
One way to reduce this ambiguity is to do an intervention. Removing the effect of B enables one to discriminate between chain and common cause models. With a chain (A→B→C), C will no longer occur, but in a common cause model (C←A→B), C will occur. People can decide between models by using interventions (Steyvers et al. Reference Steyvers, Tenenbaum, Wagenmakers and Blum2003). However, the critical feature of this Bayesian modeling is that the Bayesian component does not do the crucial intervention but simply assesses the result of the intervention. A supervisor or other mechanism decides which intervention to make (Hagmayer et al. Reference Hagmayer, Sloman, Lagnado, Waldmann, Gopnik and Schultz2007; Steyvers et al. Reference Steyvers, Tenenbaum, Wagenmakers and Blum2003). It is this decision process that is outside the Bayesian frame that is critical. The dependencies in the causal graphs may be generated by Bayesian processes, but the Bayesian analysis does not do the psychological work that solves the problem. As J&L suggest, it merely does the complex counting. Associative or connectionist models can also do this work. For example, a traditional associative net has nodes and connections of varying strengths between them (e.g., Baker et al. Reference Baker, Murphy, Vallee-Tourangeau, Shanks, Holyoak and Medin1996). And it is common practice for scientists to remove nodes in order to discover what the associative nets have learned (e.g., Cowell et al. Reference Cowell, Bussey and Saksida2006; Harm & Seidenberg Reference Harm and Seidenberg1999; Plaut Reference Plaut1995). If the supervisor were to perform similar actions, this would be an associative analogue to graph surgery. An associative analysis is not only very similar to the Bayesian computation, it also has some advantages which we discuss. Therefore, it is disingenuous to claim that this type of research in causal induction arises from or differentially supports a Bayesian perspective.
The second way to disambiguate the causal models, temporal precedence, has been the object of much associative analysis (e.g., Pavlov Reference Pavlov1927). The causal arrows in the models can be replaced with “leads to” and, if the observer can discriminate order, they could easily discriminate the common cause, A leads to B and C, from the chain, A leads to B that leads to C models. Bayesian fundamentalists researching causal induction (e.g., Lagnado & Sloman Reference Lagnado and Sloman2006) have, indeed, shown that with temporal information people can discriminate the models. However, the timing information does not arise directly from Bayesian computations. Again it is the supervisor that disambiguates the data by using timing. However, learning orders emerges easily from a connectionist perspective. Associative nets, like neuronal processes, have activations that decay with time. In (A→B→C), when C finally comes along, A activation will have decayed more than B activation, so a stronger link between B and C will form. Baetu and Baker (Reference Baetu and Baker2009) have reported a series of experiments in which they have studied how people form causal chains from experience with the individual links (generative and preventative). People are good at assembling these independently learned links into chains. But, most important, a modification of a simple associative model (the autoassociator; McClelland & Rumelhart Reference McClelland and Rumelhart1988) generates representations of causal structure, and predicts participants’ behavior.
Finally, the associative structures have a certain face validity for psychological processes that the Bayesian frames do not. In associative terms, causal chains are represented as associative strengths and not likelihood ratios. Order and timing can flow naturally from them. They represent causes and effects of different magnitudes, and not just binary (absent/present) events. Causes and activations may be weak or strong, and not just present or absent.
What does this say about J&L's thesis? First, we agree that Bayesianism must progress beyond fundamentalism. Indeed, much of the research concerning the supervisor can lead the unwary believer to the unwarranted conclusion that this work discriminates Bayesian computations from others. Second, J&L argue that the Bayesian analysis can prosper at all levels. In our rather simple case, it certainly does not account for the supervisor. It is not clear how it could in a principled way. Third, they argue that Bayesian analyses should become closely linked to psychological mechanism, and we agree; but we argue that associative structure may already be there. For instance, we are now closer to understanding how a prediction-error algorithm (e.g., Rescorla & Wagner Reference Rescorla, Wagner, Black and Prokasy1972) might be implemented in the brain (Kim et al. Reference Kim, Krupa and Thompson1998; Waelti et al. Reference Waelti, Dickinson and Schultz2001).
In conclusion, we agree with J&L that Agnostic Bayesian nets offer a powerful method for artificial intelligence and that, if elaborated, can learn about or represent any finite data set – but so could an associative net. However, the question for psychological process is one of parsimony and mechanistic plausibility, and we are not convinced that J&L have demonstrated this contribution. We would be more convinced if they had described a single instance where a Bayesian analysis produced a realistic psychological mechanism or empirical result.
Jones & Love (J&L) reveal the shortcomings of current Bayesian approaches to psychological processes. We agree with the authors on this but believe they still show too much faith in the potential of the Bayesian approach as an alternative to connectionist modeling. We illustrate the problems with a discussion of research on human causal induction. Bayesian fundamentalism has taken firm hold in this field; however, there is also a tradition of associative analysis, allowing us to consider the added value of Bayesianism.
Causal reasoning in naturalistic environments quickly becomes a mathematically complex problem requiring important auxiliary processes beyond the math. According to a Bayesian analysis, people start by representing and selecting from multiple causal structures. For example, three events A, B, and C can be described by several classes of causal model, including a causal chain in which A causes B which then causes C (A→B→C) and a common cause model in which A causes B and C (C←A→B). Observations are used to decide which is most likely. A simple Bayesian analysis has problems because some of the models can be Bayesian equivalents. For example, a deterministic chain model (A→B→C) and a deterministic common cause model (C←A→B) produce the same empirical probabilities. Either all three events are present on a trial or none are.
One way to reduce this ambiguity is to do an intervention. Removing the effect of B enables one to discriminate between chain and common cause models. With a chain (A→B→C), C will no longer occur, but in a common cause model (C←A→B), C will occur. People can decide between models by using interventions (Steyvers et al. Reference Steyvers, Tenenbaum, Wagenmakers and Blum2003). However, the critical feature of this Bayesian modeling is that the Bayesian component does not do the crucial intervention but simply assesses the result of the intervention. A supervisor or other mechanism decides which intervention to make (Hagmayer et al. Reference Hagmayer, Sloman, Lagnado, Waldmann, Gopnik and Schultz2007; Steyvers et al. Reference Steyvers, Tenenbaum, Wagenmakers and Blum2003). It is this decision process that is outside the Bayesian frame that is critical. The dependencies in the causal graphs may be generated by Bayesian processes, but the Bayesian analysis does not do the psychological work that solves the problem. As J&L suggest, it merely does the complex counting. Associative or connectionist models can also do this work. For example, a traditional associative net has nodes and connections of varying strengths between them (e.g., Baker et al. Reference Baker, Murphy, Vallee-Tourangeau, Shanks, Holyoak and Medin1996). And it is common practice for scientists to remove nodes in order to discover what the associative nets have learned (e.g., Cowell et al. Reference Cowell, Bussey and Saksida2006; Harm & Seidenberg Reference Harm and Seidenberg1999; Plaut Reference Plaut1995). If the supervisor were to perform similar actions, this would be an associative analogue to graph surgery. An associative analysis is not only very similar to the Bayesian computation, it also has some advantages which we discuss. Therefore, it is disingenuous to claim that this type of research in causal induction arises from or differentially supports a Bayesian perspective.
The second way to disambiguate the causal models, temporal precedence, has been the object of much associative analysis (e.g., Pavlov Reference Pavlov1927). The causal arrows in the models can be replaced with “leads to” and, if the observer can discriminate order, they could easily discriminate the common cause, A leads to B and C, from the chain, A leads to B that leads to C models. Bayesian fundamentalists researching causal induction (e.g., Lagnado & Sloman Reference Lagnado and Sloman2006) have, indeed, shown that with temporal information people can discriminate the models. However, the timing information does not arise directly from Bayesian computations. Again it is the supervisor that disambiguates the data by using timing. However, learning orders emerges easily from a connectionist perspective. Associative nets, like neuronal processes, have activations that decay with time. In (A→B→C), when C finally comes along, A activation will have decayed more than B activation, so a stronger link between B and C will form. Baetu and Baker (Reference Baetu and Baker2009) have reported a series of experiments in which they have studied how people form causal chains from experience with the individual links (generative and preventative). People are good at assembling these independently learned links into chains. But, most important, a modification of a simple associative model (the autoassociator; McClelland & Rumelhart Reference McClelland and Rumelhart1988) generates representations of causal structure, and predicts participants’ behavior.
Finally, the associative structures have a certain face validity for psychological processes that the Bayesian frames do not. In associative terms, causal chains are represented as associative strengths and not likelihood ratios. Order and timing can flow naturally from them. They represent causes and effects of different magnitudes, and not just binary (absent/present) events. Causes and activations may be weak or strong, and not just present or absent.
What does this say about J&L's thesis? First, we agree that Bayesianism must progress beyond fundamentalism. Indeed, much of the research concerning the supervisor can lead the unwary believer to the unwarranted conclusion that this work discriminates Bayesian computations from others. Second, J&L argue that the Bayesian analysis can prosper at all levels. In our rather simple case, it certainly does not account for the supervisor. It is not clear how it could in a principled way. Third, they argue that Bayesian analyses should become closely linked to psychological mechanism, and we agree; but we argue that associative structure may already be there. For instance, we are now closer to understanding how a prediction-error algorithm (e.g., Rescorla & Wagner Reference Rescorla, Wagner, Black and Prokasy1972) might be implemented in the brain (Kim et al. Reference Kim, Krupa and Thompson1998; Waelti et al. Reference Waelti, Dickinson and Schultz2001).
In conclusion, we agree with J&L that Agnostic Bayesian nets offer a powerful method for artificial intelligence and that, if elaborated, can learn about or represent any finite data set – but so could an associative net. However, the question for psychological process is one of parsimony and mechanistic plausibility, and we are not convinced that J&L have demonstrated this contribution. We would be more convinced if they had described a single instance where a Bayesian analysis produced a realistic psychological mechanism or empirical result.