The authors (Mareschal et al. Reference Mareschal, Johnson, Sirois, Spratling, Thomas and Westermann2007a) make a convincing case that constraints from brain, body and environment should be taken seriously when trying to explain psychological development and that computational modeling has an important theoretical role to play in such explanations. My commentary focuses on some interesting comments made in Neuroconstructivism's Box 1.1 (p. 13) concerning constructivist approaches to the modeling of development. There they raise the possibility that neural networks could mimic constructivist development with a hierarchy of networks such that the outputs of networks solving simple problems could serve as inputs to networks solving more complex problems. In this way, later and more complex knowledge could be based on, and benefit from, earlier and simpler knowledge.

Currently, most such models are designed and arranged by hand, where each network is trained separately and then combined in a designed hierarchy. The influential model of object processing featured in chapter 9 of Volume 1 is a good example of this approach (Mareschal et al. Reference Mareschal, Plunkett and Harris1999). That model neatly explains the well-known décalage in development of object permanence between looking and reaching measures (Baillargeon Reference Baillargeon1987) by noting that information processed by ventral and dorsal visual streams needs to be integrated in reaching tasks, but not in looking tasks, which can rely solely on the dorsal stream.

One limitation of such models is that much of their ability is supplied by the human modelers who cleverly arrange the training and connectivity between various network modules. In this case, one network module learns to recognize objects based on their static features while another learns to follow the trajectory of moving objects. A third network module learns to reach for objects by integrating these two skills, thus explaining why successful reaching takes longer to achieve than successful looking.

Recent work in knowledge-based neural learning raises the possibility that the design and arrangement of such hierarchical networks could be done automatically by the learning algorithm itself without human intervention (Shultz & Rivest Reference Shultz and Rivest2001; Shultz et al. Reference Shultz, Rivest, Egri, Thivierge and Dandurand2007b). The algorithm used in this work has been dubbed knowledge-based cascade-correlation (KBCC) because it extends the earlier cascade-correlation (CC) algorithm to enable the recruitment of old networks into new learning. CC is a constructive neural learner (Fahlman & Lebiere Reference Fahlman, Lebiere and Touretzky1990) that has been used to model a wide range of developmental phenomena (Shultz Reference Shultz2003). It works by starting with a simple network with only input and output units, and then recruiting single hidden units as needed to reduce network error. An interesting variant called sibling-descendant CC (SDCC) allows classic descendant candidate units to compete with sibling candidate units (Baluja & Fahlman Reference Baluja and Fahlman1994). As in ordinary CC, the candidate hidden unit whose activation correlates best with network error is the one that actually gets recruited. A descendant recruit is installed on its own layer just beyond any existing hidden units, whereas a sibling recruit is installed on the current highest layer of hidden units. Evidence indicates that SDCC produces equivalent coverage of developmental phenomena to that provided by classic CC, but with a greater variety of network topologies, most with less network depth and fewer connection weights (Shultz et al. Reference Shultz, Mysore, Quartz, Mareschal, Sirois, Westermann and Johnson2007a).

KBCC is a natural extension of CC in which previously learned networks are saved and made available to be recruited in new learning. These older networks compete with each other and with single-unit candidates in the usual fashion. Namely, the candidate whose activations correlate best with network error is the new recruit. KBCC was designed (Shultz & Rivest Reference Shultz and Rivest2001) to capture the pervasive tendency of humans to build their new learning on their existing knowledge rather than learning everything from scratch (Heit Reference Heit1994; Pazzani Reference Pazzani1991; Wisniewski Reference Wisniewski1995). Such knowledge-based constructivism can account for some of the ease and speed of human learning and for the interference effects that sometimes arise.

Prospects for simulating cognitive developmental phenomena with KBCC are good but only just beginning. A preliminary KBCC model of the benchmark balance-scale task (Shultz et al. Reference Shultz, Rivest, Egri, Thivierge and Dandurand2007b) suggests that KBCC could cover a genuine torque rule, as opposed to a mere addition rule (Quinlan et al. Reference Quinlan, van, Maas, Jansen, Booij and Rendell2007) in a terminal stage in accord with the likelihood that adolescents acquire such a torque rule by direct instruction in science classes (Siegler & Klahr Reference Siegler, Klahr and Glazer1982). Symbolic rules, such as the torque rule, which predicts that the side of the scale with higher torque (product of weight and distance from the fulcrum) will be the side that descends, can be inserted into the recruitment pool of KBCC in a neural form.

Although most KBCC simulations report faster and more accurate learning by virtue of recruiting existing knowledge (Shultz et al. Reference Shultz, Rivest, Egri, Thivierge and Dandurand2007b), one simulation demonstrates that prior knowledge can be not just helpful, but even essential, in learning. That project shows that prior knowledge of divisibility is required for networks to learn how to detect prime numbers from examples of prime and composite integers (Egri & Shultz Reference Egri, Shultz, Sun and Miyake2006).

The import of knowledge-based learning for modeling neuroconstructivism is that automatic organization of hierarchical, modular networks may now be within our grasp. Just as classic CC networks are able to construct their own topology automatically as they learn, KBCC can organize multiple networks into a hierarchy that allows the output of some older, knowledgeable networks to feed inputs of other units and networks and the outputs involved in new learning.

Letting computational models do their own thing seems preferable because it makes the modeling enterprise less artistic and more scientific. It allows the intelligence of computational systems to be taken out of the hands of skillful modelers and placed inside the algorithms themselves. In turn, this allows intelligent behavior to emerge in a more natural fashion, as opposed to being engineered by human handlers, thus affording a deeper glimpse into the way phenomena might really work. Models allowing such emergence may well be ultimately wrong, as virtually all models and theories are, but at least their workings can be more fully examined and tested in an objective, scientific manner, thus advancing the field more directly and more rapidly. In short, emergent models take constructivism more seriously.