Leech et al. describe a connectionist algorithm that reproduces several known effects in the development of analogy-making. The authors claim that this algorithm models how children develop the ability to make analogies in a manner not yet captured by previous (primarily non-connectionist) models of analogy such as Structure Mapping Theory (SMT) (Gentner Reference Gentner1983). The current version of the algorithm does not account for (the development of) the complex analogies made by adults. Moreover, Leech et al.'s target article is silent on two issues prominent in previous work on analogy, namely: (1) a computational-level theory in the sense of Marr (Reference Marr1982), that is, a precise formulation of a cognitive ability as an input-output function, of which the presented connectionist model supposedly provides an algorithmic-level implementation; and (2) the computational complexity of the proposed algorithm and/or its associated computational-level theory. In this commentary, we discuss why consideration of (1) and (2) may be essential for making progress in research programs such as Leech et al.'s.

To start, we find it useful to re-cast the problem of deriving models of cognitive development (be they algorithmic- or computational-level) in terms of satisfying various constraints. The most basic of these is the empirical constraint, that is, the model must mimic/predict observed cognitive behavior. Though this is often construed only in terms of adult cognitive behavior, the model should be able to fit performance across the different stages of development (e.g., in infancy, childhood, and adulthood) and account for any apparent discontinuities between different stages (e.g., relational shift in the case of analogy). This constraint holds for any model of natural phenomena. In the case of cognitive abilities, which develop over time, Leech et al. point out the need for a developmental constraint, that is, “all proposed mechanisms [of the model] must have a developmental origin” (sect. 5.5, para. 1). That is, the model should incorporate mechanisms which allow the ability to mature consistent with the empirical constraint. Overlooked or ignored so far is a third and equally important constraint, the computational constraint; that is, a cognitive model must satisfy both the empirical constraint and the developmental constraint while operating within the computational resource-limits imposed by the human body and its environment.

Computational complexity analysis is the tool of choice for assessing whether or not a cognitive model can satisfy the computational constraint, thereby placing such analysis at the heart of cognitive modeling. This is not to say that such analysis is easy: Though well-developed measures such as worst-case asymptotic time complexity are applicable to algorithms operating on digital computational architectures, it is not obvious which measures are most appropriate for connectionist algorithms. Potential measures include the time it takes for the network to settle, the number of training-cycles required to develop a given level of performance, and the number of nodes and layers in a network required for computing a given input-output mapping. Once defined, such measures can be used in conjunction with a suitable criterion for computational tractability (see, e.g., van Rooij Reference van Rooij2003; in press). Doing so would enable cognitive modelers such as Leech et al. to evaluate how their models' computational resource requirements scale for the larger inputs that are characteristic of real-world domains of analogizing, and to show whether or not modifications are necessary to accommodate adult-level analogizing.

Though algorithmic-level models can be evaluated against the three constraints mentioned above, there are additional benefits in specifying the computational problems that these algorithms are supposed to be solving – that is, formulating computational-level theories. The first benefit is explanatory transparency: A computational-level theory provides a precise input-output characterization of the cognitive capacity that is to be explained, which is the primary explanandum.Footnote ¹ In the absence of such a characterization, it is hard to tell if the proposed algorithmic-level theory is explaining anything at all (Cummins Reference Cummins, Keil and Wilson2000). The second benefit is explanatory power: Computational-level theories postulate organizing principles that govern cognitive abilities, which in turn give insight into the rationale of cognitive computations. This is not obviously supported by algorithmic-level theories, especially when we are dealing with connectionist algorithms (Cummins Reference Cummins1995). The third benefit is analytical power: Computational-level complexity analyses can demonstrate that no algorithm (let alone a given algorithm) meets the computational constraint for a particular computational-level theory (see also Tsotsos Reference Tsotsos1990). Moreover, the same analyses can highlight aspects of one's theory that are responsible for excessive resource demands, and as such can guide the formulation of new theories that meet the computational constraint (see van Rooij & Wareham, in press; van Rooij et al. Reference van Rooij, Stege and Kadlec2005; Wareham Reference Wareham1999).

Formulating computational-level theories of cognitive capacities is not easy, and seems to be particularly hard for connectionist architectures. Yet, such theories can be formulated (for an example, see Thagard Reference Thagard2000), and given the benefits we have identified, it may well be worth the effort. Such theories may counteract the acknowledged temptation to focus on getting connectionist algorithms to work rather than focusing on why they work (Mareschal & Thomas Reference Mareschal and Thomas2007, p. 148), and enable them to actually count as explanations (Green Reference Green2001). Such theories may also enable the exploitation of known connectionist-oriented complexity results (Bruck & Goodman Reference Bruck and Goodman1990; Judd Reference Judd1990; Parberry Reference Parberry1994), which, given the computational intractability of theories of analogy such as SMT (Veale & Keane Reference Veale and Keane1997), may be crucial in helping approaches such as Leech et al.'s scale to adult-level performance. Finally, computational-level connectionist theories may more clearly expose relationships to non-connectionist theories. For example, reading the target article, we wonder to what extent connectionist algorithms could be trained to map analogies according to the criteria set forth by SMT, and hence to what degree these approaches are complementary rather than competing. Lacking a characterization of the problem that the connectionist network is supposed to be solving, we are so far unable to tell.