1. Introduction

The view that the brain computes is a working hypothesis in cognitive and brain sciences.Footnote ¹ But what exactly is meant by the statement that the brain computes? What distinguishes computing systems, such as brains, from noncomputing systems, such as stomachs and tornadoes? Elsewhere (Shagrir Reference Shagrir2010a), I offer a tentative account according to which computing is a sort of modeling. The gist of the account is that a system computes just in case its input-output mapping relations preserve (in a sense to be specified) mathematical relations in the represented domain. My aim here is to describe how this modeling notion evolved, at least partly, from projects conducted at the University of California, San Diego (UCSD). The first phase was a critique, during the 1980s, of the “structural notion” on which the distinction between computing and noncomputing has to do, at least in part, with the structure of the mechanism (sec. 2). It is shown that this notion is inadequate in the context of neuroscience (sec. 3). The second phase was an attempt to advance an alternative account according to which computing has nothing to do with structure but only with information processing (sec. 4). The third phase was the recognition that computing is not just a mapping from representation to representation but a special sort of mapping. It is special in that it preserves, through its input-output mapping relations, mathematical relations in the represented domain (sec. 5).

2. Computing as Structure

The accepted wisdom is that the distinction between computing and noncomputing has to do, at least in part, with the structure of the mechanism. Obviously, every mechanism has some structure. But the claim is that a mechanism is a computing one only if it has unique structural properties. By structural properties is meant certain architectural properties; in effect, those holding this view associate computing mechanisms with digital architectures.

For example, Fodor characterizes computing processes as those that “apply to representations in virtue of (roughly) the syntax of representations” (Reference Fodor and Haugeland1980, 309), which for him are processes that can be characterized as “occurring in a languagelike medium” (Reference Fodor1994, 9), namely, in classical architectures (Fodor and Pylyshyn Reference Fodor and Pylyshyn1988). An example of noncomputing processes are those that occur in most connectionist machines. Newell and Simon famously associate a computer with a “physical symbol system,” which has “a specific architectural” structure (Reference Piccinini1976, 42). Newell specifies this structure in terms of five subsystems: memory, operators, control, input, and output (Reference Newell, Simon and Haugeland1980, 142; see also Harnish Reference Harnish2002, 396–97).

Cummins characterizes computing processes in terms of program execution, which, for him, means that they have the unique structure of step satisfaction: “To compute a function g is to execute a program that gives o as its output on input i just in case g(i) = o. Computing reduces to program execution. … Program execution reduces to step-satisfaction” (Reference Churchland1989, 91–92). Noncomputing processes also satisfy functions, but in contrast to the computing ones, they are not doing so in a stepwise manner. Piccinini characterizes computation as “the generation of output strings of digits from input strings of digits in accordance with a general rule that depends on the properties of the strings and (possibly) on the internal state of the system” (Reference Piccinini2008b, 34; see also Piccinini Reference Piccinini2007, 521). Computing processes thus manipulate inputs in accordance with a rule defined over the input types and their place within strings, whereas the noncomputing mechanisms are often described as nondiscrete processes governed by dynamical equations.

I use the phrase “structural notion” to refer to all those accounts that associate computing with unique architecture. These accounts can differ in detail and origin. Some accounts are anchored in logical proof-theoretic systems (Fodor Reference Fodor and Haugeland1980; see also Haugeland Reference Haugeland and Haugeland1981), others in computability theory (Cummins Reference Cummins1989; Searle Reference Searle1992), automata theory (Putnam Reference Pylyshyn1988; Chalmers Reference Chalmers1996), artificial intelligence (Newell and Simon Reference Piccinini1976), or mechanistic explanations (Piccinini Reference Piccinini2007; see also Edelman Reference Edelman2008). Nevertheless, they all reduce computing to some type of a digital architecture; they all view the digital electronic devices as paradigm examples of computing systems.

3. Computational Neuroscience: The Case against Structure

The structural notion has its virtues but also one vice: it is at odds with much of the work in connectionist cognitive science and, even more so, in computational neuroscience. This gap is highlighted, with respect to connectionist computation, by Rumelhart, McClelland, and the PDP Research Group (Reference Rumelhart and McClelland1986, 121–27; the group was centered at UCSD) and then, also with respect to computational neuroscience, by some philosophers, for example, Paul Churchland, who argues that “it has become increasingly clear that the brain is organized along computational lines radically different from those employed in conventional digital computers” (Reference Churchland1989, 156).Footnote ²

A concrete example can clarify the conceptual gap between computation and structure, at least in the context of computational neuroscience. It is known that the brain moves the eyes with quick saccadic movements and that, between saccades, it keeps the eyes still (Leigh and Zee Reference Marr2006). What is not obvious is that the brain holds the eyes still because it stores a memory of eye positions; when the memory is damaged, there is a constant drift of the eyes to a null point. The brain's memory of horizontal eye position appears to be implemented by a persistent neural activity, which is spread over several areas in the brain stem and the cerebellum. Various experimental studies conducted on humans, monkeys, cats, and goldfish show that this system converts transient eye-velocity-encoding inputs into persistent eye-position-encoding outputs. It was concluded that the system locates the new eye position by computing integration on the pulse saccadic inputs with respect to time, hence, the name oculomotor integrator (Robinson Reference Robinson1989). The motor neurons, reading out the memory, keep the eyes still in the new location.

Using the language of state space analysis, the oculomotor integrator moves from one state, S_i, which represents one eye position, E_i, to another state, S_j, which represents another eye position, E_j, by performing integration on eye-velocity-encoding inputs I (fig. 1). How does this system compute integration in arriving at the new position? The experimental findings show that when the eyes are still, the pattern of neural activity in the memory is constant in time and that, for every eye position, the pattern of activity is different and persistent. This persistence is thus explained in terms of a recurrent network with a continuous line attractor dynamics (Cannon, Robinson, and Shamma Reference Cannon, Robinson and Shamma1983; Seung Reference Seung1996, Reference Seung1998); the continuity is required for consistency with the analog, graded encoding of the eye position in the neural activity. Each point along the line encodes a different eye position. A new stimulus destabilizes the network, which gradually relaxes on a new point along the attractor line; this point encodes the current eye position (fig. 2).

Figure 1.

Dynamics of the oculomotor memory. The system consists of states S_i, each representing (dashed arrow) a different eye position E_i, and moves from one state, say S₁, to another state, say S₂, by integrating on pulse saccadic inputs, I, which encode eye velocity.

Figure 2.

State space portrait of the memory network. All trajectories flow toward a line attractor. Each point along the line is a different state S_i, and each state S_i represents a different eye position E_i. A new eye-velocity-encoding stimulus, I, forces the network to move to a new state (representing the current eye position), by integrating over the stimulus. From Seung Reference Seung1996, 13340; © 1996 by National Academy of Sciences, U.S.A.

This example highlights the difficulties of applying the structural notion of computing in neuroscience. First, the integrating mechanism is at odds with the idea that computing has to do with a digital architecture. It is true that the network consists of separable cells, but the activation of cells is described in terms of a continuous function governed by differential equations. Moreover, the space of states is not discrete, in the sense that it consists of separable attractors; the space, rather, is dense, in that it is a continuous line attractor, whereas each point along the line is, ideally, a different stable state. In addition, the dynamics of the network is not, in any obvious sense, a rule-governed or step-satisfaction process. The dynamics is described as minimization of the energy landscape. In short, the dynamics is no more digital than the dynamics of stomachs, tornadoes, or any other noncomputing system.

Second, the integrating mechanism is at odds with the more general claim, namely, that the distinction between computing and noncomputing has to do with structure. The oculomotor integrator has no obvious structural properties that distinguish it from all the noncomputing dynamics. Even the language used to describe the dynamics is not the language of automata or computability theory. The integrating process is described in the language of control theory and dynamical theory, which is the same language used to describe many noncomputing dynamical systems. So, all in all, it is very difficult to see what could be a relevant structural difference between this computing system and what we count as noncomputing. Structure of mechanism, it seems, contributes nothing to this distinction.

4. Computing as Information Processing: The CKS Version

In a series of papers, Patricia Churchland, Christoff Koch, and Terrence Sejnowski (henceforth, CKS) advance an alternative conception of computing, with the aim of making sense of the computational view in neuroscience. The gist of their account is that computing has to do with information or representation. In their words, “A physical system is considered a computer when its states can be taken as representing states of some other system” (Churchland, Koch, and Sejnowski Reference Churchland, Koch, Sejnowski and Schwartz1990, 48). They further argue that this semantic feature differentiates between computational explanations and other causal explanations: “Mechanical and causal explanations of chemical and electrical signals in the brain are different from computational explanations. The chief difference is that a computational explanation refers to the information content of the physical signals” (Sejnowski, Koch, and Churchland Reference Sejnowski, Koch and Churchland1988, 1300).

Characterizing computation as information processing is not unusual. In fact, most structural accounts assume that computing is a species of information processing. They assume that yet another constraint on computing, apart from digital architecture, is that there is “no computation without representation” (Fodor Reference Fodor1981, 122; Pylyshyn Reference Ramsey1984, 62; notable exceptions are Stich [Reference Stich1983] and Piccinini [Reference Putnam2008a]). The unusual feature in CKS's characterization is that it does not include, in addition to the semantic constraint, a structural constraint. Computing mechanisms, according to CKS, do not have unique architectural features that differentiate them from other causal and mechanical processes.

Giving up a structural constraint raises the concern that the characterization is too liberal. One worry is that being a computer is not just a matter of fact; it is, at least in part, a matter of perspective. This is because, strictly speaking, we can interpret the states of every system as representing something else. Interestingly, CKS themselves concede just that: “Whether something is a computer has an interest-relative component, in the sense that it depends on whether someone has an interest in the device's abstract properties and in interpreting its states as representing states of something else” (Churchland et al. Reference Churchland, Koch, Sejnowski and Schwartz1990, 48). “Delimiting the class of computers is not a sheerly empirical matter, and hence that ‘computer’ is not a natural kind”; whether something is a computer has to do with “the subjective and practical matter of whether we care what the function is” (Churchland and Sejnowski Reference Churchland and Sejnowski1992, 65).

I agree with CKS that, in some trivial sense, everything can be seen as computing. I argue elsewhere (Shagrir Reference Shagrir2010a), however, that this “universality feature” has no negative implications for the status of the computational approach in neuroscience. Here, I will make just three brief comments. First, we might view something as a computing system, although actually we do not. CKS tell us what it is that we do when we actually view something as a computer. So their account, if right, is valuable in that it supports the distinction between systems that we actually view as computing and those that we do not. Second, it is doubtful that adding a structural constraint makes the universality feature go away; in some trivial sense, everything can be seen as implementing an automaton (Putnam Reference Pylyshyn1988) or executing a program (Searle Reference Searle1992). Third, the universality feature does not entail that “computation is not discovered in the physics” (Searle Reference Searle1992, 225). We could simply distinguish between systems, arguably desktops, whose representational powers are “derivative” or “observer dependent” and systems, arguably brains, whose representational powers are “natural” or “observer independent” (see, e.g., Dretske Reference Dretske1988). If brains belong to the latter sort, there is no reason to think that their representational, hence computational, powers are not “intrinsic to physics.” Quite the contrary: scientists describe brains as computing because they think that brains have natural representational (hence, computational) powers and that discovering and referring to these powers is essential in explaining behavior.

The real concern with CKS's account lies elsewhere. It is that there are many processes that we view as mappings from representation to representation, yet we do not view them as computing; examples are old-fashioned slide projectors and mental processes such as dreaming and imagining (viewed at a phenomenological level). Unlike these processes, computational processes are, in a sense to be specified, formal processes. CKS, in effect, repeatedly emphasize that when we view something as a computer we also refer to the “device's abstract properties.” But then the question is, “Which abstract properties?” It does not seem the case that merely referring to the device's abstract properties (or describing it in abstract terms) will make the slide projector compute. The “formality condition,” it seems, states more than that.

5. Computing as Modeling

The modeling notion characterizes computing as a special sort of information processing. Computing is information processing in the sense that it maps representation to representation. But computing is more than that. In addition to representing entities in some domain, the system's input-output functional relations preserve mathematical relations between the entities that the inputs and outputs represent.

The characterization of the modeling notion originates, at least partly, in several philosophical works conducted at UCSD. One source is the notion of computing advanced by CKS that puts semantics rather than structure in the forefront. Another is the pioneering work of Gila Sher (Reference Sher1991, Reference Sher1996) on the foundations of logic. Sher advances a semantic, model-theoretic account of logical consequence. In this account, the formality constraint on logical consequence is not identified with the recursive-syntactic structure of the formulas or the inference rules. A consequence is formal (hence, logical) due to certain mathematical properties and relations denoted by the sentences/formulas. Thus, the consequence “everything is round and blue; therefore everything is blue” is logical due to certain set-theoretic relations, that is, inclusion, between the set of round and blue things and the set of blue things.Footnote ³

Another important source is Paul Churchland's work in neurophilosophy. Looking at neural network models, Churchland (see his collected papers: Reference Churchland2007) noticed that there is similarity or isomorphism between high-dimensional relations (e.g., geometrical congruence) in the state space of the representing network and high-dimensional relations (e.g., geometrical congruence) in the represented domain “in the world.” Thus, when exemplifying his point with road maps, Churchland writes that “it is these interpoint distance relations that are collectively isomorphic to a family of real world similarity relations among a set of features within the domain being represented” (Reference Churchland2007, 107). Churchland, however, does not associate this isomorphism directly with computing; his main objective is to show how these similarities underlie central notions in epistemology and in semantics.

An early characterization of computing in terms of modeling is presented by Rick Grush (Reference Grush, Machamer, Grush and McLaughlin2001).Footnote ⁴ Analyzing the Zipser-Andersen model (Reference Zipser and Andersen1988), Grush states that “we can suppose that the function computed by an idealized posterior parietal neuron is something like f = (e − e_P) σ(r − r_i)” (Reference Grush, Machamer, Grush and McLaughlin2001, 161). According to Grush, however, the term ‘compute’ does not just mean that the cells “are acting in accord with” the function f (159), namely, that there is some mechanism that supports this input-out behavior. When we say that the PPC (posterior parietal cortex) cells “really are computing a function” (159), we mean that in addition to their behaving in accordance with f, we take f to be the “interpretation of stimulus distance from preferred direction relative to the head” (161).

Grush's point can be explicated more easily with the example of the oculomotor integrator. In this context, we can see that the term is a description of two different (mathematical) relations (fig. 3). One is the mapping relation between the states of the neural network (oculomotor integrator), S_i and S_j. This relation is just the integration over the transient saccadic electrical inputs to the network, I, with respect to time.Footnote ⁵ Under this description (“a-interpretation” in Grush's vocabulary), the system is acting in accordance with integration. But we can note that also describes the relation between the eye positions E_i and E_j being represented by these states. This relation is just the integration of the eye velocity, I, with respect to time. Under this description (“e-interpretation”), the system is really computing integration.

Figure 3.

Modeling by the oculomotor integrator: describes both the relations between two states of the neural memory, S₁ and S₂, and the relations between the two eye positions, E₁ and E₂, that S₁ and S₂ represent.

More generally, we would say that a system is acting in accordance with a mathematical function f, when f is a description of the system's functional (input-output) relations. We would say that a system computes a mathematical function f when (and only when), in addition, it satisfies the following two conditions: (a) the states of the system are (or can be interpreted as) representing states in another domain, and (b) the functional relations between the states being represented (by the inputs and outputs) are also f.Footnote ⁶

What is the scope of the modeling notion of computing? Does it account for all the cases in which we describe something as computing? Does it cover, at the very least, the cases where we describe the nervous system as a computing system? I do not argue that the modeling notion underlies all the cases where the system is described as computing. Nor do I argue that it covers all the cases where the described system is a cognitive, or perhaps even a nervous, system. It might well be that structural accounts can be applied to cases to which the modeling account does not apply. What I do insist on, rather, is that the modeling notion can account for many cases, in neuroscience and elsewhere, some of which cannot be addressed by the structural notion.

First, a computing system does not have to represent only actual physical states; it can also represent potential, counterfactual, and imaginary states. It is hypothesized, for example, that our integrator is also used to compute outputs that encode desired eye position from inputs that encode the velocity of the head movement (Robinson Reference Robinson1989; Goldman et al. Reference Goldman, Kaneko, Major, Aksay, Tank and Seung2002). A system can also represent, as I shall indicate in a moment, abstract entities such as numbers and automata.

Second, the modeling notion covers many instances (although perhaps not all) of digital computing systems. There are models in classical cognitive science that assume that our cognitive (representational) system preserves high-dimensional relations (for discussion, see Ramsey Reference Zipser and Andersen2007, 77–92). The modeling notion also fits with machines that realize abstract automata. We can think of the “realization relation” in terms of representation, where the mathematical relations between the representing states correspond to the relations between the states of the automaton.Footnote ⁷ Thus, the modeling notion encompasses McCulloch and Pitts's (Reference Marr and Hildreth1943) neural network, whose behavior can be seen as “realizing” logical expressions.

Third, the modeling notion is related to analog computers; this indicates that the modeling notion is not invoked just in neuroscience and has respectable historical roots. The structural notion cannot tell why we view these systems as computing, as their structure is typical of the noncomputing dynamics. But the modeling notion can; in fact, the aim of the analog machines was to simulate mathematical relations of other physical systems by implementing these very relations in the computing system itself. A paradigm case of such a computing system is an analyzer designed to solve differential equations by using wheel-and-disc mechanisms that perform integration. Another example is the tide-predicting machine that determines the height of the tides by the integration of 10 principal toothed wheels that simulated the motion of the sun, moon, earth, and other factors that influence tides.

Finally, there are works in computational neuroscience that are a better fit with the modeling notion. One is the oculomotor integrator; others are sensorimotor systems such as the ones described by the Zipser-Andersen model (Reference Zipser and Andersen1988) and by Shadmehr and Wise (Reference Shadmehr and Wise2005; see discussion in Shagrir Reference Shagrir2006). Computational theories of perception also lean toward the modeling notion. Consider, for example, the theory of edge detection (Marr and Hildreth Reference Newell1980; Marr Reference McCulloch and Pitts1982, chap. 2; see discussion in Shagrir Reference Shagrir2010b). Marr describes the pertinent early visual processes as computing the zero crossings of the formula ∇²G∗I(x, y); I refers to the array of photoreceptors (retinal image), “∗” is a convolution operator, and ∇²G is a filtering operator; G is a Gaussian that blurs the image, and ∇² is the Laplacian operator (∂²/∂x² + ∂²/∂y²), which is sensitive to sudden intensity changes in the image. The structural notion does not fit easily with this analog-to-digital dynamics. But the modeling notion does. The formula describes mathematical relations at two different levels. It describes the “inner” relations, between the activity of the retinal photoreceptors (and retinal ganglion cells) and the activity of cells in the primary visual cortex (V1). But it also describes the “outer” relations between what is being represented by these cells. The formula states that the relations between light-intensity values, signified by I, and the light reflectance along physical edges are those of derivation (i.e., that sudden changes in light intensities occur along physical edges such as object boundaries).

6. Conclusion

This article has described how the modeling notion of computing evolved from various research projects conducted at UCSD. It was first noticed, during the 1980s, that the structural notion of computing is at odds with much of the work in connectionist cognitive science and computational neuroscience. This observation led Churchland, Koch, and Sejnowski to advance an alternative account that puts the emphasis on the idea that computing is information processing. Their account was gradually refined, however, in a way that respects the idea that computing mechanisms are formal processes. The outcome of this refinement is the modeling notion according to which a (computing) system preserves, through its input-output functional relations, mathematical relations in the represented domain.