Brette finds fault with the coding metaphor for neuronal activity in the brain on the basis of its disconnection with the causal structure of brain activity and its inadequate representational power. In so doing, he shows why brain activity is not compatible with a computational picture that includes coding of sensory signals, computation with those codes, and then decoding to generate behavior. The quotations in Section 4.1 even suggest that decoding must occur before the brain can interpret codes to determine action. Only in one sentence in his final section, 5.2 (para. 2), does he come near to noticing the real problem with coding: “Even if it were possible to map brain activity to computational descriptions, neural codes would not provide the adequate mapping.” He is correct about adequate mappings, but the bigger problem is the one implicit in the “even if” clause: Computational descriptions are not the way to describe what it is the brain does.
First let me clear away one objection to my argument: Yes, the brain computes if we look upon it as a device that receives sensory signals encoded as neuronal firings and emits behavioral commands also encoded as neuronal firings. I think it is useful to constrain “computation” to its nonmetaphorical usage to describe what goes on in Turing or Von Neumann computers – not to be a stickler for definition, but because the aspects in which the activity in the brain differs from the activity in those machines are precisely the things that are at the heart of the hard problems of neuroscience, the things that the computational metaphor drives researchers to look for that are not there: meanings assignable externally to neuronal firings and algorithms that describe a finite sequence of steps to get from a defined input to a defined output, that is, programs. Without externally assigned meanings and programs to operate upon them, computation is only a metaphor, in my view a big bad metaphor that has only held back the science of the brain.
Why is the computer metaphor bad? Because it inspires people to look for codes and algorithms as solutions to these basic problems instead of looking for mechanisms relevant to the brain. For example, it led Tsotsos (Reference Tsotsos2011) to the absurd, admittedly strawman, conclusion that a general unbounded visual match on an image with p pixels requires time on order O(p 22p). So the brain must be doing something else. Perhaps rather than search for visual algorithms, one could address questions like these: How does the firing of a neuron in the brain come to signify something to those neurons that receive that firing, as opposed to signifying something to the experimenter who records it? How do these firings organize themselves, as a result of experience in the world, to produce behavioral outputs that serve the survival needs of the organism, without an external programmer?
As Brette is well aware, the meaning of a neuronal spike, unlike a bit in a computer, cannot be described in isolation. Perhaps the best discussions of how neural firings come to have significance for other neurons are those provided by Harnad (Reference Harnad and Forrest1990a) and Bickhard & Terveen (Reference Bickhard and Terveen1996). It won't do just to add more codes (Brette, sect. 1, last para.). Neurons are members of assemblies that form and re-form according to the situation (Izhikevich Reference Izhikevich2006); the meaning of neuronal firing depends on context (Gilbert Reference Gilbert1996) and may differ for different recipient neurons. Analyzing neuronal firing from the point of view of an “ideal observer” is useless because neuronal firing is not just a well-defined but noisy code; for success one needs a more complicated observer, perhaps a “homunculus,” which can vary in its responses according to the total picture provided by all the other neurons in the system, interacting through their recurrent or reentrant connections. But then one has just kicked the can down the road; such a homunculus is not a computer. It is not fair to silently ascribe key elements of the performance of a brain model to components that are not included in the model, the unmodeled homunculus in the machine (Reeke and Edelman Reference Reeke and Edelman1988).
In short, coding fails because the only thing it is good for is as input to and output from algorithms. But if not algorithms, then what? The standard computer science picture of algorithms, even including those that emulate nondeterministic physical phenomena, is still the Turing machine definition: a predefined sequence of operations taken from a predefined set designed to accomplish a predefined computation. With this broad definition, algorithms can no doubt be found in the brain. But what are the predefined operations: membrane depolarization, spike firing, volume diffusion of chemical signals? How are these operations organized without a programmer: synaptic plasticity regulated by multiple chemical signals conveying states of arousal, emotions, homeostasis, reward, and punishment? And what are the predefined computations, or effective methods of performing behaviors: obtaining food, water, mates, or just some ill-defined pleasure signal? The answers to these questions are not found in algorithm theory.
Fodor and Pylyshyn (Reference Fodor, Pylyshyn, Pinker and Mehler1988) have argued persuasively that so-called connectionist models (Rumelhart et al. Reference Rumelhart and McClelland1986) are not sufficient to implement all cognitive activities of brains; symbol systems and syntactic operations on them are needed. There is no contradiction once one looks at real brains: symbol systems and syntactic operations upon them can be constructed from the signals and operations upon them that I have just argued we need to look for in the brain. The question we only have partial answers to is how this is accomplished by experience in the complex real world. Computation theory does not provide the answer to that problem. Brette's final suggestion, that the solution resides somewhere in the area of modeling the full sensorimotor loop, is indeed the best approach we know of, one perhaps most fruitfully investigated by building “neurorobots,” robots controlled by model neuronal systems. This approach has also been called “synthetic neural modeling” (Reeke et al. Reference Reeke, Finkel, Sporns, Edelman, Edelman, Gall and Cowan1990).
You have
Access
Brette finds fault with the coding metaphor for neuronal activity in the brain on the basis of its disconnection with the causal structure of brain activity and its inadequate representational power. In so doing, he shows why brain activity is not compatible with a computational picture that includes coding of sensory signals, computation with those codes, and then decoding to generate behavior. The quotations in Section 4.1 even suggest that decoding must occur before the brain can interpret codes to determine action. Only in one sentence in his final section, 5.2 (para. 2), does he come near to noticing the real problem with coding: “Even if it were possible to map brain activity to computational descriptions, neural codes would not provide the adequate mapping.” He is correct about adequate mappings, but the bigger problem is the one implicit in the “even if” clause: Computational descriptions are not the way to describe what it is the brain does.
First let me clear away one objection to my argument: Yes, the brain computes if we look upon it as a device that receives sensory signals encoded as neuronal firings and emits behavioral commands also encoded as neuronal firings. I think it is useful to constrain “computation” to its nonmetaphorical usage to describe what goes on in Turing or Von Neumann computers – not to be a stickler for definition, but because the aspects in which the activity in the brain differs from the activity in those machines are precisely the things that are at the heart of the hard problems of neuroscience, the things that the computational metaphor drives researchers to look for that are not there: meanings assignable externally to neuronal firings and algorithms that describe a finite sequence of steps to get from a defined input to a defined output, that is, programs. Without externally assigned meanings and programs to operate upon them, computation is only a metaphor, in my view a big bad metaphor that has only held back the science of the brain.
Why is the computer metaphor bad? Because it inspires people to look for codes and algorithms as solutions to these basic problems instead of looking for mechanisms relevant to the brain. For example, it led Tsotsos (Reference Tsotsos2011) to the absurd, admittedly strawman, conclusion that a general unbounded visual match on an image with p pixels requires time on order O(p 22p). So the brain must be doing something else. Perhaps rather than search for visual algorithms, one could address questions like these: How does the firing of a neuron in the brain come to signify something to those neurons that receive that firing, as opposed to signifying something to the experimenter who records it? How do these firings organize themselves, as a result of experience in the world, to produce behavioral outputs that serve the survival needs of the organism, without an external programmer?
As Brette is well aware, the meaning of a neuronal spike, unlike a bit in a computer, cannot be described in isolation. Perhaps the best discussions of how neural firings come to have significance for other neurons are those provided by Harnad (Reference Harnad and Forrest1990a) and Bickhard & Terveen (Reference Bickhard and Terveen1996). It won't do just to add more codes (Brette, sect. 1, last para.). Neurons are members of assemblies that form and re-form according to the situation (Izhikevich Reference Izhikevich2006); the meaning of neuronal firing depends on context (Gilbert Reference Gilbert1996) and may differ for different recipient neurons. Analyzing neuronal firing from the point of view of an “ideal observer” is useless because neuronal firing is not just a well-defined but noisy code; for success one needs a more complicated observer, perhaps a “homunculus,” which can vary in its responses according to the total picture provided by all the other neurons in the system, interacting through their recurrent or reentrant connections. But then one has just kicked the can down the road; such a homunculus is not a computer. It is not fair to silently ascribe key elements of the performance of a brain model to components that are not included in the model, the unmodeled homunculus in the machine (Reeke and Edelman Reference Reeke and Edelman1988).
In short, coding fails because the only thing it is good for is as input to and output from algorithms. But if not algorithms, then what? The standard computer science picture of algorithms, even including those that emulate nondeterministic physical phenomena, is still the Turing machine definition: a predefined sequence of operations taken from a predefined set designed to accomplish a predefined computation. With this broad definition, algorithms can no doubt be found in the brain. But what are the predefined operations: membrane depolarization, spike firing, volume diffusion of chemical signals? How are these operations organized without a programmer: synaptic plasticity regulated by multiple chemical signals conveying states of arousal, emotions, homeostasis, reward, and punishment? And what are the predefined computations, or effective methods of performing behaviors: obtaining food, water, mates, or just some ill-defined pleasure signal? The answers to these questions are not found in algorithm theory.
Fodor and Pylyshyn (Reference Fodor, Pylyshyn, Pinker and Mehler1988) have argued persuasively that so-called connectionist models (Rumelhart et al. Reference Rumelhart and McClelland1986) are not sufficient to implement all cognitive activities of brains; symbol systems and syntactic operations on them are needed. There is no contradiction once one looks at real brains: symbol systems and syntactic operations upon them can be constructed from the signals and operations upon them that I have just argued we need to look for in the brain. The question we only have partial answers to is how this is accomplished by experience in the complex real world. Computation theory does not provide the answer to that problem. Brette's final suggestion, that the solution resides somewhere in the area of modeling the full sensorimotor loop, is indeed the best approach we know of, one perhaps most fruitfully investigated by building “neurorobots,” robots controlled by model neuronal systems. This approach has also been called “synthetic neural modeling” (Reeke et al. Reference Reeke, Finkel, Sporns, Edelman, Edelman, Gall and Cowan1990).