The central tenet of Brette's article is that the notion of a “neural code” is a metaphor, which, despite being popular, does not provide a valid basis for theories of brain function. While we agree with many things that Brette says – and, in particular, would argue that knowing the neural code is not sufficient for an understanding of brain function – we think that this central tenet is itself unfounded. A code is, quite simply, a set of rules. One could interpret the “neural code” as the set of rules which neurons obey (we accept that the term is usually used in a somewhat narrower sense). In cryptography, a code is the set of rules which map interpretable messages into a secret form not interpretable by an enemy agent. The aim of the enemy agent is to recover the interpretable form (decode), without knowing in advance the rules. In sensory physiology, we are often concerned with understanding the rules governing how sensory events in the world map onto neuronal activity. Similarly, we do not know the rules (code) a priori – but recovering them, through experiment or otherwise, helps us to understand the system. In one case, the rules are designed by a cryptographer; in the other, they emerged through Darwinian selection. However, this has little bearing upon the task confronting the agent. Far from being a metaphor, we argue that the term neural code is literally applicable to the rules governing the relationship between environmental stimuli and neural activity, as well as to other areas of brain function such as the generation of movement. We are thus not employing “code” as a metaphor, except inasmuch as such rules may only approximate far more complex biophysics.
Brette argues that neural codes depend upon experimental context. Again, we disagree. The most relevant experimental context is the full repertoire of natural behaviour; we would perform all experiments in this context if it were feasible. Unfortunately, the world is high dimensional, and this would typically result in inadequate sampling of the variables we wish to study within the lifetime of the experiment (or experimenter). We restrict the context in order to sample sufficiently to make statistically valid conclusions (i.e., use an experimental design), attempting instead to ensure generality by also performing different experiments (e.g., with natural images as well as bars). If the neural code has in fact been completely described, the rulebook used does not depend upon context. We are mystified by the counterview proposed – that perceptual systems “build information from relations between sensory signals and actions” (Brette, abstract). Leaving aside that information cannot be “built” (we assume it is representation that is meant here), this structured internal model would, we expect, correspond exactly to our set of rules, or “code.”
We think that, to understand brain function, it is likely to be necessary to understand the neural code, as couched in these terms, perhaps with the use of quantitative techniques from information theory (see, e.g., Panzeri and Schultz Reference Panzeri and Schultz2001; Panzeri et al. Reference Panzeri, Schultz, Treves and Rolls1999). However, we would not argue that it is sufficient. As engineers, our view is that the brain is an information processing system, and to understand it, we need to know how the information is represented mechanistically. This merely sets the scene: To “reverse engineer” (disassemble to understand) the system, we also need to understand many other “design principles” (Sterling and Laughlin Reference Sterling and Laughlin2015) – the performance constraints, the organisational principles, and so forth – and finally, the specific computations that occur. To do this, we employ (and develop, where needed) additional branches of engineering, such as control theory. The autonomic nervous system perhaps provides a good illustrative example. Parasympathetic and sympathetic nerve fibres connect the internal organs of the body in a complex control system regulating heart rate, digestion, respiratory rate, and many other internal variables. Control theory would seem to be a fruitful approach to study the operations of this system. A first step towards this would be writing down the control variables, for instance, how is blood oxygenation encoded in neural activity in the parasympathetic nervous system? Understanding the neural code for the signals in this system is a necessary (but not sufficient) part of the process of understanding how the system works. Brette provides a counterexample concerning the heart, in which one might conclude that cardiac cells rate-code for running speed. However, any reasonable neural coding analysis of this system would examine vascular flow rates and pressures, obtaining the correct conclusion.
Brette suggests that it is unclear if neural codes can represent structure, giving cell assemblies as an example and noting that a labelled graph, not just a “bag of neurons” (sect. 2.5, para. 2), is required to do so. There is evidence that cell assemblies do support such structured relations. The inhibition of specific assemblies is enough to erase long-lasting behaviours, such as cocaine-driven place preference (Trouche et al. Reference Trouche, Perestenko, Ven, van de Bratley, McNamara, Campo-Urriza, Black, Reijmers and Dupret2016). Co-activation of such cell assemblies can encode goal-driven behaviours, relating both “words” (e.g., places, rewards, actions) and “syntax” (e.g., causal and contextual relations between “words”). Brette alludes to temporal structure as providing a possible solution to this problem, and in fact, there is mounting evidence that this is the case. As an example, consider sharp-wave ripples (SWRs), synchronous, brainwide, fast oscillatory events originating in the hippocampus. During SWRs, place cells are re-activated, according to their spatial tuning, in time-compressed sequences (Buzsaki Reference Buzsáki2015). Spike timing is crucial in these “replay” events, which were shown to encode previously experienced spatial trajectories and memories in rats (Maboudi et al. Reference Maboudi, Ackermann, de Jong, Pfeiffer, Foster, Diba and Kemere2018). We do not yet fully understand the “rulebook,” or neural code, underpinning such structure, but we would argue that doing so will be crucial to understanding how memories are stored, recalled, and used in brainwide networks.
In summary, for us the “neural code” is the set of rules governing how the activity of neurons relates to the state of the system in which they are situated. Knowing this rulebook is likely to be an essential step in the path towards understanding brain function.
The central tenet of Brette's article is that the notion of a “neural code” is a metaphor, which, despite being popular, does not provide a valid basis for theories of brain function. While we agree with many things that Brette says – and, in particular, would argue that knowing the neural code is not sufficient for an understanding of brain function – we think that this central tenet is itself unfounded. A code is, quite simply, a set of rules. One could interpret the “neural code” as the set of rules which neurons obey (we accept that the term is usually used in a somewhat narrower sense). In cryptography, a code is the set of rules which map interpretable messages into a secret form not interpretable by an enemy agent. The aim of the enemy agent is to recover the interpretable form (decode), without knowing in advance the rules. In sensory physiology, we are often concerned with understanding the rules governing how sensory events in the world map onto neuronal activity. Similarly, we do not know the rules (code) a priori – but recovering them, through experiment or otherwise, helps us to understand the system. In one case, the rules are designed by a cryptographer; in the other, they emerged through Darwinian selection. However, this has little bearing upon the task confronting the agent. Far from being a metaphor, we argue that the term neural code is literally applicable to the rules governing the relationship between environmental stimuli and neural activity, as well as to other areas of brain function such as the generation of movement. We are thus not employing “code” as a metaphor, except inasmuch as such rules may only approximate far more complex biophysics.
Brette argues that neural codes depend upon experimental context. Again, we disagree. The most relevant experimental context is the full repertoire of natural behaviour; we would perform all experiments in this context if it were feasible. Unfortunately, the world is high dimensional, and this would typically result in inadequate sampling of the variables we wish to study within the lifetime of the experiment (or experimenter). We restrict the context in order to sample sufficiently to make statistically valid conclusions (i.e., use an experimental design), attempting instead to ensure generality by also performing different experiments (e.g., with natural images as well as bars). If the neural code has in fact been completely described, the rulebook used does not depend upon context. We are mystified by the counterview proposed – that perceptual systems “build information from relations between sensory signals and actions” (Brette, abstract). Leaving aside that information cannot be “built” (we assume it is representation that is meant here), this structured internal model would, we expect, correspond exactly to our set of rules, or “code.”
We think that, to understand brain function, it is likely to be necessary to understand the neural code, as couched in these terms, perhaps with the use of quantitative techniques from information theory (see, e.g., Panzeri and Schultz Reference Panzeri and Schultz2001; Panzeri et al. Reference Panzeri, Schultz, Treves and Rolls1999). However, we would not argue that it is sufficient. As engineers, our view is that the brain is an information processing system, and to understand it, we need to know how the information is represented mechanistically. This merely sets the scene: To “reverse engineer” (disassemble to understand) the system, we also need to understand many other “design principles” (Sterling and Laughlin Reference Sterling and Laughlin2015) – the performance constraints, the organisational principles, and so forth – and finally, the specific computations that occur. To do this, we employ (and develop, where needed) additional branches of engineering, such as control theory. The autonomic nervous system perhaps provides a good illustrative example. Parasympathetic and sympathetic nerve fibres connect the internal organs of the body in a complex control system regulating heart rate, digestion, respiratory rate, and many other internal variables. Control theory would seem to be a fruitful approach to study the operations of this system. A first step towards this would be writing down the control variables, for instance, how is blood oxygenation encoded in neural activity in the parasympathetic nervous system? Understanding the neural code for the signals in this system is a necessary (but not sufficient) part of the process of understanding how the system works. Brette provides a counterexample concerning the heart, in which one might conclude that cardiac cells rate-code for running speed. However, any reasonable neural coding analysis of this system would examine vascular flow rates and pressures, obtaining the correct conclusion.
Brette suggests that it is unclear if neural codes can represent structure, giving cell assemblies as an example and noting that a labelled graph, not just a “bag of neurons” (sect. 2.5, para. 2), is required to do so. There is evidence that cell assemblies do support such structured relations. The inhibition of specific assemblies is enough to erase long-lasting behaviours, such as cocaine-driven place preference (Trouche et al. Reference Trouche, Perestenko, Ven, van de Bratley, McNamara, Campo-Urriza, Black, Reijmers and Dupret2016). Co-activation of such cell assemblies can encode goal-driven behaviours, relating both “words” (e.g., places, rewards, actions) and “syntax” (e.g., causal and contextual relations between “words”). Brette alludes to temporal structure as providing a possible solution to this problem, and in fact, there is mounting evidence that this is the case. As an example, consider sharp-wave ripples (SWRs), synchronous, brainwide, fast oscillatory events originating in the hippocampus. During SWRs, place cells are re-activated, according to their spatial tuning, in time-compressed sequences (Buzsaki Reference Buzsáki2015). Spike timing is crucial in these “replay” events, which were shown to encode previously experienced spatial trajectories and memories in rats (Maboudi et al. Reference Maboudi, Ackermann, de Jong, Pfeiffer, Foster, Diba and Kemere2018). We do not yet fully understand the “rulebook,” or neural code, underpinning such structure, but we would argue that doing so will be crucial to understanding how memories are stored, recalled, and used in brainwide networks.
In summary, for us the “neural code” is the set of rules governing how the activity of neurons relates to the state of the system in which they are situated. Knowing this rulebook is likely to be an essential step in the path towards understanding brain function.