Brette argues that coding is an inappropriate concept for explanations of neurocognitive phenomena. Brette identifies three properties of coding: correspondence, representation, and causality. Brette grants correspondence but rejects both representation and causality for the neural code. Although we disagree with his analyses of representation and causality, we limit our critique to the latter.
Brette's argument against causality focuses on two points. First, coding assumes that the parts of a cognitive system have separate functions. However, Brette claims that function cannot be attributed to the brain's parts. Second, coding implies linear causality for the brain. Brette argues instead that the brain features circular, coupled causality.
We argue that functions can be attributed to the parts of brains and, though brains are dynamical systems with circular causality, linear causality may still apply. We contend that the rejection of functions for parts of the brain constitutes a direct attack on the nature of explanation in cognitive neuroscience. Furthermore, the causality claim commits a category mistake, as the linear structure of the concept need not be mimicked by the causal structure of the brain. Finally, linear approximations are immensely successful in neuroscientific explanations.
Brette first argues against the assignment of decoding and encoding functions to parts of the brain. Such assignment requires the analysis of behavior “independently of the system in which the neurons are embedded” (sect. 4.1, para. 2). But such an analysis “determines a neural code that … depends neither on the goals of the animals nor on the effect of spikes on … actions” (sect. 4.1, para. 6). He concludes that the analysis of the brain by decomposition cannot proceed because “function can be meaningfully ascribed to the organism as a system, but not … to the components of this system” (sect. 4.1, para. 6). As such coding functions are defined independently of the organism's goals, Brette rejects the possibility of assigning coding functions to parts of the brain.
Brette's analysis relies on several misleading claims. First, some notions of function that do not rely on goals, such as causal role functions (Cummins Reference Cummins1975; Walsh & Ariew Reference Walsh and Ariew1996), can be attributed to parts of organisms. Second, nothing about goals prevents function ascription to the organism's parts while permitting function ascription to the whole organism. As part of a larger system, the function of the part could share the goal of the organism. Indeed, this is typical for biology, where functions are often assigned to organs – such as the circulation of blood for the heart or cleaning the blood of toxins for the kidneys – even though the goals of these functions might be for the organism. Third, encoding and decoding can take into account variables relevant to the organism's biological fitness (Rice Reference Rice2015), and goal-oriented functions are defined and ascribed with respect to those fitness-relevant variables.
Furthermore, we contend that coding and other analysis-by-decomposition models are indispensable to explanations of brain function that integrate with psychology. These models break down psychological phenomena into subfunctions for explanation (cf. Dennett Reference Dennett1981; Lycan Reference Lycan1981; Marr Reference Marr1982a). Decomposition requires that the psychological properties of subfunctions be reduced or removed when ascribing those subfunctions to component parts and proceeds recursively with finer decompositions with fewer psychological properties. At the lowest levels, functional descriptions are completely bare of psychology, yielding a reduction to neuroscience. Coding is a perfect example of such decomposition. The concept of coding implies encoding and decoding functions with reduced intentional implications, as those functions are grounded purely in probabilistic terms (Shannon & Weaver Reference Shannon and Weaver1963). While message contents still need to be determined, coding analyses of systems like the brain can result in parts that carry weaker intentional properties.
Brette next argues that the causality implied by coding does not apply to the brain. The neural code has linear causality (viz., input → encoder → decode → output), whereas the brain possesses circular, coupled causality. The brain is like a tent, where “different elements are co-determined…. In addition to the coupling of neurons, the brain itself is coupled to its environment; that is, there is circular and not linear causality” (sect. 4.4, para. 2). Linear causality refers to temporally sequential, causally related pairs of states, whereas tentlike causality refers to simultaneous, jointly causally related sets of states. In short, the causal structure implied by the neural code fails to match the causal structure of the brain.
We first note that tentlike causality is consistent with linear causality. Linear encoding-decoding relationships between each pair of elements are consistent with an overall picture of a circular, coupled causal system. Indeed, Brette arguably commits a category mistake (Ryle Reference Ryle1949): while the conceptual analysis of coding involves a linear structure (Shannon & Weaver Reference Shannon and Weaver1963), the implementations of encoding and decoding functions need not. This category mistake is further illustrated by Brette's claim that coding structures imply the “discrete temporality of an algorithm, … disconnected from physical time…. But … dynamical systems cannot in general be mapped to algorithmic descriptions” (sect. 4.4, para. 1). However, nothing about the coding metaphor entails discrete time, and one variable may encode another in the sense of carrying Shannon information without being part of an algorithm.
Brette's argument also ignores the utility of linearity and discrete time for analyzing complex systems. Continuous-time systems can be well approximated by discrete-time systems (Oppenheim & Schafer Reference Oppenheim and Schafer2013), and many equivalences exist between dynamical systems with circular causality and approximators with iterated linear causality (e.g., Funahashi et al. Reference Funahashi and Nakamura1993; Schäfer & Zimmerman Reference Schäfer, Zimmermann, Kollias, Stafylopatis, Duch and Oja2007). Furthermore, systems with tent-like causality can be approximated by linear causal elements, such as by iteratively computing local relationships (e.g., Geman & Geman Reference Geman and Geman1984; Roth & Black Reference Roth and Black2005; Van Den Oord et al. Reference van den Oord, Kalchbrenner and Kavukcuoglu2016). Deep learning research uses discrete-time architectures to model many forms of continuous behavior as well (e.g., handwriting [Graves Reference Graves2013], speech synthesis [Van Den Oord et al. Reference van den Oord, Dieleman, Zen, Simonyan, Vinyals, Graves, Kalchbrenner, Senior and Kavukcuoglu2016], video prediction [Srivastava et al. Reference Srivastava, Mansimov and Salakhutdinov2015], robotic control [Levine et al. Reference Levine, Finn, Darrell and Abbeel2016], humanoid running [Heess et al. Reference Heess, Sriram, Lemmon, Merel, Wayne, Tassa, Erez, Wang, Ali Eslami, Riedmiller and Silver2017]). Hence, complicated, ethologically relevant behavior can be analyzed with linear causality and discrete time, and this analysis is likely to remain a crucial part of building and designing models of this behavior (Santoro et al. Reference Santoro, Hill, Barrett, Raposo, Botvinick and Lillicrap2019).
Brette argues that coding is an inappropriate concept for explanations of neurocognitive phenomena. Brette identifies three properties of coding: correspondence, representation, and causality. Brette grants correspondence but rejects both representation and causality for the neural code. Although we disagree with his analyses of representation and causality, we limit our critique to the latter.
Brette's argument against causality focuses on two points. First, coding assumes that the parts of a cognitive system have separate functions. However, Brette claims that function cannot be attributed to the brain's parts. Second, coding implies linear causality for the brain. Brette argues instead that the brain features circular, coupled causality.
We argue that functions can be attributed to the parts of brains and, though brains are dynamical systems with circular causality, linear causality may still apply. We contend that the rejection of functions for parts of the brain constitutes a direct attack on the nature of explanation in cognitive neuroscience. Furthermore, the causality claim commits a category mistake, as the linear structure of the concept need not be mimicked by the causal structure of the brain. Finally, linear approximations are immensely successful in neuroscientific explanations.
Brette first argues against the assignment of decoding and encoding functions to parts of the brain. Such assignment requires the analysis of behavior “independently of the system in which the neurons are embedded” (sect. 4.1, para. 2). But such an analysis “determines a neural code that … depends neither on the goals of the animals nor on the effect of spikes on … actions” (sect. 4.1, para. 6). He concludes that the analysis of the brain by decomposition cannot proceed because “function can be meaningfully ascribed to the organism as a system, but not … to the components of this system” (sect. 4.1, para. 6). As such coding functions are defined independently of the organism's goals, Brette rejects the possibility of assigning coding functions to parts of the brain.
Brette's analysis relies on several misleading claims. First, some notions of function that do not rely on goals, such as causal role functions (Cummins Reference Cummins1975; Walsh & Ariew Reference Walsh and Ariew1996), can be attributed to parts of organisms. Second, nothing about goals prevents function ascription to the organism's parts while permitting function ascription to the whole organism. As part of a larger system, the function of the part could share the goal of the organism. Indeed, this is typical for biology, where functions are often assigned to organs – such as the circulation of blood for the heart or cleaning the blood of toxins for the kidneys – even though the goals of these functions might be for the organism. Third, encoding and decoding can take into account variables relevant to the organism's biological fitness (Rice Reference Rice2015), and goal-oriented functions are defined and ascribed with respect to those fitness-relevant variables.
Furthermore, we contend that coding and other analysis-by-decomposition models are indispensable to explanations of brain function that integrate with psychology. These models break down psychological phenomena into subfunctions for explanation (cf. Dennett Reference Dennett1981; Lycan Reference Lycan1981; Marr Reference Marr1982a). Decomposition requires that the psychological properties of subfunctions be reduced or removed when ascribing those subfunctions to component parts and proceeds recursively with finer decompositions with fewer psychological properties. At the lowest levels, functional descriptions are completely bare of psychology, yielding a reduction to neuroscience. Coding is a perfect example of such decomposition. The concept of coding implies encoding and decoding functions with reduced intentional implications, as those functions are grounded purely in probabilistic terms (Shannon & Weaver Reference Shannon and Weaver1963). While message contents still need to be determined, coding analyses of systems like the brain can result in parts that carry weaker intentional properties.
Brette next argues that the causality implied by coding does not apply to the brain. The neural code has linear causality (viz., input → encoder → decode → output), whereas the brain possesses circular, coupled causality. The brain is like a tent, where “different elements are co-determined…. In addition to the coupling of neurons, the brain itself is coupled to its environment; that is, there is circular and not linear causality” (sect. 4.4, para. 2). Linear causality refers to temporally sequential, causally related pairs of states, whereas tentlike causality refers to simultaneous, jointly causally related sets of states. In short, the causal structure implied by the neural code fails to match the causal structure of the brain.
We first note that tentlike causality is consistent with linear causality. Linear encoding-decoding relationships between each pair of elements are consistent with an overall picture of a circular, coupled causal system. Indeed, Brette arguably commits a category mistake (Ryle Reference Ryle1949): while the conceptual analysis of coding involves a linear structure (Shannon & Weaver Reference Shannon and Weaver1963), the implementations of encoding and decoding functions need not. This category mistake is further illustrated by Brette's claim that coding structures imply the “discrete temporality of an algorithm, … disconnected from physical time…. But … dynamical systems cannot in general be mapped to algorithmic descriptions” (sect. 4.4, para. 1). However, nothing about the coding metaphor entails discrete time, and one variable may encode another in the sense of carrying Shannon information without being part of an algorithm.
Brette's argument also ignores the utility of linearity and discrete time for analyzing complex systems. Continuous-time systems can be well approximated by discrete-time systems (Oppenheim & Schafer Reference Oppenheim and Schafer2013), and many equivalences exist between dynamical systems with circular causality and approximators with iterated linear causality (e.g., Funahashi et al. Reference Funahashi and Nakamura1993; Schäfer & Zimmerman Reference Schäfer, Zimmermann, Kollias, Stafylopatis, Duch and Oja2007). Furthermore, systems with tent-like causality can be approximated by linear causal elements, such as by iteratively computing local relationships (e.g., Geman & Geman Reference Geman and Geman1984; Roth & Black Reference Roth and Black2005; Van Den Oord et al. Reference van den Oord, Kalchbrenner and Kavukcuoglu2016). Deep learning research uses discrete-time architectures to model many forms of continuous behavior as well (e.g., handwriting [Graves Reference Graves2013], speech synthesis [Van Den Oord et al. Reference van den Oord, Dieleman, Zen, Simonyan, Vinyals, Graves, Kalchbrenner, Senior and Kavukcuoglu2016], video prediction [Srivastava et al. Reference Srivastava, Mansimov and Salakhutdinov2015], robotic control [Levine et al. Reference Levine, Finn, Darrell and Abbeel2016], humanoid running [Heess et al. Reference Heess, Sriram, Lemmon, Merel, Wayne, Tassa, Erez, Wang, Ali Eslami, Riedmiller and Silver2017]). Hence, complicated, ethologically relevant behavior can be analyzed with linear causality and discrete time, and this analysis is likely to remain a crucial part of building and designing models of this behavior (Santoro et al. Reference Santoro, Hill, Barrett, Raposo, Botvinick and Lillicrap2019).