Although Brette's official target is the “coding metaphor,” he often seems more broadly sceptical of the idea that patterns of neural activity can represent properties of the external environment. His scepticism is motivated by examples where, although a neuron's activity is sensitive to an environmental property and carries correlational information about that property (i.e., information in Shannon's sense), this close dependence is a fragile artefact of a particular experimental setup, prone to disappear as soon as the subject leaves the lab and re-enters the wider world.
He gives us several cautionary tales along these lines. For example, a single cone cell carries correlational information about wavelength in a laboratory setup that fixes the intensity of the light, but the correlation breaks down when the intensity is allowed to vary. A neuron in the medial superior olive may carry correlational information about the location of a sound in a laboratory setup that holds fixed intensity and frequency, but intensity and frequency will not be fixed outside the lab.
Brette argues that the use of the term coding to describe these fragile correlations is misleading: it obscures their dependence on a specific laboratory context, suggesting instead that the correlation is robust and general. He also laments a particular kind of slip: the existence of a fragile correlation leads to talk of coding, and this leads to an assumption that neural activity represents the detected property, where “representing” goes beyond merely “correlating with” and implies semantic content.
The article brings out a real conceptual problem in contemporary neuroscience: representing a feature in the environment is sometimes considered synonymous with detecting that feature in a controlled setting. Loose talk of “coding” may be part of this problem, if it encourages researchers to conflate the distinction between detection and representation. But is the only alternative abandoning all talk of representation in the brain?
A different approach is to set out an explicit and demanding set of conditions under which a pattern of neural activity genuinely represents an environmental property. This approach has been pursued in some depth in the philosophical literature on mental representation. Although philosophical accounts of mental representation are many and various, none simply identifies representation with correlation. Typical additional requirements include the following: (a) the correlation between the detected feature and the pattern of neural activity is exploited in further processing; (b) this further processing ends up guiding action and explaining success; and (c) misrepresentation is possible (e.g., when neurons fire in the absence of the stimulus they evolved to detect), and this helps explain unsuccessful action. These criteria have been discussed under different guises in the philosophical literature on causal and teleosemantic theories of mental content (Dretske Reference Dretske1981; Millikan Reference Millikan1984; Neander Reference Neander2017; Shea Reference Shea2018). With a theory of this general type in hand (such as Shea's or Neander's), there is room for a serious empirical debate about whether a given pattern of neural activity represents the property it detects.
As an illustrative example, consider mirror neurons: neurons that fire both during motor action production and during observation of motor acts in others. These neurons were first discovered in area F5 and in the inferior parietal lobule of the monkey and later found in functionally equivalent areas of the human brain (Fabbri-Destro & Rizzolatti Reference Fabbri-Destro and Rizzolatti2008). There is no doubt that their activity correlates under controlled conditions with properties of a variety of motor acts, such as grasping movements. They belong to an information-processing route that involves other brain regions and they receive information from higher visual areas. There are also well-documented contextual effects: for example, some (but not all) neurons that fire in response to a perceived grasping movement will also fire if the movement is partially occluded by a screen, but only if the agent previously saw a graspable object hidden behind the screen (Umiltà et al. Reference Umiltà, Kohler, Gallese, Fogassi, Fadiga, Keysers and Rizzolatti2001).
Here we have systematic correlation, inviting casual talk of neurons “coding” for motor acts. Yet it would be hasty to infer that the neurons represent the properties they detect. There is a long-running debate about their function in cognition and about what they represent (Cook et al. Reference Cook, Bird, Catmur, Press and Heyes2014). To settle that, we need to address a package of further questions: Do mirror neurons detect the same type of property in a variety of experimental settings and with different techniques? Are the correlations with properties of motor acts exploited in further processing? Do they guide successful action? Are there cases in which unsuccessful action can be explained by the mirror neurons misrepresenting a property of a motor act? As long as clear answers to these questions remain elusive, we should remain agnostic on the question of what features of the external world mirror neurons represent. However, the fact these questions are difficult to answer is no reason to doubt the validity and value of the concept of representation for understanding the brain. On the contrary, the concept of representation is exactly what we need to formulate and test hypotheses about the function of mirror neurons. To call for the elimination of the concept would be akin to arguing that, because it is often hard to tell whether or not a trait is an adaptation, we should eliminate the concept of “adaptation” from evolutionary biology.
In sum, Brette correctly highlights the shortcomings of the “coding metaphor” as a theory of representation in the brain. His article points to the need to replace fairly loose talk of “coding” with more precisely articulated concepts of feature detection, correlation, and representation. The concept of representation is especially onerous and should be used with great care. But to eliminate the concept altogether would be throwing out the baby with the bathwater.
Although Brette's official target is the “coding metaphor,” he often seems more broadly sceptical of the idea that patterns of neural activity can represent properties of the external environment. His scepticism is motivated by examples where, although a neuron's activity is sensitive to an environmental property and carries correlational information about that property (i.e., information in Shannon's sense), this close dependence is a fragile artefact of a particular experimental setup, prone to disappear as soon as the subject leaves the lab and re-enters the wider world.
He gives us several cautionary tales along these lines. For example, a single cone cell carries correlational information about wavelength in a laboratory setup that fixes the intensity of the light, but the correlation breaks down when the intensity is allowed to vary. A neuron in the medial superior olive may carry correlational information about the location of a sound in a laboratory setup that holds fixed intensity and frequency, but intensity and frequency will not be fixed outside the lab.
Brette argues that the use of the term coding to describe these fragile correlations is misleading: it obscures their dependence on a specific laboratory context, suggesting instead that the correlation is robust and general. He also laments a particular kind of slip: the existence of a fragile correlation leads to talk of coding, and this leads to an assumption that neural activity represents the detected property, where “representing” goes beyond merely “correlating with” and implies semantic content.
The article brings out a real conceptual problem in contemporary neuroscience: representing a feature in the environment is sometimes considered synonymous with detecting that feature in a controlled setting. Loose talk of “coding” may be part of this problem, if it encourages researchers to conflate the distinction between detection and representation. But is the only alternative abandoning all talk of representation in the brain?
A different approach is to set out an explicit and demanding set of conditions under which a pattern of neural activity genuinely represents an environmental property. This approach has been pursued in some depth in the philosophical literature on mental representation. Although philosophical accounts of mental representation are many and various, none simply identifies representation with correlation. Typical additional requirements include the following: (a) the correlation between the detected feature and the pattern of neural activity is exploited in further processing; (b) this further processing ends up guiding action and explaining success; and (c) misrepresentation is possible (e.g., when neurons fire in the absence of the stimulus they evolved to detect), and this helps explain unsuccessful action. These criteria have been discussed under different guises in the philosophical literature on causal and teleosemantic theories of mental content (Dretske Reference Dretske1981; Millikan Reference Millikan1984; Neander Reference Neander2017; Shea Reference Shea2018). With a theory of this general type in hand (such as Shea's or Neander's), there is room for a serious empirical debate about whether a given pattern of neural activity represents the property it detects.
As an illustrative example, consider mirror neurons: neurons that fire both during motor action production and during observation of motor acts in others. These neurons were first discovered in area F5 and in the inferior parietal lobule of the monkey and later found in functionally equivalent areas of the human brain (Fabbri-Destro & Rizzolatti Reference Fabbri-Destro and Rizzolatti2008). There is no doubt that their activity correlates under controlled conditions with properties of a variety of motor acts, such as grasping movements. They belong to an information-processing route that involves other brain regions and they receive information from higher visual areas. There are also well-documented contextual effects: for example, some (but not all) neurons that fire in response to a perceived grasping movement will also fire if the movement is partially occluded by a screen, but only if the agent previously saw a graspable object hidden behind the screen (Umiltà et al. Reference Umiltà, Kohler, Gallese, Fogassi, Fadiga, Keysers and Rizzolatti2001).
Here we have systematic correlation, inviting casual talk of neurons “coding” for motor acts. Yet it would be hasty to infer that the neurons represent the properties they detect. There is a long-running debate about their function in cognition and about what they represent (Cook et al. Reference Cook, Bird, Catmur, Press and Heyes2014). To settle that, we need to address a package of further questions: Do mirror neurons detect the same type of property in a variety of experimental settings and with different techniques? Are the correlations with properties of motor acts exploited in further processing? Do they guide successful action? Are there cases in which unsuccessful action can be explained by the mirror neurons misrepresenting a property of a motor act? As long as clear answers to these questions remain elusive, we should remain agnostic on the question of what features of the external world mirror neurons represent. However, the fact these questions are difficult to answer is no reason to doubt the validity and value of the concept of representation for understanding the brain. On the contrary, the concept of representation is exactly what we need to formulate and test hypotheses about the function of mirror neurons. To call for the elimination of the concept would be akin to arguing that, because it is often hard to tell whether or not a trait is an adaptation, we should eliminate the concept of “adaptation” from evolutionary biology.
In sum, Brette correctly highlights the shortcomings of the “coding metaphor” as a theory of representation in the brain. His article points to the need to replace fairly loose talk of “coding” with more precisely articulated concepts of feature detection, correlation, and representation. The concept of representation is especially onerous and should be used with great care. But to eliminate the concept altogether would be throwing out the baby with the bathwater.