We are in accord with the view of this target article that neural coding based on information theoretic concepts is insufficient to explain perceptual processing, and extend it in this commentary. Three aspects related to neural codes are discussed in the target article: correspondence, representation, and causality. We focus here primarily on the second one.

To quote from the target article, “Second, neural codes carry information only by reference to things with known meaning. In contrast, perceptual systems must build information from relations between sensory signals and actions, forming an internal model” (abstract). In other words, information theoretic neural codes acquire meaning externally, which is problematic as the target article points out, while perceptual systems must acquire meaning internally through relationships within the neural system, as the article again points out correctly. We have previously made this distinction between extrinsic and intrinsic neural coding (Lehky et al. Reference Lehky, Sereno and Sereno2013), based on similar grounds discussed in the target article.

Population coding was only touched upon lightly in the article. We propose that population codes may be central in implementing intrinsic neural coding. A number of studies on high-level visual processing have interpreted monkey neurophysiological data, using multidimensional scaling (MDS) analysis, in terms of neural populations that form high-dimensional neural response spaces (Baldassi et al. Reference Baldassi, Alemi-Neissi, Pagan, Dicarlo, Zecchina and Zoccolan2013; Eifuku et al. Reference Eifuku, De Souza, Tamura, Nishijo and Ono2004; Kayaert et al. Reference Kayaert, Biederman and Vogels2005; Kiani et al., Reference Kiani, Esteky, Mirpour and Tanaka2007; Lehky & Sereno Reference Lehky and Sereno2007; Murata et al. Reference Murata, Gallese, Luppino, Kaseda and Sakata2000; Op de Beeck et al. Reference Op de Beeck, Wagemans and Vogels2001; Rolls & Tovée Reference Rolls and Tovée1995; Sereno & Lehky Reference Sereno and Lehky2011; Reference Sereno and Lehky2018; Sereno et al. Reference Sereno, Sereno and Lehky2014; Young & Yamane Reference Young and Yamane1992). In such a neural representation space, each neuron in the population forms one axis of the space. The response to a stimulus is a point in the response space. In other words, the stimulus response is a neural response vector, which, for interpretation using MDS analysis, is simply an ordered list of neural firing rates in the population. No claim is made that the brain is necessarily implementing dimensionality reduction algorithms, but rather dimension reduction offers the possibility of visualizing low-dimensional manifolds existing within high-dimensional neural response space (for discussions, see (Lehky & Sereno Reference Lehky and Sereno2011; Sereno & Lehky Reference Sereno and Lehky2011)).

In terms of intrinsic coding, an advantage of using high-dimensional representation spaces is that a stimulus response is indicated by nothing more than raw firing rates in the population. There is no measurement of tuning curves in terms of an externally defined physical parameter or any other externally referenced yardstick. Such external referencing is essentially the fallacy of information-theoretic neural coding, as pointed out in the target article and as exemplified by neural codes mentioned in the target article (Jazayeri & Movshon Reference Jazayeri and Movshon2006; Pouget et al. Reference Pouget, Dayan and Zemel2003; Quian Quiroga & Panzeri Reference Quian Quiroga and Panzeri2009) as well as many others. Rather than external referencing, intrinsic coding uses internal referencing based on relations between stimulus representations. Relationships between stimuli are indicated by geometrical relationships between points (stimulus responses) within the high-dimensional space, such as distances in the neural response space (differences in neural response vectors). Organizing stimulus responses in terms of the neural population vectors themselves provides structure to the responses that go beyond summary statistics of a population response distribution (summary statistics such as are required in the “bag of neurons” model mentioned in the target article).

From our perspective, the way to ascribe meanings to neural representations (i.e., deal with the grounding problem) is through coupled internal representations of sensory and motor variables that become organized and consistent with each other through experience in and interaction with the environment (Lehky et al. Reference Lehky, Sereno and Sereno2013). This is essentially the same sensorimotor proposal set forth in the target article. Under our view, populations of neurons form high-dimensional representation spaces, and those representation spaces are organized, developed, and transformed as those populations project to other brain structures, whether they be primarily sensory or motor and whether feedforward recurrent processing, and/or lateral interactions occur. Such an account of neural processing follows a dynamical systems approach and does not incorporate Shannon information theory.

There are a few specific points where we are in disagreement with the target article or feel that amendments are possible. First, with respect to correspondence, we do think that contextual variables can also be encoded with an intrinsic neural population approach and have demonstrated the effects of attention modulation at a single cell in a ventral visual cortical area and its effects at a neural population level on the discriminability of shapes and accuracy of object position (Sereno & Lehky Reference Sereno and Lehky2018; Sereno et al. Reference Sereno, Lehky and Sereno2019).

Second, with respect to causality, we believe the dynamical systems account is consistent with the discrete and timed nature of spikes. Neural processing generally occurs using interactions between analog potentials within the postsynaptic dendritic tree, and not as digital spikes as observed in the axon. Spikes are used for long-distance transmission of the signal along the axon until conversion back to analog form at the next synapse for processing. Dendritic potentials, which are continuous in terms of voltage and time, are the proximate cause of neural processing, not spikes. This view of neural processing as an analog dynamical system is valid regardless of whether spikes can be described by a rate code or time code and regardless of the presence of active potentials within the dendrites.

In sum, we agree with the target article that some currently popular neural coding metaphors are inappropriate and misleading. We agree that these neural coding metaphors are not neutral. They influence the architecture of our conceptual system and limit our understanding. Better metaphors are possible. We propose an alternative intrinsic and relational neural population metaphor.