Brette provides a timely critique of the coding metaphor and suggests the alternative lies in models of the full sensorimotor loop (Ashby Reference Ashby1965; Brooks Reference Brooks1991a; Cisek Reference Cisek1999; Clark Reference Clark1997; Dewey Reference Dewey1896; Gibson Reference Gibson1979; Hendriks-Jansen Reference Hendriks-Jansen1996; Piaget Reference Piaget1963; Powers Reference Powers1973a). Indeed, the fundamental task for an organism is not to encode the environment but to complement its dynamics so the entire brain-body-environment system flows toward states supporting the organism's survival and away from those that don't.

However, the full sensorimotor loop is so complex that understanding it all is a daunting task. This is partly why the coding metaphor is so pervasive – it offers a tempting method to delineate subsystems within the loop, each with defined inputs and outputs, which can then be studied experimentally. But if splitting the loop into sensory, motor, and cognitive processes leads to artificial borders and flawed notions of coding, then how else can we subdivide the large question of behavior into smaller and more manageable questions?

One possible answer lies in evolution. The brain evolved through a long series of modifications that gradually elaborated older systems into newer ones, and the history of these modifications can be reconstructed from comparative data. With that history as a guide, one can develop theories of behavior through a stepwise process that provides a different way of dissecting behavior (Cisek Reference Cisek2019; Grossberg Reference Grossberg, Rosen and Snell1978). Instead of breaking the sensorimotor loop into modules such as “perception” and “decision-making,” we can consider how distinctions between control mechanisms emerged in evolution through differentiation and specialization of ancestral systems.

Cisek (Reference Cisek2019) briefly summarizes our evolutionary history as the continuous expansion of the behavioral repertoire by differentiating control mechanisms and extending them into progressively more abstract domains (Hendriks-Jansen Reference Hendriks-Jansen1996; Piaget Reference Piaget1963). These mechanisms operate as sensorimotor loops that ensure the brain-body-environment system flows toward desirable states. As spikes are a means of directing that flow, their activity perforce corresponds to aspects of the world, but also to the organism's needs and its policies for meeting those needs. We could call these a “pragmatic representation” – activity that doesn't describe the world but instead mediates interaction with it. While it may seem inappropriate to call this a “representation,” the term allows one to consider pragmatic representations as one end of a continuum at the other end of which lie “descriptive representations” – which still serve a pragmatic role but have at least partially detached from internal context. We can then use these concepts to interpret neural organization.

The mammalian neocortex consists of two sheets: a dorsomedial sheet sensitive to topographic space, and a ventrolateral sheet that is nontopographic (Finlay & Uchiyama Reference Finlay and Uchiyama2015). Dorsomedial neocortex is organized as a set of circuits for controlling different aspects of the animal's behavioral repertoire (Graziano Reference Graziano2016; Kaas & Stepniewska Reference Kaas and Stepniewska2016). Each of these contains a map of potential actions that compete for execution, biased by signals from other regions such as the basal ganglia and ventrolateral cortical regions, including temporal and orbitofrontal areas (Cisek & Thura Reference Cisek, Thura, Corbetta and Santello2018; Yoo & Hayden Reference Yoo and Hayden2018). In this view, behavior consists of a constant competition between potential actions present in the environment (Cisek Reference Cisek2007; Cisek & Kalaska Reference Cisek and Kalaska2010).

For example, let's consider the primate dorsal premotor cortex (PMd), long associated with planning and control of reaching (Wise Reference Wise1985). PMd neurons are selective to reach direction, speed, and amplitude, but their tuning appears to change over time and with intended speed (Churchland & Shenoy Reference Churchland and Shenoy2007), and they are modulated by attention (Lebedev & Wise Reference Lebedev and Wise2001), speed-accuracy trade-offs (Thura & Cisek Reference Thura and Cisek2016), expected probability (Thura & Cisek Reference Thura and Cisek2014), and magnitude of rewards (Pastor-Bernier & Cisek Reference Pastor-Bernier and Cisek2011) in a manner that depends on the angular distance between reach targets. This makes decoding PMd impossible unless one knows every detail of the experimental context, as Brette describes for sensory systems. However, if we interpret PMd as part of a dynamical system that guides the arm to a desirable target (Churchland et al. Reference Churchland, Cunningham, Kaufman, Ryu and Shenoy2010), then these findings make good sense. Before movement, a target must be selected, which can be accomplished through competition among recurrently connected neurons (Cisek Reference Cisek2006; Erlhagen & Schoner Reference Erlhagen and Schoner2002; Grossberg Reference Grossberg1973) biased by any factors that bear upon the animal's choice. In this view, PMd activity is amplified when monkeys make hasty guesses (Thura & Cisek Reference Thura and Cisek2016) not because it “encodes hastiness,” but because increased tonic arousal accelerates winner-take-all selection in recurrent networks (Grossberg Reference Grossberg1973). Once movement starts, these correlations vanish, and activity becomes specific to a given movement. Thus, PMd behaves just as one would expect from a dynamical system for specifying, selecting, and guiding actions, even if those actions cannot be decoded from its activity.

While dorsomedial neocortex may be specifying competing potential actions, ventrolateral regions may be detecting cues to help bias that competition – what ethologists call “key stimuli” (Hinde Reference Hinde1966). Pragmatically most important is that the activity helps to select the right thing to do in the world, not to describe the world. For example, activity related to visual information coming from an apple should be amplified by hunger and reduced by cues that indicate a predator.

In a few special cases, however, a pragmatic representation may gradually detach from its context dependence and become increasingly “descriptive.” If interaction is based on exploiting affordances in the world (Gibson Reference Gibson1979), then it is important that those affordances are often attached to specific objects and places. Thus, there is a pragmatic role for categorical processes in ventrolateral cortex that group key stimuli into activity that classifies objects, especially those of most relevance for a given species’ behavior (Leopold et al. Reference Leopold, Mitchell, Freiwald and Kaas2017). And as control is further extended to complex social interactions, so it becomes even more useful to construct encapsulated “symbols” that can mediate coupling in the agent-agent system. The words on this page are an example.

In summary, I agree with Brette's critique of the coding metaphor and his suggestion that we replace it with closed-loop sensorimotor control. However, his message could be neglected simply because abandoning coding may seem like abandoning the possibility of subdividing the large problem of behavior into manageable subproblems. Fortunately, evolution provides us with an alternative approach. Instead of subdividing behavior into information processing modules resulting from the history of psychological theories, we can resynthesize those theories by following a different kind of history: that of our own evolution.