Anil Seth’s target article argues that conscious AI is unlikely on current computational trajectories and that a form of biological naturalism offers a more plausible foundation for consciousness. I agree, and in this commentary I aim to reinforce his conclusion by arguing that the link between his anti-computationalist and pro-biological stances is one of logical necessity.
The brain-as-computer metaphor is indeed an “increasingly inconvenient fiction” (sec. 3.6), and describing mental processes computationally is not enough to say that the brain is a computer. The metaphor fails for a fundamental reason that Seth himself briefly gestures towards (footnote 19): computation is observer-dependent.
Any physical system (such as a silicon chip) that performs a computation does so only because an external observer has created a mapping between its physical states and a set of symbols, where the meaning (semantics) of those symbols is supplied by the observer, and not given by the physics of that system. Crucially, this leaves the observer free to apply an arbitrary number of different interpretations of what computation the system performs. Thus, the question “What specific computation does a given physical system perform?” is ill-posed and underdetermined without further context and knowledge about computation-external intent and purpose.
For all the cases in which we have clear answers, it is because we have knowledge about the computational intent of the designer. As an observer, we can then match our interpretation to the designer’s intent, which requires an agreement that exists outside of, and separate from, the computation performed. Concretely, for all current AI systems on digital computers, this involves two separate steps that ensure convenient, substrate-independent computation of arbitrary algorithms. First, there is an agreement of how to map the physical states of a silicon chip to a set of symbols (such as sequences of binary numbers). This is the part linking the hardware to the software. Second, there is an agreement on what those symbols mean for a given scenario, that is, whether and how these numbers are interpreted in terms of video, text, robot actuators, or something else. This is the part linking the software to an understanding what its output means with respect to the real world. Put differently, computation is a form of description sandwiched between an encoding step (from meaning to symbols and then to physical states of the computing device) and a decoding step (from the physical states of the device back to symbols and ultimately back to meaning).
If computation is logically divorced from semantics and meaning, then where does meaning come from in the first place? Who are these observers that can supply meaning? Terrence Deacon addresses this question in his book Incomplete Nature (Deacon, Reference Deacon2011), where he distinguishes between the observer-relative information in a computer and the intrinsic information that arises from teleodynamic systems – systems organised around an intrinsic goal. Only living systems, which Seth correctly identifies in sec 4.1 as autopoietic (self-producing and self-maintaining), have such an intrinsic goal: to continue existing. This grounds information and meaning for the system itself without requiring an external observer. This principle provides a deeper reason for why, as Seth concludes in sec 3.3, it is hard “to separate what they do from what they are” in biological systems.
In sec 3.7, Seth correctly states, “Simulating something… is not equivalent to realising (instantiating) that thing.” We can now see why this gap is unbridgeable for consciousness. Because computation is an observer-relative symbolic description, a computational model of a mind can only ever be a simulation. It cannot realise consciousness because it lacks the teleodynamic organisation required to ground intrinsic meaning.
The observer dependence of computation reveals an even deeper and more fundamental problem with computational functionalism than is hinted at in footnote 19. It is not just that computation is observer-dependent while consciousness is not. More fundamentally, the observer dependence makes it logically inconsistent to assert that the act of performing a computation can ever, by itself, create a specific real-world phenomenon – whether that phenomenon is consciousness or any other aspect of reality. This presents the computational functionalist with a sharp dilemma: either consciousness is not a real-world phenomenon (a stance that functionalism, with its insistence on physical realisation, was developed to oppose), or it simply cannot be created by an act of computation. This directly strikes at the heart of computational functionalism, which Seth defines as the view that computation is sufficient for consciousness (sec. 3.1).
If computation is out, what is left? We are forced to look for systems with intrinsic, non-observer-dependent properties. This leads us directly to the very features Seth highlights in his argument for biological naturalism: allostasis, metabolism, and the fundamental biological imperative to stay alive (sec. 4.1–4.2). These are not just interesting features of life; they are the very source of the intrinsic goal-directedness that allows a physical system to mean something to itself.
In summary, by following the observer-dependent nature of computation to its logical conclusion, the rejection of computational functionalism is no longer just a critique; it becomes a positive argument for biological naturalism. Consequently, rejecting the computer metaphor and embracing our nature as “living, flesh-and-blood ‘beast machines’” (sec. 4.5) are not just two sequential moves in an argument, but a single, unified conclusion. This is why I strongly second Seth’s closing sentiment (sec. 7): “A clear view of the biological basis of mind is essential to avoid overestimating our machines and, crucially, underestimating ourselves”.
You have
Access
Anil Seth’s target article argues that conscious AI is unlikely on current computational trajectories and that a form of biological naturalism offers a more plausible foundation for consciousness. I agree, and in this commentary I aim to reinforce his conclusion by arguing that the link between his anti-computationalist and pro-biological stances is one of logical necessity.
The brain-as-computer metaphor is indeed an “increasingly inconvenient fiction” (sec. 3.6), and describing mental processes computationally is not enough to say that the brain is a computer. The metaphor fails for a fundamental reason that Seth himself briefly gestures towards (footnote 19): computation is observer-dependent.
Any physical system (such as a silicon chip) that performs a computation does so only because an external observer has created a mapping between its physical states and a set of symbols, where the meaning (semantics) of those symbols is supplied by the observer, and not given by the physics of that system. Crucially, this leaves the observer free to apply an arbitrary number of different interpretations of what computation the system performs. Thus, the question “What specific computation does a given physical system perform?” is ill-posed and underdetermined without further context and knowledge about computation-external intent and purpose.
For all the cases in which we have clear answers, it is because we have knowledge about the computational intent of the designer. As an observer, we can then match our interpretation to the designer’s intent, which requires an agreement that exists outside of, and separate from, the computation performed. Concretely, for all current AI systems on digital computers, this involves two separate steps that ensure convenient, substrate-independent computation of arbitrary algorithms. First, there is an agreement of how to map the physical states of a silicon chip to a set of symbols (such as sequences of binary numbers). This is the part linking the hardware to the software. Second, there is an agreement on what those symbols mean for a given scenario, that is, whether and how these numbers are interpreted in terms of video, text, robot actuators, or something else. This is the part linking the software to an understanding what its output means with respect to the real world. Put differently, computation is a form of description sandwiched between an encoding step (from meaning to symbols and then to physical states of the computing device) and a decoding step (from the physical states of the device back to symbols and ultimately back to meaning).
If computation is logically divorced from semantics and meaning, then where does meaning come from in the first place? Who are these observers that can supply meaning? Terrence Deacon addresses this question in his book Incomplete Nature (Deacon, Reference Deacon2011), where he distinguishes between the observer-relative information in a computer and the intrinsic information that arises from teleodynamic systems – systems organised around an intrinsic goal. Only living systems, which Seth correctly identifies in sec 4.1 as autopoietic (self-producing and self-maintaining), have such an intrinsic goal: to continue existing. This grounds information and meaning for the system itself without requiring an external observer. This principle provides a deeper reason for why, as Seth concludes in sec 3.3, it is hard “to separate what they do from what they are” in biological systems.
In sec 3.7, Seth correctly states, “Simulating something… is not equivalent to realising (instantiating) that thing.” We can now see why this gap is unbridgeable for consciousness. Because computation is an observer-relative symbolic description, a computational model of a mind can only ever be a simulation. It cannot realise consciousness because it lacks the teleodynamic organisation required to ground intrinsic meaning.
The observer dependence of computation reveals an even deeper and more fundamental problem with computational functionalism than is hinted at in footnote 19. It is not just that computation is observer-dependent while consciousness is not. More fundamentally, the observer dependence makes it logically inconsistent to assert that the act of performing a computation can ever, by itself, create a specific real-world phenomenon – whether that phenomenon is consciousness or any other aspect of reality. This presents the computational functionalist with a sharp dilemma: either consciousness is not a real-world phenomenon (a stance that functionalism, with its insistence on physical realisation, was developed to oppose), or it simply cannot be created by an act of computation. This directly strikes at the heart of computational functionalism, which Seth defines as the view that computation is sufficient for consciousness (sec. 3.1).
If computation is out, what is left? We are forced to look for systems with intrinsic, non-observer-dependent properties. This leads us directly to the very features Seth highlights in his argument for biological naturalism: allostasis, metabolism, and the fundamental biological imperative to stay alive (sec. 4.1–4.2). These are not just interesting features of life; they are the very source of the intrinsic goal-directedness that allows a physical system to mean something to itself.
In summary, by following the observer-dependent nature of computation to its logical conclusion, the rejection of computational functionalism is no longer just a critique; it becomes a positive argument for biological naturalism. Consequently, rejecting the computer metaphor and embracing our nature as “living, flesh-and-blood ‘beast machines’” (sec. 4.5) are not just two sequential moves in an argument, but a single, unified conclusion. This is why I strongly second Seth’s closing sentiment (sec. 7): “A clear view of the biological basis of mind is essential to avoid overestimating our machines and, crucially, underestimating ourselves”.
Competing interests
Alexander Lerchner is a full-time employee of Google DeepMind.
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.