Anil Seth’s critique of computational functionalism as a theory of consciousness and defense of biological naturalism as an alternative are strikingly similar to those presented in BBS 45 years ago by John Searle (Reference Searle1980). “Minds, brains, and programs” was written in the depths of an AI winter, six years before the broad adoption of back-propagation (Rumelhart, Hinton, & Williams, Reference Rumelhart, Hinton and Williams1986). We are now in the midst of an AI boom, with achievements in AI considered worthy of Nobel Prizes. This suggests that something important has not changed, despite the enormous technical progress in AI, neuroscience, and the rest of biology in the past 50 years. We point to one thing that appears not to have changed: an assumption that whether something is computing is an ontological question.

Seth, like Searle, focuses on what he calls “Turing computation,” and again, like Searle, links this conceptually with Turing computability. “Turing computation,” for both authors, is computation by von Neumann machines. Seth devotes considerable discussion to architectures that are not von Neumann, but neglects one that is arguably least von Neumann, that of reservoir computing (Tanaka et al., Reference Tanaka, Yamane, Héroux, Nakane, Kanazawa, Takeda, Numata, Nakano and Hirose2019). This focus on architectural differences belies an ontological assumption: that some things are “computers” with behavior that counts as “computation” while other things are not.

Whether a process is computation, however, is not a matter of fact, but a matter of measurement and interpretation (Horsman et al., Reference Horsman, Stepney, Wagner and Kendon2014), and a given process can be interpreted as computing many different functions (Bongard & Levin, Reference Bongard and Levin2023). Measurements provide the data for interpretation, and all measurements have finite resolution; hence, computational interpretations of even “analog” processes are always discrete (Fields, Reference Fields1989; Gisin, Reference Gisin2021). The theory of Turing computability – for which the Turing Machine is just a convenient model – is the theory of functions “which could naturally be regarded as computable” (Turing, Reference Turing1937, p. 230). Constructing a computational interpretation requires computing a function (Horsman et al., Reference Horsman, Stepney, Wagner and Kendon2014), so what Turing computability constrains is our ability to interpret processes, including quantum processes (Deutsch, Reference Deutsch1985), as computations.

When “computation” is seen as a fundamentally interpretative notion, the idea that dynamical systems, which are just physical processes defined on state spaces, are not “computing” ceases to make sense. Viewing the Free Energy Principle (FEP), which describes dynamical systems, as making any kind of distinction between living and nonliving, or conscious and nonconscious, systems (Sect. 4.2) is, moreover, missing the point of the FEP (Fields, Glazebrook, & Levin, Reference Fields, Glazebrook and Levin2021; Fields, & Glazebrook, Reference Fields and Glazebrook2025). The FEP states the condition under which a system – any system – that has a Markov blanket will persist in having one. Whether a system meets this condition is a matter of measurement and interpretation; indeed, whether a system is entangled with its environment, and hence whether it has a Markov blanket, is a matter of measurement and interpretation (Zanardi, Reference Zanardi2001). The interpretation step is not optional – no system can demonstrate by measurement alone that any system, including itself, is unentangled with its environment (Fields, Glazebrook, & Levin, Reference Fields, Glazebrook and Levin2024).

All systems, therefore, can be regarded as a computers, and claims that a system exceeds the computational power of a Turing Machine – e.g. that it solves the Frame (McCarthy, & Hayes, Reference McCarthy, Hayes, Michie and Meltzer1969) or Halting (Turing, Reference Turing1937) problems – can be neither refuted nor confirmed by experiments (Fields et al., Reference Fields, Glazebrook, Marcianò and Zappala2025). The processes that we interpret as computations are, as Seth points out, what systems do. It is, however, only what systems do, what they report in some way, that we can observe. As Hawking (Reference Hawking2002) points out, our observational abilities, and hence our measurements and interpretations, are all inevitably incomplete.

Searle missed this point about doing and observability, and Seth misses it here; so have many others in the past 45 years. What we can discover, by observation and experimentation, is what systems – including ourselves – are capable of being aware of, which is just what they are capable of differentially responding to (Fields, Glazebrook, & Levin, Reference Fields, Glazebrook and Levin2021). Laptops are capable of being aware of key presses, and large language models (LLMs) are capable of being aware of text strings. Organisms – even bacteria – are capable of being aware of hundreds to many thousands of dimensions of variation in the states of their environments and bodies. Are laptops in fact aware of key presses – is there “something it is like” when a key is pressed? Suppose some rocks are not just capable of experiencing, but actually experience, nanosecond-scale pulses of light when the right kinds of cosmic rays pass through them. Does this matter? What does it change?

The key to the “real problem” of consciousness, in our view, is careful attention to what systems could be aware of. LLMs could be aware of text fragments, and maybe even of phrase-usage statistics, together with context. Rocks could be aware of heat, and bacteria quite likely are, in our view, aware of energetic and osmotic stress. None of these are anything like human consciousness.

We fully agree with Seth that humans thinking that AI systems have human-like awareness is both an actuality and a problem. Perhaps the antidote is a careful analysis of what AI systems could be aware of, setting aside the non-empirical, unanswerable question of whether they in fact are.