I am glad to have been invited to comment on this article because, when I reviewed it for BBS, although I thought it presented an excellent argument, I did not fully agree with it. Peer review isn’t a matter of agreement; but, happily, peer commentary is.

Let’s go directly to the heart of the matter. I agree with Seth when he says that real artificial consciousness (to use his felicitous term) “would need to be (in some relevant sense) living AI, though not necessarily carbon-based” (p. 13). Everything in this statement, which Seth makes more than once in his article, pivots on the question as to what the relevant sense is in which an artificial agent needs to be “alive.”

If it doesn’t need to be carbon-based, it doesn’t need to be biological. Surely this negates the essence of the term “biological naturalism,” and thereby negates a crucial aspect of Seth’s argument. In short, it sets aside everything that he says about the possibly necessary-for-consciousness properties of nerve cells, metabolism and the like.

This leaves us with the stark fact that the rest of his commitment to so-called “biological” naturalism in relation to consciousness is “grounded in active inference, cybernetics, autopoiesis, and the free energy principle” (p. 23). These remaining commitments undercut everything he says about substrate dependence, because there is nothing substrate dependent about active inference, cybernetics, autopoiesis, and the free energy principle. Seth admits as much when he says that “physically embodied robots could have energetic and structural integrity requirements entailing real though non-biological forms of homoeostasis and allostasis (but not autopoiesis)” (p. 27).

Since he admits this much, I need only show that autopoiesis, too, is substrate-independent. Consider, for example, the artificial and decidedly non-biological – indeed virtual – self-organising dynamical systems that Friston engineered (Friston, Reference Friston2013). To test whether these artificial systems were autopoietic, he selectively “lesioned” their sensory, active, and internal subsystems (i.e., he prevented these subsystems from influencing their neighbours). This produced a catastrophic loss of configurational integrity, as was illustrated in Figure 5, which showed structural disintegration of the previously self-organising systems, as the integrity of their Markov blankets was lost and their internal elements were extruded into the environment. Friston described this as “a testament to [their] autopoiesis or self-creation – in the sense that [their] self-organised dynamics are necessary to maintain [their] structural or configurational integrity” (ibid., p. 10).

Together with a team of physicists, computer scientists, roboticists, and fellow neuroscientists, I am currently attempting to engineer a real artificial consciousness which is substrate independent but which embodies all the functional (and in a broad sense, computational) dynamics that Seth committed to above, including non-biological forms of homoeostasis and allostasis. (Our agent is driven by a sophisticated form of active inference; it is chronically trying to reduce its uncertainty about how to meet its own existential needs in the environment within which it is embedded, and thereby to stay “alive.”) In this relevant sense, our agent is artificially alive. It is genuinely agentic, in the way that Seth points out most “agentic AI” is not, in that it possesses what he calls endogenous goals (p. 27). It has “skin in the game” (p. 8); it “gives a damn” (Haugeland, Reference Haugeland1998, pp. 47, 60).

Towards the end of his article, Seth argues against the engineering of such systems. He says that “the development of real artificial consciousness for its own sake should not be an explicit goal” (p. 29). He makes this point on the alarming grounds that creating real artificial consciousness risks a mass inauguration of new forms of suffering (p. 28) and that real artificial consciousness might endow a system with its own interests and values, beyond those installed by the system’s designers (p. 29). Why, then, is my team trying to develop a conscious AI?

This question is best answered by quoting Feynman: “What I cannot create, I do not understand.” (This was written on Richard Feynman’s blackboard at the time of his death in 1988.) The scientific problem as to how consciousness arises will never be solved by means of philosophical debates and thought experiments, or by adding more plausible theories to the plethora of mutually contradictory ones we already have. In short, we do indeed need to transform the “hard problem” (Chalmers, Reference Chalmers1995) into what Seth calls the “real problem.” But this requirement can only mean that we must subject our mechanistic, causal theories of consciousness to the same rigorous tests of falsification that we use in the rest of science. This can be done most directly by means of concrete experiments in which we attempt to build artificially conscious agents and then test whether we have succeeded or not.

I believe that we have reached a stage in consciousness science where the most important question is not “what should we (not) do?” but rather “how will we know (or agree) when we have done it?”. To this end, my principal co-investigator and I are drawing up a shortlist of functional and behavioural criteria that we hope a reasonable number of consciousness scientists and scholars will be able to agree upon (Solms and Shock, Reference Solms and Shockin preparation).

We have committed ourselves publicly (Solms, Reference Solms2021) to stopping our research programme just as soon as such criteria have been met, so that the relevant stakeholders can take collective responsibility for how we should all proceed from that point onwards. In other words, we fully recognise the risks that Seth identifies; but if artificial consciousness can be created, it will be created. So, it is best that it is created by pure scientists like us, “for its own sake,” rather than by commercial interests, to mention just one of the most obvious worrying alternatives.