Anil Seth’s paper provides an excellent overview of the issues surrounding the question of whether artificial intelligence could be conscious, and I agree with him on many issues; in particular, I believe that consciousness is a property only of living systems. Where we disagree is that I think all the relevant criteria for living systems could be satisfied by software running on Turing computation.

Seth repeats the “nothing gets wet in a weather forecasting computer” assertion, but that is only true because weather forecasting simulations aren’t intended to model the wetting of individual objects. If someone were to claim that no lift is produced in an aerodynamics simulation because the computer running the simulation stays firmly on the desk, we would say they were committing a category error. The relevant criterion is whether the digital plane wing experiences lift, and in a good aerodynamics simulation, it does.

I think it should be possible to create digital organisms with virtual bodies that they use to navigate virtual environments, and for those bodies to operate via internal processes that will cease if the organism cannot negotiate its environment well enough to find the resources needed to maintain them. Such organisms would satisfy the requirements for embodiment, embeddedness, active inference, allostasis, and metabolism. None of these criteria requires crossing the divide between entities inside the virtual environment and those outside it. The essence of embodiment is not whether an organism’s body can bump against my physical body; what’s important is that the organism’s body can bump against objects in its own environment. Likewise, we can meaningfully speak of entities that must actively maintain their internal non-equilibrium states while existing within a virtual environment, even if those entities are not reducing the local entropy within the computer on which the virtual environment is running.

Seth notes that the distinction between a simulation and an implementation isn’t always clear. Let’s consider the scenario of a digital simulation so detailed that the interactions between individual atoms are modeled; this is arguably the limiting case of digital simulation possible on Turing computation, if we assume that modeling quantum phenomena would require a quantum computer. Simulating an entire biological organism and a surrounding environment this way would obviously be prohibitively expensive, but if we set that issue aside momentarily, we can ask the question: Does this constitute a simulation, or an implementation? The carbon atoms within this digital simulation are incapable of bonding with carbon atoms in the physical world, but I would nonetheless argue that this digital simulation could be considered a genuine implementation in a realm that is disjoint from our own. I believe an organism existing in such a simulation would be conscious, but I don’t think it fits within Seth’s taxonomy; the consciousness created could be called substrate dependent inasmuch as it is a product of carbon atoms, but it could also be called substrate independent because those carbon atoms are simulated via Turing computation.

Now let’s consider the issue of computational cost, which is relevant for reasons beyond the practical. Broadly speaking, simulations are useful only when they have advantages over the system being simulated; it’s cheaper to test the flight characteristics of a simulated plane than it is to construct an actual plane, while a weather simulation is useful if it runs faster than the Earth’s atmosphere while remaining accurate enough to provide actionable predictions. The aforementioned scenario of an atomic-level simulation of an organism fails on these counts; it would be both more expensive and slower than the equivalent organism in the physical world. I believe a biological organism in such a digital simulation would be conscious, but if that is the only way to create a conscious digital entity, which would mean we lack a deep understanding of consciousness because we have failed to identify any useful abstractions that pertain to it. A good aerodynamics simulation is a proof that we have identified useful abstractions of aerodynamics, such as the Reynolds number and the Navier-Stokes equations, because a digital plane wing experiences lift without our needing to simulate every single air molecule.

If embodiment, embeddedness, allostasis, and active inference are useful principles when trying to understand consciousness, then those are opportunities for abstraction that would enable us to build a simulation that is faster and cheaper than an atomic-level one. For example, if we can properly understand the role that allostasis plays in consciousness, we should be able to implement a digital allostatic system that is vastly simpler than an atomic simulation, while still satisfying the requirements needed to create a genuinely conscious digital organism. This scenario might qualify as “theory-based computational” in Seth’s taxonomy, but not in the way he currently describes the category, if I’m understanding him correctly. The theories involved have yet to be formulated, but would be built around the criteria that Seth lists; even though he says these criteria point to substrate dependence, they are what would enable a substrate-independent implementation of consciousness.

I don’t believe any current artificial-life programs qualify as living systems, but I don’t think it’s impossible in principle for a computer program to qualify, and that would be the first step toward creating digital entities that are conscious. I fully agree with Seth regarding the ethical dangers of creating beings capable of suffering. However, I don’t think we will accidentally create entities with the moral status of mammals. The arguments favoring biological naturalism also suggest that the development path for digital lifeforms would roughly follow that of evolution in the natural world, starting with non-conscious biological behaviors such as reflexes and progressing toward conscious behaviors. I think it will take significant effort to create entities with even the moral status of insects. While some would argue that we should treat insects better than we do, I don’t think it would be unconscionable if we created digital organisms of comparable status. So if we decide it’s worthwhile to investigate the possibility of digital consciousness, we would have plenty of time to avoid an ethical catastrophe.