We enjoyed reading Seth’s thoughtful, and thought-provoking, article on the plausibility of conscious artificial intelligence. One of the key themes in his argument is the relationship between metabolic machinery in biological systems and the informational characterizations of autopoiesis. The relationship between information and energy is important in understanding perceptual awareness in terms of the substrate that is doing the “perceiving.” This commentary focuses upon energies of an informational sort as a means of selecting between future paths or trajectories. We suggest that the sort of curious behavior that is characteristic of embodied biological creatures described in the article might be essential element of (human-like) consciousness.

Figure 1 links key quantities in perceptual inference and energetics. The core of this is Boltzmann’s relation (Gibbs, 1902 (Reference Gibbs2010)). In short, energy can be thought of as a measure of improbability. The relationship between energy and probability lets us express the conditional dependency structures that define Markov blankets (Pearl, Reference Pearl1988), as a decomposition of the Hamiltonian (total energy) of a system into additive parts in which states internal and external to a blanket do not appear together in any term. This formalizes a boundary in which all interactions between internal and external take place via intermediate (blanket) states.

Figure 1.

The Energetics of Inference. This figure brings together some of the key concepts linking perceptual inference to its thermodynamic cost. Boltzmann’s relation links probabilities with energies (or Hamiltonians) via temperature ( $T$ ) and a constant ( $k$ ) that determines units. This relation lets us express conditional dependency structures by decomposing a Hamiltonian ( $\mathcal{H}$ ) for a joint system into a sum of terms, each of which includes only subsets of the full system. If two parts ( $\mu$ and $\eta$ ) do not appear in the same term, they are said to be separated by a Markov blanket ( $b$ ). In neurobiology, one can think of brains being separated from the outside world by a blanket comprising sensory receptors and muscles. For Boltzmann’s relation to hold over time, in the presence of random fluctuations, there must be a dissipative element to the dynamics, such that expected ( $\mathbb{E}$ ) flows appear to descend the gradients of the Hamiltonian – counteracting the dispersive effects of fluctuations. If one assumes that internal (e.g., brain) states “represent” or parameterise some “variational” distribution ( $q$ ) describing perceptions of external (world) states, then one can quantify how well those perceptions match optimal Bayesian inference based on a model ( $m$ ) using variational free energy ( $\mathcal{F}$ ), expressed here in terms of a KL-Divergence.

Figure 1. Long description

A diagram illustrating the concepts of Markov blanket, variational density, dissipative dynamics, Boltzmann probability, and perceptual inference in a system. The diagram includes several labeled sections: Markov blanket, Variational Density, Dissipative Dynamics, Boltzmann, and Perceptual Inference. The Markov blanket section shows the partitioning into internal and external states. The Variational Density section explains how internal states parameterize beliefs about external states. The Dissipative Dynamics section describes how average flows are down energy gradients. The Boltzmann section relates energy to improbability. The Perceptual Inference section discusses how beliefs change so as to approximate Bayesian posteriors, decreasing free energy. The diagram also includes visual elements such as arrows and graphs to illustrate these concepts.

Dissipative systems are characterized by a tendency for their total energy to decrease – that is, to move from less probable to more probable configurations – over time, with energy lost as heat. This means the minima of their Hamiltonians are attracting points under averaged flows (Kwon, Ao, & Thouless, Reference Kwon, Ao and Thouless2005). The internal states of a dissipative Markov blanketed system have attracting points determined by blanket, but not external, states (Friston, Reference Friston2019). If we specify an arbitrary distribution (i.e., variational density) over external states parameterized by internal states (Parr, Reference Parr2022), dissipative dynamics cause this distribution to evolve toward an attracting point in information space (Amari, Reference Amari2016; Kim, Reference Kim2018). From a neurobiological perspective, this is like saying a representation of the external world, parameterized by neural states within the brain, evolve toward some perceptual inference.

More technically, one can characterize the dynamics of inference in terms of (expected) rates of change of variational free energy – an informational quantity used in Bayesian statistics to score the fit of a model to data (MacKay, Reference MacKay1991; Penny, Reference Penny2012) – by assuming an internal model for which the attracting points for the variational density approximates the Bayesian posterior density under that model. There is a clear similarity in the form of the expected changes in the (thermodynamic) Hamiltonian and (informational) free energy, with both depending upon the gradients of the Hamiltonian for internal states (see Figure 1). However, their rates of change are not identical. The dynamics of variational free energy depend on the internal model. Understanding this model – arguably, the core problem of neurobiology – may be essential in characterizing the metabolic (e.g., mitochondrial (Lane, Reference Lane2020)) machinery that supports internal brain dynamics in terms of their awareness of the external world.

The above sketches a way to interpret the dynamics of states inside a brain as a set of beliefs about the external world (Wiese & Friston, Reference Wiese and Friston2021). To some extent, this is also true of many AI systems. Can we take this further? While most artificial systems wait for data (e.g., prompts), biological systems actively select their own. One can score the efficiency of this selection using information theory and Bayesian experimental design principles (Lindley, Reference Lindley1956). Specifically, the plausibility of a behavior that optimizes data for inference is the mutual information (Kreer, Reference Kreer1957; Shannon, Reference Shannon1948) – a.k.a., expected information gain – between blanket and external states. Given an internal model, selection of paths with high mutual information has the appearance of curiosity-driven behavior (Friston et al., Reference Friston, Adams, Perrinet and Breakspear2012; Gottlieb et al., Reference Gottlieb, Oudeyer, Lopes and Baranes2013; Itti & Baldi, Reference Itti and Baldi2009; Manohar & Husain, Reference Manohar and Husain2013; Mirza et al., Reference Mirza, Adams, Mathys and Friston2016; Yang, Wolpert, & Lengyel, Reference Yang, Wolpert and Lengyel2016). Mutual information is defined in terms of log probabilities, so has units suitable for an energy (or “action”) for planning. Perhaps more importantly, it depends on the beliefs a creature holds about the environment and favors those paths that select data, not to optimize reward or value, but to optimize our beliefs about our world.

The implications of these ideas for consciousness research are as follows. First, the energetics of the physiological (or computational) system engaged in perception constrains the form of the internal “representation” or beliefs that might underwrite awareness of the external world. Second, as curiosity-driven behavior is a function of those representations – we must know what we are uncertain about to know which action to take to resolve that uncertainty – specific beliefs imply specific patterns of behavior (see (Parr & Friston, Reference Parr and Friston2017) for an example in the context of saccadic sampling patterns in visual neglect). Taken together, behavior and biophysics provide complementary empirical constraints on the belief dynamics that might underwrite subjective awareness.

In summary, the above endorses Seth’s point that the physiology of information must be closely integrated with metabolism in natural, biological, intelligence. Perceptual awareness of a simple sort depends on the dissipation of informational and thermodynamic energies that afford an interpretation of the internal parts of a system (e.g., neurons in a brain) as performing inferences about the external world. These inferences play an important role in planning. A core feature of natural intelligence, and perhaps consciousness, is that we move our sensory apparatus to select our own data. In doing so, we both optimize our beliefs to explain our data and optimize our data to improve our beliefs.