In his defence of biological naturalism and rejection of computational functionalism, Seth shows himself to be in the grip of dualistic intuitions. Thankfully, we can read Seth’s paper as a worthwhile contribution to the ongoing society-wide conversation on how to talk about, think about, and treat advanced AI, while setting aside the metaphysical framing of his arguments.

Seth mounts a rich and persuasive case that human consciousness cannot be understood without life’s distinctive machinery – autopoiesis, allostatic control, predictive processing under the free-energy principle, and substrate-entangled dynamics. I agree that this undercuts the expectation that humanlike consciousness will “come for free” with more intelligent machines. However, we should also be cautious about drawing extrapolations from features of biological consciousness – even essential features – to the possibility of exotic forms of consciousness in artificial systems.

A.I. isn’t conscious because consciousness requires properties that only biological agents possess, which A.I. lacks. What A.I. lacks that is relevant for consciousness is not predictive processing or the minimization of free energy. A.I. lacks embodiment, direct coupling with the environment, multiscale complexity, and self-organization, e.g., plasticity, degeneracy, robustness, and the properties that enable living organisms to be highly adaptive to environmental changes.

There is a curious contradiction in Seth’s argument, or a degree of ambiguity which proves to be fatal. He endorses “biological naturalism,” but he says also that it is possible to engineer “real artificial consciousness.” Such an artificially consciousness system need not be carbon-based, he says, so long as it is embodied in a system that is artificially “alive.”

Does the free energy principle support the view that consciousness is substrate-dependent? Based on previous work (Wiese, 2024), I argue that the free energy principle can be used to clarify conditions under which a system realises, rather than merely simulates, a causally relevant property (such as consciousness). This helps to address a weakness in Seth’s argument.

I was delighted, challenged, and somewhat overwhelmed by the breadth and depth of the commentaries elicited by the target article. I am very grateful to every one of my commentators for offering such thoughtful perspectives. Here, I attempt to isolate some common themes and offer some thoughts in response. One conclusion is immediately evident: it is increasingly urgent to think clearly about the prospects for conscious AI and about the consequences of seemingly conscious AI. The commentaries also flesh out the potential for an exciting research programme investigating the substrate dependence of both consciousness and cognition.

The power of human social cognition is typically associated with mental state reasoning—a mentalistic stance. Here, we argue humans have a second equally powerful system, an institutional stance, that interprets social interactions in terms of structured sets of roles linked to rule-like behavioral expectations. Precursors of the institutional stance in other species may support social reasoning in the absence of complex models of mental states. However, the institutional stance is unique in humans in its generative capacity, supporting the creation of a myriad range of institutional structures that shape our world and enable large-scale social coordination.

Jara-Ettinger and Dunham propose that human social cognition relies on an institutional stance grounded in generative, role-structured representations. I extend this account by suggesting that institutional reasoning functions as a constraint-regulating cognitive system, balancing stability and flexibility under changing social demands and helping explain how role-based coordination remains robust without producing rigidity, overload, or breakdown.

I argue that, in light of grounding the institutional stance in role-based norms, this stance is not a causal or predictive model of social cognition. Instead, it is fundamentally normative in that it facilitates practices of holding responsible that mere predictive models would have difficulty explaining. The institutional stance, as such, facilitates robust ought-based claims and demands directed towards role-holders.

If Jara-Ettinger and Dunham are right about the pervasive role the institutional stance plays, we have to seriously consider that this stance perpetuates injustice, because it tends to reinforce, normalize, and hide problematic structures. To address the problems that come with this stance, we need to take into account history and context and ask ourselves what is morally right.

The notion of institutional stance clarifies existential problems. The institutional stance enabled the ancients to view society as stably organized and to rather easily find their place in it. Societal changes gradually complicated the pursuit of self-knowledge in this fashion, and the meaning of life emerged as a problem in the 1800s as Western civilization transformed its institutions.

Behavioural economics knows that market norms displace social norms, but has never explained why the substitution is cognitively abrupt or why some findings built on it replicate while others do not. Jara-Ettinger and Dunham’s institutional stance supplies both answers: stance adoption makes the two logics genuinely incompatible, and embeddedness in real role relationships determines whether an effect survives replication.

The Institutional Stance (IS) offers a promising account of how humans navigate institutional structures, but it suffers from a problem inherent in purely third-person frameworks: a detached observer can predict behavior aligned with institutional expectations, but cannot feel the binding grip of normative force. We propose reconceptualizing the IS as an important aspect of social cognition rooted in shared intentionality.

I extend the scope of the institutional stance analysis to a potentially surprising domain: marriage. Long-term romantic relationships, like many other institutions, often give rise to strongly prescribed roles for both parties. Over time, the resulting models and expectations of our counterparts may become sufficiently well articulated that they supplant the individual mind in front of us.

We agree with J. Jara-Ettinger and Y. Dunham that social understanding cannot be reduced to mental-state attribution, but we challenge their view that role-based expectations are inherently institutional. Institutions require collective symbolic functions, unlike expectations that arise from recurring relational patterns. These patterns – what we call naïve sociology – underlie an elementary social stance that evolutionarily predates institutional reasoning and provides a more parsimonious basis for interpreting social behavior.

Jara-Ettinger and Dunham treat the “mentalistic stance” as theory-theory, leaving the institutional stance vulnerable to a familiar reply: the tacit theory can be context-tuned to fit the surprise cases. I argue that the institutional stance is best motivated by pairing it with simulation-based mindreading and scripts, which together reduce theory-theory’s distinctive explanatory role and preserve the authors’ appeal to computational economy.

Jara-Ettinger and Dunham propose that institutional cognition tracks roles and their normative expectations. I argue that decentralized roles lacking formal authority require an additional mechanism to stabilize legitimacy. Sacred values – nonnegotiable commitments that impose costly obligations – function as role-constitutive norms that signal intrinsic devotion, thereby determining legitimate role occupancy and preventing free riding on group identities.

The authors contend that the “institutional stance” paves the way for uniquely human institutions. However, it is formalized rules that are the integral ingredients for modern institutions and sufficient for understanding people within them. We suggest that formalized rules are a technology for norm conformity that codify, legitimize, enforce, and explain permissible behavior across roles and contexts – and literally establish institutions.

We address two important questions that the target article leaves largely open. First, we explain why institutional representations on their own struggle to reduce uncertainty in social interactions sufficiently and hence require mental state inferences to calibrate predictions. Second, we explain how supplementing theories of institutions with theories of mind provides the richness necessary to refine social predictions.

The target article proposes an evolutionarily old “institutional stance,” where people make sense of others’ behavior through roles independent of individuals. We argue that representations of such role-filler-independent roles are likely not primitives. We offer an alternative: relationship-bound role representations that encode particular individuals. Role-filler independence may instead be a cultural tool that addresses coordination challenges in large-scale groups.