The challenge of artificial intelligence (AI) value alignment centers on how to align AI agents with the right set of values or principles, such that they reliably make good decisions when navigating our social world. This comment explores whether AI alignment could be grounded in resource-rational contractualism (RRC). It focuses on three questions:

1. 1. Does RRC adequately capture key normative properties of contractualist moral philosophy – providing convincing answers to moral questions?

2. 2. Can a version of the RRC framework be extended to address the challenge of AI value alignment – ensuring AI agents reliably act in accordance with human values?

3. 3. Can RRC inform the kind of moral reasoning AI agents may need, if they are to adhere to both general and case-specific moral principles?

Taking the philosophical question first, a potential disconnect emerges between contractualism, as understood in moral philosophy, and its operationalization in RRC. Whereas philosophical contractualism, exemplified by thinkers such as Hume (Reference Hume1748), Kant (Reference Kant1797), Rawls (Reference Rawls1971), Habermas (Reference Habermas1990), and Scanlon (Reference Scanlon1998), explores the nature of agreement in an impartial setting free from power asymmetries, Levine, Chater, Tenenbaum and Cushman concentrate on resource-rational bargaining among rational agents seeking mutually advantageous solutions. The outcomes of these two processes are not necessarily the same. In the latter case, those with weak bargaining power may be forced to accept conventions that do not treat them fairly (Levine et al. draw attention to this problem in Section §2.3). To understand what morality really requires of AI, we need to focus on normative principles that meet a higher standard of justification. What matters is not what people would accept, but rather, what they are owed, when interacting with AI agents. Contractualist mechanisms that equalise bargaining power must therefore remain central to the process.

RCC offers insights into the psychological mechanisms through which human beings reach collective agreement which can inform AI alignment. Human beings need to find agreement and moderate their conduct in ways that are collectively endorsed despite the existence of trade-offs between community members. Levine et al. indicate how this may be done. AI agents, empowered to take autonomous action, increasingly face a related set of dilemmas, including which group or values to prioritise. We suggest that an amended version of RRC that focuses more directly on the outcomes of impartial deliberation can be productively applied to these questions. Indeed, recent literature on AI alignment has stressed the need for AI agents to be aligned with principles resulting from fair deliberation between affected parties (Gabriel, Reference Gabriel2020; Zhi-Xuan et al., Reference Zhi-Xuan, Carroll, Franklin and Ashton2024; Gabriel and Keeling, Reference Gabriel and Keeling2025). RRC’s tripartite model provides guidance about how this could be done in practice.

To begin with, aligned AI agents must be able to understand and obey the law (Etzioni & Etzioni, Reference Etzioni and Etzioni2016). This ability corresponds to RRC’s “model-free” or precedent-based moral cognition, which generalises from rules and/or the outcome of past negotiations. However, legal compliance is not sufficient: AI agents also need to navigate the world of social and moral conventions, taking action even when the law is silent. In these cases, per RRC, hypothetical reasoning may help AI agents both identify fair solutions to moral dilemmas and predict human behaviour in similar situations. For example, Weidinger et al. (Reference Weidinger, McKee, Everett, Huang, Zhu, Chadwick and Gabriel2023) use Veil of Ignorance reasoning to identify principles that an AI agent should follow when working out a solution to distribution problems. This corresponds to “model-based” agreement. Finally, in a third set of cases involving AI alignment, “actual” deliberation among stakeholders is needed. This is crucial when the stakes are high and all parties must abide by an agreement. In the context of AI alignment, democratically inspired alignment assemblies have been proposed for this purpose (see Anthropic, 2023).

Finally, RRC may provide guidance for developing appropriate mechanisms for AI moral cognition and alignment. Crucially, AI agents need to manage finite amounts of energy, compute, and time successfully. A key challenge in machine learning therefore concerns how to build integrated models that can switch between resource-intensive reasoning approaches and less intensive modes of generation or retrieval (Zelikman et al., Reference Zelikman, Wu, Mu and Goodman2022). AI agents need to be able to decide-how-to-decide in a resource-rational manner. Applied to AI alignment, this means that AI agents need to know when to seek real-world deliberative input (i.e., asking for moral instruction), when to actively deliberate about moral questions (i.e., engage in chain-of-thought reasoning), and when to rely on precedent (i.e., drawing upon training or retrieval data; Gao et al., Reference Gao, Xiong, Gao, Jia, Pan, Bi and Wang2023). By mapping out the moral terrain and providing instruction about the type of reasoning needed to resolve different kinds of dilemmas, RRC may provide a model for AI agents that need to allocate resources across different moral tasks.

Thus, in the context of AI alignment, an integrated picture starts to emerge. An aligned AI agent will tend to seek out active direction for consequential or novel decisions affecting different stakeholders. When the stakes are lower it could use hypothetical deliberation to identify mutually acceptable and justified actions in multi-party settings. In a further set of more routine cases, it could generalise appropriately from the rules and norms of a community, leveraging accumulated knowledge in an efficient way. We therefore conclude that a modified version of RRC, more firmly committed to impartiality which promotes the equality of participants, has translational promise for AI alignment.