R2.1 Computational extensions

Resource-Rational Contractualism (RRC) aims to provide a computational framework for understanding moral cognition. Operationalizing it poses a number of foundational computational and representational challenges. A central question, posed by Chandra & Kleiman-Weiner, concerns the mental representation of “agreement,” a question that raises issues about theories of mental representation more generally. For example, proponents of a “probabilistic language of thought” (Ong et al., Reference Ong, Zhi-Xuan, Tenenbaum and Goodman2024; Piantadosi, Tenenbaum, & Goodman, Reference Piantadosi, Tenenbaum and Goodman2016) or indeed traditional symbolic AI (Newell & Simon, Reference Newell and Simon1976) may view agreements as represented in a structured, logical mentalese – what Chandra & Kleiman-Weiner term a legalese of thought. Alternatively, advocates of large language models as a metaphor for, or even computational approximation to, human reasoning (e.g., Kosinki, Reference Kosinski2024), will see agreements as primarily, or perhaps exclusively, represented in public natural languages – the same languages in which real-world contracts are articulated. We think both of these computational approaches to RRC are worth exploring, as well hybrids that capture how natural language can be used to build, extend, and communicate about structured probabilistic models (Grand et al., Reference Grand, Tenenbaum, Mansinghka, Lew and Andreas2025; Wong et al Reference Wong, Grand, Lew, Goodman, Mansinghka, Andreas and Tenenbaum2023, Reference Wong, Kwon, Zhang, Tenenbaum and Levine2025).

A second key issue is to explain how we find productive agreements among the vast multitude of unproductive ones (Yu & Thompson). Several commentators argue that this search should be conceptualized as a collective, rather than an individual, challenge. While individuals are indeed remarkably flexible at virtual bargaining, creatively simulating hypothetical agreements including solutions to moral dilemmas (Le Pargneux, Chater, & Zeitoun, Reference Le Pargneux, Chater and Zeitoun2024; Levine et al., Reference Levine, Kleiman-Weiner, Chater, Cushman and Tenenbaum2024), these mental simulations are rarely conducted de novo. Rather, each new agreement draws upon precedents established by prior agreements, enabling a dynamic process of cultural evolution. We agree with Yu & Thompson that such evolution involves not only the selection of past precedents but also their active and creative transformation (Mesoudi, Reference Mesoudi2021).

We agree with André, Fitouchi & Baumard that the computational mechanisms underpinning contractualism need not necessarily involve, or originate from, such distributed bargaining processes – whether explicit or virtual. They suggest that the visual system need not represent the principles of geometry and optics on which it tacitly relies, and perhaps moral thinking need not represent principles of contractualist thinking. In both cases, heuristic approximations may be selected directly; the broader “computational level” framework (in Marr’s, Reference Marr1982, sense) merely explains why such approximations are favored by selectional processes.

But while vision arises through relatively slow processes of natural selection, moral thinking must be driven at least in part by cultural evolution – after all, moral norms are continually in flux and show substantial variation between groups, just as with other cultural traditions (language, religions, arts and so on). And in this cultural evolution, variation appears to be not random, but shaped by continual debate – both within the individual and between individuals – about which rules and actions are acceptable. On André, Fitouchi & Baumard’s view, such debate would be largely epiphenomenal; by contrast, on our view, it is the engine driving the cultural construction of moral systems.

R2.2. Strategy selection

Bar-On, Le Pargneux, Mullen & Bauman, and Bystranowski raise the following question: given the plethora of moral mechanisms described by RRC, how do people (or, in the case of Gabriel & Franklin, AI systems) choose between them?

There is a seemingly straightforward answer: people use the mechanism that most efficiently trades off accuracy against resource use (cf., Ho, Saxe, & Cushman, Reference Ho, Saxe and Cushman2022). However, difficulties arise at the level of implementation: to determine how accurate and effortful a mechanism would be in a given context appears to require running the mechanism and comparing it with the usage of the other mechanisms – a far more computationally intensive process than simply using the most effortful and accurate mechanism to begin with (Lieder & Griffiths, Reference Lieder and Griffiths2019)!

In practice, there are several approaches people might use to heuristically select between the available mechanisms in order to circumvent this problem. For example, people may default to using a rule-following or norm-based approach and only select a more computationally intensive mechanism when the stakes are high or there is some indication that the rule might go wrong (e.g., the case falls outside the distribution that the rule was designed for, perhaps because the environment has changed), or there is some potential benefit (perhaps personal gain) from deploying extra computational power.

Bystranowski also notes that rules should only be selected when they are a plausible candidate for effective governance. For instance, situational cues may trigger the search for a rule (or the construction of one, if none exists), such as whether the situation involves coordination incentives or takes place in the public sphere. This stands out as a valuable direction for further study. Indeed, there are likely to be interesting cross-cultural differences to uncover here – some societies have a proliferation of rules whereas others have fewer; when and why rules emerge in different contexts is likely to vary (e.g., based on the level of trust in a society; Andre et al., Reference André, Fitouchi, Debove and Baumard2022; Lie-Panis et al., Reference Lie-Panis, Fitouch, Baumard and Andre2025).

Le Pargneux describes the mechanism selection problem as a struggle between reconciling partial and impartial mechanisms, and wonders if there is a principled way to determine which prevails. RRC does provide a principled solution to this worry, however. In its ideal form, RRC proposes that moral judgment should be perfectly responsive to each stakeholder’s particular circumstances and needs (and is in this sense, “partial”). However, “impartiality” gets incorporated into the moral decision-making process in various ways – including as a simplifying approximation or in considering what is generally acceptable in a community (see also section on A rejection of bargaining power).

However, practically speaking, and in agreement with Le Pargneux, we anticipate large individual and cultural differences in people’s tendencies to use more heuristic/rule-based approaches as opposed to more flexible and computationally intensive ones (see especially Trujillo et al., Reference Trujillo Jiménez, Zhang, Zhi-Xuan, Tenenbaum and Levine2025). In a low-trust society, people may default to strictly following rules (even when doing so leads to suboptimal outcomes) in order to signal adherence to the social order, increase predictability of their behavior, and so on (Andre et al., Reference André, Fitouchi, Debove and Baumard2022; Theriault, Young, & Barrett, Reference Theriault, Young and Barrett2021). Indeed, empirical studies conducted to test contractualist mechanisms of moral cognition (Levine et al., Reference Levine, Kleiman‑Weiner, Schulz, Tenenbaum and Cushman2020, Reference Levine, Kleiman-Weiner, Chater, Cushman and Tenenbaum2024; Trujillo, Reference Trujillo Jiménez, Zhang, Zhi-Xuan, Tenenbaum and Levine2025) reveal large individual variation in how subjects respond to the cases used to probe their moral capacities – though the connection to cultural differences or individual background remains untested.

Indeed, several commentaries propose that there may be forces beyond resource-rational considerations that compel people to select one moral mechanism over another.

Conway, Lam & Sedikides’, and Mullen & Bauman both posit that the selected mechanism itself might be the target of moral judgment, not just the decision it renders. Conway, Lam & Sedikides point out that the choice of mechanism may be a signal about the decision-maker’s character. Pushing this view to the extreme, we might predict that even the optimal solution to a problem of interdependent choice might be rejected by a decision-maker if it could potentially be mistaken for the result of a self-serving or otherwise biased process (Kleiman-Weiner, Shaw, & Tenenbaum, Reference Kleiman-Weiner, Shaw and Tenenbaum2017). A similar empirical prediction falls out of the critique of Mullen & Bauman, who note that people’s perceptions of the process used to arrive at a decision impacts their satisfaction with that decision. We agree and add that there are some moral decisions that could be arrived at via a range of different decision-making processes (e.g., one that is entirely self-serving versus one that benefits the self but is also universalizable, such as cutting in line when doing so benefits others; White et al., Reference White, Bhui, Cushman, Tenenbaum and Levine2024; Kwon, Tenenbaum, & Levine, Reference Kwon, Tenenbaum and Levine2023). We anticipate that the mechanism that a person infers is being used to drive a moral action will impact their judgment of the action. Collectively, the forces described in this paragraph suggest that strategy selection may sometimes be shaped by reputational consequences.

Finally, Bar-On raises a potential solution to the mechanism selection problem inspired by a theory of moral uncertainty, positing that if each of the mechanisms are held as valid possible approaches to a morally charged problem, then “maximum expected choiceworthiness” (MEC) could be used to aggregate a decision from their different recommendations. This appears directly contrary to the RRC approach, as it requires that all mechanisms must be computed, together with a final MEC procedure adjudicating between them – a highly computationally intensive endeavor. However, building on the suggestions of Conway, Lam & Sedikides’, and Mullen & Bauman, we see a potential path forward for Bar-On’s proposal, namely, that in cases with unlimited computational capacity and full information, it may be wise to additionally consider the various ways that a decision could be perceived by others (who are perhaps themselves resource-limited observers of the ultimate decision). In a world in which there are some stakeholders who prefer or expect a consequentialist choice, others who most favorably judge a rule-based one, and yet others who highly value a virtual bargaining approach, selecting the decision that is acceptable from each of these views may indeed be the best strategy. We are less clear that MEC itself is the best algorithmic solution to this problem, however, given its requirement that the values of each theory be flattened to a utility-based calculus. Instead, a moral parliament approach might better fit the bill (Moore, Choi, & Levine, Reference Moore, Choi and Levine2024; Newberry & Ord, Reference Newberry and Ord2021).

In contrast to these commentaries, Andre, Fitouchi, and Baumard propose that we overplay the difficulty of strategy selection. Calculating mutual benefit can be computationally trivial, they argue: just compute the Nash product “unless,” they write, “the situation is highly complex or informationally impoverished.” In our view, most moral cases are highly complex and informationally impoverished. Indeed, this is why so much of morality seems to be rule-based on its face and why resource-rational approximations of the contractualist ideal (such as universalization or implied valuation) can be used to override those rules. When and how these mechanisms are selected is, therefore, a matter of fundamental concern.

R2.3 Character, virtue, and reputation

Another family of proposed extensions to the contractualist framework urges us to consider person-centered moral judgments – those concerning moral character, reputation, and partner choice. Much prior work illustrates the central role that person-centered moral judgments play in our moral psychology (reviewed in Uhlmann, Pizarro, & Diermeier, Reference Uhlmann, Pizarro and Diermeier2015). What, then, is its relationship to contractualism?

One element of this relationship has been considered in some detail: an agreement with one person, or group, often occurs against the backdrop of potential alternative agreements with others. This competitive “market” for agreements with social partners (sometimes called partner choice; Barclay, Reference Barclay2016) thus determines the nature of our outside options and, therefore, the terms of the agreement itself. Thus, for instance, people might judge it fair for an employee to get a larger bonus if they have more potential employers to work for. Building on this logic, it has been suggested that the dynamics of a large market for potential cooperative partners, combined with the logic of bargaining, should lead to a general default preference for egalitarianism in resource divisions (Andre & Baumard, Reference André and Baumard2011; Baumard, Andre, & Sperber, Reference Baumard, André and Sperber2013). This fits elegantly with the suggestion by Conway, Lam & Sedikides and discussed above: whether a choice is principled (e.g., rule-based) can itself be a target of moral evaluation. Thus, for instance, the positive public evaluation of the rule “I treat all customers as bargaining equals” may offset the losses of foregone exploitation with the gains of a larger market share.

Kuhlmeir & Dunfield advocate further development of a theory of person-centered morality within a resource-rational contractualist approach. They note that, when assessing others’ reputation and potential as cooperative partners, our cognitive strategies could be organized around resource-rational principles. For instance, we could deploy very cognitively cheap associative learning mechanisms or trait-based inferences; or we could deploy a more sophisticated rich model linking enduring properties of the individual’s mental state to their actions (Ho, Saxe, & Cushman, Reference Ho, Saxe and Cushman2022; Tamir & Mitchell, Reference Tamir and Mitchell2013).

Finally, Srivastava et al raise the very interesting question of whether the mechanisms of RRC could apply to intertemporal choices within a person, as might be appropriate according to multiple selves models of human agency – models that have been popular in economics, philosophy, and psychology (Ainslie, Reference Ainslie2017; Parfit, Reference Parfit1987; Thaler & Shefrin, Reference Thaler and Shefrin1981). This is an attractive direction for future research, particularly because it may help explain why people tend to view virtues such as prudence, temperance, perseverance, ambition, or self-control as moral virtues (Foot, Reference Foot1978; Fitoucci, Andre, & Baumard, Reference Fitouchi, André and Baumard2023), even though they primarily affect the actions and outcomes for the individual concerned (rather than being concerned with interactions with, or the well-being of, others).

R2.4 Cross-cultural extensions

How universal is the RRC theory? Bar-On, for instance, points out that people in different cultures tend to rely on different suites of moral mechanisms to make judgments. This is clearly important for the theory to capture, but not necessarily an objection, given that RRC is a highly flexible framework (indeed, as noted above, Le Pargneux fears it may be too flexible). Indeed, the historical contingency of precedent and rule-based reasoning naturally predicts cross-cultural variability, and contextual differences might make certain mechanisms more efficient or effective than others (see further discussion in the section on ).

Yu & Thompson articulate another way that RRC could help explain cultural differences (rather than be undermined by them). They argue compellingly that “the psychological processes proposed in the target article play a transformation-like role in cultural evolution, leaving fingerprints of resource-rational computation in the norms we inherit” and that many culturally inherited norms can be effectively reasoned over. To put their view another way, we can think about group-level contracts layering on one another over time, and eventually crystallizing into norms, the logic of which can sometimes be inverted and excavated (as in the case of the Micronesian navigators; Asher, Reference Ascher2018).

Nonetheless, how much and which aspects of the theory are universally applicable remains an empirical question. As Maier, Cheung & Lieder also note, one possibility is that there are some moral systems that do not fit, even as approximations, into a contractualist framework, but instead have some other aim (perhaps absolutely prioritizing the commands of a supreme leader, or the will of God; see also our discussion of this in section on the Consequentialist Critique.)

For yet other societies, even this limited version of the RRC framework may not describe their normative architecture. However, if the basic building blocks of RRC are present in human cognition, then we might expect members of these societies to be able to imagine what a society governed by contractualist ideals would be like. This raises a set of questions around how far this imaginative capacity can go beyond experience and can allow individuals and groups to seek out or attempt to build (or else condemn or try to dismantle) contractualist societies very different from their own. Indeed, such thinking has been argued to underlie some of the great examples of democratic society-building in western history (Manville & Ober, Reference Manville and Ober2023), and works of contractualist moral imagination such as Thomas Paine’s Common Sense have arguably played pivotal roles in catalyzing groups to think about what might be possible if a society could recast its political foundations (somewhat along RRC lines). Indeed, the power of such imaginative thinking allays the concern of Dahl & Killen, who worry that RRC “cannot explain how humans evaluate social inequalities and injustices,” and therefore “cannot explain how people challenge existing conditions on moral grounds.” To the contrary, we propose that RRC might provide a conceptual toolkit for revolutionary thinking.

R3.1 What about those who can’t bargain?

Several commentaries ask how RRC can account for our moral concern towards those who lack any bargaining power. For instance, as Thorstand and Lloyd point out, many people feel some moral obligations to animals and to future generations. On its face, this contradicts the contractualist framework’s presumed sensitivity to bargaining power.

Our sense of moral concern for the powerless is real, and it must play an important role in any psychological theory of morality. But, before we address it, we must also question just how deeply committed people really are to the powerless. Several commentaries note our concern for animals and for future generations as especially compelling examples. Yet, most people do not treat animals with the same concern they treat other humans. To the contrary, most people actually eat non-human animals, sometimes several times a day. Indeed, we devote major economic and institutional structures to raising billions of animals in captivity in order to kill and eat them or to use them for other material purposes. Notice just how different this is from the way we treat other humans. Sadly, humans are capable of profound cruelty and indifference to each other, and happily, many people have loving relationships with their pets. Certain animals enjoy circumscribed legal protections against cruelty or extinction. Still, it would be fair to ask whether the most important explanandum for a descriptive account of human moral psychology is our occasional concern for certain animals (which contractualism struggles to explain), or instead the yawning gulf between our baseline levels of moral respect for humans and profound disregard for many animals (which contractualism explains quite clearly).

A similar point applies to future generations. It is clear that people have a strong commitment to their own children and future descendants. Contractualism isn’t needed to explain this; it follows directly from the logic of natural selection. But just how often do people act in ways that suggest a generalized, impartial concern for distant future generations? Again, we cannot help asking whether the chief explanandum is that people occasionally manifest some sort of moral commitment along these lines (which contractualism struggles to explain), or instead that it so clearly pales in comparison to our concern for close others in the present (which contractualism explains quite clearly).

Note, too, that a metaphorical extension of virtual bargaining through anthropomorphism (for animals), or through imagination and temporal abstraction (for future humans), will automatically generate some of our ethical intuitions. Thus, we can have the sense that killing an animal would be wrong because the animal would never agree to this, even though the very notion of agreement may not strictly be applicable for non-human animals.

We suggest a grounding in three basic observations that will help address egalitarian moral impulses: (1) Unquestionably, people both express and sometimes live up to egalitarian, impartial moral convictions; (2) those convictions tend to be expressed with relatively more vigor than they are applied; and (3) both their expression and application are variable across history and culture, and are perhaps especially prevalent in what are sometimes described as “WEIRD” societies (Henrich et al., Reference Henrich, Heine and Norenzayan2010).

So perhaps the real question for moral psychology is why only a minority of people give voice to impartial, egalitarian principles and why, even among these, very few earnestly live up to those principles? Before attempting to answer this question, we first consider a related series of challenges regarding bargaining power among humans in the present.

R3.2 A rejection of bargaining power?

Another group of commentaries asks whether people really consider it morally appropriate for those with more power, or superior outside options, to receive the greater share in resource division, as predicted by the Nash Bargaining Solution (NBS) and other theories of negotiation. Indeed, two commentators actually ran empirical tests designed to address this issue.

To begin, it is useful to briefly review the original scope of the NBS. It is designed to predict how people will agree to divide the surplus generated by joint work, given their “outside options” (i.e., their earnings in the absence of the agreement). The framework assumes that the agreement is binding (e.g., that it is enforced by a powerful third party such as the state) and that neither party can be forcibly coerced into agreement by the other against their interest. It predicts that people will divide resources equally when they have equivalent outside options and the resources are worth the same amount to each of them, but that relatively fewer resources will go to an individual who realizes greater gain from those resources, while more will go to those who have a superior outside option available.

In our original target article, we use the NBS to illustrate the basic idea of “mutual advantage” in a concrete setting, but do not intend it as a specific guide to the agreement-based resolution of all moral questions. Among the commentaries, we find a broad range of positions on its applicability. On the one hand, André, Fitouchi & Baumard (see also Andre et al., Reference André, Fitouchi, Debove and Baumard2022) propose that people compute the NBS to resolve moral questions, that this is the predominant mechanism of moral judgment, and that it can explain moral judgments on questions going far beyond the division of profit under binding agreement. On the other hand, Gabriel & Franklin, Malter & Metcalfe, Lieder, Cheung & Maier, Thorstand, and Lloyd question whether it does a particularly good job of predicting moral judgments even within its strict, original confines. And both Malter & Metcalfe and Lieder, Cheung & Maier present studies backing up this assertion.

How far do these results bear on RRC? Beginning with the study by Malter & Metcalfe, note that before querying participants, they provided those participants with a quotation from a dictionary on the definition of morality and asked participants to include this definition in their reasoning, potentially introducing substantial demand effects. Let us, though, put this concern to one side. Participants were presented with two scenarios. In one, a drug company charges an “exorbitant” price “much higher” than its R&D cost for lifesaving medication, which consumers pay because it “is still better than dying.” In the second, a scalper negotiates a price of $2,000 for tickets costing $200. In both cases, most participants judge that the sales are immoral.

Certainly, the default assumption of an agreement-based model of moral judgment would be that these transactions are not immoral because all parties agree to them (especially if the case if the agreements conform to the NBS). As specified, though, the cases leave so many variables undetermined (e.g., the costs and benefits of the negotiated agreement to each party in terms of their utilities, and the utility-based costs and benefits of their outside options) that it is not possible to say whether participants perceive the parties to have obtained the NBS, or to deviate from it. Given the extremity of the asymmetric bargains struck, it certainly stands out as a possibility that participants would have perceived these specific agreements as departing from the NBS, but we cannot know.

By contrast, consider a series of studies that carefully construct a scenario in which the background assumptions of the NBS apply and in which the precise bargaining outcomes and outside options are specified and parametrically varied over a wide range of settings (Le Pargneaux et al., Reference Le Pargneux, Lie-Panis, Cashman and Cushman2026). In these studies, participants were not presented with agreements as fait accompli, but instead were asked to specify the agreements that they would consider “morally best” or, in another condition, “fair.” These studies showed a close alignment between participants’ moral judgments and the NBS, which outperformed both equitarian and egalitarian alternatives, as well as several alternative theories of negotiation. This comports with other research showing contexts in which peoples’ moral judgments track the asymmetric resource distributions prediction by the NBS under manipulations of bargaining power (Le Pargneux & Cushman, Reference Le Pargneux and Cushman2025). In sum, then, we take from the study by Malter & Metcalfe one firm conclusion: The mere fact that two parties agree to an arrangement is not sufficient to assure that people judge it morally appropriate. Whether or not this arises because of departures from the NBS, or other background assumptions of the Nash bargaining game, remains uncertain.

What about the study by Lieder, Cheung & Mayer? They build on a specific example we offered in the target article, imagining a baker who has a surplus of bread and a pauper who has meager funds available to procure it. We supposed that people would consider it morally appropriate for the pauper to pay at least some amount to the baker for the bread (broadly consistent with the prediction of bargaining theory), rather than the baker giving the pauper the bread for free (as might be demanded by certain deontological or consequentialist approaches). They test this in two contexts: One in which the baker meets the pauper on the street and one in which the pauper enters the baker’s store. In the former (“non-market”) context, they find that most people think the baker will agree to give away the bread for free, while in the latter (“market”) context they find that most people think the pauper will pay full or reduced price for the bread. Although differing in several details, these findings are broadly consonant with one of the studies we described above (Roberts-Gaal et al., Reference Roberts-Gaal, Le Pargneux and Cushman2025). This study found that moral judgments tracked the NBS in a market framing where negotiation was possible, but instead favored more egalitarian solutions in an alternative “donation” framing where negotiation was not possible.

In sum, we conclude that (1) there are contexts in which the NBS guides many people’s predictions about the outcomes of negotiation, and in which it is judged a morally appropriate outcome, (2) people will often view generosity that goes beyond the NBS as commendable, and (3) there are also contexts – such as charitable donation – in which the NBS does not capture people’s predictions or moral judgments well.

Across both the previous two sections, then, we find plenty of phenomena that contractualist approaches are well suited to explain, but also normative moral impulses that challenge it. In the next sections, we take up the normative challenge from contractualist, deontic and consequentialist ethical viewpoints.

R3.3 The (normative) contractualist critique: Egalitarianism

Although the NBS clearly explains important parts of our moral psychology, some people also show some commitment to impartiality, egalitarianism, and other bulwarks against the exploitation of the vulnerable. These constitute an important challenge to our view.

One important critique comes from a certain subset of normative contractualist views. Recall that RRC builds on “contractualism” in the broad sense – referring to both “contractarian” views – which see contracts as emerging from the agreements of self-interested rational actors – and “contractualist” views, which tend to embed normative assumptions in the bargaining process (see footnote 6 of the target article). Nonetheless, our fundamental assumptions often fit better with that of contractarianism. The critique in this section comes from our contractualist flank. Several commentaries propose that one could augment our framework with contractualist features like a veil of ignorance (Thorstad) or other structural assurances of impartiality (Eibach & Grossman, Le Pargneaux, Franklin & Gabriel). These are friendly amendments, insofar as each preserves the contractualist heart of our proposal. These are amendments we are inclined to accept – but perhaps neither as completely, nor for quite the same reasons, as offered.

After all, and as we have already emphasized (see the above section on Character, Virtue, and Reputation), one compelling account of egalitarian attitudes has already been developed within the contractualist framework. It proposes that we negotiate not just over the elements of each social interaction we engage in but also over who we will engage with: we exercise some control over “partner choice” and operate in a market of potential social partners (Andre & Baumard, Reference André and Baumard2011; Baumard, Andre, & Sperber, Reference Baumard, André and Sperber2013). To maintain a reputation as a good cooperative partner, then, people signal their egalitarian values. It is consistent with this possibility that people experience a genuine, intrinsic motive to act in egalitarian, impartial ways, but also that they might also be motivated more instrumentally by the prospect of reputational enhancement. Moreover, the pressure towards such egalitarian motives will be dictated by the size of the social market for cooperative partners; thus, it might be predicted to correlate with population size and density, as well as with relational mobility. In sum, this model helps us to explain why people would sometimes exhibit egalitarian preferences, might even more vociferously express them, and why these tendencies might vary across history and culture.

Suppose that philosophical theories are shaped, in part, by the prior moral commitments of the individuals and cultures from which they arise (Greene, Reference Greene2008). On this view, the fact that many contractualist theories have auxiliary elements that ensure egalitarian outcomes – such as Rawls’s “veil of ignorance” or Scanlon’s conception of “reasonable rejection” – might reflect intuitions of morality and justice ultimately honed by the reputational demands of partner choice. For instance, Eibach & Grossman propose that people often engage in moral reasoning with a kind of generalized other. In this stance, people consider their situation from the perspective of a third party – imagining themselves having to justify their actions to a dispassionate observer or to the broader community. This resonates with some philosophical methods, but it also has a transparent connection to reputational concerns: if it matters who will agree to cooperate with you in the future, then imagining justifying your actions to those cooperators is a good strategy for assessing your action in the here and now.

Also along these lines, Thorstand proposes that adding the normative assumption of egalitarianism as a requirement to the bargaining conditions would help explain altruism and altruistic tendencies. But it is still interesting to ask why some philosophers (and some everyday people) take egalitarianism to be a normative assumption at all. Again, we propose that these kinds of intuitions may arise from long-term reputational concerns.

As Le Pargneaux rightly points out, however, the fact that contractualist models have the flexibility to explain impartial moral judgments could be viewed as a liability, since the same models clearly also have the flexibility to explain robust sensitivity to power asymmetries. How can we provide support for a theory that accommodates logically contradictory predictions? His proposed answer is to test the theory not against its predictions regarding moral judgment, but rather against its predictions regarding the processes of arriving at moral judgments. These will be especially diagnostic when they rely on explicit models of agreement, as in the case of virtual bargaining (Misyak & Chater, Reference Misyak and Chater2014). Indeed, Lloyd’s proposal that people may consider “implied consent” strikes us as a mechanistic-level account very much in keeping with Le Pargneaux’s vision. The very concept of implied consent has, at its heart, a model-based assessment of whether a social partner would agree to be treated in a certain manner. If verified, this model would provide prima facie mechanistic evidence broadly consistent with a contractualist approach.

Having considered these friendly amendments, we turn next to consider two more radical critiques.

R3.4 The deontic critique: Rights, duties, and rules

Several commentaries critique the foundations of the contractualist approach by arguing that it cannot adequately capture the deontic nature of our moral psychology – the inviolable principles, rights, duties, etc., which transcend the contingency of mutual agreement. Stanford & Poulin argue that RRC (and contractualist frameworks more generally) may not align with the widely held intuition that moral norms, demands, and obligations are imposed upon us, rather than agreed upon by groups, societies, or cultural traditions. They raise the possibility that RRC may thereby fail to engage with what is distinctive about moral thought entirely. Rompikuntla & Shivnekar and Dahl & Killen suggest that by focusing on mutual benefit, RRC may take a wrong turn: they note that people are moral reasoners (rather than mere utility-product calculators) and are highly sensitive to principles of fairness, justice, and respect for rights. Ullman suggests that there are domains where the very idea of negotiation or bargaining appears morally unacceptable. For example, he points out that there is something inherently odd about explicit negotiations – especially monetary ones – between family members.

Before taking up the specifics of these commentaries, we begin with a broader reflection on the nature of the challenge itself. On the one hand, the contractualist framework supposes that moral judgment is grounded in negotiation and agreement, and this could be construed as predicting a certain kind of “anything goes” denial of absolute moral duty or prohibition (apart, perhaps, from a foundational duty to honor agreement). On the other hand, a central ambition of the resource-rational framework that we offer is to explain how certain deontological judgments – those structured around inviolable rules – can be reconciled with contractualism. This occurs, for instance, when people negotiate over and agree to rules, or when they generalize rules from past agreements. Clearly, then, the mere existence of moral rules, duties, rights, etc., cannot contradict the theory; they are among its core explananda.

The key question that comes out of these commentaries, then, is whether we can square the resource-rational contractualist framework with the familiar sense of inviolability and self-sufficiency that characterizes deontic morality. Is it consistent with the framework that we feel that certain moral rules cannot be renegotiated and do not merely amount to matters of agreement – even if, as a matter of fact, they can and do? We suggest that the intuition that moral standards are imposed externally may be misleading. What feels like external imposition may actually reflect deeply culturally embedded agreements, the logic of which can often be uncovered, assessed, and argued over.

Indeed, we see contractualism (and more broadly the quasi-realist approach to ethics developed by Blackburn, Reference Blackburn1993) as providing an explanation for the origin of our sense that morality stems from externally imposed and non-negotiable “moral facts,” without needing to postulate the existence of any such metaphysically shadowy entities. RRC proposes that moral rules may be “cached” and therefore at first appear detached from any past agreements from which they might have emerged. Many of these rules may be in the form of general principles of respecting fairness, justice, and rights, as such rules are often mutually beneficial to society as a whole (and all, or almost all, of its members). However, such rules can still be understood and renegotiated in order to be applied to novel cases.

This may help to explain the role of reasoned argument in the moral domain rightly noted by Dahl & Killen. Disputes can become heated when they concern how a rule, law, or meaning – previously treated as settled – is to be extended to new or ambiguous cases, with implications for future behavior. And, importantly, such debates are not unconstrained expressions of preference. They are shaped by shared facts, such as past precedents, and arguments about whether the present case is sufficiently similar to or different from those precedents, what the rule is meant to achieve, and so on.

Along these lines, Zhang & Laurin ask how RRC can make sense of how people make third-party moral judgments on the agreements (and behavior) of others (sometimes framed as if conveying the viewpoint of an impartial spectator). They suggest two reasons why third-party moral judgments may diverge from the agreements made by the directly involved parties. First, a third party might believe that what the participants perceive as mutual benefit is, in fact, mutually harmful – perhaps due to misinformation, coercion, or long-term consequences that the parties themselves do not fully appreciate. Second, to reframe their perspective slightly, there may be a tension between agreements at different levels of analysis – for example, between the “local” agreements between individuals and agreed norms of a wider society. This tension may be compounded by the cached nature of societal agreements – norms that are inherited, reinforced, and applied automatically, even when their original rationale is no longer salient. We agree entirely, and to these possibilities we would add another: that third-party judgments often reflect not just concern with the specific, ad hoc agreement between individuals, but with its potential universalization (Levine et al., Reference Levine, Kleiman‑Weiner, Schulz, Tenenbaum and Cushman2020). That is, they are not merely asking whether this agreement is acceptable, but what would happen if everyone were permitted – or encouraged – to make similar agreements. This line of reasoning is common in moral discourse, especially in areas like paid surrogacy, organ markets, and assisted dying. In many such cases, people might accept that the individuals involved can reach a consensual arrangement, yet still reject the broader permissibility of such practices on the grounds that generalizing them would set a dangerous precedent. The concern is not with this agreement per se, but with the normative signal its acceptance would send.

Ullman suggests that there are domains where the very idea of negotiation or bargaining appears morally unacceptable. For example, he notes that there is an inherent oddness about explicit negotiations – especially monetary ones – between family members. We agree that there is something peculiar about spouses paying each other to pick up their own children. However, we doubt that the right diagnosis is simply that bargaining is absent in such contexts. On the contrary, a vast body of literature in social and organizational psychology – including theories such as equity theory (Walster, Walster, & Berscheid, Reference Walster, Walster and Berscheid1978) and social exchange theory (Thibaut & Kelley, Reference Thibaut and Kelley1959) – suggests that the stability and value of personal relationships often derive from their capacity to embody ongoing, mutually beneficial patterns of give and take. What seems troubling about specific and detailed negotiations in intimate or relational contexts is not the fact of bargaining itself, but the granularity and transactional focus of such bargaining. The point of a relationship is precisely that it establishes a general framework of mutual support, which does not need to be renegotiated case by case.

A useful parallel can be drawn from the literature on contracting in business contexts: namely, the distinction between transactional and relational contracts (Macneil, Reference Macneil1978). Transactional contracts tend to be highly specific, detailed, and rigid and are therefore prone to disputes – since no contract can fully anticipate all possible future contingencies, reflecting what Hart (Reference Hart1961) called the open texture of the law. By contrast, relational contract operates on the basis of general principles and mutual trust. Rather than trying to specify every detail in advance, the parties commit to shared norms and ongoing collaboration, allowing them to adapt flexibly to changing circumstances. Indeed, there is strong evidence that relational contracting is often more effective than transactional approaches, especially when interactions are varied, complex, and ambiguous (Dyer & Singh, Reference Dyer and Singh1998). So, yes, a family whose contracts are excessively transactional may be functioning poorly. But coordinating family life seems to us inherently a matter of continual renegotiation, often implicitly through virtual bargaining, to align interests and coordinate behavior. Moreover, abandonment of the entire relationship (Ullman gives the example of leaving an ill spouse for a new partner) is morally abhorrent because it is typically not a matter of mutual consent – but rather a betrayal of a prior implicit (and, in a typical marriage ceremony, explicit) agreement to stay together “for better or for worse, to death do us part.”

Similarly, Stanford & Poulin propose that certain moral principles are inviolable even if violation is agreed upon by all parties; as an example, they choose slavery. We would suggest, however, that the example of slavery is particularly apt for illustrating the contractualist standpoint: a defining feature of slavery is that slaves are not in a position to agree – and indeed would not agree – to the rules and role imposed upon them. This highlights the failure of such arrangements to meet the basic conditions of moral justification under a contractualist framework, which depends on mutual agreement. To ask, as Stanford & Poulin do, whether we could consider slavery acceptable if it were agreed upon by all parties including the enslaved is to deny the central feature of the moral violation: That those enslaved are not empowered to disagree with their social position and freely walk away (for further discussion see Gauthier, Reference Gauthier1986, Ch 7).

R3.5 The consequentialist critique: Concern for welfare

Lieder, Cheung & Maier, and Kardos question our focus on any form of contractualism as our fundamental account. Lieder, Cheung & Maier endorse the resource-rational approach to moral psychology but raise the important consideration that other moral frameworks may serve as normative ideals toward which deontological, cost-benefit, or contractualist reasoning approximates. In particular, they propose global consequentialism (Pettit & Smith, Reference Pettit, Smith, Hooker, Mason and Miller2000) as a candidate for such an overarching normative standard. We agree that this is a valuable and promising direction for further exploration. However, we suggest that contractualism constitutes a particularly natural starting point for a descriptive theory of moral cognition, insofar as it directly addresses the practical problem of coordinating interdependent social action.

Indeed, the history of morality suggests a gradually “expanding circle” from concerns with immediate family and immediate community to considering humanity, and indeed, sentient life, more broadly (Singer, Reference Singer1981). Thus, we suspect “global” ethical concerns of any kind are a late development. Moreover, we believe there is some justification for seeing contracts as especially fundamental: all societies are concerned with the coordination of behavior across people and time (Baumard, André, & Sperber, Reference Baumard, André and Sperber2013) – creating, monitoring, and enforcing agreements is surely the stuff of human social life, arising every time two or more people make a joint plan, or resolving conflicts between plans. Indeed, human social life seems a perpetual process of negotiation.

R3.6 Pluralism: Three theories or a triple theory?

Kardos raises the possibility that different types of moral consideration are more compelling in different contexts. Kardos interprets this observation as arguing for some form of moral pluralism, and we accept that this may ultimately be correct, both descriptively and normatively. But our ambition is to create for a stronger and, if viable, more explanatory account, according to which the apparent diversity of moral considerations arises from the variety of approximations to a contractualist standard that best deal with specific situations. Moreover, we wholeheartedly agree with Kardos that morality should not be interpreted in individualist terms – indeed, a contractualist viewpoint explains why: the psychology of morality is rooted not in the judgments of individuals but in systems of agreements between two or more people. And there will be a reciprocal connection between such systems of agreements and group cohesion and identity: who is “in the group” may affect who has a role in bargaining over of what should be agreed; and the group may gain its coherence through shared agreements about how to live (and dissenters may be punished, ostracized or even expelled from the group).

Indeed, Fu & Mei stress the possibility that the process of negotiation need not lead to a population aligning on a single set of agreements (or indeed, rules or values). Rather the population may polarize, or fractionate in more complex ways, into different subpopulations with divergent viewpoints, without prospect of reaching long-term consensus. From a descriptive point of view, we see this is a feature, rather than a bug, in our account: indeed, by default we should expect groups of all sizes to be renegotiating at least some of the rules by which they live – and hence potentially generating clashes within and between groups. Indeed, this is surely the normal process of “politics” whether within a family (see Ullman), sports team, firm, or nation; and the challenge of a liberal society has long been viewed as agreeing to ground-rules which are sufficiently flexible that such variety can be tolerated, and disputes resolved, with a minimum of a coercion and conflict (Berlin, Reference Berlin1969; Mill, Reference Mill1859/2003). Thus, we see the close links between group identity and moral reasoning that Fu & Mei identify as suggesting deep connections between moral and social psychology.

R4.1 AI alignment

Both Gabriel & Franklin and Huang & Evans make the timely observation that the RRC framework has the potential to interface with artificial intelligence (AI) alignment in important ways. This issue is particularly pressing when we consider a near-term future where AI agents are increasingly empowered to take autonomous action on behalf of their users. Such agents will have to navigate the complex human moral world, understand human moral laws, rules, and unspoken norms, and make decisions related to interdependent choices where many people and AI agents are impacted.

Gabriel & Franklin argue that RRC could act as a framework for AI alignment (a theme developed in Levine et al.). They rightly highlight an important distinction (also insightfully discussed by Le Pargneux) between the contractarianism in RRC, which crucially assumes that agents have different bargaining power (through differing resources, knowledge, etc), and many normative approaches in philosophy which propose that such asymmetries should be idealized away (e.g., behind a veil of ignorance, Rawls, Reference Rawls1971; or treating people as morally equal agents, Scanlon, Reference Scanlon1998) – so that players debate and form agreements on equal terms. Such an approach is particularly natural if we are considering the justification of rules for society in its entirety and from the “original position.” (Though, see also the section The (normative) contractualist critique.)

Both partial and impartial viewpoints are promising directions for AI alignment research. Minimally, bargaining power asymmetries need to be part of a descriptive account of AI alignment, because AI agents would need a framework like RRC to model the moral thinking of the humans with whom they interact.

Moreover, the triple theory embedded in RRC – which integrates consequentialist, deontological, and contractualist elements, whether in a partial or impartial iteration – may represent a significant advantage in addressing the (normative) alignment problem as well. Indeed, given the wide diversity of philosophical views on ethical theory – and the fact that all such theories are known to diverge from moral intuition in various cases, as critics are quick to highlight – it would seem unwise to align an AI system with human morality by following the dictates of any single moral theory alone. Doing so risks producing outcomes that many would find deeply unappealing. For instance, if an AI were to adopt a strictly classical utilitarian framework, it might drift towards Parfit’s famous “repugnant conclusion,” become slave to Nozick’s Utility Monster, or other pathologies (Nozick, Reference Nozick1974; Parfit, Reference Parfit1987; Smart & Williams, Reference Smart and Williams1973). And if the interests of non-human animals were inadequately accounted for, this could theoretically justify the large-scale destruction of biodiversity, preserving only what is necessary to sustain a human-habitable biosphere. Similarly, adherence to rigid deontological rules – such as an absolute prohibition on lying – could lead to catastrophic outcomes in situations involving malevolent agents or oppressive political regimes.

Just as humans appear to employ a rich array of moral reasoning strategies – sometimes integrating them to varying degrees, as reflected in approaches like the Triple Theory at the core of RRC – we propose that a morally trustworthy AI should likewise embody a plurality of moral reasoning principles (perhaps beyond the three considered here, e.g., including virtue ethics, Foot, Reference Foot1978, see also the commentary by Srivastava et al). This diversity would not only mirror human ethical practice more closely but also help mitigate the risks inherent in strict adherence to any single ethical system. Overall, we agree with Gabriel & Franklin that RRC could perhaps be seen as more than just a descriptive theory of moral cognition, but a normative framework that could guide robust AI alignment (Levine et al.).

Of course, the most natural perspective is that AI agents should function as delegates of, or proxies for, human agents, operating in the service of their interests. There are obvious dangers if AI agents develop their own independent goals – however benign they may appear – unless those goals are clearly subordinate to human values. From a contractualist standpoint, as embodied in RRC, the central aim is to view AI agents as facilitators of agreements that would be acceptable to humans under conditions of mutual justification. This perspective presupposes that AI agents do not possess ethically relevant interests of their own (though this assumption could be contested – see Gunkel, Reference Gunkel2018).

Nevertheless, Huang & Evans propose that suitably fine-tuned LLMs might act as delegates to carry out an “in silico” version of virtual bargaining, thus enabling more efficient and effective agreements between individuals and across society. The commentators are careful to note that at present, AI models may not yet have appropriately robust representations of humans to engage in virtual bargaining successfully, but they point out that the RRC framework lays out guidelines for a series of benchmarks to measure the moral capabilities of LLMs as they develop.

Soon, the ability of AI to creatively construct appropriate and mutually beneficial norms and conventions with each other (and indeed with humans) may become of central importance. Following Huang & Evans’s suggestion that human social institutions can arise bottom-up from ad hoc agreements constructed in the moment, we may similarly imagine that “ecologies” of agentic AI systems may be able to develop new norms and conventions to guide interactions with each other, which may extend or modify human moral conventions. Indeed, it seems plausible that processes such as virtual bargaining and universalization can generate new norms between AI agents, just as with humans.

There are dangers here of course. If artificial agents are able to virtually bargain in transactions with each other, the resulting outcomes need not necessarily be socially beneficial. Indeed, in the human case, private bargains that are not open to wider scrutiny can lead to insider trading, cartels, bribery, and corruption of all kinds, which are mutually beneficial to the agents inside the bargain, but socially destructive more broadly. These issues raise many questions including: Might similar issues arise for bargains between AI agents? How might such pernicious bargains be detected and regulated?

R4.2 Institutions, delegation, and legal frameworks

Several commentaries raise issues related to how institutional structures can and do relate to the claims of RRC and suggest some promising directions for extending and implementing the RRC framework.

Radkani & Saxe and Forstmann & Weiss focus on the process of delegation as its own sort of resource-rational strategy. Radkani & Saxe point out that once delegation is conceived of in this way, many of the RRC styles of thinking can be applied to the dynamics of the delegate-principal interaction. We suggest that the same logic can be applied to complex organizational structures, with elaborately defined roles, responsibilities, and hierarchies: an organization may be viewed as a way of distributing action across multiple resource-constrained agents so that they cooperate effectively.

Consequently, we should expect many aspects of morality to be relative to a person’s role, whether in a relationship, family, group, or formal organization (e.g., Earp et al., Reference Earp, McLoughlin, Monrad, Clark and Crockett2021). Forstmann & Weiss call for a “cost-of-agency-transfer parameter” in the RRC model to weigh the benefits and burdens of delegation against engaging in virtual bargaining. We see a promising line of research in exploring whether, rather than requiring an additional parameter to the RRC model, the phenomenon of agency transfer might arise endogenously from the bargaining process itself (including bargaining over who decides what is a matter for virtual, or for public, negotiation). Indeed, organizational forms (teams, hierarchies etc) might arise spontaneously as solutions to coordinating the actions and responsibilities of groups of resource-rational agents.

In this spirit, Huang & Evans suggest that many institutions have RRC-like mechanisms embedded in them, which go beyond laws and norms alone. They think of the mechanisms in RRC as not just describing the human mind, but a way of outsourcing the RRC approach to making moral decisions to society-level processes. We see this as an interesting framework within which to understand the cultural evolution of patterns of moral and social behavior: past agreements can provide publicly available precedents for future agreements (paralleling the incremental development of common law legal systems), and specific agreements can be universalized into publicly articulated rules, regulations, and laws. From this perspective, the cultural evolution of morality is a distributed computation over many individuals and generations of resource-rational agents, each primarily addressing the challenges of the moment (for a parallel perspective on the cultural evolution of language see, e.g., Chater & Christiansen, Reference Chater, Christiansen, Lappin and Bernardy2022; Christiansen & Chater, Reference Christiansen and Chater2022; see also Polanyi, Reference Polanyi1941; as noted above, this viewpoint contrasts with André, Fitouchi & Baumard).

If this is right, then perhaps RRC may provide insights into how organizations and institutional rules can be better designed to allow coordination for mutual benefit. Indeed, White & Bui suggest that RRC, while described in the target article as a descriptive theory, can be leveraged for normative purposes, whether in legal, organizational, or political contexts.