1. Introduction
Norms of cooperation and fairness pose something of a puzzle for game theoretic explanations of behavior. Cooperation is often conceived of as joint cooperation in a prisoner's dilemma or a stag hunt. But joint cooperation is not an equilibrium of a prisoner's dilemma, and it is not the only equilibrium of a stag hunt. And moreover, evolutionary analyses of the stag hunt have consistently shown that the uncooperative equilibrium is a more likely outcome of adaptive processes such as the replicator dynamics and reinforcement learning.Footnote 1 Similarly, fairness is often viewed as an equal split in a bargaining game. But models of bargaining almost always have an infinite number of equilibria,Footnote 2 and there is no immediate reason to suppose that fair division will be the equilibrium selected by a dynamic process of learning or evolution.
Simple models of bargaining involve two or more parties who have an opportunity to divide some resource between them. But the acquisition or creation of this resource lies outside of standard bargaining models; it is imagined as a pure windfall, which the participants in the interaction haggle over. This is the case in, for example, the Nash demand game. The game begins with two players demanding shares of a pie, but the model says nothing about where the pie comes from. This is also the case in Rubinstein's bargaining model. Rubinstein invites us to imagine two players alternating proposals as to how to divide a pie, but the model does not specify how the agents acquire the pie. In real life, however, it is not often that players are merely presented with a windfall. Here I explore the possibility that players bargain over a surplus that they had a hand in jointly creating. Players must first cooperate, and only then do they bargain over how the fruits of cooperation are allocated. The creation of the pie is thus a preliminary stage to the bargaining process. Cooperation here is modeled as choosing to hunt stag in a stag hunt game, and the bargaining process is simply the Nash demand game.
One might think that one could straightforwardly predict behavior in this compound game from knowledge of behavior in the constituent games, but this is wrong. It will be shown that, under the replicator dynamics, this combined game in which players cooperate and then bargain has different dynamical properties than the individual games. In particular, the compound model increases the size of the basin of attraction for fair division. And for some payoff values, unequal division in the bargaining subgame is dynamically unstable. Furthermore, for some payoffs the basin of attraction for cooperation in the stag hunt is also increased. This compound model demonstrates not only that norms of cooperation and fairness can coevolve but that the joint setting in which cooperation is a prerequisite for bargaining can be more favorable for the evolution of both cooperation and fairness than either of the individual games when taken by themselves.
2. Dividing the Fruits of Cooperation
In a stag hunt game, shown in table 1, players each have two strategies. They can choose to hunt stag or they can choose to hunt hare. If both players cooperate by jointly hunting stag, they receive a large reward. However, if either player goes it alone by hunting hare, then the stag hunt fails. Hunting hare receives a smaller reward, but the reward is independent of your partner's choice. If 
, this game has two strict Nash equilibria: 〈stag, stag〉 and 〈hare, hare〉. In the terminology of Harsanyi and Selten (Reference Harsanyi and Selten1988), the stag hunting equilibrium is payoff dominant, and the hare-hunting equilibrium is risk dominant.Footnote 3
| Stag | Hare | |
| Stag | S, S | 0, H |
| Hare | H, 0 | H, H |
Note It is assumed that 
.
The stag hunt has been repeatedly suggested as a model of the strategic interaction underlying the formation of a social contract. Rousseau (Reference Rousseau and Cranston1755/1984, 111) noted that each member of a group hunting a stag “well realized that he must remain faithful to his post; but if a hare happened to pass within reach of one of them, we cannot doubt that he would have gone off in pursuit of it without scruple.” The stag hunt also appears as the meadow-draining problem in Hume (Reference Hume1739/2000). In this story, neighbors may jointly benefit from the draining of a common meadow, but although joint cooperation is mutually beneficial, it may be difficult to coordinate on because draining your portion of the meadow takes work, and a half-drained meadow is no better than a meadow that has not been drained at all. The essence of this strategic interaction is that joint cooperation is mutually beneficial but riskier than going it alone.
Stag hunts also frequently appear in biological models as idealizations of cooperation. For instance, Bergstrom (Reference Bergstrom2002) showed that haystack models of group selection are equivalent to one-shot stag hunts played by the founding individuals of the haystacks. As another example, consider social microbes, such as Myxococcis xanthus, that coordinate feeding on larger microbial prey (Crespi Reference Crespi2001). These “wolfpack bacteria” secrete enzymes in order to digest their prey, but a single bacterium cannot excrete a sufficient quantity of the enzyme alone. In order to feed, multiple bacteria must cooperate. This appears to be a stag hunt.
In the Nash demand or bargaining game, however, players simultaneously demand portions of a pie of size C. If both their demands can be satisfied (i.e., if the demands sum to something less than or equal to C), then both players receive their demand. If their demands are incompatible, then both players receive nothing. To simplify the analysis, we can assume that the players demand 1/3, 1/2, or 2/3 of the pie. The payoff matrix for this Nash demand minigame is shown in table 2.
| 1/3 | 1/2 | 2/3 | |
| 1/3 | C/3, C/3 | C/3, C/2 | C/3, 2C/3 |
| 1/2 | C/2, C/3 | C/2, C/2 | 0, 0 |
| 2/3 | 2C/3, C/3 | 0, 0 | 0, 0 |
This game has one symmetric Nash equilibrium in pure strategies. Both players in this equilibrium demand 1/2. There are also two notable symmetric mixed equilibria. In the first of these equilibria, the players demand 1/3 with probability 1/2, and they demand 2/3 with probability 1/2. They never demand 1/2. This mixture is evolutionarily stable. There is also a totally mixed equilibrium in which the players demand 1/3 with probability 1/2, demand 1/2 with probability 1/6, and demand 2/3 with probability 1/3.
Like the stag hunt, bargaining games have also been suggested as idealizations of social contract formation. In particular, Harsanyi (Reference Harsanyi1953), Rawls (Reference Rawls1958), and Binmore (Reference Binmore1993) observe that a society's selection of a fair social contract resembles a choice (perhaps made from behind a veil of ignorance) of a societal arrangement from a set of feasible arrangements. Some of these setups may be preferred more by some members of society than others. Thus, the arrival at a social contract requires some mechanism for negotiating which one of these societal outcomes is chosen as the actual social contract or the fair arrangement of society. This negotiation can be modeled as a bargaining game.Footnote 4
This sort of strategic interaction is also common in biological models. Bargaining games have been used to explain the conflict of interest between mates in species that exhibit cooperative biparental care (Houston and Davies Reference Houston, Davies, Sibly and Smith1985; McNamara and Houston Reference McNamara and Houston2002; Johnstone and Hinde Reference Johnstone and Hinde2006). In these sorts of models, mating pairs are understood as negotiating their level of investment in their joint offspring. And territorial behavior has also been illuminated through the application of bargaining games. Periera, Bergman, and Roughgarden (Reference Periera, Bergman and Roughgarden2003) argue that some species, such as Anolis lizards, partition their home ranges in such a way as to approximate solutions to bargaining games.
The stag hunt and Nash demand minigames can be easily combined into an extensive form game in which players first simultaneously choose whether to hunt stag or hare. If either chooses hare, then both players receive the corresponding payoffs from the stag hunt game. However, if both players choose stag, then they generate some surplus of size C. After creating this surplus, the players bargain over how it is allocated. This negotiation is modeled as the Nash demand game. The extensive form structure of this game is illustrated in figure 1. To interpret this strategic interaction, one might think of early hominids choosing to cooperate in a group hunt and then, if the hunt is successful, distributing the fruit between the hunters. Or one might think of a male and a female of a species, such as hedge sparrows, that jointly raise their offspring, choosing whether to mate and, then, if they do mate, negotiating the level of parental investment each provides to their young.
Compound game.
Notice that if both players choose to hunt stag, then they end up engaged in the simultaneous-move Nash demand minigame, so that the demand game is retained as a subgame of the larger compound game. It is also worth noting that if both players cooperate and then equally divide the pie, then they receive the same payoffs as they would from the ordinary stag hunt (with 
). For this reason, the standard stag hunt is also retained in the compound game. This compound setting makes obvious an assumption implicit in the payoff matrix of the ordinary stag hung, namely, the assumption that both players profit equally from mutual cooperation. So the combined game can be viewed as relaxing assumptions present in both individual models. From the bargaining game, we are relaxing the presupposition that players are able to jointly obtain a desirable and scarce resource. And from the stag hunt, we are relaxing the assumption that whatever benefit is created through cooperation is automatically distributed equally between the cooperators.
Pure strategies in an extensive form game stipulate actions to take at each information set. Thus, each player's strategy consists of a move in the stag hunt and a move in the demand game. In other words, each player's pure strategy is a pair 〈s 1, s 2〉, where 
and 
. Notice that, when 
, any pair of strategies that has both players choose hare is a Nash equilibrium of this game. Additionally, when 
, the strategy profile in which both agents play 〈stag, 1/2〉 is a strict Nash equilibrium. In this equilibrium, players cooperate and then split the surplus equally. And when 
, the asymmetric profile in which one agent plays 〈stag, 1/3〉 and the other plays 〈stag, 2/3〉 is a strict Nash equilibrium. In this state, the agents cooperate but then unequally divide the fruits of their labor. Corresponding to this asymmetric equilibrium, there is also a mixed symmetric equilibrium in behavioral strategies. In this symmetric equilibrium, both players choose stag and then demand 1/3 with probability 1/2 and demand 2/3 with probability 1/2. There are many other mixed equilibria of this game, but they are not dynamically stable, and so I omit a description of them here.
3. The Replicator Dynamics
An obvious way to begin an investigation into the dynamics of the compound game is with the one-population replicator dynamics. The one-population continuous-time replicator dynamics are given by the set of differential equations
where x is a vector in which each element is the frequency of the ith strategy in the (infinitely large) population and A is the game's payoff matrix. Because the components of x must sum to one, the replicator dynamics for a game with n pure strategies lives in the 
-dimensional simplex. The replicator dynamics was originally proposed by Taylor and Jonker (Reference Taylor and Jonker1978) as a model of the evolution of an asexually reproducing population. But this dynamic is also suitable as a simple model of cultural evolution or social learning (see, e.g., Börgers and Sarin Reference Börgers and Sarin1997; Schlag Reference Schlag1998). For example, consider a learning scheme that has agents spontaneously revise their strategies by imitating a player chosen with probability proportional to the difference between that player's expected payoff and the average payoff of the entire population. Schlag (Reference Schlag1998) shows that, provided that limits are taken in the appropriate way, this general learning scheme yields the replicator dynamic as its aggregate behavior.
Evolutionarily stable equilibria are always asymptotically stable states in the replicator dynamic.Footnote 5 Therefore, in both the stand-alone stag hunt and the stand-alone Nash demand minigame, there are two asymptotically stable states. Phase portraits for both games are shown in figure 2. Although the replicator dynamics do not uniquely identify an equilibrium as the sole stable outcome of the evolutionary process, not all equilibria have basins of attraction with the same size. For example, in the stag hunt, the probability that a randomly selected initial condition converges to the cooperative state is 
. So if the payoff for mutual cooperation is 3 and the payoff for going it alone is 2, just 1/3 of the state space converges to mutual cooperation. Additionally, if chance mutations are added to the model, the population spends most of its time in the uncooperative state because this is the state with the larger basin of attraction (Foster and Young Reference Foster and Young1990).
Phase portraits for both the stag hunt (a) and the Nash demand game (b) in the replicator dynamic. Sinks are indicated by black circles, saddles by gray circles, and sources by open circles.
When examining the dynamics of the compound model of cooperation and bargaining, there is a sense in which incorporating three hare-hunting strategies is redundant. After all, if either player chooses hare, then the branch of the game tree corresponding to the bargaining problem is never reached. A consequence of this fact is that the three strategies 〈hare, 1/3〉, 〈hare, 1/2〉, 〈hare, 2/3〉 are behaviorally indistinguishable. This means that, for instance, an experimenter cannot determine which of these three strategies a hare hunter is employing just by observing play in the compound game. So it is convenient, at least for a first pass of analysis, to limit the state space to four strategies: hare, 〈stag, 1/3〉, 〈stag, 1/2〉, 〈stag, 2/3〉. This selection of strategies maintains all of the interesting equilibria of the extensive form but eliminates the weak Nash equilibria that are formed by mixing over the various behaviorally identical hare-hunting strategies. Payoffs are given by the strategic form representation of the extensive-form game. These strategic form payoffs are shown in table 3.
| 〈Stag, 1/3〉 | 〈Stag, 1/2〉 | 〈Stag, 2/3〉 | Hare | |
| 〈Stag, 1/3〉 | C/3 | C/3 | C/3 | 0 |
| 〈Stag, 1/2〉 | C/2 | C/2 | 0 | 0 |
| 〈Stag, 2/3〉 | 2C/3 | 0 | 0 | 0 |
| Hare | H | H | H | H |
Since there are four pure strategies in the compound game, the replicator dynamic here lives in the three-dimensional tetrahedron. As long as 
, there are two symmetric strict Nash equilibria: one that has both agents play hare and one in which both agents play 〈stag, 1/2〉. These strict equilibria are asymptotically stable. Furthermore, when 
, the mixed symmetric Nash equilibrium in which both agents play 〈stag, 1/3〉 with probability 1/2 and 〈stag, 2/3〉 with probability 1/2 is an evolutionarily stable strategy. Thus, it is also asymptotically stable in the replicator dynamics. A phase portrait of this entire three-dimensional system is shown in figure 3.Footnote 6 When the rewards of going it alone are small, the evolutionary process will take the system to one of three states: hare hunting, cooperation followed by equal division, or cooperation followed by unequal division.
Replicator dynamics for the compound cooperation/bargaining game with payoffs from table 3 when 
. Sinks are indicated by black circles, saddles by gray circles, and sources by open circles.
However, when the payoff for hare hunting is large, the dynamics of the system are fundamentally different. Specifically, when 
, the mixed Nash equilibrium corresponding to cooperation followed by unequal division is dynamically unstable. The system's phase portrait is illustrated in figure 4. This difference in the qualitative nature of the system in these two cases is due to a bifurcation that occurs at 
. When hare hunting yields this payoff, the strategy hare weakly dominates the strategy 〈stag, 1/3〉, and the two unstable rest points on the 
face of the system collide with the saddle on the 〈stag, 1/3〉 vertex and the mixed rest point corresponding the unequal division. These two rest points become a source and a saddle, respectively. And then, for 
, hare strictly dominates 〈stag, 1/3〉, and thus the latter strategy is driven to extinction on that face of the dynamics. Consequently, the mixed rest point becomes dynamically unstable.
Replicator dynamics for the compound cooperation/bargaining game with payoffs from table 3 when 
. Sinks are indicated by black circles, saddles by gray circles, and sources by open circles.
We see here that the opportunity to hunt hare in the first stage of the game effectively adds an outside option that, for some parameter values, alters the nature of the strategic interaction.Footnote 7 If the payoff for hare hunting is high enough, then unequal division does not pay. Although such distribution is asymptotically stable in the ordinary Nash demand game, it is dynamically unstable in the compound game. There is no point in cooperating if you are going to get the short end of the stick when dividing the spoils of cooperation.
But what of parameter values 
? Although unequal division remains asymptotically stable provided that the players evolve to stag hunting, it is less likely that the system converges to unequal division in the compound game than it is in the ordinary Nash demand game. Approximately 71.7% of state space evolves to the equal division equilibrium in the stand-alone bargaining game.Footnote 8 But as figure 5 shows, the frequency of equal division is greater in the compound game than in the ordinary Nash demand game for all values of H. These data were obtained by fixing 
and then varying H from .05 to 1.6. Each data point is the average of 50,000 randomly chosen initial conditions. Notice that above 
, unequal division is never seen as a result of the evolutionary process. This is because although this state is a rest point of the dynamics, it is dynamically unstable for the reasons described above. But even when H is below the bifurcation point, provided that the system evolves to cooperation (of course the system could also evolve to the uncooperative hare-hunting state), it is more likely to end at equal division here than in the stand-alone bargaining game.
Relative size of the basin of attraction of the fair division equilibrium in the compound game (dotted line) compared to that in the ordinary stand-alone miniature bargaining game (solid line). Each dot indicates the proportion of randomly chosen initial conditions that ended at fair division in the joint game out of those initial conditions that evolved to cooperation in the first stage of the game.
Additionally, it is possible to investigate the likelihood of convergence to cooperation in the compound game. In the standard stag hunt, in which the payoff for jointly hunting stag is S and the payoff for hunting hare is H, the relative size of the basin of attraction for cooperation is 
. Figure 6 shows the estimation of the size of the basin of attraction for stag hunting in the compound game (again with 
and H ranging from 0.05 to 1.6). As the plot makes clear, there is a range of H values in which the evolution of cooperation is more likely in the compound game than it is in the stand-alone stag hunt. Moreover, this range overlaps with the range in which fair division in the compound game (provided the agents evolve to cooperate) is more likely than fair division in the stand-alone bargaining game. In other words, for some parameter values the combined strategic interaction in which players must cooperate and then divide the pie is more favorable for the evolution of both cooperation and fair division than either the stand-alone stag hunt is for cooperation or the stand-alone Nash demand game is for fair division.Footnote 9
Relative sizes of the basins of attraction for cooperation in the compound game (dotted line) and cooperation in the ordinary stand-alone stag hunt (solid line). The payoff for cooperating in the stand-alone stag hunt was fixed at 
and the joint surplus created by cooperation was fixed at 
in the compound game.
4. Conclusion
One approach to studying the evolution of cooperation and fairness would be to consider both types of strategic behavior as modules that evolve separately and are then combined. This approach would entail examining the evolutionary dynamics of stag hunt games independently from the dynamics of bargaining games. This sort of analysis is appropriate for modeling interactions in which mutual cooperation does not lead to the creation of a resource that must be distributed. The alternative approach, taken here, sees cooperation and fairness as coevolved strategies in a compound game. This approach provides a perhaps more realistic idealization of situations, such as group foraging, in which the benefits of cooperation must be allocated between the cooperators. I have shown here that the two approaches give different results.
In particular, the compound stag hunt/demand game changes the prospects for the evolution of cooperation and fairness in two distinct ways. First, the option of hunting hare in the preliminary stage of the compound game functions as an outside option to the bargaining problem. The presence of this outside option makes accepting the loser's payoff in the bargaining game unattractive. Consequently, in the replicator dynamics, the likelihood of observing fair division in the bargaining subgame is greater than the likelihood of observing fair division in the stand-alone bargaining game. This is especially perspicuous when the ratio of the payoff to hare hunting to the payoff for getting the short end in the bargaining game is greater than one. For parameter values that fall in this regime, unequal division is dynamically unstable and thus never the outcome of the evolutionary process. Second, for a wide range of possible parameter values, the compound game also favors the evolution of cooperation. Namely, if the ratio of the payoff for hunting hare to the size of the public surplus created by jointly hunting stag is small, then a larger portion of phase space converges to stag hunting in the compound game than in the stand-alone stag hunt.
.
