The target article is interesting and raises many important questions about the role of costly punishment in the evolution of human sociality. Particularly relevant is the mismatch between the design of many economic experiments and the conditions under which human evolution (genetic or cultural) seems likely to occur.

Nearly all economic game experiments exploring costly punishment are explicitly designed to suppress the opportunity for retaliation and feuds. These experiments reshuffle either groups or identities from round to round and report the amount of punishment received only in aggregate. Such design features are highly unrealistic, and serve to cast costly punishment in the most positive possible light. As Guala points out, costly punishment can be disastrous in more realistic experimental settings with truly repeated interactions where retaliation is possible (Denant-Boemont et al. Reference Denant-Boemont, Masclet and Noussair2007; Dreber et al. Reference Dreber, Rand, Fudenberg and Nowak2008; Nikiforakis Reference Nikiforakis2008; Wu et al. Reference Wu, Zhang, Zhou, He, Zheng, Cressman and Tao2009). Not only can cooperators punish defectors, but defectors can also punish cooperators (Cinyabuguma et al. Reference Cinyabuguma, Page and Putterman2006; Gächter & Herrmann Reference Gächter and Herrmann2009; Reference Gächter and Herrmann2011; Herrmann et al. Reference Herrmann, Thöni and Gächter2008). Furthermore, positive reciprocity (rather than costly punishment) can effectively maintain cooperation when repetition or reputation are allowed (Dal Bó Reference Dal Bó2005; Dal Bó & Fréchette Reference Dal Bó and Fréchette2011; Fudenberg et al., in press; Milinski et al. Reference Milinski, Semmann, Bakker and Krambeck2001; Reference Milinski, Semmann and Krambeck2002; Rand et al. Reference Rand, Dreber, Ellingsen, Fudenberg and Nowak2009a; Rockenbach & Milinski Reference Rockenbach and Milinski2006; Wedekind & Milinski Reference Wedekind and Milinski2000), although see Vyrastekova and van Soest (Reference Vyrastekova and van Soest2008).

This same critique also applies to almost all evolutionary game theoretic models of costly punishment and cooperation. Many models have been proposed to demonstrate how costly punishment could promote the evolution of cooperation (Bowles & Gintis Reference Bowles and Gintis2004; Boyd et al. Reference Boyd, Gintis, Bowles and Richerson2003; Gintis Reference Gintis2000; Hauert et al. Reference Hauert, Traulsen, Brandt, Nowak and Sigmund2007; Nakamaru & Iwasa Reference Nakamaru and Iwasa2005; Reference Nakamaru and Iwasa2006; Sigmund et al. Reference Sigmund, De Silva, Traulsen and Hauert2010, Traulsen et al. Reference Traulsen, Hauert, De Silva, Nowak and Sigmund2009; Wang et al. Reference Wang, Wu, Ho and Wang2011). Yet virtually all of these models assume that only cooperators punish defectors. Punishment targeted at cooperators (“antisocial punishment”) is excluded a priori.

We feel that evolutionary models do best to include the full range of combinatorially possible strategies (of a specified level of complexity) and then to ask which are favored by natural selection. If, instead, the strategy set is restricted to only include strategies that seem logical or desirable, this can greatly affect the outcomes and potentially be quite misleading. We note that this is not unique to models based on “strong reciprocity” but also applies to most models that do not invoke group selection.

Recent theoretical work has examined the effect of retaliation and antisocial punishment, and the results have not been promising for “altruistic” punishment. When the opportunity to retaliate is added to a model based on intergroup conflict, punishment is much less effective at promoting cooperation (Janssen & Bushman Reference Janssen and Bushman2008). In such models, groups of cooperators outcompete groups of defectors; but within a single group, defectors outcompete cooperators. Costly punishment allows cooperative groups to keep out defectors (Boyd et al. Reference Boyd, Gintis, Bowles and Richerson2003). But now the second-order free-rider problem arises: Cooperators who punish are at a disadvantage relative to cooperators who do not punish. When defectors are rare, this disadvantage is small and does little to undermine cooperation. But when retaliation is possible, this exacerbates the second-order free-rider problem: Not only do cooperative punishers bare the cost of punishing relative to non-punishing cooperators, but they also incur the additional cost of being retaliated upon. Thus, punishment has only limited power to promote cooperation.

Similar results are obtained when antisocial punishment (as well as indiscriminant punishment) are allowed in a model based on spatial structure (Rand et al. Reference Rand, Armao, Nakamaru and Ohtsuki2010). In this second model, interaction and competition only occur with those nearby, thus relative payoff is key and spite is adaptive. When only cooperators can punish defectors, punishment allows cooperation to dominate: Without punishment, cooperators are at a disadvantage because they pay the cost of cooperation; but by punishing defectors, they can regain the relative advantage (Nakamaru & Iwasa Reference Nakamaru and Iwasa2005; Reference Nakamaru and Iwasa2006). When all punishment strategies are available, however, defectors can also punish cooperators (a form of anticipatory retaliation). Thus, the prosocial and antisocial punishments cancel each other out, and punishment no longer allows cooperation to proliferate. Instead, the only strategy that is globally stable is to defect and then punish cooperators.

Likewise, punishment no longer promotes cooperation in an optional Public Goods game model once antisocial punishment is allowed (Rand & Nowak Reference Rand and Nowak2011). In optional cooperation games, defectors invade cooperators, loners that opt out of the game in favor of a fixed intermediate payoff invade defectors, and cooperators invade loners (Hauert et al. Reference Hauert, De Monte, Hofbauer and Sigmund2002). Allowing cooperators to punish defectors breaks this rock-paper-scissors cycle and stabilizes cooperation (Hauert et al. Reference Hauert, Traulsen, Brandt, Nowak and Sigmund2007). But when all punishment strategies are possible, the cycle is as easily broken by defectors that punish loners, or loners that punish cooperators. Antisocial punishment is common, and punishment does not substantially increase cooperation compared to a game without punishment.

Another model considers prosocial and antisocial punishment as well as retaliation in the context of a repeated Prisoner's Dilemma game (Rand et al. Reference Rand, Ohtsuki and Nowak2009b). A Nash equilibrium analysis finds many cooperative equilibria that pay to punish defection. Yet stochastic evolutionary simulations find that selection consistently disfavors costly punishment, and these simulations show quantitative agreement with a set of behavioral experiments (Dreber et al. Reference Dreber, Rand, Fudenberg and Nowak2008). In equilibrium, costly punishment is not actually costly – just the threat is sufficient to maintain cooperation. But in the noisy world of stochastic game dynamics, mutation and (relatively) weak selection lead to heterogeneous populations: The punisher must pay. Hence, costly punishment is disfavored, and instead evolution leads to traditional tit-for-tat strategies that “punish” defection not with costly punishment, but rather with denial of future reward.

Thus, the issues raised by Guala with respect to artificial experimental designs also apply to many evolutionary game theoretic models. Initial explorations of allowing retaliation and antisocial punishment in these models find the power of punishment for promoting cooperation to be much reduced or non-existent; further work in this vein is an important direction for future study.