In their excellent article, Henrich et al. rightly caution us to be careful when we draw general conclusions from WEIRD subject pools, of which undergraduates are the most frequently used one, also in economics. My main comment is that the right choice of subject pool is intimately linked to the research question. Since the different behavioral sciences also have different research questions, the right choice of subject pool will also often be different across disciplines. In my own discipline, economics, students are actually often the best subject pool for quite a few (fundamental) research questions. Here is why I believe so.

Economic theories normally do not come with assumptions (or even caveats) about the restricted validity to only a specific group of people; that is, they (implicitly) assume “generality.” Like the assumption of selfishness, “generality” is a good assumption in the absence of rigorous data. The tools of experimental economics have been deployed to investigate the empirical relevance of the selfishness assumption (see, e.g., Fehr et al. Reference Fehr, Gächter and Fischbacher2002) and are now also used to probe the “generality assumption,” that is, the importance of variations of behavior across population subgroups within a given society (e.g., Bellemare et al. Reference Bellemare, Kröger and Van Soest2008) or across societies (e.g., Herrmann et al. Reference Herrmann, Thoni and Gächter2008).

However, my main point is this: The “right choice” of subject pool depends on the research question. If the researcher is interested in understanding behavioral variation between particular groups of people, then the right choice is running experiments with these people. The landmark study by Henrich et al. (Reference Henrich, Boyd, Bowles, Camerer, Fehr, Gintis, McElreath, Alvard, Barr, Ensminger, Henrich, Hill, Gil-White, Gurven, Marlowe, Patton and Tracer2005) is a shining example. Yet, at least in economics, substantial effort is also devoted to test formal theories or to detect interesting behavioral regularities (Bardsley et al. Reference Bardsley, Cubitt, Loomes, Moffatt, Starmer and Sugden2010; Croson & Gächter Reference Croson and Gächter2010; Smith Reference Smith2010). Because economic theories normally assume generality, any subject pool is in principle informative about whether theoretical predictions or assumptions contain behavioral validity. At that stage, generalizability to other subject pools is not (yet) an issue. Among the universe of potential subject pools to test a theory, students are often the perfect one: on average, students are educated, intelligent, and used to learning. These are very valuable characteristics because, in addition to the main aspect of a theory of interest to the researcher, economic theories often assume cognitive sophistication. It therefore makes sense to control for sophistication also by choice of subject pool (in addition to clear instructions), in order to minimize chances of confounding genuine behavioral reactions to the treatment of interest with lack of understanding of the basic decision situation.

Take recent theories of social preferences (as surveyed, e.g., in Fehr & Schmidt Reference Fehr, Schmidt, Kolm and Ythier2006) as an example. In addition to other-regarding preferences, these theories all assume cognitive sophistication. When testing these theories, the main point of interest is not to find out whether people are as cognitively sophisticated as the theories (maybe wrongly) assume, but to see to what extent other-regarding motives exist, holding everything else constant. Because students are typically above average with regard to cognitive sophistication, they are often a perfect subject pool for first tests of a theory. Moreover, students, unlike most other subject pools, are readily available (and cost effective). Experiments can therefore also easily be replicated, which is important to establish empirical regularity and hard to achieve with any other subject pool.

Of course, strictly speaking, observed results hold only for the subject pool from which evidence is collected. Generalizability is a generic issue in any empirical research (Falk & Heckman Reference Falk and Heckman2009). However, once a clear benchmark result is established, we can proceed by testing, for example, how age and life experience matter (e.g., Sutter & Kocher Reference Sutter and Kocher2007b), or how results extend to more representative subject pools (e.g., Bellemare et al. Reference Bellemare, Kröger and Van Soest2008; Carpenter et al. Reference Carpenter, Connolly and Myers2008). Along the way, researchers often establish whether and how students differ from the general population.

As Henrich et al. point out, understanding the potential influence of cross-societal (or cultural) differences in (economic) behavior is a particularly interesting direction for investigating generalizability. But it poses further challenges, in particular if socio-demographic factors matter (as some of the above-cited research suggests). The reason is that socio-demographic influences might be confounded with genuine societal or cultural differences. The problem is exacerbated the more subject pools are actually being compared. Again, to ensure that confounds are minimized, student subject pools are often the best available choice (Bohnet et al. Reference Bohnet, Greig, Herrmann and Zeckhauser2008; Herrmann et al. Reference Herrmann, Thoni and Gächter2008) to establish a clean benchmark result on how people from different societal/cultural backgrounds behave in the exact same decision situation – a fundamental question from the generality perspective of economics. The benchmark can – and should(!) – then be taken as a starting point for investigating generalizability to other social groups.