Henrich et al. provide an important piece of “consciousness raising”: They remind us that human beings are complicated and diverse, and review evidence that the variance in some types of behaviour and cognitive performance is far greater than has often been assumed. These facts have been obscured by the WEIRD-ness of the vast majority of the research subjects in behavioural science: people are not interchangeable in the same sense in which a sample of gold, whatever its origin, can be taken to the laboratory and treated as representative of gold generally. No one geographical, cultural, or socio-economic subgroup of people (least of all, perhaps, contemporary Western undergraduates) is representative of humanity. This means theories tested only or largely against a narrow range of subjects are likely to be incomplete or outright false. Behavioural scientists, in other words, have engaged in precipitate theorizing: outliers are unlikely to constrain explanatory theories correctly or inspire the right sort of hypothesizing.
To remedy this bias, ideally, we need globally representative samples of subjects in order to generate thorough descriptions of cognitive phenomena, guide hypothesis formation, and rigorously to test theories. The logistical and financial challenges of conducting research on this basis are immense. Overcoming them requires, among other things, changes in the research culture and to academic incentives (here Henrich et al.'s recommendations are spot on). In psychology, for example, the current incentives produce many small, single-institution, low-powered studies aimed at discovering novel effects. What we need, however, is not, say, evidence of yet more ways of modulating implicit associations in American undergraduates. Rather, we need large, prospectively designed, highly powered, cross-cultural studies that can answer specific questions more definitively. Medical research provides a model behavioural researchers would do well to emulate. In general and in outline, before some clinical intervention is approved for use, research must proceed through three stages. This begins with small open-label studies in Phase I; proceeds to larger, single-blind trials in Phase II; and culminates in large, multi-center, randomized, double-blind, placebo-controlled trials in Phase III. Behavioural scientists conduct too many studies in the equivalent of Phases I or II, with too few large, definitive and cross-cultural studies. There is still room for small studies – important novel phenomena undoubtedly await discovery. We merely suggest more “Phase III” research. Pooling resources, exploiting various online collaboration tools (e.g., Nielsen Reference Nielsen2008), and shifting editorial policies and research priorities should result in more large, cross-cultural studies being conducted.
That noted, Henrich et al. underplay – to the point of missing – that how the behavioural sciences research community itself is constituted introduces biases. That the subject-pool of behavioural science is so shallow is indeed a serious problem, but so is the fact that the majority of behavioural researchers are themselves deeply WEIRD. People in Western countries have, on average, a remarkably homogenous set of values compared to the full range of worldwide variability (Inglehart & Welzel Reference Inglehart and Welzel2005), and the data Henrich et al. adduce suggest similarly population-level homogeneity in cognitive styles. Moreover, academics are more uniform than the populations from which they are drawn (as the target article's Contrast 4 suggests), so it is not implausible to think behavioural scientists are even WEIRD-er than their most common subjects. Henrich et al. review a body of studies and experiments that did not strike those who designed and conducted them as focused on outliers. Intelligent scientists acting in good faith conducted, peer-reviewed, and published this research, honestly believing in many cases that it threw light on human nature. This forcefully illustrates the power of the biases on the part of researchers themselves. It also suggests that, besides widening the pool of subjects, there are significant gains to be made by broadening the range of inputs to the scientific process, including in the conception, design, and evaluation of empirical and theoretical work. Given that diverse groups are demonstrably better at some kinds of problem solving (e.g., Hong & Page Reference Hong and Page2004; Lakhani et al. Reference Lakhani, Jeppesen, Lohse and Panetta2006), as things stand, the WEIRD-dominated literature is robbed of potentially worthwhile perspectives, critiques, and hypotheses that a truly global research community could provide. Clearly, simply increasing the number of behavioural sciences researchers will, in general, be beneficial. Our key contention, though, is that the marginal benefits of additional Western researchers are much smaller than the marginal benefits of more non-Western researchers, among other things, just because they are non-Western.
The non-Western world, in short, can contribute not only additional subjects to experiment upon – the main focus of the target article's recommendations – but also additional researchers, with novel perspectives and ideas and who are less affected by WEIRD biases. (Naturally, these researchers will have biases of their own. Our claim is not that there is someone who consistently knows better than WEIRD researchers. It is that diverse groups of investigators can avoid some kinds of error.) Clearly, these researchers will have to be educated, will likely be middle class, and, since science flourishes in politically open societies, they will tend be concentrated in liberal countries. Nevertheless, additional non-Western researchers, even if they are educated and relatively wealthy, could be a boon to the behavioural sciences.
A direct and powerful way to remedy both sources of bias – too many WEIRD subjects and too few non-WEIRD researchers – is to foster research capacity in the non-Western world. Non-WEIRD researchers tend to study non-WEIRD subjects, so increasing their number will deepen the subject pool and widen the range of inputs to the scientific process at the same time. Building research capacity, however, should not merely involve collaborations led by WEIRD researchers; it should aim to generate studies led and initiated by non-Western researchers. Committed and long-term inter-institutional collaboration between Western and non-Western universities focused on remedying the deficits in the behavioural sciences literature should include internships at Western universities for non-Western researchers, stints at non-Western universities for WEIRD researchers, and extensive student exchange programmes (especially for graduate students). Unlike many existing scholarship and exchange programmes in the sciences, a key point of the necessary programmes should be for the learning to proceed in both directions.
You have
Access
Henrich et al. provide an important piece of “consciousness raising”: They remind us that human beings are complicated and diverse, and review evidence that the variance in some types of behaviour and cognitive performance is far greater than has often been assumed. These facts have been obscured by the WEIRD-ness of the vast majority of the research subjects in behavioural science: people are not interchangeable in the same sense in which a sample of gold, whatever its origin, can be taken to the laboratory and treated as representative of gold generally. No one geographical, cultural, or socio-economic subgroup of people (least of all, perhaps, contemporary Western undergraduates) is representative of humanity. This means theories tested only or largely against a narrow range of subjects are likely to be incomplete or outright false. Behavioural scientists, in other words, have engaged in precipitate theorizing: outliers are unlikely to constrain explanatory theories correctly or inspire the right sort of hypothesizing.
To remedy this bias, ideally, we need globally representative samples of subjects in order to generate thorough descriptions of cognitive phenomena, guide hypothesis formation, and rigorously to test theories. The logistical and financial challenges of conducting research on this basis are immense. Overcoming them requires, among other things, changes in the research culture and to academic incentives (here Henrich et al.'s recommendations are spot on). In psychology, for example, the current incentives produce many small, single-institution, low-powered studies aimed at discovering novel effects. What we need, however, is not, say, evidence of yet more ways of modulating implicit associations in American undergraduates. Rather, we need large, prospectively designed, highly powered, cross-cultural studies that can answer specific questions more definitively. Medical research provides a model behavioural researchers would do well to emulate. In general and in outline, before some clinical intervention is approved for use, research must proceed through three stages. This begins with small open-label studies in Phase I; proceeds to larger, single-blind trials in Phase II; and culminates in large, multi-center, randomized, double-blind, placebo-controlled trials in Phase III. Behavioural scientists conduct too many studies in the equivalent of Phases I or II, with too few large, definitive and cross-cultural studies. There is still room for small studies – important novel phenomena undoubtedly await discovery. We merely suggest more “Phase III” research. Pooling resources, exploiting various online collaboration tools (e.g., Nielsen Reference Nielsen2008), and shifting editorial policies and research priorities should result in more large, cross-cultural studies being conducted.
That noted, Henrich et al. underplay – to the point of missing – that how the behavioural sciences research community itself is constituted introduces biases. That the subject-pool of behavioural science is so shallow is indeed a serious problem, but so is the fact that the majority of behavioural researchers are themselves deeply WEIRD. People in Western countries have, on average, a remarkably homogenous set of values compared to the full range of worldwide variability (Inglehart & Welzel Reference Inglehart and Welzel2005), and the data Henrich et al. adduce suggest similarly population-level homogeneity in cognitive styles. Moreover, academics are more uniform than the populations from which they are drawn (as the target article's Contrast 4 suggests), so it is not implausible to think behavioural scientists are even WEIRD-er than their most common subjects. Henrich et al. review a body of studies and experiments that did not strike those who designed and conducted them as focused on outliers. Intelligent scientists acting in good faith conducted, peer-reviewed, and published this research, honestly believing in many cases that it threw light on human nature. This forcefully illustrates the power of the biases on the part of researchers themselves. It also suggests that, besides widening the pool of subjects, there are significant gains to be made by broadening the range of inputs to the scientific process, including in the conception, design, and evaluation of empirical and theoretical work. Given that diverse groups are demonstrably better at some kinds of problem solving (e.g., Hong & Page Reference Hong and Page2004; Lakhani et al. Reference Lakhani, Jeppesen, Lohse and Panetta2006), as things stand, the WEIRD-dominated literature is robbed of potentially worthwhile perspectives, critiques, and hypotheses that a truly global research community could provide. Clearly, simply increasing the number of behavioural sciences researchers will, in general, be beneficial. Our key contention, though, is that the marginal benefits of additional Western researchers are much smaller than the marginal benefits of more non-Western researchers, among other things, just because they are non-Western.
The non-Western world, in short, can contribute not only additional subjects to experiment upon – the main focus of the target article's recommendations – but also additional researchers, with novel perspectives and ideas and who are less affected by WEIRD biases. (Naturally, these researchers will have biases of their own. Our claim is not that there is someone who consistently knows better than WEIRD researchers. It is that diverse groups of investigators can avoid some kinds of error.) Clearly, these researchers will have to be educated, will likely be middle class, and, since science flourishes in politically open societies, they will tend be concentrated in liberal countries. Nevertheless, additional non-Western researchers, even if they are educated and relatively wealthy, could be a boon to the behavioural sciences.
A direct and powerful way to remedy both sources of bias – too many WEIRD subjects and too few non-WEIRD researchers – is to foster research capacity in the non-Western world. Non-WEIRD researchers tend to study non-WEIRD subjects, so increasing their number will deepen the subject pool and widen the range of inputs to the scientific process at the same time. Building research capacity, however, should not merely involve collaborations led by WEIRD researchers; it should aim to generate studies led and initiated by non-Western researchers. Committed and long-term inter-institutional collaboration between Western and non-Western universities focused on remedying the deficits in the behavioural sciences literature should include internships at Western universities for non-Western researchers, stints at non-Western universities for WEIRD researchers, and extensive student exchange programmes (especially for graduate students). Unlike many existing scholarship and exchange programmes in the sciences, a key point of the necessary programmes should be for the learning to proceed in both directions.