We agree with the diagnosis of Duarte et al. that political and social psychological research is sometimes biased. However, we disagree with their proposed cure (e.g., affirmative action for conservatives). According to Duarte and colleagues such biases in research can be traced back to insufficient diversity in ideas, perspectives, and research agendas. To enhance this diversity, and thereby reduce biases, the authors propose to enhance the political diversity of research groups. We are skeptical about this recommendation for several reasons: First, we do not know how much diversity would be necessary to reduce these biases. Would it be enough to include liberals and conservatives? Or should communists, fascists, or even terrorists also be included? Second, which type of diversity would be most important for reducing biases in research? Alternatively to political diversity, one may think of ethnic, cultural, religious, or disciplinary diversity. Third, we doubt that diversity within research groups necessarily reduces biases. Diversity in research groups may foster influences of hidden profile (Lu et al. Reference Lu, Yuan and McLeod2012), where the focus on common information produces minimal consensus, which creates limited and biased research. Diversity in research groups may even lead to protracted conflicts and biased research – for example, the endless debates about qualitative or quantitative research or pro- or anti-Israel attitudes. Such examples show that not all diverse research groups are productive.
In order to develop alternatives to improve social psychological science, we suggest that a more systematic analysis of the underlying processes that lead to biased research would be necessary. In addition to biases produced by methodological weaknesses (Cumming Reference Cumming2014; Simmons et al. Reference Simmons, Nelson and Simonsohn2011), we think biases in research emerge mainly because of the following reasons: Researchers often focus only on certain stimuli examples but not systematically on the whole or, at least, a broad range of stimuli (e.g., triggering events for disgust; Proch & Kessler Reference Proch and Kessler2014). They actively disbelieve what they consider as morally wrong (e.g., positive effects of authoritarianism; Kessler & Cohrs Reference Kessler and Cohrs2008). In addition, they perceive some issues as “social problems,” whereas similar other issues disappear (e.g., targets of prejudice; Kessler & Mummendey Reference Kessler and Mummendey2001). Finally, they take for granted what others also state to be true, which prevents them from actually testing their truth.
Based on these considerations, we would like to focus on one simple but often neglected way to handle the problem of biased research. Instead of producing studies with “interesting” effects, theories and concepts should be tested more systematically. We feel that such interesting effects are more likely to be published than systematic tests of theories. This is not necessarily a political phenomenon, but may lead to biased research in all social psychological fields. Systematic testing of theories requires a thorough formulation of theories and concepts. Observations and phenomena should, therefore, be analyzed in psychological terms, which are then traced back to general social psychological processes. For instance, most intergroup research has demonstrated that members of all groups (even the most arbitrary) show some biases (Tajfel et al. Reference Tajfel, Billig, Bundy and Flament1971). Taking into account such basic social psychological research would make obvious that conservatives as well as liberals would be biased, albeit both may exhibit their biases on their group-specific topics. Moreover, thorough testing of theories should include more systematic observations (see Rozin Reference Rozin2001). In addition to the orthogonal experimental design, this would include systematic variation of stimuli and an ecological representative design.
We therefore suggest taking an ecological perspective along the lines proposed by Brunswik (Reference Brunswik1955; Dhami et al. Reference Dhami, Hertwig and Hoffrage2004) more seriously. Such an approach would lead to a greater variation of (among others) triggering events, group identities, and targets of deviance and prejudice that are studied. For instance, to test the hypothesis that conservatives are more disgust sensitive than liberals, it is necessary to vary the disgust-eliciting events systematically. Thus, research should focus not only on elicitors that trigger disgust in conservatives, such as homosexuality, but also on elicitors that may trigger disgust in liberals, such as environmental pollution or animal husbandry. In fact, our own recent studies show that either conservatives or liberals can be more disgust sensitive depending on the set of triggering events (Proch & Kessler Reference Proch and Kessler2014). With such results, representative sampling becomes an important tool for further research. With representative sampling, it becomes possible to examine the domination of certain stimuli in a particular environment, which could make either conservatives or liberals more disgust sensitive. Thus, generalizations beyond a particular environment may be invalid. We should follow Brunswik's ideas of varying the environmental stimuli in order to disentangle psychological processes from content. This would be possible only by varying the content of stimuli either systematically or according to the typical distribution in a certain environment. Only then could one generalize psychological tendencies (e.g., higher disgust sensitivity) towards a particular environment.
In addition to the self-conscious inclusion of an ecological perspective in testing and sampling, we would like to add that the scope of theory building and knowledge gathering should also be broadened. Fundamental phenomena should connect diverse fields within social psychology and psychology in general. Instead of working in isolated spheres and keeping citation circles, work groups should link more closely and challenge each other in constructive ways. Interestingly, the present target article seems to focus mainly on “American” social psychology and does not refer to a “European” tradition (or diverse others) such as basic intergroup research (e.g., Tajfel & Turner Reference Tajfel, Turner, Worchel and Austin1979). Furthermore, excellent research may also broaden its scope of literature to the history of psychology, to other social sciences (e.g., anthropology, philosophy), and to all relevant written and oral sources, which would include novels and simply talking to people with various perspectives. Finally, we may also learn to take questions, concerns, and critique of young students and researchers more seriously, as they do not think along habitual lines in the manner of more experienced researchers (Luchins Reference Luchins1951). Perceived from this angle, we recommend not worrying about the composition of work groups, but rather being concerned about the scope and breadth of the research focus and evidence.
We agree with the diagnosis of Duarte et al. that political and social psychological research is sometimes biased. However, we disagree with their proposed cure (e.g., affirmative action for conservatives). According to Duarte and colleagues such biases in research can be traced back to insufficient diversity in ideas, perspectives, and research agendas. To enhance this diversity, and thereby reduce biases, the authors propose to enhance the political diversity of research groups. We are skeptical about this recommendation for several reasons: First, we do not know how much diversity would be necessary to reduce these biases. Would it be enough to include liberals and conservatives? Or should communists, fascists, or even terrorists also be included? Second, which type of diversity would be most important for reducing biases in research? Alternatively to political diversity, one may think of ethnic, cultural, religious, or disciplinary diversity. Third, we doubt that diversity within research groups necessarily reduces biases. Diversity in research groups may foster influences of hidden profile (Lu et al. Reference Lu, Yuan and McLeod2012), where the focus on common information produces minimal consensus, which creates limited and biased research. Diversity in research groups may even lead to protracted conflicts and biased research – for example, the endless debates about qualitative or quantitative research or pro- or anti-Israel attitudes. Such examples show that not all diverse research groups are productive.
In order to develop alternatives to improve social psychological science, we suggest that a more systematic analysis of the underlying processes that lead to biased research would be necessary. In addition to biases produced by methodological weaknesses (Cumming Reference Cumming2014; Simmons et al. Reference Simmons, Nelson and Simonsohn2011), we think biases in research emerge mainly because of the following reasons: Researchers often focus only on certain stimuli examples but not systematically on the whole or, at least, a broad range of stimuli (e.g., triggering events for disgust; Proch & Kessler Reference Proch and Kessler2014). They actively disbelieve what they consider as morally wrong (e.g., positive effects of authoritarianism; Kessler & Cohrs Reference Kessler and Cohrs2008). In addition, they perceive some issues as “social problems,” whereas similar other issues disappear (e.g., targets of prejudice; Kessler & Mummendey Reference Kessler and Mummendey2001). Finally, they take for granted what others also state to be true, which prevents them from actually testing their truth.
Based on these considerations, we would like to focus on one simple but often neglected way to handle the problem of biased research. Instead of producing studies with “interesting” effects, theories and concepts should be tested more systematically. We feel that such interesting effects are more likely to be published than systematic tests of theories. This is not necessarily a political phenomenon, but may lead to biased research in all social psychological fields. Systematic testing of theories requires a thorough formulation of theories and concepts. Observations and phenomena should, therefore, be analyzed in psychological terms, which are then traced back to general social psychological processes. For instance, most intergroup research has demonstrated that members of all groups (even the most arbitrary) show some biases (Tajfel et al. Reference Tajfel, Billig, Bundy and Flament1971). Taking into account such basic social psychological research would make obvious that conservatives as well as liberals would be biased, albeit both may exhibit their biases on their group-specific topics. Moreover, thorough testing of theories should include more systematic observations (see Rozin Reference Rozin2001). In addition to the orthogonal experimental design, this would include systematic variation of stimuli and an ecological representative design.
We therefore suggest taking an ecological perspective along the lines proposed by Brunswik (Reference Brunswik1955; Dhami et al. Reference Dhami, Hertwig and Hoffrage2004) more seriously. Such an approach would lead to a greater variation of (among others) triggering events, group identities, and targets of deviance and prejudice that are studied. For instance, to test the hypothesis that conservatives are more disgust sensitive than liberals, it is necessary to vary the disgust-eliciting events systematically. Thus, research should focus not only on elicitors that trigger disgust in conservatives, such as homosexuality, but also on elicitors that may trigger disgust in liberals, such as environmental pollution or animal husbandry. In fact, our own recent studies show that either conservatives or liberals can be more disgust sensitive depending on the set of triggering events (Proch & Kessler Reference Proch and Kessler2014). With such results, representative sampling becomes an important tool for further research. With representative sampling, it becomes possible to examine the domination of certain stimuli in a particular environment, which could make either conservatives or liberals more disgust sensitive. Thus, generalizations beyond a particular environment may be invalid. We should follow Brunswik's ideas of varying the environmental stimuli in order to disentangle psychological processes from content. This would be possible only by varying the content of stimuli either systematically or according to the typical distribution in a certain environment. Only then could one generalize psychological tendencies (e.g., higher disgust sensitivity) towards a particular environment.
In addition to the self-conscious inclusion of an ecological perspective in testing and sampling, we would like to add that the scope of theory building and knowledge gathering should also be broadened. Fundamental phenomena should connect diverse fields within social psychology and psychology in general. Instead of working in isolated spheres and keeping citation circles, work groups should link more closely and challenge each other in constructive ways. Interestingly, the present target article seems to focus mainly on “American” social psychology and does not refer to a “European” tradition (or diverse others) such as basic intergroup research (e.g., Tajfel & Turner Reference Tajfel, Turner, Worchel and Austin1979). Furthermore, excellent research may also broaden its scope of literature to the history of psychology, to other social sciences (e.g., anthropology, philosophy), and to all relevant written and oral sources, which would include novels and simply talking to people with various perspectives. Finally, we may also learn to take questions, concerns, and critique of young students and researchers more seriously, as they do not think along habitual lines in the manner of more experienced researchers (Luchins Reference Luchins1951). Perceived from this angle, we recommend not worrying about the composition of work groups, but rather being concerned about the scope and breadth of the research focus and evidence.