R1.1. Haidt's “show of hands” demonstration is unconvincing

Seibt, Waldzus, Schubert, & Brito (Seibt et al.), referring to Jonathan Haidt's demonstration that few social psychologists would publicly self-identify as conservative at a major conference, suggest that “The evidence for the claims of under-representation is rather weak.” That would be true if our evidence were limited to Haidt's “show of hands” (target article, sect. 2, para. 3; see also Note 7). But in our target article we presented multiple forms of evidence, including a graph showing how partisan identity and ideological orientation have both been moving leftward, and a survey of the field that included self-reports of political identity and of willingness to discriminate (Inbar & Lammers Reference Inbar and Lammers2012). Those who think that conservatives are properly represented should offer some evidence, or at least tell us where the missing conservatives can be found.

R1.2. The American political system is unusual, in ways that render our claim invalid

Hilbig & Moshagen claim that our evidence that psychologists are likelier to identify as Democrats than Republicans (Gross & Simmons Reference Gross and Simmons2007; Rothman & Lichter Reference Rothman, Lichter, Maranto, Redding and Hess2008) is invalid, because, they argue, U.S. Democrats are actually moderates or centrists when their policy views are examined in relation to policy views in Europe and elsewhere.

Cross-national comparisons of extremism are notoriously difficult. (What criteria should we use in judging whether left-wing Democrats are to the right of the Tony Blair/Gordon Brown wing of the British Labor Party?) But even if we treat the claim as largely true, it does not alter our conclusion that social psychology leans left. Our conclusions do not hinge on a single source of data, as we present evidence from multiple sources that social psychologists are politically homogeneous (see Figure 1 of the target article). This includes party identification, ideological identification, and attitudes towards policy issues (see Inbar & Lammers Reference Inbar and Lammers2012). Neither Hilbig & Moshagen nor Seibt et al. present evidence showing that social psychologists are more aptly characterized as moderates or centrists. We also note that it is ever convenient to characterize one's own ideology as centrist or “moderate,” while casting the other side as the true ideologues. Moreover, our target article presented direct evidence and examples of biased research, which is ultimately the core issue. Hilbig & Moshagen did not address that evidence.

What would change our view? When research starts to report that large minorities (say, 20%) of social psychologists are members of moderate right or libertarian (classic liberal) parties (e.g., European Christian Democrats, American Republicans, or Coalition Australian parties), and that they identify as non-left, and support non-left policies, our view that social psychology lacks diversity will change. For now, such data do not exist.

We found Bilewicz, Cichocka, Górska, & Szabó's (Bilewicz et al.'s) data fascinating and a valuable step toward understanding the ideological characteristics of social psychologists outside of Western democracies. Their data from Eastern Europe suggest that social psychologists are not universally left-wing, at least on economic issues, especially in countries in which the Left is still tainted by its association with communism. But we find it noteworthy that, like U.S. social psychologists, the Western and Eastern European social psychologists in this sample report being “liberal” (leftist) in their positions on social issues – those most likely to inform and distort social psychological research. Indeed, these data largely mirror those of Inbar and Lammers (Reference Inbar and Lammers2012), who find that, whereas there is some variation among social psychologists on economic and foreign policy issues, there is little variation on social issues.

R3.1. Conservatives are just not interested in – or capable of – conducting social psychological science

Several commentators repeated the argument for self-selection of liberals into (and conservatives out of) social psychology that we addressed in our target article. Eagly notes that liberals are more attracted to the type of progressive social change highlighted by social psychology. Gelman & Gross highlight how people self-select into environments that promise ideological fit, using the military as a career more attractive to conservatives.Footnote ²

Hibbing et al. argue that we ignore fundamental differences between liberals and conservatives that explain conservatives' self-selection out of social psychology. We agree that there are important differences between liberals and conservatives, and indeed some of own research has shown fundamental differences in liberals' and conservatives' values and moral beliefs (e.g., Crawford Reference Crawford2012; Reference Crawford2014; Graham et al. Reference Graham, Haidt and Nosek2009). We included a section in our target article (sect. 5.3) describing the evidence that “differences in interest” were relevant, and likely account for some portion of the underrepresentation of non-liberals.

That said, we are unconvinced that these differences are robust or reliable enough to produce the types of career-determining decisions that Hibbing et al. suggest. One problem is that Hibbing et al.'s argument collapses social and economic conservatism, despite abundant evidence of the importance of this distinction (e.g., Feldman & Johnston Reference Feldman and Johnston2014; Malka & Soto Reference Malka and Soto2015). Indeed, much of the evidence in favor of Hibbing et al.'s thesis has relied on measures of social conservatism, but has been generalized to “conservatism” writ large. Second, meta-analytic treatments of behavioral measures of psychological rigidity (van Hiel et al. Reference Van Hiel, Onraet and De Pauw2010) have produced much less robust results than did the Jost et al. (Reference Jost, Glaser, Kruglanski and Sulloway2003) meta-analysis of self-report measures (for details, see Jussim et al., in press a). Finally, a recent set of studies showing ideological differences in avoiding dissonance-arousing situations (Nam et al. Reference Nam, Jost and Van Bavel2013) have proven difficult to replicate (Brandt & Crawford Reference Crawford2014; Crawford et al., in preparation b).

As we said in our target article, we agree that self-selection contributes to the political homogeneity of the field. This is, however, a chicken-and-egg problem: Are social psychological phenomena inherently more attractive to liberals? Or have pressures to conform to liberal norms influenced the questions that social psychologists pose – and the phenomena they discover? If there were more lines of inquiry attractive to conservatives, or at least more lines of inquiry that didn't have obvious liberal values embedded within them, would social psychology attract more conservatives? These are open questions, and one way to find out is to change how we conduct our work and frame our hypotheses. This is precisely the change we proposed in our target article.

Although we showed the implausibility of claims that cognitive differences between liberals and conservatives are sufficient to explain the massive ideological lopsidedness of the field, some commentators have endorsed such claims. For example, Beit-Hallahmi argues that conservatives are religious and conformist, and that religiosity and conformity are the antithesis of science. This exaggerated view of conservatism ignores the multi-dimensionality of political orientation (and it is consistent with research findings that partisans often exaggerate opponents' positions; see Jussim et al. [in press a] for a review). We reviewed evidence (Kemmelmeier Reference Kemmelmeier2008) in our target article that cognitive ability is negatively related to social conservatism (akin to the religiosity that Beit-Hallahmi mentions) but positively to economic conservatism. Beit-Hallahmi argues that the fact that liberal academics have risen to the top is evidence of their intellectual superiority. We would invite Beit-Hallahmi to consider the following turnabout test – would this argument be accepted as valid if it pertained to female or African American scientists' positions in STEM (i.e., science, technology, engineering, and mathematics) fields?

Relatedly, there seem to be several misconceptions surrounding just who we suggest join the ranks of social psychology. We acknowledge, as several commentators noted (Kessler, Proch, Hechler, & Nägler [Kessler et al.]; Wright), that we did not clearly define political diversity in our target article. First, we reiterate, we are not calling for Nazis, KKK members, terrorists, anti-Semites, racists, Creationists, or other political-religious extremists (as variously questioned or insinuated by, e.g., Beit-Hallahmi; Pfister & Böhm; Ross; Seibt et al.). In fact, we are not even necessarily calling for conservatives – instead, we clearly state throughout our target article (e.g., recommendations in section 6) that we are calling for non-liberals, which could certainly include conservatives, but also moderates, libertarians, apolitical people, and so forth. We agree with Funder that “reasonable conservatives” (implicit in Ross's mention of moderate, fiscal conservativesFootnote ³ ) would be quite welcome, but we cringe at the idea that a left-leaning field should get to judge what counts as “reasonable” conservatism. (We would also want to know whether applicants to graduate programs who are on the left would be limited to “reasonable liberals,” or whether applicants on the far left would still be admitted.) Further, Funder's call for “reasonable” conservatives implies that typical conservatives are unreasonable.

R3.2. Calling for increased political diversity is premature and not data-driven

Pfister & Böhm and Gelman & Gross are skeptical of the applicability to social psychology of the findings reviewed in our target article that increased diversity improves decision-making (Crano Reference Crano2012; Mannix & Neale Reference Mannix and Neale2005). Gelman & Gross acknowledge the leftward tilt of the field, but want more evidence that diversity improves science before calling for more diversity. (It is tempting to replace political diversity with gender diversity and see if the authors would feel the same way about the need for more research.) These authors suggest that we cannot apply the results from small-scale organizational diversity to large-scale diversity (such as in the field of social psychology). Yet the goal of much social science experimentation is to design small-scale studies to understand and generalize to larger populations. For example, should we not seek to apply lab-based evidence that anthropomorphizing environmental protection causes increases environmentalism (Ahn et al. Reference Ahn, Kim and Aggarwal2014) to inform large-scale environmental protection campaigns? Should we not seek to apply lab-based evidence that superordinate goals improve intergroup relations (Gaertner et al. Reference Gaertner, Mann, Murrell and Dovidio1989) to large-scale settings?

R3.3. Increasing political diversity will cause unanticipated problems

Several commentators raised concerns about epistemic costs that could result from political diversity. Ditto, Wojcik, Chen, Grady, & Ringel (Ditto et al.) offer several concerns, from a splintering of the field into ideological camps (also expressed by Kessler et al.) and the creation of new ideologically homogeneous journals, to a transformation from a field that produces more left-leaning conclusions to one that is muddled in moderation and an “equivalency bias.” Relatedly, several commentators (Pinker; Pfister & Böhm; Seibt et al.; Wright) express concern with a “cancelling out” approach by which conservatives were recruited to “cancel out” the biases of liberal social psychologists (especially if such practices led to quotas for conservative psychologists; Pinker).

Some clarification of our point is necessary here. We are not calling for a field in which there are equal numbers of right-wing extremists to “cancel out” the left-wing extremists – and we regret the use of this “cancel out” phrase in our target article (sect. 4.1, para. 4), which likely led to confusion. We would simply prefer a field in which there are enough non-liberals to provide checks and balances on the types of motivated reasoning that undercut the quality of theories, methods, statistical decision-making, and interpretation. Our desired end-state is not a field in which ideological battles rage, but rather one that is intellectually honest, tolerant, and dynamic. We find little fault with van der Vossen's vision of a depoliticized scientific field (although we disagree that political agnosticism among the members of our field is requiredFootnote ⁴ ), or with Pinker's vision of a field of politically disinterested inquiry (see also Tybur & Navarette; Winegard, Winegard, & Geary [Winegard et al.]). Indeed, Baumeister speaks from his own personal experience of trying to personally disengage from politics, and how it has made him more open to data, whatever its political implications.

Complete value neutrality may be an impossible ideal, yet it may still be worth quixotically striving for, because the price of abandoning the quest is our slippery-slope degeneration into an anything-goes, advocacy-driven pseudo-science, with a methodological façade of rigor (Tetlock Reference Tetlock1994). Political diversity is important not to provide equal time or to cancel out one side; it is important to ensure that politically popular notions presented as science are subjected to sufficient skeptical scrutiny to maximize the chance of “accepted science” actually being valid.

R3.4. Other forms of diversity are as (or more) important than political diversity

Motyl & Iyer did not challenge the importance of political diversity, but they did argue for the importance of other types of diversity, such as methodological or religious. We agree, but point out that more diverse religious perspectives would likely produce more diverse political perspectives, given the relationship between religiosity and political orientation, at least among the politically engaged (Malka et al. Reference Malka, Lelkes, Srivastava, Cohen and Miller2012). We acknowledge the validity of Seibt et al.'s argument that there are many sources of political diversity, besides conservatism, which are lacking from social psychology. To reiterate our target article's argument, we are interested in increasing the number of non-liberal voices in social psychology, including centrists, libertarians (classic liberals), the politically apathetic, or those whose political views do not easily fall out on a left/right spectrum.

Along with Ross and Seibt et al., Binning & Sears argue that gender and ethnic diversity are more important than political diversity (though they provide no justification for this claim). Pfister & Böhm and Pinker extend this argument by suggesting that political orientation should not be a protected type of diversity such as gender or ethnicity, because political orientation is not immutable. Comparative discrimination, beliefs about what attributes merit protection, and judgments of importance of various types of biases are all beyond the scope of our article or so subjective that we see no scientific/empirical way to address them. We leave readers to reach their own conclusions about these issues, hopefully informed by our article's points without being constrained by them.

Further, we were surprised by Pfister & Böhm's argument that the improvement of deliberation and outcomes is not a function or purpose of diversity efforts. Rather, they declare that the real purpose of diversity is justice and equality. However, Redding highlights that many legal arguments in favor of efforts to increase demographic diversity rely on the fact that such diversity provides differences of perspectives. We think the following thought experiment makes our point: Imagine that the last female Supreme Court justice is about to retire. Should the President go to great lengths to find a woman to nominate? If so, is it only for the sake of justice and equality, or do we also believe that diversity of perspectives and life experiences will improve the functioning of a deliberative body?

In sum, we agree that many forms of diversity are important, but we repeat that the epistemic benefits of diversity come more from viewpoint diversity than from demographic diversity (Menz Reference Menz2012; Williams & O'Reilly Reference Williams and O'Reilly1998). We have norms in our field that strongly encourage demographic diversity; yet, as we pointed out in our target article (and as Binning & Sears acknowledge), there are norms in our field that discourage political diversity, one of the most important forms of viewpoint diversity.

R3.5. Political diversity is not necessary for protecting the field from political bias

Several commentators recognize that political homogeneity can create biased assumptions and research, but question (or are at least agnostic about) whether increased diversity is the solution. For example, Brandt & Proulx provide several additional questionable interpretive practices (QIPs), which they cleverly rename “QTIPS” (for “questionable theoretical and interpretive practices”) that can harm theory construction and testing. Washburn, Morgan, & Skitka (Washburn et al.) suggest adversarial collaboration and additionally provide a helpful checklist that researchers can use to self-monitor against committing QIPs.

Binning & Sears argue that interdisciplinary collaboration can increase diversity of thought and improve social psychological science. Kessler et al. go one step further by arguing that such collaboration can increase the breadth and scope of social psychological theory. This is a point echoed by Tybur & Navarette, who suggest that evolutionary psychology, a discipline which has been tarred as “conservative” (see Seibt et al.) despite the left-wing personal beliefs of many of its practitioners, can be a model for social psychology. Specifically, they argue that a social psychology with a preponderance of theories (versus an overarching one, like natural selection) allows scholars to cherry-pick the theory most amenable to their values and expectations.

Along similar lines, Kessler et al. suggest encouraging ecological theory testing and exemplify this strategy with their own research demonstrating that when elicitors of disgust among liberals (e.g., environmental pollution) are included in study materials, the typical relationship between conservatism and disgust sensitivity can be reversed. Pinker likewise encourages scholars to focus on grander theory rather than “interesting” effects. Washburn et al. suggest that social psychologists take advantage of their preponderance of theories by designing studies to provide compelling and fair tests of competing hypotheses. Shweder argues that the ideals of Merton's epistemic community should be sufficient to address the problems we raised, and he fears that any sort of ideological screening of scholars (including mild forms of affirmative action such as greater outreach) might backfire and damage the intellectual climate of the field.

We agree with these commentators when they argue that political bias is not inevitable, and social psychologists may be capable of policing themselves and one another to avoid the biased blind spots and embedded assumptions we highlighted in our target article. Indeed, we noted many examples in our target article of ideologically balanced research that did not necessarily involve non-liberal social psychologists. Even if our article does not end up leading to any increase in the political diversity of the field, an increased awareness of these issues (and scholars' attempts to mitigate them) should help improve the science. At the same time, it is important to recognize the limits of people's abilities to recognize their own biases (e.g., Pronin et al. Reference Pronin, Lin and Ross2002).

That social psychologists can limit their political biases falls far short of ensuring that all will. As long as the field is so politically homogeneous, and as long as all of its members are not capable of preventing such biases, the field will have a tendency to produce findings that support left-wing values and narratives. Thus, we still think that increasing the number of non-liberals in social psychology will improve theory and research. But we also recognize that this is just one way to protect our science from political bias, and we thank the many commentators for their efforts to highlight the challenges in implementing our recommendation.

R5.1. A proposal for checking political biases in discussions of political bias

R5.2. Final (revised and augmented) recommendations

So, what do we recommend for researchers interested in engaging in good faith attempts to protect themselves and their field from political biases? We summarize our original recommendations and now add the most constructive ones based on the commentaries:

1. 1. Acknowledge the problem and raise awareness about it.

2. 2. Seek feedback from non-liberals.

3. 3. Expand organizational diversity statements to include politics.

4. 4. Add a statement to your own academic website acknowledging that you encourage collaboration among people of diverse political views.

5. 5. Eliminate pejorative terms referring to non-liberals; criticize others' scholarship when they use those terms. As an editor or reviewer, do not permit such terms to pass without comment.

6. 6. Avoid “leakage” of political hostilities or presumptions (including jokes) when functioning in any teaching or research capacity, but especially around students and junior colleagues.

7. 7. Encourage young scholars who are not liberals to pursue careers in social psychology.

8. 8. Be alert to double standards. Use turnabout tests to reveal bias.

9. 9. Support adversarial collaborations that encourage competing ideological camps to explore the boundary conditions on each other's claims, in joint data collection and model building efforts.

10. 10. Assign in classes, especially for graduate students, the growing scholarship taking social psychology and related disciplines to task for having a scientific problem stemming from political bias (Brandt et al. Reference Brandt, Reyna, Chambers, Crawford and Wetherell2014; Crawford Reference Crawford2012; Eagly Reference Eagly1995; Reference Eagly and Arkin2011; Inbar & Lammers Reference Inbar and Lammers2012; Jussim Reference Jussim2012a; Reference Jussim2012b; Jussim et al., in press a; Redding Reference Redding2001; Tetlock Reference Tetlock1994). Teach eliminating such biases as a core component of methods, validity, and scientific integrity.

11. 11. Use Washburn et al.'s checklist in one's own work, especially in politicized areas.

12. 12. Use Popperian falsification. If you are a liberal social psychologist, to guard against potential bias, seek to falsify rather than confirm your preferred prediction.