THE HOI POLLOI OF IR, AS CAPTURED BY TRIP

In reporting the results from the 2004, 2006, 2008 surveys, the TRIP project authors note that in the perception of the survey respondents, “the proportion of constructivist literature, which explores the social construction of reality and the role of norms and identity in international politics, has risen over time. . . . Respondents . . . in 2008 believed that it comprised 17% of the literature” (Maliniak et al. Reference Maliniak, Oakes, Peterson and Tierney2011, 443). Similarly, the analysis by TRIP staff of journal data, based on coding articles in 12 journals—APSR, AJPS, BJPS, EJIR, IO, IS, ISQ, JCR, JPR, SS and WP—as Realist, Liberal, Constructivist, Marxist, or Nonparadigmatic, also finds that the percentage of constructivist articles in top journals has risen from nearly zero in the 1980s to more than 10% by 2006.

Subsequent surveys, which include results from contexts other than the US, paint an even more favorable picture for Constructivism as a paradigm, and social constructivism as an ontological position (with epistemological implications). For example, in the 2012 surveys 22% of all global respondents choose constructivism as the main paradigm in which they work. With the exception of New Zealand, Norway, and Hong Kong, in all national surveys, Constructivism has the most popular support of all paradigms thus selected. Even in the US, where it shares top billing with Liberalism, Constructivism has 20% of the support of those surveyed (Maliniak, Peterson, and Tierney Reference Maliniak, Peterson and Tierney2012, 27). There are also reasons to believe that this number does not in fact capture all respondents who subscribe to a broadly constructivist understanding of the world. There are those who follow other approaches and paradigms besides Constructivism who assume that “the identity of agents and the reality of institutions are socially constructed” (Maliniak et al. Reference Maliniak, Oakes, Peterson and Tierney2011, 443, fn 17). Most strands of Feminism and English school, for example, also start with this assumption. These paradigms were selected by 4% and 2% of all global respondents in the 2012 survey (Maliniak et al. Reference Maliniak, Peterson and Tierney2012, 27). Furthermore, there may yet be others whose approaches fall under the “other” and “I do not use paradigmatic analysis” categories, 15% and 22% of global respondents respectively, that also subscribe to the basic assumption about the social construction of agent identities and the reality of institutions. ^{Footnote 2} This view is supported by the answers to question 20, where 47% of all respondents in the 2012 survey selected “My work does not assume the rationality of actors” in contrast to “I employ a rational choice framework” (7%) and “My work is broadly rationalist, but I do not employ a strictly rational choice framework” (46%; Maliniak et al. Reference Maliniak, Peterson and Tierney2012, 26). ^{Footnote 3} Answers to question 26 are also indicative of a similar trend, with 53% of global respondents declining to pick positivism as their epistemological position (32). In other words, while we cannot be certain due to reasons having to do with survey design, there are strong indicators that nearly half of all global respondents subscribe to broadly constructivist assumptions about agency.

Why should this matter? Because once we recognize this possibility, the mirage of parity between how the field is practiced and how it is represented in “top” journals evaporates. At first glance, all seems all is right with the field in this regard: as noted above, the 2011 ISQ article on TRIP reports that about 10–15% of the articles in top journals reflect the constructivist paradigm, a number which seems to be more or less in line with the self-reporting of respondents: around 20%. However, when we take into account that the TRIP assessment coded “an article as constructivist if its authors assume that the identity of agents and the reality of institutions are socially constructed” (Maliniak et al. Reference Maliniak, Oakes, Peterson and Tierney2011, 443, fn. 17), then we have to acknowledge that in the TRIP journal assessments, not only Constructivist articles but articles with any broadly social constructivist approach would have been coded as Constructivist by TRIP. In stark contrast to how other paradigms are defined, ^{Footnote 4} Constructivism is defined very minimally in the coding. This choice therefore both overcounts the actual number of published articles that adopt a proper Constructivist framework (as it has evolved within IR) and obscures how unrepresentative the top journals are of the field as it is practiced both in the US and globally. (I will deal with the question whether this is merited separately below.)

The TRIP assessment of journal articles also finds a great increase of articles that the authors code as nonparadigmatic, from about 30% in the 1980s to more than 60% in the 2000s. However, unlike in the respondents’ survey, we cannot assume that the nonparadigmatic articles category could capture any approach that broadly subscribes to social constructivist assumptions. If they did, presumably they would have been coded as Constructivist by the TRIP project. Thus, one could reasonably speculate that the “nonparadigmatic” articles continue the positivist tradition of realism and liberalism’s “neo” variants, and express similar assumptions about how the world works, but simply do not indicate very formal paradigmatic affiliations. This speculation is supported by TRIP’s own findings that “there is less methodological and virtually no epistemological diversity”; that “when we compare this publication pattern with our survey data, we see evidence of bias: the percentage of articles using quantitative methods is vastly disproportional to the actual number of scholars who identify statistical techniques as their primary methodology”; and that “when we look at the research that is published by the major journals, 90% of articles in 2006 were positivist, up from 58% in 1980” (Maliniak et al. Reference Maliniak, Oakes, Peterson and Tierney2011, 439). These patterns raise the troubling possibility that nearly half of the global field as it practiced is represented barely as a minority in the top journals of the discipline, whereas other half is represented as a dominant majority, thereby reproducing the myths of “mainstream” or “real” IR.

ARE OUR “TOP JOURNALS” REPRESENTATIVE OF THE FIELD?

To get a better sense of the articles in top journals coded in the TRIP database, I engaged in a simple counting exercise of my own focused on International Organization, which is considered to be the top journal of the subfield by a broad consensus. Following the TRIP model (and picking up from their 2007 cut-off point for the 2011 article), I collected data from the first and third issues of each year; however, I also tried to pay attention to whether an article was operating explicitly or implicitly in the IR constructivist paradigm ^{Footnote 5} (Constructivism) or simply allowed for the possibility of social construction of (aspects of) international relations (constructivism). Because in many cases these judgments are subjective, in liminal cases I biased the coding against my own argument: i.e., I chose the risk of overcounting openness to constructivist arguments over the possibility of undercounting. What I found is the following: only 5 (6%) out of the 82 articles in my dataset could be classified as operating in anything resembling the Constructivist paradigm. Another 15% could possibly be classified as allowing for or at least not entirely ruling out the possibility of social construction, so I coded them as being open to constructivist arguments; I coded the cases where this was only a possibility as 0.5. It should be noted, however, that only one article in this second category did not subscribe to either a rationalist or positivist epistemology (or both). To put it another way: if we treat constructivism not as a paradigm but as an ontological position with certain epistemological implications, only 7% of the articles in IO qualify (even when we include the thinnest versions of Constructivism), with the rest subscribing to rationalism or positivism (or both). Given that almost half of the discipline globally does not understand the field in the latter sense, as reported by TRIP, this should give us at least some pause.

Unfortunately, there is no reason to believe that IO is unique in this regard. While this remains to be verified, anecdotal sense suggests that the same exercise could be repeated for other journals in the TRIP database, and with the possible exception of EJIR, it is unlikely that the patterns revealed would be drastically different. If so, then the real question becomes how we should respond to these patterns as a discipline.

Figure 1

Articles in IO by Paradigm, 2008–2015

Figure 2

Articles in IO by Epistemology, 2008–2015

SHOULD OUR TOP JOURNALS BE REPRESENTATIVE OF THE FIELD?

Allowing for the possibility that the journal environment worsened for constructivism after TRIP took its snapshot of journals in 2011 does not change the overall picture: on the one hand, in the field we have the growing (or at least stable) global popularity of Constructivism as a paradigm (23% in the 2014 survey; 20% of US respondents in the 2012 survey) as well as anti-positivist approaches in general (52% in the 2014 survey), and on the other hand, we have their dwindling representation in top journals (likely less than 10%). ^{Footnote 6} The mismatch is stark enough to be a cause for concern. It resembles other patterns of stratification and exclusion that we consider problematic in academia, such as the pattern of growing numbers of women and minorities at the undergraduate or even graduate levels of education versus their near absence from professorial ranks.

As in the gender stratification example, there may of course be causes other than conscious bias that explain the situation. Furthermore, there is no evidence to suggest that the editors of IO or any other top journal are deliberately discriminating against Constructivism or other anti- or non-positivist approaches. ^{Footnote 7} However, we also know from sociology that when the pattern of stratification is this stark, all sorts of structural advantages accrue to the in-group: individuals may “voluntarily” take themselves out of the running, for example, by not submitting articles to top journals, just as women “voluntarily” drop out of the workforce. This may further perpetuate the status quo by making it more difficult for those who stay in: as reviewers and editors are usually drawn from the pool of authors who previously published with the journal, even with the best of intentions they may not “get” articles with alternative approaches, inadvertently creating a higher barrier to publication, reminiscent of the phenomenon of women needing to have better curricula vitae than men to get equal treatment (see Steinpreis, Anders, and Ritzke Reference Steinpreis, Anders and Ritzke1999). Lacking access to top journals also has other professional repercussions in terms of impact on citation factors (e.g., articles published in IO are more likely to be cited both in IO and other journals), not to mention on tenure and promotion.

Some may argue that perhaps those who work within the Constructivist paradigm or within an antipositivist vein in general publish less in top journals because they simply are not as research-active as those in other camps. Yet, in the 2014 TRIP survey, 90% of all respondents report spending more than 10% of their time on research, and 64% of respondents report spending more than 25%. The programs of field conferences such as ISA and EISA also clearly attest to the fact the “nonmainstream” 50% of the field is not research inactive. In any case, the issue of who gets to spend time on research versus teaching cannot be cleanly separated from the issue of who gets to publish in top journals. Even allowing that working with positivist methodologies allows for higher frequency of output, frequency and quality of output are not necessarily linked. In other words, the difference in frequency does not seem great enough to explain by itself the disparity between the-field-as-it-is-practiced and the-field-as-it-is-represented in the top journals.

Finally, there is the argument that what gets published in top journals is simply better research. Short of opening a whole can of philosophy of science worms, let me nevertheless note that as a discipline we do not have very good ways of deciding what is good research outside of its publication venue, so there is a real risk of tautology in that argument. It cannot be shown, for instance, that what gets published is more relevant to policy makers, because by most accounts the gap between the policy world and IR is growing. In the 2012 survey, 37% of global respondents (and 42% of US respondents) said the gap was growing, and 39% said the gap was stable; only 23% said the gap was shrinking, with 2% answering there was no gap (p. 66). Perception itself is not evidence of the gap but is strongly indicative. We especially do not have a way of comparing across paradigms or epistemologies. Ultimately however, as with other patterns of social stratification, the main problem is not who gets in, but who gets left out. This is a pity because with a slight adjustment of perspectives, a more inclusive type of excellence ^{Footnote 8} is possible without at all sacrificing principles of meritocracy.