OPEN ACCESS AND CITATION ADVANTAGE

Since at least the 1980s, subscription prices of US academic publications have been rising faster than the rate of inflation (Dingley Reference Dingley2005). As a result, academic institutions—particularly college and university libraries—have had to alter purchasing strategies for acquiring and/or renewing databases, journals, monographs (i.e., books), and other academic resources (Greco et al. Reference Greco, Wharton, Estelami and Jones2006). This situation, commonly referred to as the “serials crisis,” has had an impact across the spectrum of academic publishing (Greco et al. Reference Greco, Jones, Wharton and Estelami2007). In response to the crisis, some called for scholars to reconsider publishing with inflationary and overpriced journals (Parks Reference Parks2002) and for academic libraries to make access to them as economically efficient as possible (Pascarelli Reference Pascarelli1990). However, no alternative has shown as much traction as OA publishing.

For the purposes of this study, we use the classic definition of OA, in which an article “is available online to be read for free by anyone, anytime, anywhere—as long as they have Internet access” (Crawford Reference Crawford2011, 1). Despite the broad definition, we have narrowed the OA field by focusing solely on articles published in top political science journals. None of these journals is freely available in toto (which is referred to as “gold” OA). By definition, this restricts our study to green OA, which means articles that have been self-archived by the author and/or a sponsoring institution.

OA advocates argue that disciplines will benefit from free access to information (Calise and De Rosa Reference Calise and Rosa2008; Papin–Ramcharan and Dawe Reference Papin–Ramcharan and Dawe2006) and that the increased use of OA publishing will help to stem the serials crisis (Calise and De Rosa Reference Calise and Rosa2008; Papin–Ramcharan and Dawe Reference Papin–Ramcharan and Dawe2006; Harnad and Brody Reference Harnad and Brody2004; May 2005)In addition, advocates argue that OA will increase research efficacy as measured by citation counts and/or the citation impact factor. Despite the budgetary imperative to reduce costs and the potential citation advantage for authors, buy-in for OA publishing has largely been limited to disciplines in the physical and natural sciences as well as engineering and mathematics. Because these disciplines have been heavily invested in OA for many years, they have been the focus of the preponderance of research into the OACE.

Although researchers generally have found a positive correlation between OA and research impact, note that results are varied and the issue is not settled (Doty Reference Doty2013). For example, Lawrence (Reference Lawrence2001) found a citation advantage for OA articles in computer sciences, whereas Anderson et al. (Reference Anderson, Sack, Krauss and O'Keefe2001) found no citation advantage for OA articles in medicine. In subsequent research, Harnad and Brody (Reference Harnad and Brody2004) reported a citation advantage for OA articles in mathematics and physics, whereas Kurtz et al. (Reference Kurtz, Eichhorn, Accomazzi, Grant, Demleitner, Henneken and Murray2005) found no advantage in astrophysics. More recent research, however, has provided more support for the conventional wisdom that OA benefits citation counts. For example, a four-discipline study (i.e., economics, applied mathematics, ecology, and sociology) found that the OACE is positive across all four disciplines but that the degree varies among them (Norris et al. Reference Norris, Oppenheim and Rowland2008). Also, in recent unpublished research, McCabe and Snyder (Reference McCabe and Snyder2014) found a citation advantage for science journals with OA publishing models, although this increase was possibly a “superstar effect” because it was found primarily in the higher-tier journals.^{Footnote 4}

Despite our previous reference to a study that included economics and sociology (Norris et al. Reference Norris, Oppenheim and Rowland2008), only a few OACE studies have included the social sciences. This is commonly attributed to the low adoption of OA in these disciplines. However, recent scholarship indicates that the prevalence of OA publishing is increasing rapidly in the social sciences generally and political science specifically (Nentwich Reference Nentwich2008). Indeed, a recent study (Gargouri et al. Reference Gargouri, Larivière, Gingras, Carr and Harnad2012, 5) estimates that between 1998 and 2006, approximately 28% of social science articles were OA; that average increased to 36% when the authors studied the data from 2005 to 2010. In addition, the number of OA political science journals listed in the Directory of Open Access Journals increased by more than 50% between 2010 and 2013 (Bjørnshauge et al. Reference Bjørnshauge, Brage, Brage and Jørgensen2013; Calise et al. Reference Calise, Rosa and Marin2010).^{Footnote 5} Online self-archiving and archiving in institutional repositories complicate the efforts to quantify the volume of political science articles that are OA; the estimates vary from 5% (Hajjem et al. Reference Hajjem, Harnad and Gingras2005) to 30% (Antelman Reference Antelman2006). Despite this ambiguity, it is generally accepted that between the increasing number of OA journals and the more liberal self-archiving policies that many journals are adopting, the percentage of political science articles that are OA is increasing (Nentwich Reference Nentwich2008).

However, as in the natural sciences, the literature presents varied results on the efficacy of OA in different social science disciplines. For example, Antelman (2004, 375–76) found that whereas political science accounted for only 29% of the OA articles, those articles received the highest citation advantage in her study. Similarly, in a 10-discipline study that included political science, Hajjem, Harnad, and Gingras (Reference Hajjem, Harnad and Gingras2005) found that OA articles consistently have more citations than non-OA articles from the same journal and year. However, Evans and Reimer (Reference Evans and Reimer2009) found that the social sciences receive negligible benefit from OA. Most recently, Xia and Nakanishi (Reference Xia and Nakanishi2012) found that anthropologists receive a significant citation advantage from OA publishing independent of journal ranking (i.e., no superstar effect). Although mixed, the greater volume of positive findings in the social sciences leads us to our first hypothesis regarding OA in political science, as follows:

• H₁: OA articles will be cited at higher rates than articles that are TA only.

PUBLISHER SELF-ARCHIVING POLICIES AND OACE

As noted previously, the two primary approaches that academic publishers take to OA publishing are “gold OA” and “green OA.” Gold OA is provided by the journals, often at a cost to researchers or sponsoring institutions. Conversely, green OA is provided by either institutional and/or subject repositories or by individuals posting to their personal or academic websites (Suber Reference Suber2012, 53). Recently, gold OA was the subject of a symposium at the International Studies Association, the results of which were published in International Studies Perspectives (Gleditsch Reference Gleditsch2012). In the symposium, Mehlum (Reference Mehlum2012) makes a strong case for gold OA, citing concerns regarding equality of access for researchers in developing countries.^{Footnote 6} He also discusses economic and pricing concerns, noting that research could be treated as a public good.

However, a strong counterargument is articulated by both Thompson (Reference Thompson2012) and Gleditsch (Reference Gleditsch2012). Although neither author is opposed to OA publishing per se, both note a considerable economic downside for professional organizations and editorial offices. A professional association may receive as much as one third of its operating budget from publishers’ payments for exclusive publication rights to its journal (Thompson Reference Thompson2012). In addition, editorial offices are kept afloat by publisher subsidies and royalties; if association-sponsored journals were to transition to gold OA, Gleditsch (Reference Gleditsch2012) argued, those editorial offices and the quality-control service they provide would be endangered. Thus, gold OA poses serious concerns for associations and their flagship journals.

However, only a fraction of journals—approximately 5% in 2004—would fit the definition of gold OA (Harnad and Brody Reference Harnad and Brody2004). Indeed, only an estimated 2% of all OA articles published in 2011 were in gold OA journals; in the social sciences, the number decreased to less than 1% (Gargouri et al. Reference Gargouri, Larivière, Gingras, Carr and Harnad2012, 6). In contrast, an estimated 90% of journals were “green OA,” giving permission to authors in those journals to self-archive some version of their article (Harnad and Brody Reference Harnad and Brody2004). Within the green OA category, publishers have adopted a wide range of copyright and self-archiving policies. Among the larger publishers of political science journals (e.g., Wiley, Oxford, Cambridge, Elsevier, and Sage), the most common policies fall into either the permissive or the restrictive category. For our purposes, “permissive” publisher policies allow authors to self-archive any version of their paper, including the publisher’s PDF. “Restrictive” publisher policies allow authors to self-archive only the preprint (i.e., prior to peer review) version. When publishers explicitly allow self-archiving of the published version of an article, it is reasonable to expect that more authors and institutions will do so. This leads to our secondary hypothesis, as follows:

• H₂: Permissive copyright and self-archiving policies will lead to higher mean citation rates.

DATA AND METHODS

Data Collection

We began by examining the JCR data to determine which journals were consistently ranked (by impact factor) in the top 20 political science journals between 2007 and 2011. Only eight journals are ranked in the top 20 in all five years (table 1). We use the JCR rankings to select our data sample because impact factor is an indicator of researchers’ use of a specific journal (Antelman Reference Antelman2004). In addition, we used only journals that were highly ranked during a five-year period because the JCR impact-factor score can be unduly influenced by a few highly cited articles (Seglen Reference Seglen1997). As a result, each year’s top 20 rankings included journals that benefited from a high number of citations for a small number of articles rather than a high number of citations for the journal as a whole. Given these issues in the impact-factor calculation, a high impact factor in one JCR report is not a guarantee that a journal is regularly consulted in the discipline. However, high impact factors during a five-year period are a reasonable indicator that political scientists consistently cited articles from the eight journals at high rates.

Table 1

Annual Journal Citation Ranks, by Journal (2007–2011)

After comparing the 2007–2011 JCR reports to determine which journals were top-ranked in all five years, we gathered all of the articles published in them in 2007 and 2008. Then we used the Publish or Perish (PoP) software program (Harzing Reference Harzing2007) to query Google Scholar and retrieve the citation counts for each article in the dataset. We excluded book reviews, letters to the editor, conference programs and proceedings, presidential addresses, and membership meeting notes. We included replies to other authors because they are often cited in literature reviews. In the event that the PoP query returned duplicate citation-count records, the record with the lower count was excluded from the sample. We then used PoP’s built-in Google Scholar queries to determine whether each article in the sample is openly available, in any format, online. These formats, as defined in the Sherpa-RoMEO Publisher Copyright Policies & Self-Archiving Database, include the following:

• • Preprint: the version of the article before peer review

• • Postprint: the version of the article after peer review, with revisions made

• • Publisher’s postprint: the publisher’s PDF version (postpublication)

We then used Sherpa-RoMEO to determine whether the journal is subject to permissive or restrictive self-archiving policies.^{Footnote 8} The end result was a database of 727 observations, each representing a single article and its citation count.

Descriptive Statistics

Our results must be understood in the context of the data sample; therefore, it is important to include descriptive statistics. As noted previously, our sample size was 727 articles. Of those, 404 articles (55.5%) were OA in some form. This is surprising given that much of the extant literature indicates that the social sciences have not adopted OA publishing at high rates. Although it is true that no mainstream political science journals have converted to gold OA (Bjørnshauge et al. Reference Bjørnshauge, Brage, Brage and Jørgensen2013), the data presented here indicate that individual political scientists are making publications OA at fairly high rates. Overall OA frequency is reported in table 2a.

Table 2a

Open Access Frequency

Note: N = 727. Percentages may not total 100% due to rounding.

It is remarkable that whereas the majority (more than 75%) of OA articles in the sample were publisher PDFs, more than 45% of them were from restrictive publishers. This may indicate that authors are either ignorant of or indifferent to the provisions of their publisher copyright agreements.

It is remarkable that whereas the majority (more than 75%) of OA articles in the sample were publisher PDFs, more than 45% of them were from restrictive publishers. This may indicate that authors are either ignorant of or indifferent to the provisions of their publisher copyright agreements. However, it also may indicate that because citations of unpublished work are uncommon in political science, authors self-archive the publisher PDF in the hope of more citations. Furthermore, of the 410 articles published by permissive publishers, 48% were not self-archived. This indicates that political scientists are not taking full advantage of author-friendly copyright agreements. Table 2b shows OA frequency by publisher-policy classification. Table 2c presents the variation in OA frequency and publisher-policy classification by journal.

Table 2b

Open Access Frequency Rates and Publisher Policies

Notes: N = 727. Percentages may not total 100% due to rounding.

**P = Permissive; R = Restrictive.

Table 2c

Open Access Frequency Rates and Publisher Policies by Journal

Note: *Percentages rounded to the nearest whole number. ** P = Permissive; R = Restrictive.

The descriptive statistics provide prima facie evidence that OA articles are cited more frequently than non-OA articles (figure 1 and table 3). As shown in table 3, the mean citation rate in the sample size is 51. For OA articles, the mean citation rate is 70 and for TA articles, it is 28; thus, the number of citations is approximately two and a half times higher for OA articles in this sample. As shown in table 3, the mean citation rates for OA articles are higher not only across the full sample but also within samples for each journal. The table also shows wide variation in the mean citation rates between journals, from an average difference of only 18 citations between OA and TA articles in Public Opinion Quarterly to an average difference of more than 100 citations in Annual Review of Political Science.

Figure 1

Citation Rates by Open Access Status

Table 3

Comparison of Mean Citation Rates between Open Access and Toll Access Articles