DESIGN

A useful means of examining the topography of political science research is through a content analysis of articles in leading political science journals. Although such an exercise is not without its problems, the output of leading periodicals provides a broadly accurate guide to the character of the academic mainstream, and hence to the scope of its engagement with religious issues. The first, and most obvious, difficulty involved in undertaking such a review, given the absence of any objective measurement of publication quality and a corresponding lack of consensus on what the “leading” journals might actually be, is to determine the most appropriate sample. One way to do this is by using data provided by the ISI Web of Knowledge, which ranks periodicals according to various measurement indices. One of the most useful of these indices is a five-year impact factor based on citation reports.Footnote ¹ Although ranking periodicals in this way is not unproblematic (Hargens Reference Hargens2000),Footnote ² this method yields a good medium term representation of the leading journals within the discipline. It also avoids the potentially greater difficulties (such as lack of replicability and comparability with other disciplines) associated with constructing a sample using more subjective criteria (Hix Reference Hix2004; McLean et al. Reference McLean, Blais, Garand and Giles2009).

This study involved a content analysis of the 20 highest-ranked political science periodicals, based on the ISI's 2009 measurement of five-year impact factor, for 11 full years from the beginning of 2000 to the end of 2010 (for comparative purposes the same process was also completed for the 20 highest-ranked sociology periodicals).Footnote ³ To limit the analysis to substantive articles, those papers less than six pages in length—typically consisting of book reviews, short replies, rejoinders, research notes, and editorials—were, as a rule, excluded from the sample.Footnote ⁴ All main and original articles, along with other published materials such as review articles, commentaries, and replies of at least six pages in length, were included. After an analysis of the title, abstract, and keywords, these published materials were then coded according to the centrality of religious issues to their content.Footnote ⁵ Articles featuring religious issues as one of the principal or direct themes of analysis were coded as “primary” items. Articles dealing with religious issues either indirectly or as a minor part of the analysis, or for which religious variables featured as but one among a range of explanatory factors or discussion points, were coded as “secondary.”

For the most part this coding schema was relatively clear, but at times some subjective interpretation was required. Although infrequent, this occurred most often in cases, such as discussions of the Arab-Israeli conflict, when it was uncertain if the main theme of a article should be classified as dealing principally with “ethnic” or “religious” matters. When such issues were treated primarily in nonreligious terms, for example, used as a demonstration of social reconciliation processes, or of “ethno-nationalist” conflict, then the material was not coded as belonging to either of the primary or secondary classifications (examples include Horowitz 2002; and Ben-Yehuda and Sandler Reference Ben-Yehuda and Sandler2003). When the title, abstract, or keywords mentioned factors explicitly related to religion (such as “sectarianism” or “creationism”), however, or when these referred to direct religious markers (such as divisions between “Christians” and “Muslims”), or contained ambiguous markers (such as “antisemitism”) with no clear indication as to whether these were intended to denote “ethnic” or “religious” terms, then the material was coded as being religious according to the “primary” or “secondary” schema.

ANALYSIS

The sample drawn from the 20 highest-ranked political science journals produced a total of 7,245 published items. Of these, a total of 97 were coded as “primary” articles, being directly engaged with religious issues as a principal or leading theme of analysis. This resulted in a mean average of 1.34%, or one primary coded article for every 75. An additional 87 articles were coded as dealing with religion as a “secondary” issue. Thus, out of the entire sample, only 184 articles were classed as engaging with religion in any way, producing an average of 2.54%, or one article for every 39 articles.

The political science journal with the highest proportion of primary coded articles was Public Opinion Quarterly, with 12 articles from a total of 332 producing an average of 3.61%. The periodical with the highest total number of primary coded articles was the Journal of Politics, with 16, although this journal also published a greater overall number of articles than any other journal included in the sample, with a total of 708. Five journals—a quarter of the entire sample—did not contain any primary coded articles at all, and an additional six (making a total of 11 journals altogether) had an average of less than 1%. Taking all primary and secondary coded articles together, the journal with the highest sum total, and also the highest overall proportion, was the Journal of Peace Research, with 25 articles from a total of 437 producing an average of 5.72%. A total of five periodicals had an average of less than 1% for all primary and secondary coded items combined. These findings are set out in table 1.

Table 1

Political Science Journals Ranked by Average Number of Primary Coded Articles

How, then, are these results to be interpreted in light of claims that political science has overlooked religion? Certainly the overall averages of only 1.34% for primary coded articles and 2.54% for all articles relating to religion, are not impressive. But can these averages be regarded as being unusually low? A key problem in answering this question, of course, is that this deals with a relative measure, and raises this obvious, subsequent question: low in relation to what?

One way to answer this question is by comparing the number of political science articles concerning religion with those dealing with other subjects. Attempting to classify published materials in terms of their principal themes relies on a greater degree of subjective judgment and interpretation than a simple primary/secondary coding schema. Nevertheless, the results provide an illustrative guide to the topological distribution of key issues within the discipline. To get a sense of this (and because a detailed examination of every article included in the sample was beyond the scope of the study), the analysis focused on the publication record of three randomly selected periodicals (two drawn from the upper section, and one from the lower section of table 1).

The first of these journals was the American Political Science Review, ranked first according to the ISI's five-year impact factor, and sixth in terms of the average number of primary coded articles. This journal published 10 primary and two secondary coded articles during the 11-year period, the combined total of 12 giving an average of 2.61%, but published three times as many articles, a total of 37 (8.04%) on issues relating to political parties; 42 articles (9.13%) on issues relating to war, conflict, and terrorism; and 43 articles (9.35%) on political institutions. A total of 56 articles (12.17%) involved the theme of “democracy,” and 59 articles (12.83%) dealt with issues related to voting, almost five times the number published on religion. Notably, too, the journal published more articles (13) on the workings of the US Supreme Court than it did both primary and secondary coded items combined.

The content of the second journal to be studied, the International Studies Quarterly, ranked tenth by the ISI and seventh by the average number of primary coded articles, followed a similar theme. Containing six primary coded items (1.55%) and 13 articles on religion in total (3.37%), this journal published 31 pieces (8%) on issues relating to “human rights,” 39 articles (10.1%) relating to issues of “democracy,” and 51 articles (13.2%) on international institutions. By far the largest categories, however, were those relating to economic issues (including trade, foreign investment, and global finance), which accounted for no fewer than 80 articles (an average of 20.7%). The theme of war, conflict, and terrorism, which totalled 103 publications, an average of 26.7%, is almost eight times greater than the average for primary and secondary articles combined.

The final periodical analyzed was Governance. Ranked 14th in the ISI listings, Governance contained no primary coded articles for the sample timeframe (from a total of 259 publications), and produced only one secondary coded item. In contrast, the journal published 11 articles (4.2%) on environmental issues, 17 articles (6.6%) on the theme of policy transfer or learning, 24 pieces (9.3%) involving the European Union, and 28 articles (10.8%) on institutional change or reform. More prominently, the journal also published a total of 47 articles (18.1%) on institutions generally (excluding issues of institutional change), and 50 articles each (19.3%) dealing with the themes of policy change and bureaucracy or administration.

These results would seem to provide a clear demonstration of the extent to which religious issues have been sidelined relative to those considered to be more central to the discipline of political science. However, to obtain a more fine-grained picture, all 97 primary coded items were also categorized in two additional ways: first, according to the principal topics that these addressed, and second, by the disciplinary subfields with which these were engaged. In the first of these, grouping primary coded articles according to their subject matter revealed a high level of concentration. Of the 10 most popular themes, 21 articles (21.7%) were related to issues of violence, conflicts and terrorism; 20 articles (20.6%) were directly related to issues about Islam; and 17 articles (17.5%) were related to US politics and society. In contrast, the topic of women and gender featured ninth, with just four articles (an average of 4.1%), while life and death issues (such as abortion and euthanasia) came tenth, with just three articles (averaging 3.1%).

The classification of published materials according to their relevant subfield was based on the eight subdivisions of political science set by the UK's Economic and Social Research Council (ESRC) (2007): institutions, political theory, comparative politics, political behavior, political economy, public policy, international relations, and methodology. Although articles could be classified as belonging to more than one category, the distribution of primary items was also highly focused on a few areas. By far and away the largest of these areas was “political behavior” (a broad category including a range of topics, such as voting, public opinion, identity issues, and terrorism), which accounted for almost two-thirds (63.9%) of all primary coded papers. In second place, though far behind political behavior, was the subfield of “political institutions,” with which almost two-fifths (17.5%) of all primary coded items were engaged. Comparative politics and political theory (the only other subfields to involve more than 10% of articles) ranked third and fourth with 13.4% and 10.3% of primary coded articles respectively. The least populated category was “political economy,” with only one article. Details of these analyses are presented in tables 2 and 3.Footnote ⁶

Table 2

Primary Coded Articles by Main Themes

Table 3

Primary Coded Articles by Subfield

In comparison to the broad spread of issues that are addressed by high-ranking periodicals, the extent of political science engagement with religion would, on the basis of the above, indeed appear to be low. Compounding this, articles engaging with religious issues also tend to be concentrated both in particular subject areas and disciplinary subfields. How does political science stand in relation to other social scientific disciplines? Here, a useful benchmark for comparison is with sociology, the social science discipline most actively involved in the study of religion. For comparative purposes, the methodology outlined earlier was repeated for the 20 highest-ranked sociology journals, again involving a content analysis of titles, keywords, and abstracts from the beginning of 2000 to the end of 2010. This produced a total sample of 6,896 articles, of which 242 were coded as primary items, an average of 3.51%. The combined total for all primary and secondary coded articles was 339, an average of 4.92%. The average proportion of primary coded papers published by sociology periodicals, then, was more than two-and-a-half times that produced by political science journals. The average proportion of both primary and secondary coded items combined was almost two times higher. This gap is all the more notable when it is considered that sociology outlets specifically designed to advance the study of religion within the discipline, such as Review of Religious Research, Sociology of Religion, and Journal for the Scientific Study of Religion, are not included in the sample. Furthermore, no sociology journal included in the sample was without a single primary coded publication (compared to a total of five political science journals), and only four sociology periodicals had an average publication rate for primary coded items of less than 1% (compared to 11 in the political science sample). The results of the content analysis for sociology journals, and a direct comparison to political science periodicals, are shown in tables 4 and 5.

Table 4

Sociology Journals Ranked by Average Number of Primary Coded Articles

* The inclusion in the ISI rankings of the Cornell Hotel and Restaurant Administration Quarterly was considered to be anomalous, because this publication deals principally with issues of leisure and business management rather than sociological matters per se. The journal was thus excluded from the sample.

Table 5

Political Science and Sociology Journals Compared

A comparison to sociology is also revealing in other ways. If both political science and sociology samples are combined, then only one political science journal (Public Opinion Quarterly) acquires a place in the top 10 ranked by the average number of primary coded items. If the rankings are ordered in terms of the average number of all primary and secondary coded items combined, then only two political science journals appear in the top ten (Journal of Peace Research and Political Psychology). In neither case does any political science periodical penetrate the top six (see table 6).

Table 6

Political Science and Sociology Journals Combined

PROBLEMS

The preceding analysis reveals and underscores the degree to which leading political science journals have overlooked religious issues. However, before drawing any firm conclusions, several potential methodological difficulties need to be addressed. The first is research design: in examining titles, keywords, and abstracts, the research design may have been insufficiently robust to accurately capture the content of all of the articles included in the sample. Thus, the objection may be that a number of articles dealing with religious issues could have been overlooked, thereby giving a lower estimation of political science engagement with religion. Although the possibility cannot be ruled out, concerns about overlooking relevant articles are more likely to apply to articles dealing with religion as a secondary issue, given that articles engaging with religion as a primary theme of analysis, for the most part, have their religious content clearly highlighted. Moreover, given the large number of articles contained in the sample, it is unlikely that the number of any articles overlooked would have been sufficient to distort the results. This potential problem is also unlikely to have affected the comparison between political science and sociology periodicals given the equal likelihood of articles being overlooked in each sample.

Another possibility, however, is that the results of the study may be an artifact of the selected sample, and that periodicals outside the sample may contain a significantly greater volume of articles involving religion. Without a more comprehensive examination, this could be the case, although again, this situation is unlikely for two reasons. First is the extremely small number of specialist political science periodicals in this area outside the sample, with the only notable outlet, Politics and Religion, publishing its first issue in 2008 (another, less prominent, journal, also called Politics and Religion, based in the Centre for the Study of Religion and Religious Tolerance, began in 2007). Second, the spread of results in the political science sample shows no relationship between a journal's position in the ISI rankings and the number of papers published on religion. Indeed, half of the top 10 periodicals ranked according to the average number of primary coded items were from outside the top 10 when ranked according to the ISI's five-year impact factor (see table 1). Comparing the top and bottom of the sample ranked in this way also revealed no significant difference in the number of articles involving religion. Indeed, if anything, journals appearing in the top 10 of the ISI rankings were more likely, albeit only slightly, to have published articles involving religion than those in positions 11 through 20 (see table 7). Because journals with lower ISI rankings were no more likely to contain articles on religion than those rated more highly, there is no reason, on this basis, to suppose that journals outside the sample, and hence lower in the ISI rankings, will be any more likely to contain a higher proportion of articles on religion.

Table 7

Political Science Journals Compared by ISI Rankings

Another potential problem is that scholarly research on religion may have been shaped by the geographical distribution of the periodicals contained in the sample, and by the respective geographical differences within political science itself. One of the most notable divisions here concerns North American and European approaches to the discipline: the former tending to be dominated by positivist approaches and the latter tending to a more historicist, theoretical, and normative approach (Marsh and Savigny Reference Marsh and Savigny2004). As such (notwithstanding editorial relocations), perhaps journals based in the United States may have been more or less likely to publish research on religion than those from Europe; this may have consequently shaped or distorted the findings.Footnote ⁷ However, the figures show no appreciable differences in the proportion of articles involving religion based on this criteria. The US and European journals contained in the sample were split almost equally, at 11 and 9, and published a total of 3,569 and 3,465 papers, respectively. Of these, US journals published a total of 47 primary coded articles, producing an average of 1.24%. European journals published 50 primary coded articles, with an average of 1.44%. The figures for all primary and secondary coded articles were 88 for US periodicals and 96 for those based in Europe, giving averages of 2.33% and 2.77% respectively. Thus, European journals had a slightly higher publication average for papers relating to religion (both primary coded and overall), however the differences between the two are not significant, only 0.2% higher for primary coded articles, and 0.44% greater for all articles involving religion (see table 8).

Table 8

Political Science Journals Compared by Geographical Location

The North American journals (all US) were: American Journal of Political Science, American Political Science Review, Annual Review of Political Science, Comparative Political Studies, Governance, International Studies Quarterly, Journal of Conflict Resolution, Political Analysis, Political Psychology, Quarterly Journal of Political Science.and Studies in Comparative International Development. The European journals (all from England unless stated) were: British Journal of Political Science, European Journal of Political Research (Netherlands), European Union Politics, Journal of Peace Research (Norway), Journal of Politics, Political Communication, Political Geography, Public Opinion Quarterly, and Review of International Political Economy.

One final difficulty to consider is that the findings presented may not accurately reflect the chronological distribution of articles on religion. A possible problem with claiming that political science has generally ignored the issue, then, is that simply taking the overall average number of published articles over the 11-year period may conceal significant variation over time. This issue can be problematic if the number of articles engaging with religious topics has substantially increased recently. To capture the chronological distribution, the contents of the sample were thus examined across four periods: 2000–01, 2002–04, 2005–07, and 2008–10. The dividing line of the initial period was set at 2001 to register any potential “September 11 effect” on the pattern of publications.

The analysis of the chronological distribution of primary and secondary coded items shows no significant increase in the average number of articles engaged with religion, but it does indicate an increase after 2001. Although the number of publications involving religion consistently increased during the 11 years (from 27 in 2000–01 to 57 in 2008–10), and although the average proportion of primary coded items also rose, from 0.95% during 2000–01 to 1.5% during 2008–10, the overall proportion of articles did not substantially change. The average number of combined primary and secondary items pursued a fluctuating trajectory during the decade before ending in a position not substantially greater from that which it had started, recording an average of 2.58% for 2008–10 compared to 2.34% for 2000–01. It is interesting that the largest rise in the proportion of articles dealing with religion occurred in the immediate post-2001 period, with an increase in the average proportion of primary articles from 0.95% to 1.26%, and a rise in the average proportion of primary and secondary articles together from 2.34% to a peak of 2.69%. Although this increase suggests a possible result of September 11, any impetus given to the study of religion does not appear to have been sustained. While the proportion of primary articles continued to rise during the subsequent period, from 2005–07, the proportion of primary and secondary articles combined fell to 2.47%. A more detailed and longer-term study is needed to fully explore this point.

A further comparison with sociology on these issues is also revealing. As with political science, the number of sociology articles engaged with religion increased during each period, from 29 primary coded articles in 2000–01 to 80 during 2008–10, but, in contrast to political science, the overall proportion of sociology articles with religious content also grew consistently. The average number of primary coded items rose from 3.07% in 2000–01 to 4.03% in 2008–10, while the number of primary and secondary coded items combined rose from 4.31% in 2000–01 to 5.44% in 2008–10. This increase shows that the gap between political science and sociology also widened over the 11-year period (notwithstanding a brief narrowing during the early part of the decade), rising from a differential of 2.12% in 2000–01 to 2.53% in 2008–10 in the average number of primary coded items, and from 1.97% to 2.86% for primary and secondary coded items combined. These statistics are presented in tables 9 and 10, and figure 1.

Table 9

Political Science Articles Involving Religion over Time

Table 10

Sociology Articles Involving Religion over Time

* The columns headed “gap” denote the percentage differential between sociology and political science published articles.

Figure 1

Political Science and Sociology Articles Involving Religion Compared over Time (by Average Percentages)