Political Science Journals in Comparative Perspective: Evaluating Scholarly Journals in the United States, Canada, and the United Kingdom

James C. Garand; Micheal W. Giles; André Blais; Iain McLean

doi:10.1017/S1049096509990205

Political Science Journals in Comparative Perspective: Evaluating Scholarly Journals in the United States, Canada, and the United Kingdom

Published online by Cambridge University Press: 25 September 2009

André Blais and

James C. Garand: Affiliation:
Louisiana State University
Micheal W. Giles: Affiliation:
Emory University
André Blais: Affiliation:
Université de Montréal
Iain McLean: Affiliation:
Nuffield College, Oxford University

Article contents

Abstract
DATA AND METHODS
EMPIRICAL RESULTS
CONCLUSION
Footnotes
References

Rights & Permissions

Abstract

In this article we report the results from a new survey of political scientists regarding their evaluations of journals in the political science discipline. Unlike previous research that has focused on data from the United States, we conducted an Internet survey of political scientists in the United States, Canada, and the United Kingdom. We present data on journal evaluations, journal familiarity, and journal impact, both for our entire sample (N = 1,695) and separately for respondents from each of the three countries. We document the overall hierarchy of scholarly journals among political scientists, though we find important similarities and differences in how political scientists from these three countries evaluate the scholarly journals in the discipline. Our results suggest that there is a strong basis for cross-national integration in scholarly journal communication, though methodological differences among the three countries may be an impediment.

Information

Type: The Profession
Information: PS: Political Science & Politics , Volume 42 , Issue 4 , October 2009, pp. 695 - 717
DOI: https://doi.org/10.1017/S1049096509990205 [Opens in a new window]
Copyright: Copyright © The American Political Science Association 2009

Political scientists communicate through a wide variety of scholarly media, including books, journal articles, convention papers, and electronic papers. Among these, books and journal articles hold a preeminent place in scholarly communication networks, but it is also the case that not all book publishers and scholarly journals are created equal. Rather, there is a well-documented hierarchy among both book publishers (Goodson, Dillman, and Hira Reference Goodson, Dillman and Hira1999) and scholarly journals (Giles and Wright Reference Giles and Wright1975; Giles, Mizell, and Paterson Reference Giles, Mizell and Paterson1989; Garand Reference Garand1990, Reference Garand2005; Garand and Giles Reference Garand and Giles2003; Crewe and Norris Reference Crewe and Norris1991), with some scholarly outlets viewed as more significant than others. Some political scientists differentiate the contributions and importance of various scholarly journals using so-called objective criteria such as citation counts and impact factors (Christenson and Sigelman Reference Christenson and Sigelman1985; Hix Reference Hix2004; Giles and Garand Reference Giles and Garand2007), while others use subjective self-reports based on surveys of political scientists (Giles, Mizell, and Paterson Reference Giles, Mizell and Paterson1989; Garand Reference Garand1990; Garand and Giles Reference Garand and Giles2003). There is evidence that, properly conceived, subjective and objective indicators of journal impact are at least moderately highly correlated (Giles and Garand Reference Giles and Garand2007).

What do we know about how political scientists evaluate scholarly journals in the discipline? First, most of the work on subjective journal evaluations has been conducted in the United States, and there is a discernible pattern of journal rankings among American political scientists. Some journals are regularly awarded with elite status; these are journals that are familiar to high proportions of political scientists and receive high marks for the rigor and quality of research that they publish. Other journals are targeted to more specialized audiences (and hence are not broadly familiar to political scientists), are perceived as less rigorous in their scholarly standards, or both, and these journals typically receive less favorable evaluations from American political scientists.

Second, there is some systematic variation among American political scientists in how they evaluate scholarly journals. Political scientists differ by subfield specialization and methodological orientation in their familiarity with and evaluations of scholarly journals (Garand and Giles Reference Garand and Giles2003; Garand Reference Garand2005). However, these differences occur primarily in how scholars rank the elite disciplinary journals, with political scientists shuffling the order of highly ranked journals depending on their subfield and methodological approach. When one considers subfield and methodological differences in evaluations, familiarity, and impact for a broad set of political science journals, one finds that there are more similarities than differences. While scholars who concentrate on American politics may differ from those who study international relations, comparative politics, or political theory in the journals that they tout as being the leading journals in the discipline, there are similarities in evaluations across the broad spectrum of journals (Garand Reference Garand2005).

Perhaps the biggest shortcoming in this literature is its almost exclusive focus on journal evaluations by American political scientists. While we know a fair amount about how American political scientists evaluate scholarly journals, we know little about patterns of journal evaluations among political scientists in other countries. Simply, very little work has been done in terms of comparative, cross-national research on how political scientists evaluate scholarly journals. The most noteworthy exception is Crewe and Norris (Reference Crewe and Norris1991), who replicate the work of Garand (Reference Garand1990) and Giles, Mizell, and Paterson (Reference Giles, Mizell and Paterson1989) for a sample of British political scientists. They find considerable differences in how British and American political scientists evaluate political science journals. There are some journals that emerge on both British and American lists, but British political scientists evaluate several British and/or subfield journals very differently than do American political scientists. This work provides tantalizing evidence of country effects in how scholars evaluate the professional media in political science. Given the globalization of the discipline of political science, it is important to describe and understand how political scientists in different countries evaluate the scholarly media in which they communicate with other scholars.

In this article we build on previous research on subjective journal evaluations and respond to the need for cross-national comparison. We replicate previous work by Giles and Garand (Giles and Wright Reference Giles and Wright1975; Giles, Mizell, and Paterson Reference Giles, Mizell and Paterson1989; Garand Reference Garand1990, Reference Garand2005; Garand and Giles Reference Garand and Giles2003) by reporting results based on an online survey in which we ask political scientists to evaluate a wide range of political science journals. Hence our results are directly comparable to those reported in previous research by Giles and Garand and their collaborators. We use both political scientists' evaluations of scholarly journals and their familiarity with these journals to create a measure of “journal impact” that is comparable to those reported in previous studies. However, in addition to respondents from the United States, we include a sample of respondents from the United Kingdom and Canada, and this permits us to make explicit comparisons among political scientists from these three countries. The result is a broader picture of how political scientists evaluate the major journals through which they communicate the findings of their research.

DATA AND METHODS

The present study is based on responses to an Internet survey of political scientists conducted in the United States, Canada, and United Kingdom during 2007. We started with the questionnaire that was the basis for the work of Garand and Giles (Reference Garand and Giles2003) on journal evaluations among American political scientists. Garand and Giles asked political scientists about their evaluations of 115 political science journals, as well as a variety of demographic, career, and attitudinal variables. In the present survey we ask respondents to evaluate 92 political science journals, and we adapt other components of the Garand-Giles survey to reflect differences across the three countries. We use English versions of the survey in each country, and in Canada we provide respondents with the option of answering the questionnaire in French. Overall, these data permit us to provide ratings of political science journals on three important dimensions (i.e., evaluation, familiarity, and impact) and to explore the key determinants of these ratings.

Sampling Issues

Our population of interest is political scientists who are affiliated with political science departments housed in universities that grant the Ph.D. in political science. This stands in some contrast to the work of Garand and Giles, who included a large subsample of faculty from Ph.D.-granting departments but also a subsample of political scientists from departments that do not grant the Ph.D.

For the United States, we obtained a list from the American Political Science Association (APSA) of members affiliated with political science departments that offer the Ph.D. in political science. The total number of American political scientists on the list was 3,486; of these, 1,134 provided usable responses, for a response rate of 32.53%.

For Canada, where many political scientists are not members of the Canadian Political Science Association (CPSA), we could not rely on CPSA membership lists. Instead, we started with a list of all Ph.D.-granting political science departments in Canada, consulted the Web site for each department to compile an initial faculty list, and followed up by contacting all departments to see whether our list needed to be updated. Based on this procedure we generated a list of 607 names of faculty at Canadian Ph.D.-granting departments. Of these, we received usable responses from 196, for a response rate of 32.3%.

For the United Kingdom, the population of interest is defined by the annual Political Studies Association directory. From data supplied by heads of departments, the directory lists all academic staff in political science and cognate departments in the UK, whether or not they are PSA members. It also lists PSA (and British International Studies Association) members in institutions outside of political science departments. As a result, the Directory likely overstates the true population of political scientists in Ph.D.-granting departments, making difficult the calculation of a firm response rate. Based on the Directory we identified 1,943 political scientists in Ph.D.-granting departments, of whom 432 responded for a response rate of 22.2%. However, because the denominator likely includes a large number of non-political scientists who are unlikely to respond to our survey, this figure understates the actual response rate. By comparison, about 1,000 political scientists are entered by the universities as research-active political scientists in the 2008 Research Assessment Exercise, so the true unobservable population lies between 1,000 and 1,943. The 22.2% figure is hence a lower boundary on the response rate for the UK sample.

Our total sample consists of 1,695 respondents, with 64% from the United States, 11% from Canada, and 24% from the United Kingdom.

Survey Administration

The present expert survey is the fifth in a series initiated in 1975, with the most recent installment published in 2003 by the two U.S. authors of this article (Giles and Wright Reference Giles and Wright1975; Giles, Mizell, and Paterson Reference Giles, Mizell and Paterson1989; Garand Reference Garand1990; Garand and Giles Reference Garand and Giles2003). In these works Giles and Garand conducted surveys of political scientists in the U.S., with respondents asked to evaluate the quality of journals on a scale ranging from 0 (poor quality) to 10 (outstanding). Garand (Reference Garand1990) and Garand and Giles (Reference Garand and Giles2003) combined data on mean journal evaluations and the proportion of respondents who were familiar with each journal to create a measure of journal impact, measured as:

$\displaylines{ {\rm Journal Impact} = {\rm Journal Evaluation} \cr + ({\rm Journal Evaluation} * {\rm Journal Familiarity)} \cr}$

This measure has a theoretical range of 0 (poor mean evaluation and no journal familiarity) to 20 (perfect mean evaluation and perfect familiarity). The authors reason that the most important journals in political science are those that are both (1) highly regarded for the quality of the work that they publish and (2) highly visible to the broadest group of political scientists. By combining quality and familiarity measures into a single scale, Garand and Giles created an impact measure that has a high level of face validity and that is fairly highly correlated (r = 0.656) with citation-based measures of journal impact.Footnote ¹

Our survey was administered by the Public Policy Research Laboratory, an academic survey-research center located at Louisiana State University. Respondents from the United States, United Kingdom, and Canada were sent e-mails with a link to the survey, which was tailored to the language and academic customs of each country. After an initial period of receiving responses from our sample, we sent a second reminder e-mail to respondents.

The survey was divided into four sections. First, all respondents received in the e-mail solicitation a cover letter that included a brief description of the project, a confidentiality statement, and a statement relating to human-subjects review by the institutional review board (IRB) at Louisiana State University. Second, we included a series of questions designed to gather descriptive information, including country of origin, highest degree received, age, sex, academic rank, field and subfield interests, and methodological approaches. Third, we included a section with open-ended questions in which respondents could identify journals (1) to which respondents would submit “a very strong paper on a topic in your area of expertise,” and (2) that respondents “read regularly or otherwise rely on for the best research in your area of expertise.” This provides an alternative way of measuring respondents' evaluations of political science journals, as developed by Garand and Giles (Reference Garand and Giles2003). Finally, we included a section in which we asked respondents to evaluate “journals in terms of the general quality of the articles it publishes.” We used a scale from 0 (poor) to 10 (outstanding) and asked respondents to evaluate each of 92 journals with which they might be familiar. We also asked respondents specifically to indicate if they were familiar with each of these journals, as well as whether or not they have ever published an article in each journal.

Journal List

The selection of journals for the survey required a balance between (1) including sufficient journals to achieve adequate coverage across journal types, specialty areas, and countries of origin and (2) not including so many journals as to burden survey respondents and reduce the response rate below acceptable levels. This was a difficult task, because there are many more journals of interest to political scientists than there is room in a survey of manageable length. We approached this task in four steps. First, we included all of the journals in the Garand and Giles (Reference Garand and Giles2003) survey that were also classified as political science, public administration, or international studies by the Institute for Scientific Information (ISI). Second, given the emphasis of the Garand and Giles list on American journals, we included all of the journals included in Hix's (Reference Hix2004) UK-originated bibliometric study. This resulted in the inclusion of 84 journals: (1) 45 journals that were in the appropriate ISI categories and included in the studies of both Garand and Giles and Hix; (2) 19 journals that were only included in the Hix study; and (3) 20 journals that were not included in the Hix study but were in the appropriate ISI category and included in the Garand and Giles study. Third, we included all journals that were familiar to at least 20% of the respondents to the Garand and Giles study regardless of their ISI categorization. This step added only four journals to the list.Footnote ² Finally, each of the authors was allowed to nominate journals for inclusion but agreement of three of the four authors was required for a nominated journal to be added to the list. Only three journals were added to the list through this procedure.Footnote ³

This procedure resulted in the exclusion of several new journals that are already well recognized, as well as some other journals with strong reputations but that do not meet the criteria to which we agreed at the outset of the study. In some cases we received inquiries from journal editors or other interested scholars, asking why a given journal was excluded from our study. However, our goal was to establish clear criteria for selecting journals and follow those criteria with only very limited exceptions. We did give respondents the opportunity to list additional journals, both at the end of the closed-answer section of the survey and in the open-ended questions about journal submission and journal-reading preferences.

EMPIRICAL RESULTS

Consistency of Journal Impact, Evaluation, and Familiarity Measures over Time

As a starting point, we consider the degree to which there is stability in our measures of journal impact, evaluation, and familiarity from one survey to the next. Garand and Giles (Reference Garand and Giles2003) find a considerable amount of consistency in these three measures from the 1989 Giles, Mizell, and Paterson study (Garand Reference Garand1990) to the 2001 survey conducted by Garand and Giles (Reference Garand and Giles2003), and our findings are very similar. We examined the scatterplots and estimated correlation and regression coefficients for the relationships between 2001 and 2007 measures of journal impact, journal evaluations, and journal familiarity for the 69 journals that are included on both the 2001 and 2007 surveys. Since the 2001 survey was based on a sample drawn only from the United States, we include here only the U.S. respondents to our 2007 survey.

Based on these scatterplots—which are available in Online Appendix Figures 1–3Footnote ⁴—as well as the correlation and regression coefficients, it is clear that there is considerable stability over time in the relative positions of journals in their impact, evaluations, and familiarity. The relationship between 2001 and 2007 measures of journal impact is very strong (r = 0.950, b = 1.226, t = 25.00), suggesting that how political scientists assess the impact of scholarly journals in 2001 is closely aligned with how they evaluate the impact of scholarly journals in 2007. The same can be said for journal evaluations, where the correlation is slightly smaller but still very high (r = 0.890, b = 1.146, t = 16.00). There is also a very strong relationship between the 2001 and 2007 journal familiarity measures (r = 0.939, b = 1.081, t = 22.25). Simply, these results reveal that the highest (lowest) journals in terms of evaluation, familiarity, and overall impact in 2007 were also the highest (lowest) impact, evaluation, and familiarity journals in 2001.

Journal Impact in Three Countries

In Table 1 we report the relative journal impact scores and rankings for respondents from all three countries combined, as well as separately for respondents from the United States, United Kingdom, and Canada. The relative impact score is the impact score for each journal, divided by the impact score for the highest-ranked journal in each relevant comparison group. A more detailed summary of mean journal evaluations, journal familiarity, and journal impact for all respondents is found in Online Appendix 1, and separate detailed country-specific summary tables are found in Online Appendix 2.

Table 1.

Evaluations, Familiarity, and Impact Ratings for Political Scientists from the United States, United Kingdom, and Canada, 92 Selected Journals, 2007

[*] Denotes journals associated with one of the three national political science associations for the countries from which our sample was selected: American Political Science Association (USA), Canadian Political Science Association (Canada), and Political Studies Association (UK).

Several important findings emerge from these results. First, the elite journals among political scientists from all three countries combined are similar to those found in previous studies of political scientists in the United States. This no doubt reflects the fact that almost two-thirds of respondents are from the United States, and we have already documented the continuity in ratings for this group. Once again, the American Political Science Review, American Journal of Political Science, and Journal of Politics emerge as the top three journals in the discipline, with the British Journal of Political Science ranked a close fourth. These are general readership journals that are both highly regarded for the quality and rigor of the research that they publish and broadly familiar to a wide range of political scientists. The remainder of the top 10—including International Organization, World Politics, Comparative Politics, Comparative Political Studies, International Studies Quarterly, and the Journal of Conflict Resolution—comprises journals that represent the leading general journals in the fields of comparative politics and international relations. When one moves to the second tier, one finds journals of different stripes, including general journals (e.g., Perspectives on Politics, Political Research Quarterly, PS: Political Science and Politics), broad-based subfield journals (e.g., Foreign Affairs, Political Theory, International Security), and well-regarded specialty journals (e.g., Legislative Studies Quarterly, Electoral Studies).

Second, careful readers will note that there is a gradual drop-off in relative impact scores for journals across the distribution. While there seems to be a strong consensus that there are certain journals that are among the elite journals in political science, in many cases the journal impact scores of lower-ranked journals are not very far removed from the scores for high-ranked journals. For instance, 53 journals have relative impact scores that are at least 50% of the impact score for the American Political Science Review, the journal ranked first among the combined respondents from the three countries in this study. As Giles and Garand (Reference Giles and Garand2007) suggest, measures of journal impact based on citations tend to have a small number of elite journals accompanied by a rapid drop-off, with a small number of journals being cited a great deal and most journals with small citation counts. However, the more gradual drop-off in relative journal impact suggests that publications in many journals are viewed by political scientists in favorable terms, even if articles in those moderately ranked journals are not cited extensively by other scholars.

Third, there is considerable similarity in how political scientists in the United States, United Kingdom, and Canada perceive the impact of scholarly journals in the discipline, though there are also some noteworthy differences. Among the top 10 journals for all respondents combined, nine are found on the American top-10 list, seven are found on the UK list, and eight are found on the Canadian list. For American political scientists, Perspectives on Politics moves into the top 10, replacing International Studies Quarterly, which is still ranked eleventh. Among political scientists from the United Kingdom, three journals in their top 10—European Journal of Political Research (eighth), Political Studies (third), and the European Journal of International Relations (tenth)—are not found in the overall top 10; UK political scientists downgrade the rankings of the Journal of Politics (from third to thirteenth), International Studies Quarterly (from nineth to seventeenth), and the Journal of Conflict Resolution (from tenth to thirty-first). For Canadian political scientists, the International Studies Quarterly and Journal of Conflict Resolution are replaced in the top 10 by the European Journal of Political Research (ranked tenth by Canadian political scientists) and the Canadian Journal of Political Science (ranked seventh). Overall, there is a fair amount of agreement among political scientists in these three countries in terms of the journals that they place in the elite category.

One can observe the similarity of journal rankings for political scientists across country by examining the scatterplots of relative journal rankings for each pair of countries. In Figures 1–3 we present scatterplots of relative journal impact for the USA-Canada, USA-UK, and UK-Canada pairings, respectively. As one can readily see, there is a strong relationship among the journal impact measures for country pairings, with the correlations ranging from a low of 0.567 (for the United Kingdom and United States) to 0.787 (for the United Kingdom and Canada). There are some clear outliers in these scatterplots. The Canadian Journal of Political Science has a much lower impact score in the United States than in Canada (Figure 1), and Political Studies has a much lower impact score in the United States than in the United Kingdom (Figure 2). In comparing journal impact in the United Kingdom and Canada (Figure 3), we find that Canadian political scientists give much higher scores to the Canadian Journal of Political Science, Canadian Public Policy, and Canadian Public Administration, while political scientists from the United Kingdom give higher impact scores to Political Studies and Democratization than their Canadian counterparts. Despite these outliers, for the most part there is a reasonably strong relationship between journal impact scores among the three countries. In fact, the reliability for an additive scale based on the separate impact scores from these three countries is quite high (α = 0.869), and a principal components factor analysis of these three country-specific scores reveals a single factor (eigenvalue = 2.398, variance explained = 0.799), with all factor loadings in excess of 0.85. Hence we find considerable similarity in journal impact across these three countries, even when there is some different ordering of journals, particularly at the highest levels.

Figure 1.

Scatterplot for Relationship between USA Journal Impact and Canada Journal Impact

r = 0.737

Figure 2.

Scatterplot for Relationship between USA Journal Impact and UK Journal Impact

r = 0.563

Figure 3.

Scatterplot for Relationship between Canada Journal Impact and UK Journal Impact

r = 0.782

Journal Evaluations

We remind the reader here that the impact scores that we described in the foregoing section are based on two components: (1) journal evaluations, based on respondents' evaluations of 92 journals on a scale ranging from 0 (poor) to 10 (outstanding), and (2) journal familiarity, based on the proportion of respondents who indicate that they are familiar enough with a given journal to evaluate it. The cross-national differences and similarities in journal impact documented in the previous section can be decomposed into evaluation and familiarity components, so it is important to explore country-specific patterns on these two variables to consider the basis for cross-national differences in journal impact.

We begin with journal evaluations. In Table 2 we report the mean evaluation scores for the top 30 journals, and in Figure 4 we present a histogram and kernel density plot of the distribution of mean evaluation scores for all 92 journals. A complete table of mean evaluations for all 92 journals can be found in Online Appendix 3. Turning first to Figure 4, we find that the distribution of mean journal evaluations is skewed to the right, with a relatively small number of journals evaluated very highly (i.e., above a mean of 7.0), a small number of journals evaluated as below average (i.e., below 5.0), and most journals falling into the mid-range (between 5.0 and 7.0). Political scientists in these three countries have identified an elite set of journals that are evaluated particularly favorably for the quality of their contributions to the scholarly literature in political science.

Table 2.

Political Scientists' Subjective Evaluations, Top 30 Highest-Ranked Journals, All Countries Combined and Separately for Each Country, 2007

Note: Figures represent the mean evaluatin score on a scale from 0 (poor) to 10 (outstanding).

Figure 4.

Distribution of Evaluations for 92 Political Science Journals on 11-point Scale, All Respondents

Note: The curve overlaying the histogram is a kernel density function.

What journals are evaluated most favorably? In previous studies Garand (Reference Garand1990) and Garand and Giles (Reference Garand and Giles2003) have found that some journals that are familiar to only narrow subsets of respondents earn surprisingly strong mean evaluations from these small groups of readers, and the result is that using only mean evaluations overstates the discipline-wide impact of these journals. The results in Table 2 are closer to the conventional wisdom, though as before there are also some surprise results. First, the journals among the top 10 are generally those that one would expect to see in such a list, including the leading general journals—American Political Science Review (ranked first), American Journal of Political Science (third), Journal of Politics (fourth), and the British Journal of Political Science (sixth)—and the leading subfield journals—International Organization (second), World Politics (fifth), Political Theory (seventh), and Comparative Political Studies (nineth). But the list also includes some journals that are ranked lower in terms of overall impact, especially Philosophy and Public Affairs and Political Analysis; these journals are thought of highly by the relatively small proportion of respondents who are familiar enough to provide an evaluation, but they are somewhat less familiar to political scientists than some other journals on the list.

It is particularly interesting to note that the previous evaluation of the American Political Science Review reported by Garand and Giles (Reference Garand and Giles2003) was considerably lower than that reported in Table 2. In the 2002 survey, the APSR was ranked seventeenth in terms of its evaluation by American political scientists; for many political scientists this was a shockingly low mean evaluation for what is generally viewed as the flagship journal of the discipline. However, in these results based on our 2007 survey, the APSR is ranked first in terms of its mean evaluation. Perhaps the relatively lower ranking in 2002 reflected criticisms of the APSR that prevailed at the time among some political scientists, most notably the adherents to the Perestroika movement within the political science discipline (see, for example, Smith Reference Smith2002).

The remainder of the top 30 list includes a mix of journals that are both favorably evaluated and familiar to a large proportion of political scientists (e.g., International Studies Quarterly, Journal of Conflict Resolution), as well as journals that are highly evaluated for the quality of their published work but by relatively narrower readerships (e.g., History of Political Thought, Journal of Policy Analysis and Management, Political Behavior). For some journals, political scientists are both familiar with the journal and evaluate the quality of its contributions very favorably; these journals score highly on impact, evaluation, and familiarity measures. In other cases, a relatively small number of political scientists are familiar with a given journal, but the narrow readership evaluates the quality of scholarly contributions in favorable terms; here the impact of the journal on the broader political science profession is limited by the small number of political scientists who are familiar with the journal.

What about the effects of country on the evaluations of political science journals? Do political scientists from the United States, Canada, and the United Kingdom evaluate political science journals differently? In Table 3 we report the mean evaluations for each of the 25 political science journals, ranked by journal impact as reported in Table 1. The list provides a mix of general, subfield, specialty, and country-specific journals that earn high journal impact rankings. We report the overall mean evaluation for each of these journals, the mean evaluation for respondents from each of the three countries in our study, and the results of an analysis of variance (ANOVA) analysis of journal evaluations. Our ANOVA analysis has the 11-point journal evaluation scale as the dependent variable, with country serving as the independent variable.

Table 3.

Mean Evaluations of Selected Political Science Journals, by Country of Respondent, 2007

Note: Figures represent the mean evaluation score on a scale from 0 (poor) to 10 (outstanding). Figures in parentheses are th enumber of respondents used to calculate the mean evaluation score.

*** prob(F) < .01

** prob(F) < .05

* prob(F) < .10

We find a wide variety of country-specific effects, ranging from strong country effects to null effects. First, the three highest-ranked journals—the American Political Science Review, American Journal of Political Science, and Journal of Politics—are general-interest journals that are highly regarded by scholars from all three countries. However, there is a significant difference in how political scientists from these three countries evaluate these three journals. In each case, the mean evaluation is highest for political scientists from the United States, and the result is an F test signifying that the differences in means for these three countries are statistically significant. Even though these journals are highly regarded for political scientists from all three countries, all three journals are associated with national and regional political science associations from the United States, and this results in a significantly higher evaluation for American political scientists.

Second, there are several journals on our list that are associated with a specific country or that have a European identity, and political scientists typically differ in their evaluations of these journals. The European Journal of Political Science and Political Studies stand out in this regard. Political scientists from Canada and the United Kingdom evaluate both of these journals more highly than do American political scientists, and in both cases these differences are highly significant. Another journal that falls into this category but that is not on the top-25 list is the Canadian Journal of Political Science (ranked thirtieth in Table 1); in this case Canadian political scientists evaluate this journal significantly more favorably than political scientists from the United States and United Kingdom. On the other hand, the British Journal of Political Science is evaluated equally by political scientists from each of the three countries, so it is not inevitable that a country designation differentiates political scientists in their evaluations of a journal.

Third, there is a wide range of other journals for which there are significant differences for political scientists across the three countries. Several journals in the fields of comparative politics and international relations have evaluations that differ from one country to the next. International Organization, Comparative Politics, Comparative Political Studies, Foreign Affairs, and International Security all exhibit significant differences, with Canadian political scientists most likely to evaluate these journals favorably. Interestingly, Perspectives on Politics and PS: Political Science and Politics are evaluated more favorably by Canadian political scientists, exceeding the evaluation score given by political scientists from the United Kingdom and, most surprisingly, the United States. Political scientists from Canada and United Kingdom are more likely to evaluate the European Journal of Political Science, Electoral Studies, Political Studies, and Philosophy and Public Affairs more favorably than American political scientists; of these, only Political Studies is evaluated most favorably by political scientists from the United Kingdom.

We are struck by the journals for which systematic differences among political scientists from different countries are not observed. In addition to the British Journal of Political Science, the differences among political scientists are not statistically significant for World Politics, International Studies Quarterly, Journal of Conflict Resolution, Legislative Studies Quarterly, Political Theory, Political Analysis, Political Research Quarterly, Public Opinion Quarterly, and Journal of Theoretical Politics. The intellectual and scholarly appeal of these journals is broad based and similar for political scientists from all three countries.

Finally, how are journal evaluations among political scientists in one country related to journal evaluations among political scientists in other countries? We inspected the scatterplots of mean journal evaluations for the USA-Canada, USA-UK, and UK-Canada pairings, respectively (see Online Appendix Figures 4–6). As is the case for journal impact, there is a strong relationship among the journal evaluation measures for each country pairing. The correlations are all quite high, ranging from 0.692 to 0.815, and there are relatively few outliers in each of the scatterplots. American political scientists evaluate the American Political Science Review, American Journal of Political Science, and the Journal of Politics more highly than what would be predicted by journal evaluations by political scientists from Canada and the United Kingdom, but for the most part journal evaluations in one country can be predicted reasonably well by knowledge of journal evaluations in the other two countries. Here again, the reliability for an additive scale based on the separate journal evaluation scores is high (α = 0.888), and a single factor emerges from a principal components factor analysis of these three items (eigenvalue = 2.498, variance explained = 0.833). Across the full range of journals, we find a high degree of similarity in how political scientists from the United States, Canada, and the United Kingdom evaluate these 92 scholarly journals.

Journal Familiarity

In addition to respondents' evaluations of the quality of articles published in a given journal, the impact of that journal in the political science discipline is also a function of the degree to which political scientists are familiar with the research found in articles published with a journal. In Figure 5 we present the histogram and kernel density plot for the distribution of familiarity scores for all 92 journals in our study, and in Table 4 we report the proportion of respondents who report being familiar with the top 30 journals. A complete table of journal familiarity results for all 92 journals and for all three countries can be found in Online Appendix 4.

Figure 5.

Distribution of Levels of Familiarity for 92 Political Science Journals, All Respondents

Note: The curve overlaying the histogram is a kernel density function.

Table 4.

Political Scientists' Journal Familiarity, Top 30 Highest-Ranked Journals, All Countries Combined and Separately for Each Country, 2007

Note: Figures represent the proportion of respondents who are sufficiently familiar with a given journal to provide an evaluation.

The histogram for our journal-familiarity variable reveals that the distribution is heavily skewed to the right, with only a few journals exhibiting familiarity levels in excess of 0.50. The lion's share of journals is familiar to between 20% and 50% of respondents. It appears that a relatively small number of journals have the potential to make a discipline-wide impact, with most journals having a potential impact on selected subgroups of political scientists based on subfield, methodological approach, or other criteria.

Among the 30 journals with which the respondents are most familiar, we find that there are only 15 journals for which a majority of respondents indicate familiarity. The American Political Science Review is the journal with the highest level of familiarity, with 93.4% of respondents indicating that they are familiar with the flagship journal of the discipline. Four other journals are familiar to at least 75% of political scientists—the American Journal of Political Science (87.5%), British Journal of Political Science (81.6%), Journal of Politics (79.1%), and PS: Political Science and Politics (76.7%). These are the journals with the potential to have the strongest overall impact on the discipline; in fact, four of these five journals are also among the most favorably evaluated journals, so this helps to explain their strong overall impact (Table 1). Among the journals familiar to over 50% of respondents, there is a mix of general interest journals (e.g., Perspectives on Politics, Political Research Quarterly, Political Science Quarterly) and major subfield journals (e.g., Comparative Politics, International Organization, Comparative Political Studies). Finally, the remainder of the list comprises journals familiar to over 40% of respondents, including specialty journals (e.g., Legislative Studies Quarterly, Electoral Studies, Public Choice), a few general interest journals (e.g., Polity, Social Science Quarterly), and journals with an audience targeted to a specific country (e.g., Canadian Journal of Political Science, Political Studies). Beyond these journals, most journals are familiar to less than 40% of political scientists in these three countries.

How do levels of journal familiarity vary for political scientists across the three countries in our study? In Table 5 we report the proportion familiar for each of the top 25 high-impact political science journals (as reported in Table 1), both in total and broken down by country. Furthermore, we include in Table 5 the likelihood ratio (LR) χ² test of the cross-national difference in the proportion of respondents who are familiar with each journal.

Table 5.

Familiarity with Selected Political Science Journals, by Country of Respondent, 2007

Note: Figures represent the proportion of respondents who are sufficient familiar with a journal to provide an evaluation. Figures in parentheses are the number of respondents used to calculate the mean evaluation score.

*** prob(F) < .01

** prob(F) < .05

* prob(F) < .10

These results reveal considerable differences in journal familiarity for political scientists from the United States, United Kingdom, and Canada. Of the 25 top-impact journals, 19 exhibit a significant difference (at the .05 level) in familiarity levels across countries. This includes the “big three” journals (i.e., American Political Science Review, American Journal of Political Science, and the Journal of Politics), other general journals (e.g., Perspectives on Politics, Political Research Quarterly), various high-impact subfield journals (e.g., International Organization, International Studies Quarterly), and specialty journals (e.g., Legislative Studies Quarterly, Political Analysis). The only high-impact journals that are equally familiar to political scientists from all three countries are the British Journal of Political Science, Comparative Politics, Comparative Political Studies, Political Theory, International Security, and Philosophy and Public Affairs. Three of these journals are explicitly cross-national in focus, and two are in the field of political theory and philosophy.

It is noteworthy that among the 19 journals for which there is a significant difference in familiarity for political scientists from different countries, the level of familiarity is highest among American political scientists for 16 of these journals. The only exceptions are the European Journal of Political Science and Political Studies (both of which are more familiar to political scientists from the UK and Canada) and Politics and Society (which is more familiar to Canadian political scientists). This may partially be explained by a general USA effect, whereby the overall level of familiarity is higher among American political scientists than those from other countries. Across all 92 journals, the mean level of familiarity is 34.6%, with political scientists from the United States showing the highest familiarity (35.8%), followed by those from Canada (34.4%) and the United Kingdom (32.1%). Even among the top 25 journals, the overall mean familiarity is 47.5%, with political scientists from the United States (49.9%) exhibiting higher familiarity than those from Canada (44.5%) and the United Kingdom (43.2%). This USA effect is likely due to the fact that a large proportion of high-impact journals originates or is published in the United States.

We also consider the interrelationships for journal-familiarity levels among political scientists from the United States, Canada, and the United Kingdom. We examined the scatterplots for the relationships between journal familiarity levels for country pairs (see Online Appendix Figures 7–9), and it is here that we begin to observe a slight deterioration in the strength of the relationships between countries. The correlations for journal familiarity are lower than those for journal impact and journal evaluations, ranging from a low of 0.453 (for the USA-UK relationship) to a high of 0.734 (for the United Kingdom and Canada). Clearly, political scientists from the United States and the United Kingdom do not share the same levels of familiarity for many scholarly journals, while political scientists from the United Kingdom and Canada tend to exhibit similar levels of journal familiarity.

Finally, even though the cross-national interrelationships involving journal familiarity are not as strong as those involving journal evaluations and journal impact, the similarity in journal familiarity is strong enough that the familiarity variables for the United States, Canada, and the United Kingdom fit together well into a scale. The reliability for an additive scale based on the separate journal-familiarity scores is lower than that for journal impact and evaluation measures, but is still quite high (α = 0.823). Moreover, a single factor emerges from a principal components factor analysis of these three items (eigenvalue = 2.235, variance explained = 0.745). Although the interrelationships between journal familiarity levels for the United States, Canada, and the United Kingdom are not as strong as those observed for journal impact and journal evaluation, there remains a fairly high level of similarity in the extent to which political scientists from these three countries are familiar with scholarly journals.

The Relationship between Journal Evaluation and Journal Familiarity

How closely are journal evaluation and journal familiarity related to one another? Are journals that receive a favorable evaluation from political scientists also the most familiar to political scientists? In Figure 6 we present a scatterplot of the relationship between these two variables for our full sample. The relationship is moderately strong for the combined sample (r = 0.553), and the correlations calculated separately for political scientists from each country range from 0.376 (for Canada) to 0.624 (for the United States). (See Online Appendix Figures 10–12.) Overall, it appears that the journals that are most favorably evaluated are typically the ones that are most familiar to political scientists in the United States, Canada, and the United Kingdom.

Figure 6.

Scatterplot for Relationship between Journal Evaluation and Journal Familiarity

r = 0.554

There are some important outliers, however. Some journals are well known to political scientists but have only moderate evaluations; PS: Political Science and Politics, Foreign Affairs, Perspectives on Politics, Presidential Studies Quarterly, and Political Science are journals with mean evaluations that are below what one would predict based on their level of familiarity to political scientists. These journals earn higher impact scores based on their widespread familiarity among political scientists, rather than on political scientists' evaluations of the importance and quality of scholarly work that they publish. On the other hand, some journals are familiar only to moderate numbers of political scientists but have mean evaluations that are above what would be predicted by their level of familiarity; these journals include Political Analysis, Political Theory, History of Political Thought, and Philosophy and Public Affairs. Journals in this category are known to more specialized audiences who give them high marks for the quality of the work that they publish.

Preferred Journal Submissions

In addition to closed-ended questions, Garand and Giles (Reference Garand and Giles2003) report the results from open-ended questions in which they ask respondents to which journal they would submit a “very strong paper” that they had written in their field of expertise. This question is designed to give respondents an alternative way of thinking about journal evaluations by eliciting from them information about how they prioritize the journals in which they would like to see their own research published. We include a similar item in our 2007 survey, and we gave respondents the opportunity to indicate up to three journals to which they would like to submit their best work. In Table 6 we report the number of first, second, and third references, as well as the number of total mentions. We also report the number of “ratings points” based on three points for a first reference, two points for a second reference, and one point for a third reference.

Table 6.

Political Scientists' Preferences for Journal Submissions of High-Quality Manuscript, All Countries Combined, 2007

Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences for submission of a high-quality manuscript. Rating points is based on giving three points for a first rating, two points for a second rating, and one point for a third rating. Relative rating is the rating points for each journal as a share of the rating points for the American Political Science Review.

As one can see from Table 6, there is again a clear-cut ranking of political science journals, with many of the highly ranked journals on this list also found among the elite journals from the close-ended items reported in Table 1. The top five journals to which political scientists prefer to send their best work are the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics, followed by the International Studies Quarterly, British Journal of Political Science, Comparative Politics, Comparative Political Studies, and Political Theory. Not surprisingly, the journals to which political scientists prefer to send their best work are also the journals that earn high impact ratings in the close-ended items. When political scientists are thinking about where to send their best work, they consider the leading general journals and broad-based subfield journals.

Two other aspects of this table warrant attention. First, we point out that political scientists vary considerably in how they prioritize political science journals as possible outlets of their work. Respondents mentioned 201 different journals as their first choice to submit their best work, with 277 different journals mentioned as their second choice and 323 different journals mentioned third. Many of these journals are mentioned only once or twice, but the fact remains that political scientists see a wide variety of possible outlets for their best research papers.

Second, we note the dominant position of the American Political Science Review, American Journal of Political Science, and (to a lesser extent) the Journal of Politics on this list. The second-place journal, the American Journal of Political Science, has only 54.1% of the rating points as the American Political Science Review, and the Journal of Politics has only 34.7% of the rating value of the APSR. The journals International Organization and World Politics have 22.3% and 21.0% of the rating value of the APSR, but no other journal approaches 20% of the rating points of the APSR. When political scientists think about the highest-priority outlets for their best work, the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics dominate the list. In fact, these five journals represent 55% (886 of 1,597) of the first-choice journals to which scholars prefer to send their high-quality work, and the top 20 journals represent 75.5% (1,205 of 1,597) of the most preferred outlets for their best work. While political scientists cite quite a few different journals as journals to which they prefer to submit high-quality manuscripts, the fact is that there are a small number of journals that are cited by a high percentage of political scientists.

Political scientists from our three countries exhibit some interesting differences in terms of the journals to which they prefer to send their best work. In Table 7 we report the top five journal-submission preferences for political scientists from the United States, United Kingdom, and Canada. As one can see, the ranking for political scientists from the United States closely tracks the overall impact ranking presented in Table 1. American political scientists prefer to send their best work to the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics; the only journal missing from those with the highest impact (Table 1) is the British Journal of Political Science. For political scientists from the United Kingdom and Canada, there are some differences. Canadian political scientists drop from their top five the Journal of Politics, add the Canadian Journal of Political Science, and shuffle the order for other journals, though the American Political Science Review is also the journal to which Canadian political scientists would prefer to send their best work. Political scientists from the United Kingdom are the outliers here. The American Political Science Review and International Organization are ranked third and fourth as preferred scholarly outlets. However, Political Studies and the British Journal of Political Science earn the top two positions, and the Review of International Studies captures the fifth position. Overall, it appears that political scientists from Canada and the United States have similar preferences about where they should send their best work, but political scientists from the United Kingdom have less overlap with their colleagues from across the Atlantic.

Table 7.

Political Scientists' Preferences for Journal Submissions of High-Quality Manuscript, Top Five by Country, 2007

Preferred Reading Sources

An alternative way of determining political scientists' journal priorities is to ask respondents to identify the journals that they “read regularly or otherwise rely on for the best research” in their areas of scholarly interest. We included a question in our 2007 survey to capture these journal-reading priorities. In Table 8 we report the results for this item, coding up to five responses for each survey respondent. We also report the number of total mentions, as well as the number of rating points based on five points for a first reference, four points for a second, reference, and so on.

Table 8.

Political Scientists' Preferences for Journal Reading, All Countries Combined, 2007

Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences as one that they “read regularly or otherwise rely on for the best research” in their areas. “Rating points” is based on giving five points for a first rating, four points for a second rating, three points for a third rating, and so on. “Relative rating” is the rating points for each journal as a share of the rating points for the American Political Science Review.

Here again, we see a fairly clear pattern of journal ratings. The ranking of the first five journals on this list—the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics—is identical to the ranking for the journal-submission preferences, and the second five journals are the same, albeit with a slightly different ranking. There is clearly a strong similarity in the results for journal-submission and journal-reading preferences. Indeed, for the 64 journals with at least five references on each of the journal-submission and journal-reading items, the correlation is almost perfect (r = 0.985).

Once again we find that political scientists cite a large number of journals for their first, second, third, fourth, or fifth preferences for journal reading. Political scientists rely on many different journals for the best work in their fields of study. However, many of these journals are cited by only one or a few political scientists. As is the case for journal-submission preferences, there is a small group of journals that dominate the list of journal-reading preferences, though there is a wider distribution of journals that political scientists read than to which they submit their work. We find that the top five journals make up 41.7% (662 of 1,587) of the first preferences, and the top 20 journals make up 62.0% (984 of 1,587) of the first preferences. There is also a bit smaller drop-off as one moves from the American Political Science Review to lower-ranked journals. The American Journal of Political Science has 68.1% of the rating value of the APSR, and the Journal of Politics has 47.5% of the APSR rating value. Other journals among the top 20 journals have a journal-reading rating value that is higher than the journal-submission rating value reported in Table 6. The bottom line is that political scientists spread their journal-reading preferences around just a bit more than their journal-submission preferences, though there is a discernible rank order of journals in terms of political scientists' reading preferences. Political scientists are somewhat more “elitist” in terms of where they would submit their best manuscripts than in where they read for the best work in their fields of study.

Finally, in Table 9 we report the top five journal-reading preferences for political scientists from the United States, United Kingdom, and Canada, and once again we find that political scientists differ across country in the journal-reading preferences. The ranking for scholars from the United States is nearly identical for journal-submission and journal-reading preferences; the only difference is that World Politics and International Organization switch positions in Table 9. Canadian political scientists retain the same top five journals but shuffle the positions of the top three; the Canadian Journal of Political Science is the most read journal by Canadian political scientists, followed by the American Political Science Review and International Organization. Finally, for political scientists from the United Kingdom, the Journal of Common Market Studies replaces the International Organization in the fourth position, with the other four journals retaining their lofty rankings. Here again, political scientists from the United Kingdom are outliers, sharing only two of their five top journals in terms of reading preferences with their American and Canadian counterparts.

Table 9.

Political Scientists' Preferences for Journal Reading, Top Five by Country, 2007

A Multivariate Analysis of Journal Evaluations

What explains variation in individuals' evaluations of political science journals? One of the questions left unresolved in this analysis is whether country differences (or similarities) are really due to the effects of other variables. For instance, Blais et al. (Reference Blais, McLean, Garand and Giles2008) find that there are systematic differences among political scientists from different countries in terms of the methodological approach that they adopt for their research and professional work. Garand and Giles (Reference Garand and Giles2003) find that methodological approach is related to the evaluations that political scientists give to some journals, particularly those that adopt a clear methodological identity. It is possible that country differences in journal evaluations may actually reflect the distribution of methodological preferences among political scientists from those countries. Other confounding effects may be possible as well, so it is important to explore the influence of country on journal evaluations, controlling for the effects of other variables.

We developed and tested a regression model for a small, select group of political science journals. Obviously, we cannot report regression results for all 92 journals, so we selected journals for inclusion in this analysis to represent a mix of general, subfield, specialty, and country-specific journals. Our models include several sets of independent variables: (1) subfield variables, representing political scientists' subfield designations, with comparative politics the excluded category; (2) methodological-approach variables, including quantitative methods, qualitative methods, and formal theory, with normative theory the excluded comparison category; (3) demographic attributes of age and gender; and (4) country variables, represented by separate binary variables for the United Kingdom and Canada and with the United States the excluded category. This model permits us to estimate the effects of country on journal evaluations, controlling for other variables found by Garand and Giles (Reference Garand and Giles2003) to be related to general journal evaluations.

In Table 10 we report the results for our regression models for 14 journals, including the American Political Science Review, American Journal of Political Science, Journal of Politics, British Journal of Political Science, International Organization, World Politics, Comparative Politics, Comparative Political Studies, International Studies Quarterly, Journal of Conflict Resolution, Perspectives on Politics, Political Theory, Canadian Journal of Political Science, and Political Studies. Turning first to the country-specific variables, we find that only 25% (seven of 28) of the country coefficients are statistically significant at conventional levels. Hence there is relatively little evidence that political scientists from the United States, Canada, and the United Kingdom differ in their evaluations of these political science journals, once we control for the effects of other variables in our model. British political scientists are slightly more favorably oriented to the American Journal of Political Science and, as one might expect, Political Studies, and they are less favorable toward the Journal of Politics, World Politics, Comparative Political Studies, Perspectives on Politics, Political Theory, and the Canadian Journal of Political Science. Canadian political scientists do not differ from American political scientists in their evaluations of any of our 14 journals, though the Canada coefficient for the Political Studies is positive and significant at a more relaxed .10 level of significance. These results suggest that any simple differences that are observed in how political scientists from the United States, Canada, and the United Kingdom evaluate scholarly journals are due primarily to the effects of other variables, such as subfield or methodological orientation.

Table 10.

OLS Regression Estimates for Models of Journal Evaluations, Selected Journals, All Countries Combined, 2007

*** prob(t) < .01

** prob(t) < .05

* prob(t) < .10

Note: Comparative politics is the excluded group among the subfield variables, and normative theory is the excluded group among the methodological approach variables.

APSR: American Political Science Review
AJPS: American Journal of Political Science
JOP: Journal of Politics
BJPS: British Journal of Political Science
IO: International Organization
WP: World Politics
CP: Comparative Politics
CPS: Comparative Political Studies
ISQ: International Studies Quarterly
JCR: Journal of Conflict Resolution
POP: Perspectives on Politics
PT: Political Theory
CJPS: Canadian Journal of Political Science
Political Studies: Political Studies

We observe quite a few significant coefficients for the subfield variables in our models, though on balance only 28.6% (48 of 168) of the subfield coefficients achieve statistical significance at the .05 level. Scholars who list international politics as a subfield are more favorable than comparative politics scholars (who comprise the excluded comparison group) in how they evaluate International Organization, International Studies Quarterly, and the Journal of Conflict Resolution, three international relations journals that are among the top 10 journals in terms of overall impact; on the other hand, they are less favorable in their evaluations of the Journal of Politics, the British Journal of Political Science, and World Politics. We find that American politics scholars tend to evaluate the Journal of Politics in favorable terms but are significantly less positive in their evaluations of International Organization and Comparative Political Studies. Scholars who list Canadian politics and British politics as one of their subfields of interest are more likely to evaluate the Canadian Journal of Political Science and Political Studies in positive terms, and scholars specializing in Canadian politics give more positive evaluations to the British Journal of Political Science, Comparative Politics, and the International Studies Quarterly. Moreover, area studies and comparative politics scholars differ in their evaluations of some journals; specifically, area-studies scholars are significantly less favorable in their evaluations of the American Political Science Review but are more favorable in their evaluations of World Politics, Comparative Politics, and Comparative Political Studies. Finally, political theorists exhibit quite a few significant differences from comparative politics scholars; theorists give less favorable evaluations to the American Journal of Political Science, International Organization, Comparative Political Studies, and Journal of Conflict Resolution, but they are more favorably oriented toward Perspectives on Politics, Political Theory, the Canadian Journal of Political Science, and Political Studies.

Of the four clusters of independent variables in our models, our methodological-approach variables have the most consistent effects on journal evaluations, with 50% (21 of 42) of the coefficients surpassing standard levels of statistical significance. Further, at least one methodological-approach variable achieves statistical significance for 12 of the 14 journal models in Table 10. There are some interesting patterns of coefficients for our methodological-approach variables, and these coefficients can help to map the methodological profile of our various journals. We find that quantitative scholars give systematically more favorable evaluations to the American Political Science Review, American Journal of Political Science, Journal of Politics, and British Journal of Political Science than do normative theorists, who comprise the excluded (comparison) group; these general-interest journals are significantly more positively evaluated by scholars with some quantitative orientation than other scholars. Qualitative scholars differ significantly from normative theorists in how they evaluate several journals. Scholars adopting a qualitative research orientation are significantly less favorable in their evaluations of the American Political Science Review, American Journal of Political Science, Journal of Politics, British Journal of Political Science, and the Journal of Conflict Resolution; each of these journals publishes quite a bit of quantitative research, and this emerges in how qualitative scholars evaluate these journals. On the other hand, qualitative scholars give more favorable evaluations to World Politics, Comparative Politics, and Comparative Political Studies, at least two of which are known as outlets for publishing qualitative research. For formal-theory scholars, the American Political Science Review draws favorable evaluations, no doubt because it publishes highly regarded work in formal theory; on the other hand, we note that formal-theory scholars give less positive marks to the Journal of Politics, Comparative Politics, International Studies Quarterly, Political Theory, and Political Studies.

The cases of International Organization and Perspectives on Politics provide an important contrast. On one hand, for International Organization none of the methodology coefficients is statistically significant, suggesting that scholars from all methodological traditions evaluate this journal in similar terms. On the other hand, Perspectives on Politics represents a stark case of the methodological divide in political science. This journal is evaluated significantly less positively by quantitative and formal-theory political scientists, with qualitative scholars giving significantly more favorable evaluations. Taken as a whole, it appears that Perspectives on Politics is favorably evaluated by qualitative and normative-theory scholars, but is much less favorably evaluated by quantitative and formal-theory scholars. This is in keeping with the perception by many political scientists that Perspectives on Politics was created to appeal to (mostly qualitative) critics of the American Political Science Review.

Finally, we also include two demographic characteristics, age and gender, as independent variables in our models. The gender coefficient is significant for only three journals. Controlling for the effects of other variables, women political scientists are more positive in their evaluations of Comparative Politics, Perspectives on Politics, and Political Theory. Women are indistinguishable from men in how they evaluate the other political science journals in Table 10. The age variable was expected to capture generational influences on journal evaluations, and we find that a small number of the age coefficients achieve conventional levels of statistical significance. Older political scientists are less favorably oriented toward the Journal of Politics, International Organization, Comparative Political Studies, and Political Theory. There does not appear to be a systematic factor that ties these negative evaluations together, since this list includes journals from a variety of subfields and methodological approaches.

CONCLUSION

The purpose of this article is to build on previous research relating to how political scientists evaluate the quality and impact of scholarly journals in the discipline. Scholars who study journal impact in the social sciences often differentiate objective (i.e., bibliographic or citation-based) approaches and subjective (i.e., survey-based) approaches for evaluating how various scholarly journals influence intellectual discourse in the political science profession. As Giles and Garand (Reference Giles and Garand2007) point out, these two approaches provide complementary information about journal impact, and ultimately there is a moderately strong relationship between the rankings of journals using these two approaches.

This article clearly adopts the subjective approach, but with an important twist. In most previous studies scholars have conducted surveys of political scientists from the United States, so relatively little is known about how American political scientists differ from those from other countries in how they evaluate political science journals. Here we report the results of a survey that we conducted of political scientists from the United States, Canada, and the United Kingdom, and this permits us to make explicit comparisons among political scientists from these three countries. Moreover, our data permit us to consider how journal evaluations in 2007 are similar to those observed in previous studies.

What do we learn about journal evaluations, familiarity, and impact? First, as is the case in previous research, we find that there is a fairly clear ranking of political science journals. This ranking is fairly robust across the different types of measures that we use throughout this study. There is a small group of elite journals that are both broadly familiar to and evaluated very favorably by political scientists, and these journals earn high marks for their subjective impact on the political science discipline. These journals include the three major general journals from the United States (i.e., the American Political Science Review, American Journal of Political Science, and Journal of Politics), a major general journal from the United Kingdom (i.e., British Journal of Political Science), and several broad-based subfield journals, such as International Organization, Comparative Political Studies, International Studies Quarterly, among others. It is these journals that draw considerable attention in the political science discipline, and publications in these journals are more likely to be read and cited and to become part of the scholarly discourse in the profession. Other journals have some combination of high familiarity or strong positive evaluations, but not both, and these journals form a second tier of scholarly journals on our list. Many of these journals are highly regarded specialty journals (e.g., Legislative Studies Quarterly, Electoral Studies) or well-known general journals with moderate evaluations (e.g., Perspectives on Politics, Political Research Quarterly, or Foreign Affairs). Finally, there are other journals that are at the periphery of political science, insofar as they are not widely read and/or not highly evaluated.

Second, there are some differences in how political scientists from the United States, Canada, and the United Kingdom perceive the quality and impact of scholarly journals, but most of these differences are fairly modest. If one looks at the top-10 lists for each of these three countries, there is a fair amount of similarity in the journals that make the list, though there are notable exceptions. Everyone thinks highly of the American Political Science Review, American Journal of Political Science, British Journal of Political Science, International Organization, and a small number of other journals, but there are also some journals (e.g., European Journal of Political Research, Political Studies, Perspectives on Politics, Canadian Journal of Political Science, Journal of Common Market Studies) that have country-specific appeal. Taken as a whole, there is more broad similarity in journal evaluations than differences for political scientists across these three countries. The correlation for journal-impact measures among these three countries is fairly high.

Finally, we estimated a series of regression models for a small subset of 14 journals and find little systematic country differences in how political scientists evaluate these journals. Although there are some mean differences across countries that emerge in our ANOVA analyses, many of these differences disappear when we include control variables representing subfield specialties, methodological approaches, demographic attributes, and publication status. It appears that most of the differences among political scientists from the United States, Canada, and the United Kingdom are due to methodological differences across these three countries. Political scientists' methodological approach has been found to relate to evaluation of some journals (Garand and Giles, Reference Garand and Giles2003), and there is clear evidence of substantial methodological differences for political scientists from the United States, Canada, and the United Kingdom (Blais et al. Reference Blais, McLean, Garand and Giles2008). When the methodological differences among political scientists from the United States, Canada, and the United Kingdom are filtered out, the coefficients representing country-specific effects become statistically non-significant. The underlying basis for differences in political scientists' methodological approaches across these three countries is the subject of ongoing research.

All of this raises some questions about the possibility of greater cross-national integration of political science. While there is a broad-based consensus about journal evaluation, familiarity, and impact among political scientists from all three countries, there are some notable differences, and the methodological divide for political scientists from the United States, Canada, and the United Kingdom could possibly create a barrier to further integration of cross-national research. How political scientists from different countries but working on similar topics can be brought to communicate with each other may have geographic underpinnings, but it is also possible that methodological differences may serve as the greatest barrier.

Footnotes

¹ When we remove the journals representing the field of international relations, which appear to be operating in a different citation environment/culture, the correlation between the bibliometric and reputational ranks of the 76 remaining journals is 0.795.

² These journals are Law and Society Review, Presidential Studies Quarterly, Review of Politics, and Social Science Quarterly. We do not include in this step journals familiar to 20% or more in the Garand and Giles study but central to another discipline, that is, American Economic Review, American Sociological Review, and American Journal of Sociology.

³ These journals are Political Analysis, Canadian Public Administration, and Canadian Public Policy.

⁴ “For the sake of brevity, we have created an online appendix in which we present tables and figures representing various auxiliary analyses. This online appendix can be found at the Cambridge University Press Web site: http://journals.cambridge.org/action/displayJournal?jid=PSC.

References

Blais, Andre, McLean, Iain, Garand, James C., and Giles, Micheal W.. 2008. “The Methodological Divide in Political Science.” Paper presented at the 2008 annual meeting of the Canadian Political Science Association, Vancouver, British Columbia, June 4–6.Google Scholar

Christenson, James, and Sigelman, Lee. 1985. “Accrediting Knowledge: Journal Stature and Citation Impact in Social Science.” Social Science Quarterly 66: 964–75.Google Scholar

Crewe, Ivor, and Norris, Pippa. 1991. “British and American Journal Evaluation: Divergence or Convergence.” PS: Political Science and Politics 24: 524–30.Google Scholar

Garand, James C. 1990. “An Alternative Interpretation of Recent Political Science Journal Evaluations.” PS: Political Science and Politics 23: 448–51.Google Scholar

Garand, James C. 2005. “Integration and Fragmentation in Political Science: Exploring Patterns of Scholarly Communication in a Divided Discipline.” Journal of Politics 67: 979–1005.CrossRef Google Scholar

Garand, James C., and Giles, Micheal W.. 2003. “Journals in the Discipline: A Report on a New Survey of American Political Scientists.” PS: Political Science and Politics 36: 293–308.Google Scholar

Giles, Micheal, Mizell, Francie, and Paterson, David. 1989. “Political Scientists' Journal Evaluations Revisited.” PS: Political Science and Politics 22: 613–17.Google Scholar

Giles, Micheal, and Wright, Gerald. 1975. “Political Scientists' Evaluations of Sixty-Three Journals.” PS: Political Science and Politics 8: 254–57.CrossRef Google Scholar

Giles, Micheal, and Garand, James C.. 2007. “Ranking Political Science Journals: Reputational and Citational Approaches.” PS: Political Science and Politics 40: 741–51.Google Scholar

Goodson, Larry, Dillman, Bradford, and Hira, Anil. 1999. “Ranking the Presses: Political Scientists' Evaluations of Publisher Quality.” PS: Political Science and Politics 32: 257–62.Google Scholar

Hix, Simon. 2004. “A Global Ranking of Political Science Departments.” Political Studies Review 2: 293–313.CrossRef Google Scholar

Smith, Rogers. 2002. “Should We Make Political Science More of a Science or More About Politics?” PS: Political Science and Politics 35: 199–201.Google Scholar

Table 1.Evaluations, Familiarity, and Impact Ratings for Political Scientists from the United States, United Kingdom, and Canada, 92 Selected Journals, 2007

Figure 1.Scatterplot for Relationship between USA Journal Impact and Canada Journal Impactr = 0.737

Figure 2.Scatterplot for Relationship between USA Journal Impact and UK Journal Impactr = 0.563

Figure 3.Scatterplot for Relationship between Canada Journal Impact and UK Journal Impactr = 0.782

Table 2.Political Scientists' Subjective Evaluations, Top 30 Highest-Ranked Journals, All Countries Combined and Separately for Each Country, 2007

Figure 4.Distribution of Evaluations for 92 Political Science Journals on 11-point Scale, All RespondentsNote: The curve overlaying the histogram is a kernel density function.

Table 3.Mean Evaluations of Selected Political Science Journals, by Country of Respondent, 2007

Figure 5.Distribution of Levels of Familiarity for 92 Political Science Journals, All RespondentsNote: The curve overlaying the histogram is a kernel density function.

Table 4.Political Scientists' Journal Familiarity, Top 30 Highest-Ranked Journals, All Countries Combined and Separately for Each Country, 2007

Table 5.Familiarity with Selected Political Science Journals, by Country of Respondent, 2007

Figure 6.Scatterplot for Relationship between Journal Evaluation and Journal Familiarityr = 0.554

Table 6.Political Scientists' Preferences for Journal Submissions of High-Quality Manuscript, All Countries Combined, 2007

Table 7.Political Scientists' Preferences for Journal Submissions of High-Quality Manuscript, Top Five by Country, 2007

Table 8.Political Scientists' Preferences for Journal Reading, All Countries Combined, 2007

Table 9.Political Scientists' Preferences for Journal Reading, Top Five by Country, 2007

Table 10.OLS Regression Estimates for Models of Journal Evaluations, Selected Journals, All Countries Combined, 2007

Garand Supplementary Material

Appendix.pdf

PDF 137.2 KB

Article contents

Political Science Journals in Comparative Perspective: Evaluating Scholarly Journals in the United States, Canada, and the United Kingdom

Abstract

Information

DATA AND METHODS

Sampling Issues

Survey Administration

Journal List

EMPIRICAL RESULTS

Consistency of Journal Impact, Evaluation, and Familiarity Measures over Time

Journal Impact in Three Countries

Journal Evaluations

Journal Familiarity

The Relationship between Journal Evaluation and Journal Familiarity

Preferred Journal Submissions

Preferred Reading Sources

A Multivariate Analysis of Journal Evaluations

CONCLUSION

Footnotes

References

Garand Supplementary Material

Save article to Kindle

Save article to Dropbox

Save article to Google Drive

Reply to: Submit a response

Your details

You have entered the maximum number of contributors

Conflicting interests