Why Has the Problem Become Worse?

• The number of U.S. journals has increased (see attached list of new [U.S. and non-U.S.] journals). Particularly impressive is the number of specialty journals that have been created over the past 10 years. This is in addition to a spate of journals that began roughly 20–25 years ago, such as Legislative Studies Quarterly, Political Psychology, American Politics Quarterly (now Research), Political Behavior, Electoral Studies, Political Theory, Journal of Policy and Management, and so on.
• Other journals have expanded the number of issues published each year (see Appendix) or have grown in page count while still publishing the same number of issues each year.
• The number of non-U.S.-based journals has increased, and submissions and reviewing can now be done worldwide. Journals such as the Journal of Elections, Public Opinion and Parties, and the European Journal of Political Theory join older ones such as the British Journal of Political Science, Scandinavian Political Studies, and the European Journal of Political Research. Globalization of research has made us more aware of research abroad, and email and the Internet have facilitated the use of reviewers worldwide. Though I have no way of knowing for sure, I suspect that submissions to U.S. journals by non-U.S. political scientists are up more than the reverse and that the use of U.S. reviewers by non-U.S. journals is greater than the reverse (for some information on the first point, see www.isq.unt.edu/2005_report.pdf, p. 8).
• The number of journals in related fields has also increased—including such diverse entries as Social Choice and Welfare, Election Law Journal, Theory and Research in Social Education, Political Geography, and the International Journal of Public Opinion Research.
• Some (many?) journals now use three reviewers instead of two. The movement to additional reviewers has come about, in part, from the pressure to reduce turnaround time for reviewing (as editors can reject manuscripts upon receiving two negative reviews).
• As turnaround time has decreased, researchers sometimes send their work to journals that are unlikely to accept it—simply to get objective feedback or on the off-chance that the paper will be accepted.
• The discipline has become more specialized. On average, this must make it more difficult to find appropriate reviewers for a given manuscript and probably results in more turndowns and therefore more overall requests.
• Research in political science has become much more methodologically and mathematically sophisticated. This also makes it difficult for editors to find suitable reviewers and puts pressure on those reviewers who are capable of evaluating advanced work. At times it leads to additional reviews, requested solely or chiefly to check on the correctness of the material rather than on its overall quality.
• Fewer papers, it seems, are accepted outright. The “revise-and-resubmit” process requires a second round of reviews.
• The discipline has become more journal-centric, with more pressure to publish journal articles rather than (only) books—or at least to accept journal articles as reflecting genuine scholarly contributions.
• Graduate students are now strongly encouraged to publish their research, sometimes collaboratively with their professors and sometimes on their own. The increased number of graduate student authors is not likely matched by their increased numbers in the reviewer pool.

Possible Solutions

If editors and reviewers are in fact overburdened, there are a number of actions that can be considered. None is a panacea, and there are objections to each of them. I do not try to evaluate them fully, instead listing only the most obvious points about each.¹

The first set of ideas is aimed at reducing the overall number of reviews that are done:

• Increase the number of manuscripts returned without reviewing, perhaps with assistance from the editorial board. Objection: less feedback to authors; possibly some poor decisions.

• Use some form of “screening review”—an initial review to determine whether a full review is warranted. Objection: longer turnaround times; possibly some poor decisions on the screening round.

• Charge a submission fee. Objection: if it is high enough to dissuade marginal submissions, it may be unfair to researchers at certain kinds of institutions.

• Urge journals (perhaps by inter-journal agreements) to use only two referees. Objection: turnaround time may rise.

• Require graduate students who submit papers to include a letter from an advisor attesting to the advisor's judgment that the paper is a plausible candidate for publication in the journal. Objection: makes graduate students second-class citizens.

The next proposals might increase the size of the reviewer pool:

• Pay reviewers when they submit on-time reviews. Objection: could encourage on-time, but poorly done reviews.

• Publish a list of late reviewers. Objection: unfair when tardiness is due to unavoidable circumstances; disincentive to agree to review.

• Have editors share information about reviewers' habits, including names of people who refuse to do any reviewing or who say they will but do not follow through.²

  “[W]e also notice that there is a sizable minority in the discipline who take advantage of the fruits of the system as contributors, while failing to contribute to it as reviewers” (www.isq.unt.edu/2005_report.pdf, pp. 13–14).

  Objection: might violate people's privacy rights.
• Do not allow people to publish papers if they don't review manuscripts in a timely fashion. Objection: could lead to type I (fail to publish papers from people who had legitimate excuses) and type II errors (publish papers from people who do timely but poor reviews).

The next proposals might reduce the number of refusals to review and hence the overall number of requests to review (if more editors meant better-targeted selection of potential reviewers), but their greatest effect would occur if the larger number of editors meant that they (collectively) took on more responsibility for reviews (or returning manuscripts without outside reviews):

• Use multiple editors for different sub-topics (as Legislative Studies Quarterly does). Objection: coordination problems.

• Use a system of associate editors who make reviewing assignments and semi-final or final decisions about acceptance. Objection: coordination problems; may require substantial work of a large number of people; greater costs.

The next proposal might lead to fewer journals altogether or at least slow the growth of new journals:

• Study citation patterns of journal articles as a way of judging the value of so many publications. Objection: citations are a controversial measure of value.

The last proposal would reward especially good reviewers:

• Establish an annual award, sponsored by individual journals, associations, or journal publishers, to recognize individuals who write numerous, good, timely reviews. Objection: coordination problems if awarded by an association or by publishers; possibly some cost.

Some Suggestions for Interim Improvements

For reviewers:

• Respond promptly to requests to review. It is highly irresponsible not to respond at all.
• If you are unlikely to review a manuscript, say so at the outset.
• If you say “no,” suggest appropriate readers; make more than one suggestion because one's first suggestion may well be the author.
• Follow through on promises. If you say you will review a manuscript, do it. Editors report that the worst hold-ups occur when reviewers “string along” an editor, promising and promising again to file a review and never doing so.³

  ISQ reports that a shockingly high percentage of requests to review—nearly 40%—are turned down, “either by declining to review, by failing to respond after numerous queries often sent to multiple e-mail addresses, or by not filing a review after they had agreed to do so” (www.isq.unt.edu/2005_report.pdf, p. 12). Kim Hill, then coeditor of AJPS, reported less formally on the Political Methodology listserv (see note 1 above) that many individuals declined invitations to review, presumably because of the frequent requests they received, and that of those to whom they did send papers, “somewhere between a quarter and a third, never provide[d] us with reviews” (3/24/05).

• Send in completed reviews promptly (see first suggestion for editors).
• (Of course) Do your professional duty and review manuscripts.

For editors:

• Publicize policies about how frequently you use a given reviewer. I believe most editors try to limit the number of reviews to one per person per year, though exceptions occur when a manuscript directly addresses a given person's research (in which case it is considered inappropriate not to send it to the person in question) and for revise-and-resubmits, which often go back to an original reviewer. If this is the case and reviewers came to believe it, they would be less likely to delay sending in their reviews (so as to avoid getting another request soon).

• When you send material via email, include the journal title (most likely in abbreviated form) in the file name. Otherwise reviewers receive files entitled “ms 0521,” “evaluation form,” or some such. After even a day, one can forget what these files are and whether they are even legitimate (as opposed to spam).

• Make the above information the first part of the title so that multiple files from the same journal reside next to one another when alphabetized.

• (Of course) Promptly acknowledge receipt of each review (if not done automatically via a web-based system).

We all benefit from good reviewing processes—ones that provide prompt, thoughtful, and helpful (to editor and author) reviews. If we are overburdened, the likelihood of establishing or maintaining such processes is reduced. But how to keep the burden in check is not obvious. I urge widespread discussion of both the problem and its possible solutions. And, since this is a problem for the discipline, not simply for one or two journals, I especially urge the APSA, perhaps initially through its Committee on Publications, to try to find a solution to the problem before it becomes even worse.

Author Bio

Richard G. Niemi is Don Alonzo Watson Professor of Political Science at the University of Rochester, where he has taught for 40 years and has reviewed countless manuscripts.