Ideally, peer review is a means to improve communication among scholars as well as between scholars and the public. Most obviously, peer review serves as a critical means of quality control. Through peer review, important questions are raised, assumptions are questioned, and close scrutiny is directed toward the entire research process, from the framing of the research question, to the gathering of evidence, to the generalizability of conclusions.
More broadly, however, the peer review process also plays an essential gatekeeping role by ensuring that the best research in the discipline is showcased in prominent publications. Reviewers are responsible for making numerous determinations that go beyond research quality. For example, is this an important question; should anyone care about these findings; and, if so, what are their broader implications? By virtue of publication decisions, peer review also directs greater resources and attention to certain research questions than to others.
Given that the peer review process plays such an essential role in setting the research agenda, in identifying quality research, and in encouraging effective communication of important findings to the public, it is ironic that it is relegated entirely to volunteer labor. The peer review process is made possible by scholars who donate their time and expertise without any expectation of reward, remuneration, or recognition. Moreover, not all scholars contribute equally to this endeavor, and some do not review at all.
THE PROBLEM
As the number of publication outlets has increased, the demands on those who review manuscripts have grown. The quality and integrity of the entire scientific process depend on the relatively frail assumption that scholars are willing to provide one of their most valuables resources—their time—for nothing in return except perhaps the satisfaction of contributing to the discipline. If we are to maintain our commitment to producing discoveries that are based on defensible and credible scientific knowledge, a way to incentivize scholars’ contributions to this process is long overdue.
After all, everyone is incredibly busy; this is a basic fact of academic life. Nevertheless, maintaining high research standards requires that we contribute to the collective good by spending many hours reviewing other people’s manuscripts. This part of our job does not appear on our CVs, and we are not credited by our employers for the time we spend reviewing the work of others. It is a particularly thankless task because the more reviews that a given scholar does—and the more thoroughly and conscientiously that he or she does them—the more new requests that scholar is likely to receive. It is well known that no good deed goes unpunished: those who review regularly for journals and take the task seriously receive still more requests as a result of acquiring a reputation as a responsible reviewer. Moreover, the broader one’s interests, the more requests one is likely to receive from diverse journals.
On the one hand, saying “yes” to these requests seems like the right thing to do; on the other hand, doing so has costs that are steep in terms of time. In addition, academics in lower-status positions are more likely to be uncomfortable saying “no” to an editor, whereas more senior colleagues may not review any manuscripts. The reward system for reviewing manuscripts is obscure at best, with some scholars reporting a fear that declining to do so will result in negative repercussions from journal editors, whereas excessive reviewing could jeopardize their own research agendas. Some scholars indicate that the newer automated requests for review are easier to decline than an e-mail direct from an editor. As a result of these uncertainties, an unrepresentative subset of researchers tends to handle most of the manuscript reviews. This is unfair not only to these reviewers but also to the authors of the articles being reviewed.
This tension over how much to do or not to do to facilitate a healthy balance plagues individual scholars, but it is an even more serious ongoing problem for journal editors who need to obtain prompt and thorough reviews of manuscripts from scholars in the field. Although there is no shortage of scholars who want to submit papers to journals, journal editors do not have a surplus of prompt and reliable reviewers.
At best, reviewing is an unacknowledged and unrewarded professional duty. Nevertheless, it is central to claims about the legitimacy of research in our field. A lack of prompt and thorough reviewers is every journal editor's primary problem, and the lack of high quality reviews lowers the value of published work within the discipline.
THE PROPOSED SOLUTION
To provide incentives for reviewers, I propose a review exchange (REX) system in which those submitting manuscripts to a journal must have accumulated credits by reviewing the work of others as “advance payment” for their own submissions. The idea is that authors must have at least three reviewer credits before submitting a manuscript (i.e., the replacement cost for the three manuscript review requests pertaining to their own manuscript).
Ideally, this system would be at least discipline-wide so that the three reviews that comprise the “credits” need not have been done for the same journal. Otherwise, the probability of having reviewed an adequate number of manuscripts for precisely the right journal for one’s own research would be inconveniently small. In addition, establishing this system for only one or a small subset of the discipline’s journals would create strange last-minute demands to editors to review for a particular journal. This is not the type of long-term, ongoing cooperation that would best enhance the review process.
Under this new system, scholars would seek out opportunities to review if they plan to submit manuscripts for review themselves. Journal editors also would acquire credits for their considerable service to the field (an even more thankless task!), and these credits could be used for future submissions to any journal.
This system would prevent free-riders from publishing in a discipline’s journals without offering their own services as a reviewer. Any journal would be welcome to join the REX provided it (1) agrees to award credits to reviewers who return reviews of acceptable quality on time, and (2) requires those who submit manuscripts to “spend” REX credits as a condition of having their manuscript reviewed. As discussed later in this article, editors would have discretion in deciding when a review was of acceptable quality and timeliness to receive a credit.
Under this system, scholars would seek out opportunities to review if they plan to submit manuscripts for review themselves. Journal editors also would acquire credits for their considerable service to the field (an even more thankless task!), and these credits could be used for future submissions to any journal.
The REX system would exist as an online platform similar to the system used by Time-sharing Experiments for the Social Sciences (TESS). It would interface with the major online journal management systems (e.g., ScholarOne and Editorial Manager) so that editors could seamlessly track reviewers’ contact information, areas of expertise, and REX credits. Currently, only a few such journal management systems are used in political science; therefore, facilitating a link between the database that editors already are using and the REX data should not be difficult. Journal editors then would have access to a system that allows awarding credits when reviews of sufficient quality are submitted, as well as subtracting credits from an author when a manuscript goes out for review. Authors of multi-authored manuscripts can pool their REX resources to “pay” for their review. Individual political scientists would have read-only access to REX to check their current credits and to offer their services to editors when they are particularly eager to acquire additional credits (see table 1a).
Sample REX Screen: Logged on from the perspective of individual authors and reviewers

This system would have the additional advantage of making shared information about reviewers’ activities and expertise available to all journal editors who join the system, thereby greatly increasing their pool of potential reviewers. For a new or smaller journal, this access to information about reviewers’ areas of expertise would be especially valuable. Many scholars, including myself, dread having to repeatedly fill out this information for each journal that requests that I review a manuscript.
Even for well-established journals, the constant need for conscientious reviewers is the most difficult part of an editor’s job. Authors understandably complain when a reviewer is late or does not spend adequate time carefully reviewing their manuscript.
Editors also need timely reviews. They know that scholars’ careers depend on receiving prompt reviews and editorial decisions. Without a pool of scholars willing to provide reviews, editors must strike a balance between high-quality reviews with constructive substantive advice and “quick and dirty” reviews that may facilitate faster decisions.
Before describing the technical details of how the REX system would operate, I address the two most obvious questions. First, it is worth considering whether the current system is truly broken and needs fixing. Those who have served recently as journal editors are in the best position to comment on this matter. However, my informal impression is that editors would appreciate having more volunteers to review manuscripts and to provide additional information that would help editors to locate and select the most appropriate reviewers for a specific manuscript.
Currently, scholars have a strong disincentive to fill out the multiple journal forms indicating their interests and areas of expertise. After all, this is likely to result in still more requests for reviews that they may feel guilty about declining. However, if this information were centrally gathered and needed to be supplied only once (not for each additional journal), it would be less time-consuming for reviewers.
This system should expedite the review process in multiple ways. For example, through the centralized REX system, reviewers could indicate when they are available to receive requests. I suggest a simple signaling system consisting of red, yellow, and green indicator lights. If reviewers are on sabbatical, overwhelmed with a backlog of reviews, or otherwise preoccupied, they simply can indicate their unavailability by posting a red light on the REX system. Unlike in the current system—which requires notifying each journal editor if the reviewer is even temporarily unavailable—this information would be conveyed simultaneously to all journals. This eliminates the need for editors to wait for a response from a potential reviewer before contacting another (see table 1b).
Sample REX Screen: From the perspective of journal editors, additional fields on existing manuscript review platforms

If scholars are eager to accumulate REX credits for the purpose of submitting their own studies, they can indicate this availability using the green light. Yellow lights would be reserved for reviewers who are currently selective, but who nevertheless would review a manuscript that was in their area of expertise and of great interest. As a result of these signals, editors would waste less time eliciting responses from reviewers who are either unwilling or unavailable.
Of course, reviewers could still refuse a manuscript that involved a conflict of interest or a topic for which they were ill-suited. Overall, however, editors’ jobs should become easier as a result of this system and authors should receive more timely responses to their submissions, resulting in their satisfaction as well.
Overall, however, editors’ jobs should become easier as a result of this system and authors should receive more timely responses to their submissions, resulting in their satisfaction as well.
The second question is whether there is an alternative incentive system that might work better or be easier to manage. To date, the only other systems of which I am aware involve compensating reviewers and/or requiring authors to pay for the privilege of having a manuscript reviewed. For multiple reasons, these approaches seem less desirable than the type of currency offered by REX. By favoring those with more disposable income who can afford to submit manuscripts to many journals, what is eventually published by systems that require payment would be unfairly skewed to senior scholars already successful at publishing (or independently wealthy). This is a clearly undesirable scenario. Some editors suggest that such a system would make authors more cautious about the quality of work they submit for review. It would be especially unfair to junior faculty members who have fewer funds to spend on submissions but have the most urgent need to publish.
Payment “in kind” through manuscript reviews has the same advantage as requiring authors to make monetary payments; that is, authors will be cautious about wasting their REX credits on a manuscript that is not yet ready for external review. However, this also makes the “payment” required equivalent for all scholars. Whereas making monetary payments would be more difficult for some than others, time spent reviewing a manuscript is equally costly for both. Establishing a separate currency through REX credits makes the proportion of resources required to submit an article approximately equal.
The other option (i.e., paying reviewers) is similarly problematic in that journals seldom have the budget necessary to compensate an amount sufficient to incentivize reviewers. Having one’s hard work acknowledged by other scholars is appreciated, but token payments are unlikely to make them more eager to serve as reviewers.
TECHNICAL ISSUES
Certain technical and practical issues must be resolved to ensure the success of this project. Most important, who would fund and manage the REX platform? Startup funds would be necessary to launch the program; however, this would be on a scale that the American Political Science Association (APSA) could fund if it wanted to do so. As I learned with the TESS platform, almost all of the costs derive from setting up the initial system and then fine-tuning it during the first year of operation. Afterwards, people simply need to continue to use it so that it becomes an increasingly valuable resource to the academic community as more participating journals and information about potential reviewers are acquired. Thereafter, minimal annual contributions from journals that join the system should be sufficient to sustain it.
How would the system start up when no one has any initial credits for the purpose of submitting a manuscript? To start the currency flowing, I suggest that when a journal first joins the system, it should give credits to all reviewers who served the journal in the past two years. Given the banking-system parallel, an equivalent to the Federal Reserve Chair would need to monitor the system in this initial period to ensure that enough people are eligible to submit manuscripts and that supply and demand are in balance.
To receive more credits, scholars need only ensure that their contact information and expertise on the REX system is up to date and that they are listed as an “available” reviewer so that editors can request their services. Those who provide low-quality, cursory reviews are unlikely to be selected as future reviewers. Most important, those who do not take their task seriously can be denied credit for their review if, in an editor’s judgment, they did not devote sufficient time and attention to it. Most journals already maintain a system for tracking reviewers’ reputations, but the incentives are precisely the opposite of what would be expected: those with a strong reputation receive still more requests and those who do a poor job are not asked to do more.
This consideration raises the more general issue of whether the costs and credits for reviewing should be uniform. For example, if a manuscript is rejected based on only two reviews, should the cost to the author be two credits rather than three? The simplest system would be to charge the same number of credits for every submission (i.e., three). However, serious feedback on research is a valuable commodity, and three reviews generally provide more than two. Therefore, I suggest that at the time of submission, an author is debited three credits; however, if a decision is made without three reviews, one credit can be refunded. Likewise, journal editors can be awarded credits based on the volume of manuscripts they handle, thereby providing concrete benefits for what is an otherwise thankless task.
What about overdue reviews? How would these be addressed in the REX system? Unfortunately, late reviews are the norm in our discipline, and the REX system should discourage it. If a review is overdue, editors would have the option of allowing the reviewer to continue until a second preestablished deadline and still receive half credit for the review. Alternatively, the reviewer could be replaced so that another has an opportunity to earn credits. If editors need an additional quick review to replace a slacker reviewer, they also could incentivize a shorter turnaround time by offering more than one credit for a particularly quick review. Allowing editors the discretion to make these decisions will provide the flexibility they need to make the system work.
If editors need an additional quick review to replace a slacker reviewer, they also could incentivize a shorter turnaround time by offering more than one credit for a particularly quick review.
Similarly, journals could incentivize high quality reviews by offering credit bonuses to their highest quality reviewers. These credits would allow conscientious reviewers to benefit from their efforts by being required to do fewer reviews in the future when they want to submit an article. Those who are diligent in this important task should be rewarded for their efforts.
How much would it cost to set up the REX system? The answer depends on information that currently is not available but easily could be acquired if there is sufficient interest. The fact that the REX system may be based on similar existing online systems (e.g., TESS) would minimize the necessary programming time. However, ensuring that REX interfaces with the major journal manager systems would require additional programming so that specific information fields can be established and viewed from any platform. It is essential that REX be well integrated in current online systems so that editors are not required to take additional steps to make use of this information. A web address where authors and reviewers could log on to check their current credit status would be established. It also would be linked to each journal that joins REX so that it can be accessed from the submission screen of participating journals as authors submit manuscripts for review. Similarly to how a debit card checks available resources before approving a purchase, the REX utility would check and encumber credits as an author submits online to a journal, thereby automatically preventing submissions from those who have not accumulated credits.
Several concerns have been raised in response to this general idea. Some people would remain unincentivized because they are primarily book authors, are in administration, or are inactive researchers. It is true that people who plan to discontinue their research agendas for any reason may not be as motivated to review under this system as those who plan to continue as active research scholars. However, I argue that this also makes them less desirable reviewers. Under the REX system, they will be no less motivated than they are now, and the majority of scholars should be more motivated. The same would be true for those who strictly publish books; they could choose to review journal articles or not, as they see fit. However, they should be no less motivated to do so than they are under the current system, in which no one is offered any remuneration.
Research on incentive systems suggests certain cases in which incentives work well initially but thereafter people become less likely to act without payment. Could this be a potential problem for REX? I do not see how it could adversely affect REX subscriber journals or their reviewers because they would consistently offer “payment” for this service. However, it should have adverse effects on reviewer take-up rates for non-REX journals, thereby encouraging those journals to quickly join the alliance.
Although there definitely will be unforeseen glitches in how REX motivates scholars as authors, reviewers, and editors, I believe most will agree that this system would be more equitable than what is currently in place. At present, many scholars actively avoid requests for reviews rather than seek them out; the REX system would prevent this behavior. REX also would mean that the most active research scholars within the discipline are among the most active reviewers. This is entirely appropriate and consistent with the goals of high quality research.
How many journals would need to join to initiate this system? In my opinion, two or three high volume journals within the discipline would be necessary for REX to get off the ground. After that, increasingly more specialized journals would opt in because of the advantages for editors, the quicker turnaround times, and the significantly larger database of clearly available and interested reviewers; the incentives would be far greater to volunteer their time. Moreover, reviewers naturally will prefer to review for REX journals, thereby pressuring more journals to join once the system is established. After two to three prestigious, high volume journals join REX, a tipping point will be reached in which virtually all journals will want to be REX journals. After all, why review for a non-REX journal when there are clear incentives to do so for the REX collective? The advantages for both editors and reviewers will pressure journals to join the REX system relatively quickly. Moreover, the supply of REX credits will become more elastic as more journals participate.
After two to three prestigious, high volume journals join REX, a tipping point will be reached in which virtually all journals will want to be REX journals. After all, why review for a non-REX journal when there are clear incentives to do so for the REX collective?
CONCLUSION
Overall, I believe the REX system has the potential to become a “win-win-win” strategy. It would benefit APSA to sponsor it because of the prestige associated with being a leader in improving the scientific review process. Moreover, APSA clearly would be providing a valuable resource to the field as a whole. The REX system well may be adopted by many other disciplines after they observe a successful working model.
Journal editors would constitute a second set of winners as a result of this endeavor. Their jobs will become easier and more rewarding when they have the leverage needed to encourage high quality, prompt manuscript reviews. The third set of winners would consist of scholars who review for and submit manuscripts to journals in the field. As authors, they will receive timelier, better quality reviews. In addition, scholars will be able to contribute their fair share—but no more—by eliminating the free-rider problem.
The goal of the REX system is to create a more efficient, higher quality, and more equitable manuscript review system than is currently in place. Technologically, it should not be difficult to set up, and the goal would be a system that is both secure and easy to use for authors, reviewers, and editors. To my knowledge, this would be an innovative first for the social sciences; to date, I am not aware of any discipline that has built or used a similar system. After REX is integrated in the major manuscript review platforms, it would be easy to offer the service to journals in other disciplines for a modest annual fee, which would cover the costs of ongoing maintenance and system updates.
I especially would appreciate reactions to this proposal from current and past journal editors. What problems and/or additional advantages do you foresee? What other technical issues would need to be resolved?


