What is open peer-review?

Historically, scientific peer review is either double-blind or single-blind, where the reviewer’s identity remains hidden. More recently, some psychology journals have adopted variants of an open peer-review policy, which discloses reviewer identity to the authors (e.g., Frontiers in Psychology; Suls & Martin, Reference Suls and Martin2009). Some other journals employ an optional open-review policy, where authors may choose to have their reviews published and identified (e.g., Collabra: Psychology). Open-review policies are becoming more prevalent in the broader scientific community. Over 70 journals from BioMed Central (BMC), an open-access publisher, have adopted some form of open-review policy. Despite its growing popularity, few journals in industrial and organizational (I-O) psychology and related fields have an open peer-review policy. In our review of 24 I-O psychology journals, we observed that they all uniformly enforce a double-blind review policy.

Advantages of open peer review

Open review increases the accountability of reviewers, which affects their judgment and decision-making processes (Lerner & Tetlock, Reference Lerner and Tetlock1999). Organizational research, for example, has shown that rater accountability improves both the reliability and accuracy of performance judgments (Mero & Motowidlo, Reference Mero and Motowidlo1995). Accountability also fosters greater cooperation and generosity in social interactions (Bateson, Nettle, & Roberts, Reference Bateson, Nettle and Roberts2006). Finally, behavioral economic research shows that signing one’s name—electronically or physically—can reduce unethical and immoral behaviors (Shu, Mazar, Gino, Ariely, & Bazerman, Reference Shu, Mazar, Gino, Ariely and Bazerman2012). In the context of reviewing, this work suggests that accountability may encourage more constructive and developmental reviews. Furthermore, it could deter reviewers from making ethically dubious recommendations, such as encouraging authors to drop unsupported hypotheses or generate new ones based on the results (i.e., hypothesizing after the results are known [HARKing]; Murphy & Aguinis, Reference Murphy and Aguinis2019). In a meta-analysis of seven randomized control trials, Bruce, Chauvin, Trinquart, Ravaud, and Boutron (Reference Bruce, Chauvin, Trinquart, Ravaud and Boutron2016) found that signed reviews are higher in quality than anonymous reviews.

An extension of signed reviews is published reviews, which is practiced by several open-access publication outlets such as Meta-Psychology and PLOS One. At the time of publication, the review content and reviewer names are included along with the journal article, thus giving recognition for the reviewers’ work. Consequently, other scholars may use published reviews as a potentially valuable resource, thereby further enhancing the scientific utility of the peer-review process. The added social and professional incentives of published reviews could motivate scholars to conduct high-quality reviews.

Why might reviewers not sign their reviews?

Despite its benefits, many reviewers are hesitant to sign their reviews and are more likely to refuse review requests if they are required to sign their names (van Rooyen, Delamothe, & Evans, Reference van Rooyen, Delamothe and Evans2010). A recent study of over 350 faculty members found that fear of retaliation is the primary concern with signing reviews, with women reporting higher concerns than men (Lynam, Hyatt, Hopwood, Wright, & Miller, Reference Lynam, Hyatt, Hopwood, Wright and Miller2019). Indeed, 15% of faculty in the aforementioned study who had signed a review felt retaliated against for a past signed review. Fear of retaliation may be especially evident among junior faculty, who may worry that a signed negative review could have adverse implications for their future tenure letters, working relationships, and reviews of their own manuscripts. These concerns are consistent with findings that tenured faculty are more likely (57%) to have signed at least one review than their untenured colleagues (29%) (Lynam et al., Reference Lynam, Hyatt, Hopwood, Wright and Miller2019). Notably, men are also more likely to sign their reviews (61%) than women (37%).

Regardless of gender or academic rank, hesitation to sign reviews due to a fear of retaliation is consistent with research on workplace incivility, which consists of subtle and ambiguous behaviors that harm targeted employees, whether intentional or not (Cortina, Reference Cortina2008). Reviewers are concerned that the author may perceive even the best-intentioned, but critical, review as an act of incivility, which, in turn, could provoke a backlash from the authors and launch a spiral of incivility. It appears much of the concern with signing one’s reviews is amplified when the review is negative. Potential solutions to overcoming this barrier may rest on editorial oversight and policy.

Editorial policy implications

The fear of negative social consequences may deter reviewers from signing their reviews. To this point, we agree with the authors that some of the responsibility for precipitating competent and civil reviews rests on the editors, rather than the reviewers. As noted by Godlee (Reference Godlee2002), editors must take full responsibility for editorial decisions, rather than “hide behind the spuriously heightened authority of anonymous reviewers” (p. 2796). We add that, by shifting the decision responsibility to the editor, reviewers may feel less pressured to provide editorial recommendations. Instead, they can focus on identifying the strengths and limitations of the article and leave the editorial decision to the action editors. Consequently, reviewers may feel empowered to provide critical reviews while assuaged of their fears of retaliation. This shift in responsibility calls for greater editor training.

Attitude toward open reviews is changing for the better (Ross-Hellauer, Deppe, & Schmidt, Reference Ross-Hellauer, Deppe and Schmidt2017). Editors and publishers can accelerate this change through policy implementations. First, editors could encourage reviewers to sign their reviews by making it a prominent option on their journal website (e.g., Collabra: Psychology). Furthermore, editors may consider using behavioral “nudges,” such as an opt-out policy where open identity is the default option: reviewers must opt-out of sharing their identity (e.g., Johnson & Goldstein, Reference Johnson and Goldstein2003). Journal may also consider publishing reviews alongside journal articles. In addition to deterring uncivil reviews, the publication of peer reviews is consistent with the broader open-science agenda. Publishing peer reviews improves the transparency of the editorial process and gives recognition to high-quality reviewers. In our view, these policy implementations give reviewers the choice to remain anonymous and simultaneously embolden open-review practices.