In their focal article, Köhler et al. (Reference Köhler, González‐Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020, p. 16) point out that reviewers need to balance their “dual roles as gatekeepers … with the generative and developmental role of peers….” The proposed framework also includes the implicit assumption that all proposed competencies are ultimately compatible with each other. Although the framework features a thoughtful and thorough list of necessary competencies, it may also be regarded as too optimistic. Each of the behaviors that is in line with these competencies can be seen as part of a role that needs to be fulfilled concurrently with other roles and ideally by the same person during the reviewing task. We argue that such roles include inherent role conflicts and role ambiguity between different goals and tasks the reviewer is assigned. For instance, it is not clear if the combination of scouting for statistical quality and at the same time for novel findings is always realistic; or, if socialization into the field can always be reconciled with advancement to the field.
One may differ about which roles should be paramount, and the selection put forward in this framework is arguably a suitable compromise. To some degree, peer review has roles that could be in line with each other, in which case one could use role theory (Biddle, Reference Biddle1986; Ebbers & Wijnberg, Reference Ebbers and Wijnberg2017) to develop a job performance scale to measure the reviewers’ performance (also see Welbourne, Johnson, & Erez, Reference Welbourne, Johnson and Erez1998), an element that would make the suggested framework perhaps more rigorous. However, there is inherent conflict and ambiguity between the different roles in peer review, and tensions around peer review are likely the result from this inherent role conflict. Extensive research has shown that role conflict and ambiguity are negatively related to job performance (Rizzo, House, & Lirtzman, Reference Rizzo, House and Lirtzman1970; Tubre & Collins, Reference Tubre and Collins2000), which likely extends to peer review.
In an effort to make reviews more reliable, Köhler et al. (Reference Köhler, González‐Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020) suggest that reviewers should not alternate between roles and fill all roles simultaneously. While this approach sounds appealing at first glance, it may be difficult to implement in practice. For instance, a reviewer of an article may find only one role particularly suitable for his/her background and expertise. He/she may be encouraging the authors of the article in a certain direction that would potentially lead to a theoretically more rigorous contribution, while in the case of a different article, he/she may point out the many problems in the theoretical approach and thereby disqualify, for instance, the publication of an article.
In what is named “mechanical skills,” Köhler et al. (Reference Köhler, González‐Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020, p. 13) state in relation to commentary provided by reviewers: “So, we need to communicate realistically what needs to be improved and what can be improved in future work or in a next step.” And in terms of “professionalism,” the reviewer needs to be constructive. All these elements would suggest that reviewers should encourage authors in a certain direction. As a potential concern, Köhler et al. point out the issue of reviewers “overstepping their roles” (p. 4) as one of the main flaws of current review practice. This issue is exemplified in the more extreme form of hijacking an article or ghost authoring the final product. It is also acknowledged that the expertise of a reviewer may push authors in a particular direction. Köhler et al. concede that there is “some gray area” (p. 7) but that certain behaviors are clearly unwanted. This latter comment points to the general problem that the proposed framework does not sufficiently address. Role conflict is a gray area, and we suggest to expand the competency framework that is offered to future or current reviewers and assist in overcoming the ambiguities and conflicts inherent to peer review.
Reducing role conflict
Without addressing role conflict in peer review, the competency framework still addresses the various improper behaviors that need to be avoided by all reviewers. In that sense, it is a useful and comprehensive reminder, but it is not necessarily sufficient as a promising way forward in guiding the complex task of peer review. Literature on role conflict suggests several ways to complement this framework. Ashforth and Saks (Reference Ashforth and Saks1996) point out that systematic socialization tactics, such as clearly communicated expectations, may reduce role conflicts. The focal article serves as an important first step in the socialization of new reviewers. Following Floyd and Lane (Reference Floyd and Lane2000), we recommend two additional avenues that may further reduce role conflict: increasing autonomy and increasing formalization.
Reviewers typically already enjoy high autonomy, but role conflicts are not necessarily explicitly addressed. One solution could be that reviewers indicate their own role priorities in the review process when they submit their reviews. Subsequently, the editor weighs the representation of certain roles by different reviewers in order to reach a balanced overall review.
Alternatively, one may increase formalization in the review process. For instance, one could include guidelines for editors regarding how to address problems of role conflict, in which case the solutions may be largely at the discretion of the editor. For instance, editors could prioritize certain roles for a reviewer when they send out a reviewing request. In such a case, the prioritization of different roles is likely informed by the reviewer’s expertise. A possible advantage of this procedure is that editors not only ensure that all necessary roles are covered, but that they also strive to assign nonconflicting roles to reviewers. Although editors may already implicitly or explicitly assign specific roles when they choose reviewers with different expertise, an explicit formalization further assists in reducing possible role conflicts.
Philosophy of science has acknowledged that conflict is central to scientific progress (Kuhn, Reference Kuhn1977). Practically, however, role conflicts may frustrate the review process when they are not recognized and addressed either by guidelines or by formal review processes. Increasing awareness and ideally an explicit focus on the reduction of role conflicts may further enhance the merit of the already useful guidelines set forth by Köhler et al. (Reference Köhler, González‐Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020).
You have
Access
In their focal article, Köhler et al. (Reference Köhler, González‐Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020, p. 16) point out that reviewers need to balance their “dual roles as gatekeepers … with the generative and developmental role of peers….” The proposed framework also includes the implicit assumption that all proposed competencies are ultimately compatible with each other. Although the framework features a thoughtful and thorough list of necessary competencies, it may also be regarded as too optimistic. Each of the behaviors that is in line with these competencies can be seen as part of a role that needs to be fulfilled concurrently with other roles and ideally by the same person during the reviewing task. We argue that such roles include inherent role conflicts and role ambiguity between different goals and tasks the reviewer is assigned. For instance, it is not clear if the combination of scouting for statistical quality and at the same time for novel findings is always realistic; or, if socialization into the field can always be reconciled with advancement to the field.
One may differ about which roles should be paramount, and the selection put forward in this framework is arguably a suitable compromise. To some degree, peer review has roles that could be in line with each other, in which case one could use role theory (Biddle, Reference Biddle1986; Ebbers & Wijnberg, Reference Ebbers and Wijnberg2017) to develop a job performance scale to measure the reviewers’ performance (also see Welbourne, Johnson, & Erez, Reference Welbourne, Johnson and Erez1998), an element that would make the suggested framework perhaps more rigorous. However, there is inherent conflict and ambiguity between the different roles in peer review, and tensions around peer review are likely the result from this inherent role conflict. Extensive research has shown that role conflict and ambiguity are negatively related to job performance (Rizzo, House, & Lirtzman, Reference Rizzo, House and Lirtzman1970; Tubre & Collins, Reference Tubre and Collins2000), which likely extends to peer review.
In an effort to make reviews more reliable, Köhler et al. (Reference Köhler, González‐Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020) suggest that reviewers should not alternate between roles and fill all roles simultaneously. While this approach sounds appealing at first glance, it may be difficult to implement in practice. For instance, a reviewer of an article may find only one role particularly suitable for his/her background and expertise. He/she may be encouraging the authors of the article in a certain direction that would potentially lead to a theoretically more rigorous contribution, while in the case of a different article, he/she may point out the many problems in the theoretical approach and thereby disqualify, for instance, the publication of an article.
In what is named “mechanical skills,” Köhler et al. (Reference Köhler, González‐Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020, p. 13) state in relation to commentary provided by reviewers: “So, we need to communicate realistically what needs to be improved and what can be improved in future work or in a next step.” And in terms of “professionalism,” the reviewer needs to be constructive. All these elements would suggest that reviewers should encourage authors in a certain direction. As a potential concern, Köhler et al. point out the issue of reviewers “overstepping their roles” (p. 4) as one of the main flaws of current review practice. This issue is exemplified in the more extreme form of hijacking an article or ghost authoring the final product. It is also acknowledged that the expertise of a reviewer may push authors in a particular direction. Köhler et al. concede that there is “some gray area” (p. 7) but that certain behaviors are clearly unwanted. This latter comment points to the general problem that the proposed framework does not sufficiently address. Role conflict is a gray area, and we suggest to expand the competency framework that is offered to future or current reviewers and assist in overcoming the ambiguities and conflicts inherent to peer review.
Reducing role conflict
Without addressing role conflict in peer review, the competency framework still addresses the various improper behaviors that need to be avoided by all reviewers. In that sense, it is a useful and comprehensive reminder, but it is not necessarily sufficient as a promising way forward in guiding the complex task of peer review. Literature on role conflict suggests several ways to complement this framework. Ashforth and Saks (Reference Ashforth and Saks1996) point out that systematic socialization tactics, such as clearly communicated expectations, may reduce role conflicts. The focal article serves as an important first step in the socialization of new reviewers. Following Floyd and Lane (Reference Floyd and Lane2000), we recommend two additional avenues that may further reduce role conflict: increasing autonomy and increasing formalization.
Reviewers typically already enjoy high autonomy, but role conflicts are not necessarily explicitly addressed. One solution could be that reviewers indicate their own role priorities in the review process when they submit their reviews. Subsequently, the editor weighs the representation of certain roles by different reviewers in order to reach a balanced overall review.
Alternatively, one may increase formalization in the review process. For instance, one could include guidelines for editors regarding how to address problems of role conflict, in which case the solutions may be largely at the discretion of the editor. For instance, editors could prioritize certain roles for a reviewer when they send out a reviewing request. In such a case, the prioritization of different roles is likely informed by the reviewer’s expertise. A possible advantage of this procedure is that editors not only ensure that all necessary roles are covered, but that they also strive to assign nonconflicting roles to reviewers. Although editors may already implicitly or explicitly assign specific roles when they choose reviewers with different expertise, an explicit formalization further assists in reducing possible role conflicts.
Philosophy of science has acknowledged that conflict is central to scientific progress (Kuhn, Reference Kuhn1977). Practically, however, role conflicts may frustrate the review process when they are not recognized and addressed either by guidelines or by formal review processes. Increasing awareness and ideally an explicit focus on the reduction of role conflicts may further enhance the merit of the already useful guidelines set forth by Köhler et al. (Reference Köhler, González‐Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020).