Peer review may be one of the most important duties in our profession (Grand et al., Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018; Köhler et al., Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020). Many of the suggestions made in the focal article by Köhler et al. (Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020) would indeed benefit the peer review process and the science and practice of industrial and organizational psychology. However, if reviewers were to act on these suggestions, it is possible that they would need to expend a great degree of additional time, effort, and resources to successfully complete many of them (i.e., seeking out additional information when expertise is lacking, giving actionable and developmental feedback). Reviewers have multiple goals and responsibilities they must manage in addition to engaging with the peer review process. Indeed, they must often make tradeoffs between writing thorough reviews and completing other highly rewarded work (i.e., publishing one’s own research). To encourage others to adopt the suggestions presented by Köhler et al. (Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020), I posit that incentives are a necessary consideration when soliciting high quality reviews using the competency framework. To support this assertion, I draw from previous research on motivation and provide suggestions for potential incentives. I then close with the suggestion to solicit responses from a wide variety of academics and practitioners across vocational areas and at different levels of their careers to determine which factors would incentivize them to serve as reviewers in general and which ones would incentivize them if they were to use tenants outlined in the competency framework.
Reviewing may be one of a professional’s many goals
The degree to which professionals would be willing to adopt the proposed framework may depend on their ability and desire to successfully complete the goals implicitly outlined therein. Assuming there is interest in using the framework, reviewers would need to complete goals related to thorough and developmental reviewing as they commit to and regulate between other goals that may be in competition for their resources. For academics, this may involve integrating these increased reviewing responsibilities with their research, teaching and advising responsibilities. For practitioners, this may involve integrating these responsibilities in with their project goals, expectations from clients, and responsibilities to their managers and direct reports. For both, the goal of getting involved with a more extensive and time-consuming peer review process may come into conflict with goals related to career advancement or work/life balance. This tension is best exemplified by the Multiple Goal Pursuit Model (MGPM; Vancouver, Weinhardt, & Schmidt, Reference Vancouver, Weinhardt and Schmidt2010). When individuals must choose between competing goals, the MGPM posits that the expected utility of each of the competing goals is considered. Expected utility is a function of the degree to which an individual subjectively places value on the goal (e.g., valence) and the degree to which an individual feels he/she has the resources needed to achieve the goal (e.g., expectancy; Vancouver et al., Reference Vancouver, Weinhardt and Schmidt2010). Even if professionals themselves place high expected utility on completing reviews as outlined by the competency framework, the organizations and universities in which they work often do not reward this behavior. Indeed, many of the rewards outlined in the focal article were geared toward self-development of the reviewer with almost no mention of tangible rewards (Annesley, Reference Annesley2012; Köhler et al., Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020). Because reviewing is not highly rewarded in general, adding additional labor to the reviewing process may result in reluctance to adopt the proposed framework.
While Köhler et al. (Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020) provide compelling suggestions, many of them would increase the amount of time and effort allocated to each review. As the authors aptly mention, there are few formal incentives for academics to participate in the peer review process as it is currently, and far fewer (if any) for practitioners. However, when time and resources must be allocated toward multiple goals, there is evidence that individuals will put their time and energy toward pursuing the rewarded goal (Schmidt & DeShon, Reference Schmidt and DeShon2007). Thus, I propose that incentivizing reviewing may encourage the adoption of this framework and higher quality reviews overall. In the next section, I highlight potential incentives for reviewers. My hope is that these suggestions will spark additional ideas regarding ways to incentivize peer reviewing, and specifically, adoption of the tenants outlined in the competency framework.
Potential incentives
Reviewers’ internal motivation
It is highly possible that some reviewers will be internally motivated to use the framework and meet the goals outlined therein (Deci & Ryan, Reference Deci and Ryan2000; Ryan & Deci, Reference Ryan and Deci2000). They may do this because (a) they find the work to be fun and enjoyable (intrinsic motivation), (b) they find the work to be personally important to them (identified regulation), or (c) they find that doing the work is important to the way they view themselves or important to their beliefs (integrated regulation). The current system functions largely in part due to individuals reviewing because they are internally motivated to do so. Particularly in the absence of implementing other incentives, internal motivation could be key for academics and practitioners adopting the competency framework.
Acquiring additional knowledge and skills during the reviewing process
Some individuals may be motivated to adopt the new reviewing framework for the purpose of self-development and gaining mastery over other potential knowledge and skills including: (a) giving effective, developmental feedback, (b) learning additional content knowledge, and (c) learning additional analytic capabilities. To best meet some of the goals outlined in the competency framework, reviewers are encouraged to seek additional information beyond the article under their review (i.e., examining the original source of the measures) and provide constructive feedback using a supportive tone. Broadening one’s content knowledge and analytic repertoire could enrich the work of both academics and practitioners. In addition, learning to provide developmental feedback well can also serve individuals in both lines of work, as professionals are often expected to give feedback to others (i.e., graduate students, direct reports, colleagues). Thus, adopting the framework may be particularly valuable for individuals hoping to engage in self-development in these areas.
Including reviewing experience in hiring and promotion decisions
Beyond reviewing awards and lines on one’s CV, there are few, if any, incentives for engaging in reviewing for the purpose of career advancement. When hiring for tenure track positions or promoting assistant professors to associate professors, the quantity and quality of one’s reviews are hardly ever examined or factored in. This is not only a disservice to the peer review process, but to the field more broadly. Providing developmental reviews, as outlined by the competency framework, can help train academic reviewers to give constructive feedback, which is key for a variety of functions that faculty complete (e.g., advising graduate students, grading papers). This valuable skillset should be highly sought after by hiring committees in institutions of higher learning. By weighting reviewing more heavily in the hiring and promotion processes of university systems, it directly signals that the work is important, and it will encourage applicants to pursue said work. One way to assess this in potential job candidates may include requiring them to list and detail their previous experience with reviewing for publications, conferences, and grants. Some journals are even partnering with services that keep track of peer review contributions in a format that allows reviewers to easily present this information to potential employers (Johnston, Reference Johnston2015). Although it is possible that some non-academic organizations could include reviewing work in their hiring and promotion processes, this suggestion is geared more toward professionals in academic settings.
Including reviewing work in grant award decisions
Similar to my previous suggestion, it could be helpful to factor previous peer reviewing work into the grant selection process. Asking grant applicants to provide information about their professional service in this capacity and weighting it highly in the selection process once again signals that this work is important and valued. Professionals, particularly academics, may be more willing to adopt the goals outlined in the competency framework if funding sources were to consider the quantity and quality of one’s reviews as a factor in the funding decision.
Subsidize society dues or conference fees for reviewers
Subsidizing professional society dues or subsidizing conference attendance fees for high performing reviewers may be a worthwhile incentive that directly assists the careers of burgeoning academics and practitioners. Indeed, these incentives may be most influential for individuals earlier in their careers or in tenuously funded positions. Covering a portion of society dues may allow reviewers to have access to research articles and discounted rates for professional meetings among other professional benefits. Similarly, covering a portion of conference fees rewards reviewers by allowing them to network with others in the field and view the latest research without having to pay to attend. Providing practical incentives such as these may encourage more reviewers to adopt the proposed competency framework.
Further examination of incentives for peer review
Although I have provided suggestions for incentives that may encourage others to adopt the competency framework, I believe it would be helpful to seek additional feedback from a broader array of professionals. Rather than speculate about what may motivate reviewers to adopt the new framework, it may be helpful to collect information from a wide subsample of individuals who are currently involved in our professional societies (e.g., SIOP, AOM) to examine the variety of ways in which individuals in different positions would be incentivized to participate in peer review process.
I recommend collecting responses from experienced reviewers, newer reviewers, graduate students, and practitioners at various levels their careers to determine (a) the degree to which they are motivated to engage in peer reviewing, (b) why they would (or would not) engage in it, (c) the degree to which they would be open to adopting the competency framework when reviewing future work, and (d) whether there are other incentives they would be interested in for serving as peer reviewers. I believe it will be particularly important to solicit information regarding what will motivate practitioners to engage with the peer review process. As the authors acknowledge, outside of intrinsic motivation or mastery over additional content areas, there are few if any incentives for practitioners to engage in the process. Directly asking for practitioner input regarding rewards and incentives may be a helpful next step.
Conclusion
I stand in agreement with Köhler et al. (Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020) that peer reviewing is arguably one of the most important activities in our field. The authors make strong suggestions for improving the process, and I believe their suggestions can be bolstered through better incentivizing the peer review process. I have provided a few suggestions for incentives that may encourage people to adopt the framework presented in the focal article, and I invite future, empirical investigation of incentive structures for the peer review process, both within our science and across scientific fields.
You have
Access
Peer review may be one of the most important duties in our profession (Grand et al., Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018; Köhler et al., Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020). Many of the suggestions made in the focal article by Köhler et al. (Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020) would indeed benefit the peer review process and the science and practice of industrial and organizational psychology. However, if reviewers were to act on these suggestions, it is possible that they would need to expend a great degree of additional time, effort, and resources to successfully complete many of them (i.e., seeking out additional information when expertise is lacking, giving actionable and developmental feedback). Reviewers have multiple goals and responsibilities they must manage in addition to engaging with the peer review process. Indeed, they must often make tradeoffs between writing thorough reviews and completing other highly rewarded work (i.e., publishing one’s own research). To encourage others to adopt the suggestions presented by Köhler et al. (Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020), I posit that incentives are a necessary consideration when soliciting high quality reviews using the competency framework. To support this assertion, I draw from previous research on motivation and provide suggestions for potential incentives. I then close with the suggestion to solicit responses from a wide variety of academics and practitioners across vocational areas and at different levels of their careers to determine which factors would incentivize them to serve as reviewers in general and which ones would incentivize them if they were to use tenants outlined in the competency framework.
Reviewing may be one of a professional’s many goals
The degree to which professionals would be willing to adopt the proposed framework may depend on their ability and desire to successfully complete the goals implicitly outlined therein. Assuming there is interest in using the framework, reviewers would need to complete goals related to thorough and developmental reviewing as they commit to and regulate between other goals that may be in competition for their resources. For academics, this may involve integrating these increased reviewing responsibilities with their research, teaching and advising responsibilities. For practitioners, this may involve integrating these responsibilities in with their project goals, expectations from clients, and responsibilities to their managers and direct reports. For both, the goal of getting involved with a more extensive and time-consuming peer review process may come into conflict with goals related to career advancement or work/life balance. This tension is best exemplified by the Multiple Goal Pursuit Model (MGPM; Vancouver, Weinhardt, & Schmidt, Reference Vancouver, Weinhardt and Schmidt2010). When individuals must choose between competing goals, the MGPM posits that the expected utility of each of the competing goals is considered. Expected utility is a function of the degree to which an individual subjectively places value on the goal (e.g., valence) and the degree to which an individual feels he/she has the resources needed to achieve the goal (e.g., expectancy; Vancouver et al., Reference Vancouver, Weinhardt and Schmidt2010). Even if professionals themselves place high expected utility on completing reviews as outlined by the competency framework, the organizations and universities in which they work often do not reward this behavior. Indeed, many of the rewards outlined in the focal article were geared toward self-development of the reviewer with almost no mention of tangible rewards (Annesley, Reference Annesley2012; Köhler et al., Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020). Because reviewing is not highly rewarded in general, adding additional labor to the reviewing process may result in reluctance to adopt the proposed framework.
While Köhler et al. (Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020) provide compelling suggestions, many of them would increase the amount of time and effort allocated to each review. As the authors aptly mention, there are few formal incentives for academics to participate in the peer review process as it is currently, and far fewer (if any) for practitioners. However, when time and resources must be allocated toward multiple goals, there is evidence that individuals will put their time and energy toward pursuing the rewarded goal (Schmidt & DeShon, Reference Schmidt and DeShon2007). Thus, I propose that incentivizing reviewing may encourage the adoption of this framework and higher quality reviews overall. In the next section, I highlight potential incentives for reviewers. My hope is that these suggestions will spark additional ideas regarding ways to incentivize peer reviewing, and specifically, adoption of the tenants outlined in the competency framework.
Potential incentives
Reviewers’ internal motivation
It is highly possible that some reviewers will be internally motivated to use the framework and meet the goals outlined therein (Deci & Ryan, Reference Deci and Ryan2000; Ryan & Deci, Reference Ryan and Deci2000). They may do this because (a) they find the work to be fun and enjoyable (intrinsic motivation), (b) they find the work to be personally important to them (identified regulation), or (c) they find that doing the work is important to the way they view themselves or important to their beliefs (integrated regulation). The current system functions largely in part due to individuals reviewing because they are internally motivated to do so. Particularly in the absence of implementing other incentives, internal motivation could be key for academics and practitioners adopting the competency framework.
Acquiring additional knowledge and skills during the reviewing process
Some individuals may be motivated to adopt the new reviewing framework for the purpose of self-development and gaining mastery over other potential knowledge and skills including: (a) giving effective, developmental feedback, (b) learning additional content knowledge, and (c) learning additional analytic capabilities. To best meet some of the goals outlined in the competency framework, reviewers are encouraged to seek additional information beyond the article under their review (i.e., examining the original source of the measures) and provide constructive feedback using a supportive tone. Broadening one’s content knowledge and analytic repertoire could enrich the work of both academics and practitioners. In addition, learning to provide developmental feedback well can also serve individuals in both lines of work, as professionals are often expected to give feedback to others (i.e., graduate students, direct reports, colleagues). Thus, adopting the framework may be particularly valuable for individuals hoping to engage in self-development in these areas.
Including reviewing experience in hiring and promotion decisions
Beyond reviewing awards and lines on one’s CV, there are few, if any, incentives for engaging in reviewing for the purpose of career advancement. When hiring for tenure track positions or promoting assistant professors to associate professors, the quantity and quality of one’s reviews are hardly ever examined or factored in. This is not only a disservice to the peer review process, but to the field more broadly. Providing developmental reviews, as outlined by the competency framework, can help train academic reviewers to give constructive feedback, which is key for a variety of functions that faculty complete (e.g., advising graduate students, grading papers). This valuable skillset should be highly sought after by hiring committees in institutions of higher learning. By weighting reviewing more heavily in the hiring and promotion processes of university systems, it directly signals that the work is important, and it will encourage applicants to pursue said work. One way to assess this in potential job candidates may include requiring them to list and detail their previous experience with reviewing for publications, conferences, and grants. Some journals are even partnering with services that keep track of peer review contributions in a format that allows reviewers to easily present this information to potential employers (Johnston, Reference Johnston2015). Although it is possible that some non-academic organizations could include reviewing work in their hiring and promotion processes, this suggestion is geared more toward professionals in academic settings.
Including reviewing work in grant award decisions
Similar to my previous suggestion, it could be helpful to factor previous peer reviewing work into the grant selection process. Asking grant applicants to provide information about their professional service in this capacity and weighting it highly in the selection process once again signals that this work is important and valued. Professionals, particularly academics, may be more willing to adopt the goals outlined in the competency framework if funding sources were to consider the quantity and quality of one’s reviews as a factor in the funding decision.
Subsidize society dues or conference fees for reviewers
Subsidizing professional society dues or subsidizing conference attendance fees for high performing reviewers may be a worthwhile incentive that directly assists the careers of burgeoning academics and practitioners. Indeed, these incentives may be most influential for individuals earlier in their careers or in tenuously funded positions. Covering a portion of society dues may allow reviewers to have access to research articles and discounted rates for professional meetings among other professional benefits. Similarly, covering a portion of conference fees rewards reviewers by allowing them to network with others in the field and view the latest research without having to pay to attend. Providing practical incentives such as these may encourage more reviewers to adopt the proposed competency framework.
Further examination of incentives for peer review
Although I have provided suggestions for incentives that may encourage others to adopt the competency framework, I believe it would be helpful to seek additional feedback from a broader array of professionals. Rather than speculate about what may motivate reviewers to adopt the new framework, it may be helpful to collect information from a wide subsample of individuals who are currently involved in our professional societies (e.g., SIOP, AOM) to examine the variety of ways in which individuals in different positions would be incentivized to participate in peer review process.
I recommend collecting responses from experienced reviewers, newer reviewers, graduate students, and practitioners at various levels their careers to determine (a) the degree to which they are motivated to engage in peer reviewing, (b) why they would (or would not) engage in it, (c) the degree to which they would be open to adopting the competency framework when reviewing future work, and (d) whether there are other incentives they would be interested in for serving as peer reviewers. I believe it will be particularly important to solicit information regarding what will motivate practitioners to engage with the peer review process. As the authors acknowledge, outside of intrinsic motivation or mastery over additional content areas, there are few if any incentives for practitioners to engage in the process. Directly asking for practitioner input regarding rewards and incentives may be a helpful next step.
Conclusion
I stand in agreement with Köhler et al. (Reference Köhler, González-Morales, Banks, O’Boyle, Allen, Sinha and Gulick2020) that peer reviewing is arguably one of the most important activities in our field. The authors make strong suggestions for improving the process, and I believe their suggestions can be bolstered through better incentivizing the peer review process. I have provided a few suggestions for incentives that may encourage people to adopt the framework presented in the focal article, and I invite future, empirical investigation of incentive structures for the peer review process, both within our science and across scientific fields.