Zwaan et al.'s otherwise excellent review of issues related to replications paints a rather bleak picture of credit given to researchers involved in replication studies. Even though the status quo is described accurately – that is, replications will not be valued as much as traditional research by hiring and tenure committees – tools and interventions exist that could change this situation (“Rewarding negative results,” 2017). We recently introduced a new approach to incentivize replications and provide credit to researchers performing high-quality replications of seminal studies. Tapping into the long tradition of scientific awards, we have designed a Replication Award that can be implemented by scientific communities, journals, institutes or departments. In addition to providing a detailed protocol on how to effectively solicit submissions, score them, and announce the winner (Gorgolewski et al. Reference Gorgolewski, Nichols, Kennedy, Poline and Poldrack2017b) we have also implemented it within the Organization for Human Brain Mapping (OHBM). The first OHBM Replication Award was presented in June 2017 to Wouter Boekel for his thorough investigation of 17 brain-behaviour relationships (Boekel et al. Reference Boekel, Wagenmakers, Belay, Verhagen, Brown and Forstmann2015). This study found evidence confirming only one of the investigated relationships (correlation between real-world social network size and grey matter volume in the amygdala (Kanai et al. Reference Kanai, Bahrami, Roylance and Rees2012). No evidence for previously reported effects was found for eight relationships, and results were inconclusive for the remaining eight.
The effectiveness of the OHBM Replication Award in the context of the promotion of replications has been evaluated by a community survey (Gorgolewski et al. Reference Gorgolewski, Nichols, Kennedy, Poline and Poldrack2017a). Of 226 respondents who were familiar with the award, 49% declared that it made them more likely to perform replications in the future, and 41% declared that it made them more likely to publish replication studies. Because prizes, awards, and other honours are often listed on curricula vitae and taken into consideration during hiring and tenure decision, we hope that this award will also have a positive influence on the careers of researchers performing replications.
We look forward to more organizations adopting Replication Awards following the lead of OHBM. The key to making replication mainstream is to provide incentives for researchers and to elevate replications to the status of a first class citizen among other scientific outputs. Replication Awards can provide such incentives a form that is already widely used in the context of evaluating careers. In the future, editorial boards and reviewers should adopt more proactive policies to facilitate the publication of replication studies (similar to NeuroImage: Clinical [Fletcher & Grafton Reference Fletcher and Grafton2013]). We also hope that new publishing initiatives such as the one initiated by the OHBM this year will facilitate this change in the future.
Zwaan et al.'s otherwise excellent review of issues related to replications paints a rather bleak picture of credit given to researchers involved in replication studies. Even though the status quo is described accurately – that is, replications will not be valued as much as traditional research by hiring and tenure committees – tools and interventions exist that could change this situation (“Rewarding negative results,” 2017). We recently introduced a new approach to incentivize replications and provide credit to researchers performing high-quality replications of seminal studies. Tapping into the long tradition of scientific awards, we have designed a Replication Award that can be implemented by scientific communities, journals, institutes or departments. In addition to providing a detailed protocol on how to effectively solicit submissions, score them, and announce the winner (Gorgolewski et al. Reference Gorgolewski, Nichols, Kennedy, Poline and Poldrack2017b) we have also implemented it within the Organization for Human Brain Mapping (OHBM). The first OHBM Replication Award was presented in June 2017 to Wouter Boekel for his thorough investigation of 17 brain-behaviour relationships (Boekel et al. Reference Boekel, Wagenmakers, Belay, Verhagen, Brown and Forstmann2015). This study found evidence confirming only one of the investigated relationships (correlation between real-world social network size and grey matter volume in the amygdala (Kanai et al. Reference Kanai, Bahrami, Roylance and Rees2012). No evidence for previously reported effects was found for eight relationships, and results were inconclusive for the remaining eight.
The effectiveness of the OHBM Replication Award in the context of the promotion of replications has been evaluated by a community survey (Gorgolewski et al. Reference Gorgolewski, Nichols, Kennedy, Poline and Poldrack2017a). Of 226 respondents who were familiar with the award, 49% declared that it made them more likely to perform replications in the future, and 41% declared that it made them more likely to publish replication studies. Because prizes, awards, and other honours are often listed on curricula vitae and taken into consideration during hiring and tenure decision, we hope that this award will also have a positive influence on the careers of researchers performing replications.
We look forward to more organizations adopting Replication Awards following the lead of OHBM. The key to making replication mainstream is to provide incentives for researchers and to elevate replications to the status of a first class citizen among other scientific outputs. Replication Awards can provide such incentives a form that is already widely used in the context of evaluating careers. In the future, editorial boards and reviewers should adopt more proactive policies to facilitate the publication of replication studies (similar to NeuroImage: Clinical [Fletcher & Grafton Reference Fletcher and Grafton2013]). We also hope that new publishing initiatives such as the one initiated by the OHBM this year will facilitate this change in the future.