In this commentary, I argue that the important debates in our discipline (e.g., whether and how to replicate) necessarily belong in scientific journals and should not be restricted to the blogosphere or the social media universe. I concede that the issue I am raising (i) is not at odds with Zwaan et al. (in fact, their explicit goal is to move the debate to a journal) and (ii) does not address the main content of their target article (I completely agree with all of Zwaan et al.'s claims about replication), but instead focuses on a seemingly minor point that was mentioned in the target article (i.e., the outlet in which the debate takes place). However, I am convinced that the outlet is of critical relevance here.
First and most important, I believe that most mainstream scientists still read scientific journals more frequently and more intensely than they follow social media. Thus, it is simply more efficient to publish fresh ideas in journals to gain optimal access to “the silent majority” whom authors would like to convince. A perfect example here is the success of the “False-Positive Psychology” article published in Psychological Science (Simmons et al. Reference Simmons, Nelson and Simonsohn2011; see also Simmons et al. Reference Simmons, Nelson and Simonsohn2018). A few additional examples that readily come to mind are the publication of the results of the “Replication Project: Psychology” in Science (Open Science Collaboration 2015), the – regrettably renamed – “Voodoo Correlations” paper in Perspectives on Psychological Science (Vul et al. Reference Vul, Harris, Winkielman and Pashler2009), the “Scientific Utopia” article in Psychological Inquiry (Nosek & Bar-Anan Reference Nosek and Bar-Anan2012), and the mind-boggling “Political Diversity” paper in Behavioral and Brain Sciences (Duarte et al. Reference Duarte, Crawford, Stern, Haidt, Jussim and Tetlock2015).
Of course, it is certainly difficult and all too often very frustrating to try to publish innovative ideas or critiques of established theories in journals because the thorny peer-review process sometimes seems to be abused by established scholars in their roles of reviewers and editors in efforts to block innovations and criticism. By contrast, all ideas can quickly and without filtering be published in blogs, and there have been several additional clever arguments put forward in favor of blogs over journals (e.g., open data, code, and materials, open reviews, no eminence filter, better error correction, and open access; Lakens Reference Lakens2017). On the other hand, established scholars sometimes complain about, for example, a lack of reflection, a lack of peer advice, impulsivity, personalized debates, and personal accusations triggered by the features of social media. Although I believe that the “tone debate” has been largely exaggerated – “Don't dish it out if you can't take it” – there is some evidence that intellectual opponents and especially third parties might be more efficiently convinced if the arguments are presented in a friendly tone. Thus, the more formal and down-to-earth tone used in scientific journals might in fact be helpful for convincing others. Similarly, mainstream journals are, in general, still more highly respected than most social media outlets. Thus, especially more conservative scholars will trust arguments exchanged in journals more than those that come from debates fought out in blogs.
This should by no means be interpreted to mean that blogs and social media do not have their merits in the replication debate and beyond. To the contrary: They are fast, they are subjective, they are mostly short and to-the-point, they may be provocative, and so forth. My argument is instead that the important debates in our discipline (e.g., whether and how to replicate) should not be restricted to these media but should also be published in established mainstream journals. Although such journals are necessarily somewhat slower, they offer another form and style and can potentially present a more elaborated form of the argument. If one mainstream journal rejects your paper, please try another (and so on). There are also newly founded – not yet so well-established – journals such as Collabra, Metapsychology, or Advances in Methods, and Practices in Psychological Science (to name just a few) that might be alternatives in the face of repeated publication failure in more traditional journals.
Taken together, the formal publication of well-crafted and clever articles (e.g., this one on replication in BBS) seems to offer the best and most efficient way to reach a maximal audience and especially to convince as yet undecided individuals to, for example, join the replication movement in order to make replication mainstream, thereby providing one contribution (out of many possible ones) to psychology's renaissance (Nelson et al. Reference Nelson, Simmons and Simonsohn2018).
In this commentary, I argue that the important debates in our discipline (e.g., whether and how to replicate) necessarily belong in scientific journals and should not be restricted to the blogosphere or the social media universe. I concede that the issue I am raising (i) is not at odds with Zwaan et al. (in fact, their explicit goal is to move the debate to a journal) and (ii) does not address the main content of their target article (I completely agree with all of Zwaan et al.'s claims about replication), but instead focuses on a seemingly minor point that was mentioned in the target article (i.e., the outlet in which the debate takes place). However, I am convinced that the outlet is of critical relevance here.
First and most important, I believe that most mainstream scientists still read scientific journals more frequently and more intensely than they follow social media. Thus, it is simply more efficient to publish fresh ideas in journals to gain optimal access to “the silent majority” whom authors would like to convince. A perfect example here is the success of the “False-Positive Psychology” article published in Psychological Science (Simmons et al. Reference Simmons, Nelson and Simonsohn2011; see also Simmons et al. Reference Simmons, Nelson and Simonsohn2018). A few additional examples that readily come to mind are the publication of the results of the “Replication Project: Psychology” in Science (Open Science Collaboration 2015), the – regrettably renamed – “Voodoo Correlations” paper in Perspectives on Psychological Science (Vul et al. Reference Vul, Harris, Winkielman and Pashler2009), the “Scientific Utopia” article in Psychological Inquiry (Nosek & Bar-Anan Reference Nosek and Bar-Anan2012), and the mind-boggling “Political Diversity” paper in Behavioral and Brain Sciences (Duarte et al. Reference Duarte, Crawford, Stern, Haidt, Jussim and Tetlock2015).
Of course, it is certainly difficult and all too often very frustrating to try to publish innovative ideas or critiques of established theories in journals because the thorny peer-review process sometimes seems to be abused by established scholars in their roles of reviewers and editors in efforts to block innovations and criticism. By contrast, all ideas can quickly and without filtering be published in blogs, and there have been several additional clever arguments put forward in favor of blogs over journals (e.g., open data, code, and materials, open reviews, no eminence filter, better error correction, and open access; Lakens Reference Lakens2017). On the other hand, established scholars sometimes complain about, for example, a lack of reflection, a lack of peer advice, impulsivity, personalized debates, and personal accusations triggered by the features of social media. Although I believe that the “tone debate” has been largely exaggerated – “Don't dish it out if you can't take it” – there is some evidence that intellectual opponents and especially third parties might be more efficiently convinced if the arguments are presented in a friendly tone. Thus, the more formal and down-to-earth tone used in scientific journals might in fact be helpful for convincing others. Similarly, mainstream journals are, in general, still more highly respected than most social media outlets. Thus, especially more conservative scholars will trust arguments exchanged in journals more than those that come from debates fought out in blogs.
This should by no means be interpreted to mean that blogs and social media do not have their merits in the replication debate and beyond. To the contrary: They are fast, they are subjective, they are mostly short and to-the-point, they may be provocative, and so forth. My argument is instead that the important debates in our discipline (e.g., whether and how to replicate) should not be restricted to these media but should also be published in established mainstream journals. Although such journals are necessarily somewhat slower, they offer another form and style and can potentially present a more elaborated form of the argument. If one mainstream journal rejects your paper, please try another (and so on). There are also newly founded – not yet so well-established – journals such as Collabra, Metapsychology, or Advances in Methods, and Practices in Psychological Science (to name just a few) that might be alternatives in the face of repeated publication failure in more traditional journals.
Taken together, the formal publication of well-crafted and clever articles (e.g., this one on replication in BBS) seems to offer the best and most efficient way to reach a maximal audience and especially to convince as yet undecided individuals to, for example, join the replication movement in order to make replication mainstream, thereby providing one contribution (out of many possible ones) to psychology's renaissance (Nelson et al. Reference Nelson, Simmons and Simonsohn2018).