Grand et al. (2018) provide a critical treatment of the need for industrial and organizational (I-O) psychologists to take active steps to ensure that ours is a robust and reliable science of organizations, the spirit of which the present authors wholeheartedly agree. One path toward ensuring that I-O psychology is rightly recognized as a robust science is by embracing the principles of Open Science. We argue that the move toward Open Science is beneficial for all six of the defining principles outlined by Grand et al. Open Science increases transparency, facilitates replication, and contributes to the accumulation and integration of research by increasing accessibility. Each of these factors helps to foster systematic, rigorous scientific practices, which, in turn, can lead to greater insights and understanding of organizational phenomenon through deductive, inductive, and abductive approaches to theory generation and refinement, ensuring that our field remains relevant for the foreseeable future. In this commentary, we highlight several initiatives that already exist to promote Open Science and explore others only alluded to by Grand et al.
The Open Science Framework
Among the many services provided by the Open Science Framework (OSF), one of the most important is the ability to preregister study designs before conducting research (Nosek et al., Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Contestabile2015). Although pre-registrations can vary in detail and comprehensiveness, they generally allow researchers to state explicitly (a) what research questions they are attempting to answer, (b) their specific hypotheses regarding these research questions, (c) the detailed methods by which they will attempt to answer these questions, and (d) the statistical plan they have for analyzing their results (Munafò et al., Reference Munafò, Nosek, Bishop, Button, Chambers, du Sert and Ioannidis2017). By stating these things prior to conducting a study, researchers are able to demonstrate transparency about the process undergone to examine a topic. Preregistration has a long history in I-O psychology, as most graduate programs require students to “preregister” their theses and dissertations through a rigorous review conducted before data collection begins.
As the preregistration process has become more common in recent years, a number of I-O psychology journals (e.g., Journal of Business and Psychology, Journal of Personnel Psychology) now accept registered reports for publication. These reports undergo peer review before the study or studies are conducted, with a focus on study rationale, hypotheses, and proposed design and analyses. Should a registered report be approved during this initial peer review, the manuscript detailing the completed project is generally assured publication in the approving journal if the authors demonstrate that they followed their preregistered design in its entirety and reached reasonable conclusions regarding their findings. Although preregistration may require additional effort on the parts of editors and reviewers, it is our belief that, in the long run, such efforts undertaken earlier in the research process will result in higher quality publications.
In addition to preregistration and registered reports, the OSF also supports archiving of study materials and data from completed studies, which allows for straightforward replications of completed projects. Although many elements of a study may be described in the methods section of an empirical paper, the full scope of the study design has historically been unavailable without requesting such materials from the original authors (Munafò et al., Reference Munafò, Nosek, Bishop, Button, Chambers, du Sert and Ioannidis2017). When these materials are open access through the OSF, research teams can move to replicate findings without consuming the time or resources of the original authors. Additionally, such transparency allows researchers to share effective materials that may be adapted into novel inquiries that build on previous findings.
Like many other services (e.g., ResearchGate, Academia.edu), the OSF facilitates the sharing of manuscripts or published works. However, in contrast to the aforementioned services, the OSF does these things in a fully open-access environment, requiring no subscription, affiliation, or fees to access these materials. By design, the OSF is fully open to researchers and the public alike, and no one entity or person stands to profit financially from this sharing (as is the case with other services). Collectively, this open-access, nonprofit design facilitates robust science with fewer opportunities for bias or conflicts of interest.
In sum, Grand et al.’s (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) recommendations to facilitate robust science in I-O psychology are not being put forth into a vacuum: The structures necessary to fully implement these recommendations have already been created and are simply waiting to be embraced by the I-O psychology community.
Open Science, the Reproducibility Crisis, and Questionable Reporting Practices
Two of the primary motivating factors that have contributed to the ongoing discussion around robust science are the so-named reproducibility crisis (e.g., Baker, Reference Baker2016) and concerns over questionable reporting practices (QRPs)Footnote
1
(Wigboldus & Dotsch, Reference Wigboldus and Dotsch2016; Woo, O'Boyle, & Spector, Reference Woo, O'Boyle and Spector2017). These phenomena are critical hurdles that I-O psychology must overcome to maintain its status as a robust science. We argue that a move toward open science can help to alleviate concerns over the dearth of direct replications reported in the literature and poor communication of research findings.
In science, our principle mechanism for promoting direct replication is the clarity of information provided in the research report (Nosek et al., Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Contestabile2015; Open Science Collaboration, 2015). Conventional norms in published psychological research “encourage” authors to provide an adequate description of the measures, design characteristics, and analyses conducted so that others might replicate a given study. However, even a cursory glance at the organizational literature reveals a maelstrom of opaque descriptions of the methods, instruments, and analyses used. Although we would like to believe that most authors strive to provide sufficient and well-written instructions to allow for direct replication (or, at minimum, conceptual replication when materials are proprietary or methods/samples are difficult and costly to reproduce), one major barrier to full transparency restrains even the most well intentioned authors: article page limits.
We argue that, in the age of online publishing, page limits are an archaic practice that is largely an artifact of the physical printing process (Nosek & Bar-Anan, Reference Nosek and Bar-Anan2012). Open science can help authors to overcome the potential barriers of page limits through one simple fact: Most, if not all, academic journals now maintain an online repository of the articles published in that journal. Shifting from paper to online journals shakes up norms around paper length (Nosek & Bar-Anan, Reference Nosek and Bar-Anan2012). With limits to the number and length of articles largely reduced in online publishing, journals are free to publish direct replications that at one time might have been “deemed ‘not newsworthy’ or a ‘waste of space’” (Nosek, Spies, & Motyl, Reference Nosek, Spies and Motyl2012, p. 617). Note that we are not suggesting that journals abandon the notion of page limits entirely; no reader wants to slog through a meandering manuscript full of unnecessary prose, hyperbole, and circuitous logic. Page limits force authors to be concise in their descriptions. However, there are tradeoffs to be considered, and we would argue that the present standards for publication do not promote the values of transparency and openness sina qua non for robust scientific practice.
Apart from increasing typical page limits for online publications, the prevalence and ease of use of online repositories for study materials and data is of paramount importance for increasing transparency and openness in scientific reporting. Many journals encourage authors to include materials (e.g., tables, syntax) in online supplements, and recent announcements by the American Psychological Association (APA, 2017) and the Association for Psychological Science (APS, 2016) indicate that major publication outlets will be encouraging such supplements. However, few journals require the inclusion of such supplemental materials for the purposes of replication. For instance, journals who do not wish to increase page limits might instead require authors to submit supplemental documentation detailing an extended treatment of the procedures used and analyses conducted (including exploratory analyses not reported in the manuscript) as a condition of publication. Moreover, editors might ask reviewers to comment on these supplemental materials during the review process—a practice that often is left to the discretion of individual reviewers.
Another mechanism for promoting replicability and reducing concerns over QRPs is the move toward publicly archiving data for (pre-)published research. As mentioned, open science encourages and, in some cases, requires sharing data from completed studies. Archiving data not only facilitates direct replication and reanalysis of existing data but can also foster secondary research efforts, such as meta-analyses, or novel research questions and analyses unexplored by the initial study authors (Nosek et al., Reference Nosek, Spies and Motyl2012; Piwowar & Vision, Reference Piwowar and Vision2013).
Undoubtedly, data sharing is one of the most contentious and controversial aspects of the movement toward open science. Some might fear that sharing data will lead to their being “scooped” or otherwise unable to publish certain findings because someone else has done so first. However, to date, there seems to be no evidence of this type of thing occurring (LeBel, Campbell, & Loving, Reference LeBel, Campbell and Loving2017). Yet there do seem to be distinct advantages to publicly archiving data, such as greater numbers of publications arising from a single data set and researchers having both their own manuscripts and their data independently cited (Piwowar & Vision, Reference Piwowar and Vision2013), both of which benefit a researcher's reputation and standing in the field. Although there are cases in which data sharing or open-access data archiving may not be ideal, or even possible (e.g., protecting client confidentiality, dealing with sensitive data, dealing with proprietary data), this concern is not unique to I-O psychology, as clinical science and other areas of applied psychology also face similar concerns (Tackett et al., Reference Tackett, Lilienfeld, Patrick, Johnson, Krueger, Miller and Shrout2017). In such cases where confidentiality concerns arise, alternative means of open data may be needed (e.g., releasing only part of a data set, only sharing data with reviewers or editors). Fortunately, it is likely that the OSF will be expanding to include such data protections in the near future (Tackett et al., Reference Tackett, Lilienfeld, Patrick, Johnson, Krueger, Miller and Shrout2017).
Publishing Open Science
Another important aspect to promoting open science is accessibility to peer-reviewed, open-access journals. Although other fields of psychology have embraced open access publishing, options are more limited for I-O psychology researchers. Notably, the journal Personnel Assessment and Decisions (PAD) is fully peer reviewed and open access. Other journals have open access options that allow authors to pay a fee to add open access for articles that are already accepted. For example, Journal of Business and Psychology (JBP) allows authors to purchase open access rights for their accepted articles (currently $3,000), allowing those articles to be shared freely to all, even to those without subscription access. Currently, out of the 1,202 articles available online at JBP, 21 (1.7%) articles are open access (https://link.springer.com/journal/10869).
PAD and JBP offer a contrast of two different models for peer-reviewed, open-access publishing. Although online publishing has removed many of the previous costs of journal publication (e.g., printing and mailing costs), there are still significant costs for online publishing. These include marketing, editorial assistance, and software for hosting the journal. For JBP and similar journals, the cost of publishing is transferred to the authors who choose to publish their articles as open access. This can create a financial challenge for authors and institutions, as this is an increased cost at a time when many university budgets are being cut. For those who have federally funded grants, the costs of publishing in open access are often built into the grants. Yet for others the money must come from already stretched department and university budgets.Footnote
2
PAD has a different model in which the journal is subsidized by the International Personnel Assessment Council (IPAC) and uses a platform supported by the Bowling Green State University Libraries, ScholarWorks. This subsidy allows for 100% of the peer-reviewed articles in the journal to be published without cost to individual authors, though it does depend on the continued generosity of the supporting organizations.
Peer-reviewed, open-access journals promote open science by allowing free dissemination of articles—an especially important benefit to I-O psychology, where non-academics often find it difficult to access important scientific information due to the prohibitive cost of subscribing to journals or even purchasing individual articles. We applaud the Society for Industrial and Organizational Psychology's (SIOP) Research Access program that allows members to purchase access to the EBSCO database of articles for $50 per year, thus leveraging group pricing to lower the journal cost to members.
Unfortunately, when navigating the open access journal landscape, many bad actors of dubious quality seem to exist solely as profit centers. The phrase academic spam is now used to describe the frequent emails that many of us receive that offer us a chance to publish in a particular journal or special issue, often associated with a hefty personal cost. The wild west of online publishing may unnecessarily scare people away from publishing in journals that exist solely online. The rules, however, for evaluating the quality of an open-access journal are the same as evaluating traditional journals: What is the reputation of the editorial board members, and does the journal publish frequently cited research?
Moving Toward a More Open Science of I-O Psychology
Collectively, I-O psychology as a science must actively pursue the tenants of open science. This will include actions and changes in mindsets on the part of many, if not all, of the stakeholder groups outlined in Grand et al.’s (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) Table 2.
To promote the move toward open science, editors and reviewers must encourage the journals on whose boards they serve to adopt open access options for archiving and sharing data, materials, and manuscripts. Fortunately, many APA (2017) and APS (2016) journals have begun to leverage existing tools for archiving, such as PsyArXiv (https://psyarxiv.com/), an online repository of preprints made available by the OFS. Similarly, editors and reviewers must also encourage authors to make study materials and data available to interested consumers of their research. For this shift to occur, authors, themselves, must recognize the potential benefits of open science practices, such as citations arising from published data archives (Piwowar & Vision, Reference Piwowar and Vision2013).Footnote
3
Journals can also take steps to encourage authors to engage in open science practices. Indeed, a number of APS journals have begun to offer “badges” to acknowledge published articles for which the authors have made study materials and/or data open access (Kidwell et al., Reference Kidwell, Lazarević, Baranski, Hardwicke, Piechowski, Falkenberg and Nosek2016).
As a professional organization, SIOP is uniquely positioned to promote open science practices. In interfacing with various publication outlets for I-O psychology research, SIOP can encourage journals to adopt some, or all, of the Transparency and Openness Promotion (TOP) guidelines set forth by the Center for Open Science (Nosek et al., Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Contestabile2015). Although not all journals may choose to adopt all practices (e.g., pre-registered reports, data sharing), many of the guidelines recommended by this group are easily implemented (e.g., requiring authors to explicitly state within the manuscript whether or not data, materials, and syntax are made available, and how readers can obtain these materials). In addition to lobbying journals to adopt open-science practices, SIOP can take direct steps to work together with journals that offer open access options by sponsoring needs-based funding assistance (as with IPAC's sponsorship of PAD) and by encouraging journals that do not currently support open access to offer open-access alternatives to interested authors. Similarly, SIOP can assert its dedication to open science by collaborating with the OSF, mirroring the initiatives undertaken by APA and APS. Finally, SIOP can lead the way by making its own publications fully open access (e.g., IOP) to the public.
In conclusion, to promote the principles of robust science laid out by Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018), we argue that I-O psychology should adopt the principles of open science, increasing transparency and openness at all stages of the research process, from preregistration to publishing open access to data archiving. This will not be an easy shift in our collective norms as a science, but it is a necessary one. Thankfully, many of the tools necessary to facilitate this culture shift are already available and ready for the taking. Only by changing our collective culture and by teaching and openly discussing the principles of open science can we hope to remain a robust and relevant science of organizations.
Grand et al. (2018) provide a critical treatment of the need for industrial and organizational (I-O) psychologists to take active steps to ensure that ours is a robust and reliable science of organizations, the spirit of which the present authors wholeheartedly agree. One path toward ensuring that I-O psychology is rightly recognized as a robust science is by embracing the principles of Open Science. We argue that the move toward Open Science is beneficial for all six of the defining principles outlined by Grand et al. Open Science increases transparency, facilitates replication, and contributes to the accumulation and integration of research by increasing accessibility. Each of these factors helps to foster systematic, rigorous scientific practices, which, in turn, can lead to greater insights and understanding of organizational phenomenon through deductive, inductive, and abductive approaches to theory generation and refinement, ensuring that our field remains relevant for the foreseeable future. In this commentary, we highlight several initiatives that already exist to promote Open Science and explore others only alluded to by Grand et al.
The Open Science Framework
Among the many services provided by the Open Science Framework (OSF), one of the most important is the ability to preregister study designs before conducting research (Nosek et al., Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Contestabile2015). Although pre-registrations can vary in detail and comprehensiveness, they generally allow researchers to state explicitly (a) what research questions they are attempting to answer, (b) their specific hypotheses regarding these research questions, (c) the detailed methods by which they will attempt to answer these questions, and (d) the statistical plan they have for analyzing their results (Munafò et al., Reference Munafò, Nosek, Bishop, Button, Chambers, du Sert and Ioannidis2017). By stating these things prior to conducting a study, researchers are able to demonstrate transparency about the process undergone to examine a topic. Preregistration has a long history in I-O psychology, as most graduate programs require students to “preregister” their theses and dissertations through a rigorous review conducted before data collection begins.
As the preregistration process has become more common in recent years, a number of I-O psychology journals (e.g., Journal of Business and Psychology, Journal of Personnel Psychology) now accept registered reports for publication. These reports undergo peer review before the study or studies are conducted, with a focus on study rationale, hypotheses, and proposed design and analyses. Should a registered report be approved during this initial peer review, the manuscript detailing the completed project is generally assured publication in the approving journal if the authors demonstrate that they followed their preregistered design in its entirety and reached reasonable conclusions regarding their findings. Although preregistration may require additional effort on the parts of editors and reviewers, it is our belief that, in the long run, such efforts undertaken earlier in the research process will result in higher quality publications.
In addition to preregistration and registered reports, the OSF also supports archiving of study materials and data from completed studies, which allows for straightforward replications of completed projects. Although many elements of a study may be described in the methods section of an empirical paper, the full scope of the study design has historically been unavailable without requesting such materials from the original authors (Munafò et al., Reference Munafò, Nosek, Bishop, Button, Chambers, du Sert and Ioannidis2017). When these materials are open access through the OSF, research teams can move to replicate findings without consuming the time or resources of the original authors. Additionally, such transparency allows researchers to share effective materials that may be adapted into novel inquiries that build on previous findings.
Like many other services (e.g., ResearchGate, Academia.edu), the OSF facilitates the sharing of manuscripts or published works. However, in contrast to the aforementioned services, the OSF does these things in a fully open-access environment, requiring no subscription, affiliation, or fees to access these materials. By design, the OSF is fully open to researchers and the public alike, and no one entity or person stands to profit financially from this sharing (as is the case with other services). Collectively, this open-access, nonprofit design facilitates robust science with fewer opportunities for bias or conflicts of interest.
In sum, Grand et al.’s (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) recommendations to facilitate robust science in I-O psychology are not being put forth into a vacuum: The structures necessary to fully implement these recommendations have already been created and are simply waiting to be embraced by the I-O psychology community.
Open Science, the Reproducibility Crisis, and Questionable Reporting Practices
Two of the primary motivating factors that have contributed to the ongoing discussion around robust science are the so-named reproducibility crisis (e.g., Baker, Reference Baker2016) and concerns over questionable reporting practices (QRPs)Footnote 1 (Wigboldus & Dotsch, Reference Wigboldus and Dotsch2016; Woo, O'Boyle, & Spector, Reference Woo, O'Boyle and Spector2017). These phenomena are critical hurdles that I-O psychology must overcome to maintain its status as a robust science. We argue that a move toward open science can help to alleviate concerns over the dearth of direct replications reported in the literature and poor communication of research findings.
In science, our principle mechanism for promoting direct replication is the clarity of information provided in the research report (Nosek et al., Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Contestabile2015; Open Science Collaboration, 2015). Conventional norms in published psychological research “encourage” authors to provide an adequate description of the measures, design characteristics, and analyses conducted so that others might replicate a given study. However, even a cursory glance at the organizational literature reveals a maelstrom of opaque descriptions of the methods, instruments, and analyses used. Although we would like to believe that most authors strive to provide sufficient and well-written instructions to allow for direct replication (or, at minimum, conceptual replication when materials are proprietary or methods/samples are difficult and costly to reproduce), one major barrier to full transparency restrains even the most well intentioned authors: article page limits.
We argue that, in the age of online publishing, page limits are an archaic practice that is largely an artifact of the physical printing process (Nosek & Bar-Anan, Reference Nosek and Bar-Anan2012). Open science can help authors to overcome the potential barriers of page limits through one simple fact: Most, if not all, academic journals now maintain an online repository of the articles published in that journal. Shifting from paper to online journals shakes up norms around paper length (Nosek & Bar-Anan, Reference Nosek and Bar-Anan2012). With limits to the number and length of articles largely reduced in online publishing, journals are free to publish direct replications that at one time might have been “deemed ‘not newsworthy’ or a ‘waste of space’” (Nosek, Spies, & Motyl, Reference Nosek, Spies and Motyl2012, p. 617). Note that we are not suggesting that journals abandon the notion of page limits entirely; no reader wants to slog through a meandering manuscript full of unnecessary prose, hyperbole, and circuitous logic. Page limits force authors to be concise in their descriptions. However, there are tradeoffs to be considered, and we would argue that the present standards for publication do not promote the values of transparency and openness sina qua non for robust scientific practice.
Apart from increasing typical page limits for online publications, the prevalence and ease of use of online repositories for study materials and data is of paramount importance for increasing transparency and openness in scientific reporting. Many journals encourage authors to include materials (e.g., tables, syntax) in online supplements, and recent announcements by the American Psychological Association (APA, 2017) and the Association for Psychological Science (APS, 2016) indicate that major publication outlets will be encouraging such supplements. However, few journals require the inclusion of such supplemental materials for the purposes of replication. For instance, journals who do not wish to increase page limits might instead require authors to submit supplemental documentation detailing an extended treatment of the procedures used and analyses conducted (including exploratory analyses not reported in the manuscript) as a condition of publication. Moreover, editors might ask reviewers to comment on these supplemental materials during the review process—a practice that often is left to the discretion of individual reviewers.
Another mechanism for promoting replicability and reducing concerns over QRPs is the move toward publicly archiving data for (pre-)published research. As mentioned, open science encourages and, in some cases, requires sharing data from completed studies. Archiving data not only facilitates direct replication and reanalysis of existing data but can also foster secondary research efforts, such as meta-analyses, or novel research questions and analyses unexplored by the initial study authors (Nosek et al., Reference Nosek, Spies and Motyl2012; Piwowar & Vision, Reference Piwowar and Vision2013).
Undoubtedly, data sharing is one of the most contentious and controversial aspects of the movement toward open science. Some might fear that sharing data will lead to their being “scooped” or otherwise unable to publish certain findings because someone else has done so first. However, to date, there seems to be no evidence of this type of thing occurring (LeBel, Campbell, & Loving, Reference LeBel, Campbell and Loving2017). Yet there do seem to be distinct advantages to publicly archiving data, such as greater numbers of publications arising from a single data set and researchers having both their own manuscripts and their data independently cited (Piwowar & Vision, Reference Piwowar and Vision2013), both of which benefit a researcher's reputation and standing in the field. Although there are cases in which data sharing or open-access data archiving may not be ideal, or even possible (e.g., protecting client confidentiality, dealing with sensitive data, dealing with proprietary data), this concern is not unique to I-O psychology, as clinical science and other areas of applied psychology also face similar concerns (Tackett et al., Reference Tackett, Lilienfeld, Patrick, Johnson, Krueger, Miller and Shrout2017). In such cases where confidentiality concerns arise, alternative means of open data may be needed (e.g., releasing only part of a data set, only sharing data with reviewers or editors). Fortunately, it is likely that the OSF will be expanding to include such data protections in the near future (Tackett et al., Reference Tackett, Lilienfeld, Patrick, Johnson, Krueger, Miller and Shrout2017).
Publishing Open Science
Another important aspect to promoting open science is accessibility to peer-reviewed, open-access journals. Although other fields of psychology have embraced open access publishing, options are more limited for I-O psychology researchers. Notably, the journal Personnel Assessment and Decisions (PAD) is fully peer reviewed and open access. Other journals have open access options that allow authors to pay a fee to add open access for articles that are already accepted. For example, Journal of Business and Psychology (JBP) allows authors to purchase open access rights for their accepted articles (currently $3,000), allowing those articles to be shared freely to all, even to those without subscription access. Currently, out of the 1,202 articles available online at JBP, 21 (1.7%) articles are open access (https://link.springer.com/journal/10869).
PAD and JBP offer a contrast of two different models for peer-reviewed, open-access publishing. Although online publishing has removed many of the previous costs of journal publication (e.g., printing and mailing costs), there are still significant costs for online publishing. These include marketing, editorial assistance, and software for hosting the journal. For JBP and similar journals, the cost of publishing is transferred to the authors who choose to publish their articles as open access. This can create a financial challenge for authors and institutions, as this is an increased cost at a time when many university budgets are being cut. For those who have federally funded grants, the costs of publishing in open access are often built into the grants. Yet for others the money must come from already stretched department and university budgets.Footnote 2 PAD has a different model in which the journal is subsidized by the International Personnel Assessment Council (IPAC) and uses a platform supported by the Bowling Green State University Libraries, ScholarWorks. This subsidy allows for 100% of the peer-reviewed articles in the journal to be published without cost to individual authors, though it does depend on the continued generosity of the supporting organizations.
Peer-reviewed, open-access journals promote open science by allowing free dissemination of articles—an especially important benefit to I-O psychology, where non-academics often find it difficult to access important scientific information due to the prohibitive cost of subscribing to journals or even purchasing individual articles. We applaud the Society for Industrial and Organizational Psychology's (SIOP) Research Access program that allows members to purchase access to the EBSCO database of articles for $50 per year, thus leveraging group pricing to lower the journal cost to members.
Unfortunately, when navigating the open access journal landscape, many bad actors of dubious quality seem to exist solely as profit centers. The phrase academic spam is now used to describe the frequent emails that many of us receive that offer us a chance to publish in a particular journal or special issue, often associated with a hefty personal cost. The wild west of online publishing may unnecessarily scare people away from publishing in journals that exist solely online. The rules, however, for evaluating the quality of an open-access journal are the same as evaluating traditional journals: What is the reputation of the editorial board members, and does the journal publish frequently cited research?
Moving Toward a More Open Science of I-O Psychology
Collectively, I-O psychology as a science must actively pursue the tenants of open science. This will include actions and changes in mindsets on the part of many, if not all, of the stakeholder groups outlined in Grand et al.’s (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) Table 2.
To promote the move toward open science, editors and reviewers must encourage the journals on whose boards they serve to adopt open access options for archiving and sharing data, materials, and manuscripts. Fortunately, many APA (2017) and APS (2016) journals have begun to leverage existing tools for archiving, such as PsyArXiv (https://psyarxiv.com/), an online repository of preprints made available by the OFS. Similarly, editors and reviewers must also encourage authors to make study materials and data available to interested consumers of their research. For this shift to occur, authors, themselves, must recognize the potential benefits of open science practices, such as citations arising from published data archives (Piwowar & Vision, Reference Piwowar and Vision2013).Footnote 3 Journals can also take steps to encourage authors to engage in open science practices. Indeed, a number of APS journals have begun to offer “badges” to acknowledge published articles for which the authors have made study materials and/or data open access (Kidwell et al., Reference Kidwell, Lazarević, Baranski, Hardwicke, Piechowski, Falkenberg and Nosek2016).
As a professional organization, SIOP is uniquely positioned to promote open science practices. In interfacing with various publication outlets for I-O psychology research, SIOP can encourage journals to adopt some, or all, of the Transparency and Openness Promotion (TOP) guidelines set forth by the Center for Open Science (Nosek et al., Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Contestabile2015). Although not all journals may choose to adopt all practices (e.g., pre-registered reports, data sharing), many of the guidelines recommended by this group are easily implemented (e.g., requiring authors to explicitly state within the manuscript whether or not data, materials, and syntax are made available, and how readers can obtain these materials). In addition to lobbying journals to adopt open-science practices, SIOP can take direct steps to work together with journals that offer open access options by sponsoring needs-based funding assistance (as with IPAC's sponsorship of PAD) and by encouraging journals that do not currently support open access to offer open-access alternatives to interested authors. Similarly, SIOP can assert its dedication to open science by collaborating with the OSF, mirroring the initiatives undertaken by APA and APS. Finally, SIOP can lead the way by making its own publications fully open access (e.g., IOP) to the public.
In conclusion, to promote the principles of robust science laid out by Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018), we argue that I-O psychology should adopt the principles of open science, increasing transparency and openness at all stages of the research process, from preregistration to publishing open access to data archiving. This will not be an easy shift in our collective norms as a science, but it is a necessary one. Thankfully, many of the tools necessary to facilitate this culture shift are already available and ready for the taking. Only by changing our collective culture and by teaching and openly discussing the principles of open science can we hope to remain a robust and relevant science of organizations.