Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) address an important and timely concern with scientific research with implications well beyond industrial and organizational (I-O) psychological science. As they describe a comprehensive ecosystem responsible for robust science, they have alluded to the notorious scientist–practitioner gap in our field. In this commentary I propose how that gap relates to the various stakeholders in Grand et al.’s ecosystem, and suggest professional associations take on an elevated role in the communication of science.
Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) describe the consequences and incentives authors face that threaten robust science (e.g., “winning the publishing game”). Similarly, practitioners face constraints related to intellectual property, copyrights, trade secrets, or sensitive company data that may prevent them from participating more fully in the ideal scientist–practitioner model. To realize the goals and propositions Grand et al. have described, we will need to understand and address the motivations of each of the stakeholder groups. Additionally, Grand et al.’s ecosystem extends well beyond I-O psychologists. Ensuring that robust science is a priority among all stakeholder groups will be a serious but not insurmountable challenge. This is an especially prominent issue when we consider the way in which I-O psychology science is communicated to nonscientific audiences.
Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018, p. 17) acknowledge that “. . . there is a need to understand the various stakeholders . . . so that the roles these entities play in promoting and sustaining a healthy science are better understood.” Not only do we need to understand the differences among stakeholders, we should acknowledge that not everyone considered a stakeholder in this ecosystem would agree they have a stake in fostering a healthy science.
For example, the rapid development of hiring and talent assessment products by computer science professionals has, in some instances, completely bypassed the field of I-O psychology. Providers selling data-mining algorithms are approaching the same corporate customers as traditional talent assessment providers. It's unlikely that providers from the tech space identify with the concerns of fostering robust science in the same way the I-O psychology community does (e.g., maximum prediction may supersede consideration of constructs, if they are considered at all). In other instances, these providers are buttressing their solutions with in-house I-O psychology researchers as they mature. However, their primary motivation is to defend and sell a product, not to advance or communicate the science of psychology.
Even among more traditional I-O psychology practitioners, the profit motive sometimes conflicts with the goals of vigorous science. For example, most widely used personality assessment instruments are built around some form of the five-factor model of personality, but each practitioner is obligated to describe and market his or her proprietary model distinctly. To the end user, it may not be transparent that different providers are largely selling measurement of the same basic constructs, which directly conflicts with the principle of openness and transparency. The technical documentation on the assessments may describe the convergent validity with other commonly used instruments, but there will be more individuals learning about psychological science from the practitioner's marketing team than from a technical report.
Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) describe the opaqueness the end user of scientific research faces. Indeed, the gatekeeping of authors, editors, and reviewers prevents the reader from knowing what research questions were not reported or what hypotheses returned null results. However, I argue that the end consumer of research is even further toward the “end” than Grand et al. imply. The individuals who make decisions about what science-based solutions are used in practice (i.e., the people who write the checks) are not likely to read any of our academic journals. Instead, their ongoing education comes in the form of whitepapers, conferences, or otherwise much more attractive presentations from consultants. Whether the consultant is an advisory firm or a direct solution provider, the scientific research consumed by corporate customers is likely to filter through a stakeholder with a profit motive and not necessarily an organization sharing the same goals as the original authors of the research.
Just as editors and reviewers are the critical arbiters of scientific knowledge from author to author, practitioners and professional associations are largely responsible for the practical knowledge consumers have of I-O psychology science. In these two channels of dissemination and communication, there are sometimes conflicting motives, which may hinder the promotion of robust science. I believe there is room for professional associations such as Society for Industrial and Organizational Psychology (SIOP), Academy of Management (AOM), and local affiliates to take a more public and prominent role as an impartial source of truth. As Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) mention, this kind of public education is already in the mission statements of most professional associations and should be promoted. This is not to imply that practitioners are doing a poor job of communicating I-O psychology science. As Grand et al. suggest, when they do it well, practitioners secure a competitive advantage through their dedication to rigorous science.
Some psychologists have suggested that professional organizations that represent practitioners operate more as guilds than as defenders of robust science. The growth of the Association for Psychological Science (APS) from the American Psychological Association (APA) is evidence of this division. However, I do not believe this has to be such a concern for I-O psychologists. Professional societies such as SIOP could be of service to all its members—scientists and practitioners alike—by more actively recognizing good science, and calling out junk science.
The recommendations Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) have for improving the robustness of science from an academic point of view are practical and irrefutable. I think a similar level of emphasis can be dedicated to ensuring that the corporate customer is properly educated as well. Professional associations are uniquely positioned to educate consumer audiences with the benefit of unbiased credibility and with members from the scientific community who can responsibly translate science to a practical audience.
You have
Access
Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) address an important and timely concern with scientific research with implications well beyond industrial and organizational (I-O) psychological science. As they describe a comprehensive ecosystem responsible for robust science, they have alluded to the notorious scientist–practitioner gap in our field. In this commentary I propose how that gap relates to the various stakeholders in Grand et al.’s ecosystem, and suggest professional associations take on an elevated role in the communication of science.
Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) describe the consequences and incentives authors face that threaten robust science (e.g., “winning the publishing game”). Similarly, practitioners face constraints related to intellectual property, copyrights, trade secrets, or sensitive company data that may prevent them from participating more fully in the ideal scientist–practitioner model. To realize the goals and propositions Grand et al. have described, we will need to understand and address the motivations of each of the stakeholder groups. Additionally, Grand et al.’s ecosystem extends well beyond I-O psychologists. Ensuring that robust science is a priority among all stakeholder groups will be a serious but not insurmountable challenge. This is an especially prominent issue when we consider the way in which I-O psychology science is communicated to nonscientific audiences.
Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018, p. 17) acknowledge that “. . . there is a need to understand the various stakeholders . . . so that the roles these entities play in promoting and sustaining a healthy science are better understood.” Not only do we need to understand the differences among stakeholders, we should acknowledge that not everyone considered a stakeholder in this ecosystem would agree they have a stake in fostering a healthy science.
For example, the rapid development of hiring and talent assessment products by computer science professionals has, in some instances, completely bypassed the field of I-O psychology. Providers selling data-mining algorithms are approaching the same corporate customers as traditional talent assessment providers. It's unlikely that providers from the tech space identify with the concerns of fostering robust science in the same way the I-O psychology community does (e.g., maximum prediction may supersede consideration of constructs, if they are considered at all). In other instances, these providers are buttressing their solutions with in-house I-O psychology researchers as they mature. However, their primary motivation is to defend and sell a product, not to advance or communicate the science of psychology.
Even among more traditional I-O psychology practitioners, the profit motive sometimes conflicts with the goals of vigorous science. For example, most widely used personality assessment instruments are built around some form of the five-factor model of personality, but each practitioner is obligated to describe and market his or her proprietary model distinctly. To the end user, it may not be transparent that different providers are largely selling measurement of the same basic constructs, which directly conflicts with the principle of openness and transparency. The technical documentation on the assessments may describe the convergent validity with other commonly used instruments, but there will be more individuals learning about psychological science from the practitioner's marketing team than from a technical report.
Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) describe the opaqueness the end user of scientific research faces. Indeed, the gatekeeping of authors, editors, and reviewers prevents the reader from knowing what research questions were not reported or what hypotheses returned null results. However, I argue that the end consumer of research is even further toward the “end” than Grand et al. imply. The individuals who make decisions about what science-based solutions are used in practice (i.e., the people who write the checks) are not likely to read any of our academic journals. Instead, their ongoing education comes in the form of whitepapers, conferences, or otherwise much more attractive presentations from consultants. Whether the consultant is an advisory firm or a direct solution provider, the scientific research consumed by corporate customers is likely to filter through a stakeholder with a profit motive and not necessarily an organization sharing the same goals as the original authors of the research.
Just as editors and reviewers are the critical arbiters of scientific knowledge from author to author, practitioners and professional associations are largely responsible for the practical knowledge consumers have of I-O psychology science. In these two channels of dissemination and communication, there are sometimes conflicting motives, which may hinder the promotion of robust science. I believe there is room for professional associations such as Society for Industrial and Organizational Psychology (SIOP), Academy of Management (AOM), and local affiliates to take a more public and prominent role as an impartial source of truth. As Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) mention, this kind of public education is already in the mission statements of most professional associations and should be promoted. This is not to imply that practitioners are doing a poor job of communicating I-O psychology science. As Grand et al. suggest, when they do it well, practitioners secure a competitive advantage through their dedication to rigorous science.
Some psychologists have suggested that professional organizations that represent practitioners operate more as guilds than as defenders of robust science. The growth of the Association for Psychological Science (APS) from the American Psychological Association (APA) is evidence of this division. However, I do not believe this has to be such a concern for I-O psychologists. Professional societies such as SIOP could be of service to all its members—scientists and practitioners alike—by more actively recognizing good science, and calling out junk science.
The recommendations Grand et al. (Reference Grand, Rogelberg, Allen, Landis, Reynolds, Scott and Truxillo2018) have for improving the robustness of science from an academic point of view are practical and irrefutable. I think a similar level of emphasis can be dedicated to ensuring that the corporate customer is properly educated as well. Professional associations are uniquely positioned to educate consumer audiences with the benefit of unbiased credibility and with members from the scientific community who can responsibly translate science to a practical audience.