Idiosyncratic empirical observations may represent interesting results, but if they do not occur in more than a few instances, they do not advance the frontiers of knowledge (Helfat, Reference Helfat2016). This is why it is imperative for scientific progress that academic findings unveil stable and repeatable patterns of reality rather than idiosyncrasies. Hence, reproducibility forms the cornerstone of scientific progress. Given this significance, it is worrying that studies repeatedly find the reproducibility of published scientific findings in some academic disciplines are as low as 15 to 30 percent (Begley & Ellis, Reference Begley and Ellis2012; Camerer et al., 2018; Ioannidis, Reference Ioannidis2005; Prinz, Schlange, & Asadullah, Reference Prinz, Schlange and Asadullah2011), leading to serious questions regarding the credibility of science.Footnote 1 To address this, researchers have recently begun to investigate questionable research practices (QRPs) as an important cause of low reproducibility, with the aim of understanding the emergence of QRPs and how to identify and avoid such practices (Aguinis, Cascio, & Ramani, Reference Aguinis, Cascio and Ramani2017; Goldfarb & King, Reference Goldfarb and King2016; Hollenbeck & Wright, Reference Hollenbeck and Wright2017).Footnote 2 Fiedler and Schwarz (Reference Fiedler and Schwarz2015) identify various forms of QRPs, such as neglecting to report all dependent measures, deciding to collect more data after results are found, p-hacking, failing to clarify the context, manipulating results, and falsifying data.
However, even if many forms of QRP are indeed a potential threat to the credibility of science, aside from clear misconduct, there are nevertheless behaviours that lie in the grey area of more or less acceptable practice (Butler, Delaney, & Spoelstra, Reference Butler, Delaney and Spoelstra2017). While clear misconduct is assumed to be relatively rare, practices in the grey area seem to be much more prevalent (Steneck, Reference Steneck2006). In such cases, researchers either do not believe they are doing anything wrong, or they regard their actions as trivial, and possibly excusable, poor practice. In this regard, Simonsohn, Simmons, and Nelson (Reference Simonsohn, Simmons and Nelson2015) argue that QRPs are also used by honest researchers when they face uncertainty about which method to deploy. To overcome this uncertainty, they may utilise statistical approaches that lead to better—that is, more publishable—results.
So far, there have only been a few attempts to investigate QRPs from a normative point of view to understand the point at which an acceptable practice, like applying multiple statistical approaches or reformulating models and hypotheses after analysis, becomes misconduct (Sterba, Reference Sterba2006). The current literature is generally methodical in approach, using techniques to uncover existing QRPs but neglecting, to a large degree, normative statements (Aguinis et al., Reference Aguinis, Cascio and Ramani2017; Allen, Ball, & Smith, Reference Allen, Ball and Smith2011; Anderson, Reference Anderson2007; Banks, Rogelberg, Woznyj, Landis, & Rupp, Reference Banks, Rogelberg, Woznyj, Landis and Rupp2016b; Butler et al., Reference Butler, Delaney and Spoelstra2017; Fiedler & Schwarz, Reference Fiedler and Schwarz2015; John, Loewenstein, & Prelec, Reference John, Loewenstein and Prelec2012; Murphy & Aguinis, Reference Murphy and Aguinis2019). The few normative approaches rely on universal science norms to explain how a scientist should behave rather than detailing the grey area of QRPs (Anderson, Martinson, & Vries, Reference Anderson, Martinson and Vries2007).
Although studies addressing QRPs make assumptions about the normative qualities of QRPs, the ethical foundations of these qualifying statements remain vague or implicit. In this regard, Fiedler and Schwarz (Reference Fiedler and Schwarz2015: 7) emphasise that “assessing scientific practice and even more so efforts to improve adherence to norms of good practice are important.” While this is undoubtedly true, little is known about the norms of good practice and how they should guide an individual researcher if they are lacking a reliable and comprehensible theoretical foundation. Everyone who claims questionable data treatment exists must have at least some normative standards in mind when explaining how an actual practice deviates. What the discussion fails to do is make these assumptions explicit and subject to examination.
The use of vague normative grounds has severe implications for progress in dealing with QRPs. Today, it is mostly unclear when a practice that emerges from a researcher’s freedom to explore multiple analytical strategies crosses over to become one that is harmful and morally unjustifiable. This lack of clarity adds to a researcher’s difficulty in drawing a precise line between acceptable and unacceptable research practices. Moreover, most current discussions assume a binary distinction among research practices; that is, there are bad practices and good practices but not much in between. However, this simple, dualistic view, where only good researchers and crooked researchers exist, neither reflects reality nor allows for deeper debate on the normative quality of data-analysis practices in quantitative science research (Pybus, Reference Pybus1983). This is why recommendations to make scientists more accountable by making data publicly available (Sijtsma, Reference Sijtsma2016), replicating data regularly (Schmidt, Reference Schmidt2009), improving statistical knowledge (Wagenmakers, Reference Wagenmakers2007), or creating open and collegial environment and training (Anderson, Reference Anderson2007) have not produced the desired results.
In this article, we argue that the moral quality of a researcher’s practice is complex and depends on multiple and partly conflicting norms, such as the freedom of science, professional standards, and norms in an academic field, as well as responsibility towards those trusting and relying on published results. While there is extensive literature on QRPs generally, there are few insights that help to erase the confusion and vagueness that exist around a morally justifiable practice that becomes a QRP.Footnote 3 This is why the current debate can benefit vastly by specifying the normative foundations and ethical principles on which all data collection, analysis, and presentation should be built. As we will show, it is not enough to simply refer to professional standards since these need to be interpreted by the individual researcher in a given context and situation (Anderson, Reference Anderson2007). Instead, it is the interpretation that needs normative grounding. Hence, this normative grounding of QRPs in ethical theory is the main contribution of this article.
In what follows, we argue that the answer to the question of where to draw the line between justifiable and unjustifiable research practice lies in both the moral limits to a researcher’s freedom and the structures of scientific discipline. Accordingly, our research questions are:
RQ1: What are the guiding moral principles of an individual researcher’s practices?
RQ2: How do the collective norms and rules of an academic field relate to the normative quality of one’s research practices?
Our article contributes to knowledge in this field by developing deontology-based reasoning for how to define QRPs from an ethical standpoint. Our main contribution is that we add a normative dimension to a mainly technical discussion on QRPs and its impact on a scientific discipline. Specifically, we contribute to the discussion on the ethics of science production by providing a framework with three properties. First, we provide a stable normative ground rooted in discourse to evaluate concrete research practices. Our normative reasoning pays attention to the complexity of everyday research by providing a structured way to assess a particular research practice, such as p-hacking. Second, we demonstrate that progress in dealing with concerns over questionable practices requires a normative foundation for the rules and structures that can be exclusively rooted neither in individual action nor at the collective level. We will provide reasoning that pays attention to the interdependencies between individuals and the system in which they are operating. Third, we are able to address academic freedom in a way that limits a researcher’s action while simultaneously ensuring the individual researcher benefits from this limitation. We provide a currently absent theoretical perspective from which to reflect on common practices and identify critical actions. Overall, this article intends to build a foundation for further developing general guidelines and the eventual establishment of a universal code of conduct.
ETHICAL THEORY AND THE CURRENT DEBATE ON QUESTIONABLE RESEARCH PRACTICES
Brady and Dunn (Reference Brady and Dunn1995) observe that the difficulty in applied ethics is the acquisition of a stable theoretical basis. The authors criticise a theoretical ‘free-for-all’ mentality in ethical theorising, dominated by eclecticism instead of secured by a common foundation. While this is true for many debates on business ethics, it is also true for the discussion on ethics and science production. However, speaking of eclecticism assumes, at least to a certain degree, that instead of a single paradigm or set of assumptions, argumentation draws upon multiple theories, styles, or ideas to gain complementary insights into a subject. But in the current debate on QRPs, even arbitrary or eclectic application of normative theories are often not observable. While some authors build their arguments on ethics theories (Altman, Reference Altman1980; Fiedler & Schwarz, Reference Fiedler and Schwarz2015; Steneck, Reference Steneck2006), others do not provide their normative reasoning (Aguinis et al., Reference Aguinis, Cascio and Ramani2017) or even their opinion in lieu of ethical theorising. Some put forward data treating behaviours as a technical matter, highlighting the role of editorials rather than an explicitly moral issue (Butler et al., Reference Butler, Delaney and Spoelstra2017). Hence, we observe two patterns of dealing with theoretical foundations of QRPs: the implicit or explicit avoidance of ethical grounding; and eclecticism.
An absence of ethical foundations is observable, especially in quantitative-driven sciences, which are likely to adopt a research frame that tries to explain or substantiate the phenomena of behaviour and action on positivistic grounds. Actions are commonly understood as the conscious choice of means to realise intended aims of the action. The normative quality then depends on the reasons that guide someone’s actions (Habermas, Reference Habermas1984). However, the “reasons cannot be observed objectively or directly ‘from outside’ and cannot be examined in the sense of a positivistic research paradigm” (Scherer & Palazzo, Reference Scherer and Palazzo2007: 1104). This is why a purely positivistic approach to QRPs is not adequate for understanding the phenomenon.
Yet, eclecticism also does not provide stable theoretical foundations for science ethics; the dominant interest in the field seems inclined towards building a diversity of perspectives, as opposed to identifying a common core of theory (Brady & Dunn, Reference Brady and Dunn1995). Such eclecticism bears the threat of an ethical ‘anything goes’ in data treatment, since it provides leeway for choosing the ethics reasoning that justifies one’s practices. In this regard, Allen et al. (Reference Allen, Ball and Smith2011) observe that non-tenured researchers tend to be less strict on QRPs than tenured scientists. This raises the question of whether interpreting norms and standards less strictly allows the satisfaction of ‘incentive-type’ structures in science, such as ‘publish or perish’ (Clapham, Reference Clapham2005), because less strict norms make it easier to reach personal goals. Other studies show researchers justify their questionable practices with regard to editors’ requests for reformulating hypotheses or research models (Butler et al., Reference Butler, Delaney and Spoelstra2017; Fiedler & Schwarz, Reference Fiedler and Schwarz2015). In need of ethical guidance, institutional actors at a collective level, such as editors and journals, become moral institutions with the power to justify QRPs. This is worrying because dropping variables in response to an editorial request does not automatically establish legitimacy among the wider scientific community. Editors and journals may be responsible for why certain publishing norms are in place. However, their practices are—as with the practices of every individual researcher—subject to adherence to good moral standards. The fact that a practice is normal or commonplace does not automatically establish its moral legitimacy. Again, a clear ethical foundation would help to disentangle norms and standards from actors and further specify their role in a scientific community.
Further, complying with academic standards and norms depends not only on one’s personal beliefs but also on expected reactions of peers (Böckenholt & van der Heijden, Reference Böckenholt and van der Heijden2007; Cialdini, Reference Cialdini2007). In a study on the prevalence of QRPs, John et al. (Reference John, Loewenstein and Prelec2012) analyse the engagement in and awareness of researchers regarding different kinds of doubtful behaviour. They show that researchers observe QRPs among their peers in nearly all institutions and at a significant level. Additionally, researchers assume a “surprisingly high percentage” (John et al., Reference John, Loewenstein and Prelec2012: 530) of misconduct within their respective fields as a whole, and also admit their own engagement in such practices. This indicates the potential of serious damage to a scientific field if the peer community falls short of providing an ethical foundation, and instead serves as a (bad) example for justifying science practices. Again, the actual behaviour of peers cannot justify a particular practice, and hence does not provide reliable and stable ethical grounds.
Nevertheless, there are also problematic tendencies from the other extreme position towards standards in sciences. Attempts to secure science are also threatened by calls for too narrow descriptive norms, which subject good scientific practices to the highest level of scientific scrutiny. This may facilitate ‘black-and-white’ thinking, where researchers follow either a norm (intended for good science) or are already committing fraud. As Fiedler and Schwarz (Reference Fiedler and Schwarz2015) argue, this leads to an ironic situation in which almost everybody violates the norms of good scientific practice (Cialdini, Reference Cialdini2007).
In light of these developments and the specific absence of a sound and reliable theoretical basis for addressing QRPs, in what follows we unfold a discussion based on discourse ethics. We will demonstrate that the discourse provides a sound foundation that is free from pick-and-choose norms or standards, as well as from reference to the behaviour of peers and other actors in the community. From this perspective, research practices can be evaluated on stable normative principles. We further point out how our theoretical foundation may inspire moving on in the debate on QRPs.
QUESTIONABLE RESEARCH PRACTICES
As discussed earlier, QRPs are practices embodied in science production; they are difficult to justify as they make the produced science less reproducible and are, ultimately, potential threats to science. Within the class of QRPs, a significant degree of weak reproducibility is blamed on a practice called p-hacking, where the p refers to the p-value, a common index for the strength of evidence (Head, Holman, Lanfear, Kahn, & Jennions, Reference Head, Holman, Lanfear, Kahn and Jennions2015). We use p-hacking for illustrative purposes in this article, as one of the exemplary QRPs that has received significant attention in both academia and public debate (Anonymous, Reference Anonymous2013; Head et al., Reference Head, Holman, Lanfear, Kahn and Jennions2015; Simonsohn et al., Reference Simonsohn, Simmons and Nelson2015). For instance, p-hacking occurs if a researcher reaches results with an insufficient p-value, but rejects the fact that the theoretical reasoning is not supported by the data. Instead, the researcher changes the method of analysis, adds controls, or eliminates outliers post hoc. If this approach eventually leads to a sufficient p-value, some might consider these to be p-hacked results. Thus, the collective term p-hacking stands for a variety of techniques to reach publishable results to illustrate our reasoning.
P-hacking and the Questionable Application of Research Techniques
Null hypothesis significance testing (NHST) is a commonly used method to objectively separate interesting findings from background noise (Fisher, Reference Fisher1925). NHST allows researchers to decide whether to accept or reject a null hypothesis; that is, a statement that claims no relationship between two variables. However, this decision requires a cut-off value, which differentiates between acceptance and rejection. In NHST, a p-value of 0.05 usually serves as such a cut-off value. Accordingly, findings on a smaller p-level are claimed to be “statistically significant” or called ”positive findings.” P-values larger than the cut-off value of 0.05 are “statistically nonsignificant,” or called “negative findings.” In the logic of NHST, results with a manipulated p-value have the potential to cause two types of errors: a Type 1 error incorrectly rejects a true null hypothesis, and a Type 2 error fails to reject a false null hypothesis.
P-hacking represents a class of behaviour in which the selection of statistically significant estimates for publication is not based on appropriate theorising but trial-and-error attempts to reach a sufficient (i.e., publishable) threshold for the p-value (Simonsohn, Nelson, & Simmons, Reference Simonsohn, Nelson and Simmons2014). Others label it “data-dredging,” “snooping,” “fishing expeditions,” “significance-chasing,” and “double-dipping” (Nuzzo, Reference Nuzzo2014). For our purpose, an impartial understanding of p-hacking is the one suggested by Simmons, Nelson, and Simonsohn (Reference Simmons, Nelson and Simonsohn2011) as “trying multiple things” until you get the desired result. It is helpful because such a generic definition accepts, in its broadest sense, that in some instances the results may contribute to a better understanding of the phenomenon. Also, a researcher trying multiple things is actually desirable, because scientific progress does not depend on repeatedly doing the same thing but on creativity regarding methods and analytical tools, even if this does contain the threat of QRPs.
Thus, a research process may involve multiple forms of p-hacking that do not automatically lead to publication bias but have the potential to do so. The application of multiple statistical analyses, the changing of data-eligibility specifications, the termination of data collection once a sufficient level of significance is reached, and the selective reporting of only those effects that appear as statistically significant are just a few of the many possible approaches to reaching the desired result (Cumming, Reference Cumming2014; Gadbury & Allison, Reference Gadbury and Allison2012). Further, many opportunities for attaining significant results are also possible at the post-analysis stage. For example, researchers can decide which variables to include in the model, how to handle statistical outliers, whether to include or exclude covariates, and whether to exclude, combine, or split the data set. Consequently, they may fail to report all the relevant variables; they may retreat data after the first round of analysis; or, as a last resort, they may exclude statistical outliers—all practices that increase the likelihood findings cannot be reproduced (Bakker & Wicherts, Reference Bakker and Wicherts2014; Bastardi et al., Reference Bastardi, Uhlmann and Ross2011; Head et al., Reference Head, Holman, Lanfear, Kahn and Jennions2015; Hutton & Williamson, Reference Hutton and Williamson2000).
Given that p-hacked results imply that a statistically significant finding may reflect selective reporting rather than a true effect, such a QRP represents a potentially serious threat to the progress of scientific disciplines. Progress in science is built on both empirical and theoretical previous knowledge. If a discipline accumulates a body of knowledge that does not necessarily reflect the phenomenon of interest, it generates a bias that affects theory construction in multiple ways. Consequently, knowledge transformation based on biased theories may lead to recommendations or practical applications that are ineffective. A supposed relation that is not the state of nature may advocate for the mistaken rejection of a theory. Alternatively, researchers may try to develop new theories with the goal of explaining non-existent effects.
Science Production and P-hacking in Practice
The fact that QRPs, such as p-hacking, potentially threaten the credibility of science and, therefore, the work and reputation of researchers themselves raises the question: Why are researchers engaged with such practices in the first place? To answer this question, we have to first consider the publication logic of academic journals, and then the generation of career capital in academic labour markets (Allen et al., Reference Allen, Ball and Smith2011; Banks et al., Reference Banks, O’Boyle, Pollack, White, Batchelor, Whelpley, Abston, Bennett and Adkins2016a).
Prestigious academic journals tend to disproportionately publish statistically significant results (Emerson, Warme, Wolf, Heckman, Brand, & Leopold, Reference Emerson, Warme, Wolf, Heckman, Brand and Leopold2010; Fanelli, Reference Fanelli2012; Song, Eastwood, Gilbody, Duley, & Sutton, Reference Song, Eastwood, Gilbody, Duley and Sutton2000). This trend has led to some scholars claiming significance as a kind of gold standard in science (Nuzzo, Reference Nuzzo2014). However, even if there are various studies pointing out that research is not most appropriately represented and summarised by p-values (Cohen, Reference Cohen1994; Ioannidis, Reference Ioannidis2005; Ziliak & McCloskey, Reference Ziliak and McCloskey2008), the notion that only significant results have the chance of being published in a top-tier journal drives researchers to “chase the significant” (Ferguson & Heene, Reference Ferguson and Heene2012). Overall, a journal’s publishing practice facilitates two kinds of publication bias. First is what is referred to as the “file drawer effect” (Rosenthal, Reference Rosenthal1979), which arises through the lower publication rate of non-significant results. This effect is responsible for the tendency of journals to be filled with studies that have significant results on the basis of a 0.05 level (as discussed earlier) and, consequently, are likely to contain Type 1 errors or false positives. This results in a situation in which “the ‘file drawers’ back at the lab are filled with 95 percent of the studies that show non-significant results” (Rosenthal, Reference Rosenthal1980: 18). As a consequence, some null hypotheses are, in fact, true, and the association being studied does not exist; yet, the 5 percent of studies that (by chance) show a statistically significant result are published instead (Rosenberg, Reference Rosenberg2005). Second, a bias derives from the reaction of researchers to the journal’s reluctance to publish nonsignificant results. By being aware of this publishing policy, a scholar’s strategy to get the work published is to make several attempts to reach a desired p-value, which leads—at least potentially—to questionable practices.
Further, researchers constantly mention situations in which pressures in the academic system push them to adopt questionable behaviours. Such situations seem to be widespread, as described by Head et al. (Reference Head, Holman, Lanfear, Kahn and Jennions2015), who found that in fourteen out of fourteen—that is, all disciplines investigated—p-hacking, as one of the major QRPs, is an existing practice. Specifically, the cause claimed for commonplace p-hacking practices is the overall incentive structure of current scientific practices (Fanelli, Reference Fanelli2012). Indeed, the current academic structure is likely to overestimate publication outputs in contrast to researchers’ other tasks, like managing complex scientific projects and solving cutting-edge problems. This may contribute to the broader issue. Moreover, by its very nature, research is insecure since the outcome cannot be predicted and the risk of wasting time, money, and effort is all part of conducting a (large) research project. Adding to this, a researcher’s own research impact and the findings are intrinsically linked to career progression. For non-tenured and early career researchers, who are placed under enormous pressure to publish, significant findings are especially important to maintain competitiveness in the academic labour market. Hence, it is not surprising that highly competitive academic fields show more counternormative behaviour than more cooperative fields (Anderson, Ronning, Vries, & Martinson, Reference Anderson, Ronning, Vries and Martinson2010).
In summary, p-hacking is a QRP that can be seen as a response, even if not the only one, to the pressure from an evaluation system that places high value on personal publication rates in acknowledged journals. Compelled to “publish or perish,” the salability of findings becomes the most important dimension of a study, which shifts the interest of researchers towards getting their research into a high-profile journal rather than getting the research done correctly (Clapham, Reference Clapham2005). The current system does not reward high costs for publishing false results, while the price for not publishing may cost researchers their careers (Anonymous, Reference Anonymous2013). In such an environment, p-hacking could appear as a rational and strategically required behaviour, which is an aspect that we will explore in depth later.
NORMATIVE QUALITY OF QUESTIONABLE RESEARCH PRACTICES
Following norms, rules, or regulations is relevant to being legitimatised to produce research findings that are recognised by the community. In this regard, norms are “collective expectations for and understandings of appropriate and desired behaviour within a given social system” (Anderson et al., Reference Anderson, Ronning, Vries and Martinson2010: 366). However, inherently, a finite list of norms, which are ultimately representative of the normative system ‘science,’ cannot be established because most norms are latent and only implicitly guide researchers’ behaviour. Consequently, some researchers follow Durkheim (Reference Durkheim2008), who suggests quantifying norms’ significance by evaluating the extent of moral outrage and negative response to behaviour that violates the norm in question (e.g., Zuckerman, Reference Zuckerman and Smelser1988). For this reason, a QRP such as p-hacking can be classified as violating norms as it receives criticism from inside and outside science. This overall sentiment indicates that many researchers state that such practices violate basic norms. However, many QRPs also refer to a researcher’s academic freedom, which is undoubtedly a central element of science as a social system. A researcher must have the freedom to explore the phenomenon under investigation with various analytical approaches. Limiting researchers in their selection or treatment of data can be seen as restricting their choice of analytical approach, which might hinder the innovative progress of science.
Ethical Dimensions of Questionable Research Practices
A researcher’s freedom allows a variety of approaches, methods, and analytical tools to be applied to investigate a phenomenon of interest. While doing research indisputably depends on a researcher’s unique freedom (Tierney & Blumberg, Reference Tierney and Blumberg2007), this is a relative freedom that goes hand in hand with the researcher assuming responsibility for their individual actions. In this regard, few would argue that academic freedom includes the right to exploit this freedom for personal benefit, if this subsequently harms the credibility of science. If we only consider the outcome of a behaviour, we have to admit that p-hacked findings do not always harm progress, because findings generated by questionable practices may (by accident) represent the actual state of affairs. However, the correctness of the outcome cannot be evaluated a priori and without proper statistical analysis. Therefore, focusing exclusively on the outcome of an action, such as in the tradition of consequentialism, does not help determine whether misconduct has occurred before the results are presented. Thus, we refer to deontological ethics to highlight the intention that guides a researcher’s practice as the central condition for the analysis of the normativity of their practice.
To outline our reasoning, we refer to Figure 1. The black dot in the middle can be understood as the initial and neutral situation of every new research project; that is, before any action or outcome. We especially emphasise personal advantage/disadvantage and advantage/disadvantage for the scientific community as the intention/outcome behind any research practice. Hitherto, we argue there are research behaviours that move the initial situation in the direction of A. For instance, if we understand each of the four fields, the one from the neutral situation (the black dot in the middle) to personal advantage and science advantage as a space of possible research practices, the space around A becomes a desired research practice from a normative standpoint. All research actions in this space lead to scientific progress and, at the same time, to personal benefit for the individual researcher. In this space, the self-oriented and purely career-driven behaviour of a researcher is not an issue, since it leads to an advantage for all. In some sense, this space is analogous to Adam Smith’s hypothetical free market space, where the self-interest of all actors leads to a benefit for all. As such, actions that attempt to generate significant results are not questionable, even if a researcher uses practices like statistical outlier removal, which may qualify under certain circumstances such as p-hacking. We note that the C and D categories include the application of tools, methods, and practices that produce a disadvantage for the researcher, which likely leads to their rare occurrence. However, whatever disadvantages the individual incurs do not automatically disadvantage the scientific community. For instance, space D contains crosschecking, replication of findings, and the like—all practices that journals are reluctant to publish but that would contribute to the state of the art of sciences. Thus, while D contains a set of actions that may indeed act to increase research reliability and progress, it remains unappealing from a researcher’s point of view as time and financial investment in these practices is not beneficial. Therefore, for the purpose of this study, we focus particularly on the spaces A and B. More specifically, we investigate how to locate the fine line between these two spaces. B is of particular interest if it comes to QRPs or clear fraud in science productions. The case of Diederik Stapel may illustrate this (Crocker, 2011). Stapel admitted he had fabricated data for studies that had high media resonance for years. This media attention brought him considerable notoriety in the scientific community and public recognition beyond the borders of his discipline. Clearly, he benefited from his practices while science itself was harmed. In what follows, we demonstrate that whether or not a certain research behaviour is classified as an element of A or B highly depends on the research context, the researcher’s intention, and the subject under investigation. While an example like Stapel’s is easy to decode, the complexity of the scientific process often turns up much more difficult cases.
Localisation of Questionable Research Behaviour
Determining the Field of Questionable Research Practices
To establish the legitimacy of a particular research practice, two distinct entities need to be considered: the individual researcher, and the structures of the scientific system in which the researcher operates. Hence, actions are limited by the moral rules that apply to the individual and by the structures of the system in which the individual interacts with others. Accordingly, we refer to discourse ethics, which allows the sciences to be investigated as a set of actions that needs to adhere to certain moral standards rooted in the human capacity for rationality. Discourse ethics sees science as a system for which it proposes procedures for deciding on moral norms. The moral assessment of a particular practice takes into account its consequences for a wider community (McNaughton & Rawling, 2007). Discourse ethics allows further reflection on the coordination of actions within a community. These mechanisms to coordinate actions of researchers represent the inner structure of science. What follows, from a discourse ethics point of view, is that the coordinating mechanism in society is ultimately the discourse.
DEONTOLOGICAL ETHICS PERSPECTIVE ON RESEARCH PRACTICES
Ethics, in general, is concerned with an action performed by an agent and its outcome. Thus, every ethic, in some fashion, must treat the outcome, act, and actor (Koehn, 1995). This is why some authors distinguish between ethics focusing on the character traits of the actor (e.g., virtue ethics), the outcome of an act (e.g., utilitarianism), and the act itself. (e.g., deontological ethics) (Graham, 2010). In particular, deontological perspectives fall within the domain of moral theories that guide and assess the choices of what we ought to do. The theory is concerned with the quality of an act. It sees the duty to act in accordance with this quality as paramount, regardless of what the outcomes might be (Frankenna, 1973).
The deontological perspective we are building on originates from Kant’s Groundwork for the Metaphysics of Morals (1785) and Critique of Practical Reason (1788). According to Kant, a moral theory needs to address the question of what is “unconditionally good” (Wood, Reference Wood1999). He further argues that only one thing must be considered as unconditionally and truly good and this is “goodwill.” For deontologists following Kant, what makes a choice right is the degree to which it is guided by goodwill. In Kant’s terms, goodwill is wholly determined by moral law (i.e., moral norms) (Bowie, Reference Bowie2017). Kant further understands moral law as a system of a priori moral principles. Individuals with goodwill form their intentions solely in accordance with what a rational individual ought to do. Hence, the good is formed of intentions and is independent from consequences of an action (O’Neill, Reference O’Neill1990).
Habermas’s (Reference Habermas1984, Reference Habermas1987) theory of communicative action builds on Kant’s idea of moral norms but locates them in the community rather than the individual. It provides a powerful development of Kant’s basic notion of deontological ethics. Human beings are fundamentally dependent on the ability to coordinate actions; that is, the coordination of what ought to be and how to find efficient ways to reach this state (Habermas, Reference Habermas1984, Reference Habermas1987). Habermas’s theory introduces the discourse as means to define collectively desirable behavior.
In what follows, we introduce the concept of discourse as a coordinating mechanism that limits one’s behaviour to actions based on concern for other people rather than purely oneself. Discourse further tackles the normative content of structures and rules within a given system, such as the sciences in general and their subsystems. In this sense, the discourse allows identification of how one should behave if one has the intention to do something good—in other words, how a researcher ought to do research if they have the intention to contribute to the sciences.
Publishing Process as a Link Between Individual Researcher and the Scientific Community
Publishing research results is a way of entering into a discourse with peers; hence, it does not refer to individual values but to a collectively shared set of norms (Edwards & Potter, Reference Edwards and Potter1993; Gilbert et al., Reference Gilbert, Rasche and Arnold2007). Consequently, legitimate research practices cannot be justified by the individual researcher; that is, no individual can draw a line between space A and B without considering the scientific community, as science is a collective act and not an individual one. This becomes apparent in the fact that researchers publish their findings, a process that transcends the individual sphere towards a wider community. While conducting research exclusively for oneself is a value-free act as long as no other entity is involved, the individual matter ends if one claims normative justifiable practices against someone else. By its very nature and thus inescapably, this is a communicative act.
The heart of discourse ethics builds on the idea that universal validity, as “a moral norm, cannot be justified in the mind of an isolated individual” (Gilbert et al., Reference Gilbert, Rasche and Arnold2007: 190). The essence of science is the public good (David, Reference David2004), and open discourse is the institution to investigate norms regarding its production among all those who build the relevant ‘public.’ A particular treatment of data analysis or reporting of findings that claims normative correctness depends upon the mutual consensus achieved by the individuals debating whether such a practice should become the standard in an academic discipline (Habermas, Reference Habermas1998). The deontological roots of discourse ethics claim a conceptual abstraction comprising an individual’s desire to justify the basic structures and rules of a given system through agreement within the community. This necessary search for agreement allows the introduction of rules and structures that may limit the freedom of a researcher. Thus, we see the scientific community—understood in a very broad sense as all those individuals with an interest in the results—as the major reference point for the quality of a researcher’s data treatment.
Discourse and Legitimate Structures of Science
Strategic Action Versus Communicative Action
Academic freedom involves the production of research findings with the intention to build a career, a reputation, and status. In fact, these motivations have the power to facilitate scientific progress; hence, they are desirable as long as they are located in A, as depicted in Figure 1. However, this has no power to explain the moral dimension of the practices used to reach these goals. Some QRPs, such as p-hacking, for instance, may be an effective way to reach one of those goals, but may harm the academic field, as depicted in B. Thus, we need an institution that ensures a researcher’s freedom is used for the benefit of science and not only for personal gain. To explore this in depth, we distinguish strategic action from communicative action (Habermas Reference Habermas1984, Reference Habermas1987).Footnote 4
Strategic action is a goal-oriented technique. It combines all actions that lead to a desired outcome, regardless of their approval by the community. As such, it is a force that attempts to transform personal disadvantage into personal advantage, or to create an even greater advantage from an already existing advantage. Because it is a technical matter (i.e., how to do things), the evaluation of a strategic action has to take place in reference to the effectiveness and efficiency of the achievement of objectives. This is why it is a technique rather than a moral act. It is the means to an end, such as career progression, reputation, and status within the academic community. Hence, strategic action is concerned with neither advantages nor disadvantages for science, but strictly with one’s self-interest. In the logic of discourse, this would be the case if the interest of a minority favours its own benefits and gets priority in the discourse regarding which rules should govern an academic field. If someone presents arguments strategically in favour of their position only, they commonly intend to lead a discourse—not for the sake of the argument, but for manipulating, impressing, or influencing. Such intention is seen as strategic action, with the primary goal of individual benefit. In the logic of Figure 1, it pushes in the direction of personal advantage. However, this is not an issue in itself since the opportunity to end up in A rather than in B remains. Taken together, QRPs such as p-hacking are strategic actions, which is questionable if it is a means to reach individual gains, regardless of whether a scientific discipline suffers.
The concept of communicative action is oriented towards reaching a consensus regarding the rules and structure of an academic discipline. Consensus in the context of Habermas’s discourse ethics refers to the exchange of arguments and the possibility of peer agreement. For this reason, the exchange of arguments needs a certain degree of truthfulness, as without consensus it is not possible. The underlying idea is that communicative action allows personal advantage, but only if the favoured rules and structures also lead to an advantage for science as a whole, which needs to be demonstrated in a process based on open communication. Communicative action requires discourse within an academic field on the research process, tools, methods applied, and the illustration of a comprehensive picture of doing research in the academic field. Oriented on discourse, any attempt to push an individual, strategically motivated agenda within open (discourse) structures must fail if one tries to canvas an all-inclusive picture of the data-analysis process. Accordingly, if an academic discipline would be governed by a publication process that promotes the interests of a group of individuals (e.g., editors, reviewers, researchers) at the expense of the scientific community, we may expect that such structures would not receive hypothetical (or actual) approval from the scientific community. This is because it is a hard-to-reach argument among groups of actors as to why one group should take personal advantage while the discipline simultaneously suffers damage. The communicative element means that if such rules or norms are openly communicated, there is a high possibility someone will raise good arguments against these rules. If we accept that arguments have regulatory power, we also accept that enforcing these rules and norms can only happen by overruling those with better arguments; an action that cannot claim to be just and hence cannot produce a legitimate outcome. In the logic of Figure 1, communicative action is a regulatory power that ensures the push from strategic action ends up in A rather than in B. This property clearly allows self-interest and personal benefit, but it is restricted by the common good.
The centre of discourse ethics provides these properties with a “framework for critically examining social accountability standards and their ethical assumptions” (Gilbert et al., Reference Gilbert, Rasche and Arnold2007: 189). It outlines an avenue to universally moral consensus on how rules and norms should be designed (Habermas, Reference Habermas1990, Reference Habermas1996). Therefore, we have to link both concepts to the principles of discourse ethics.
Habermas’ Theory of Communicative Action
The theory of communicative action is based on discourse ethics and the assumption that human beings reach greater insight into matters concerning themselves if they engage in discourse with others. In our case, insights concern our knowledge and how its production should be legitimised by presenting arguments, justifying statements, and asking and answering questions (Walseth & Schei, Reference Walseth and Schei2011). However, findings, rules, and norms for the publishing process are concerned with both kinds of discourse elements (i.e., strategic and communicative elements). To distinguish between the interests of the scientific community and the self-interest of actors in this community, we are particularly concerned with the preconditions and requirements of a discussion guided by communicative action; that is, a discourse that ensures the power of strategic action unfolds in the direction of A. The degree to which a researcher orientates research behaviour to these requirements is directly indicative of how the research approach aligns with normative research standards. To better understand how communicative action and consensus are possible, Habermas (Reference Habermas1990) developed a normative interpretation of speech acts, which are the basic elements of every form of communication, such as constative, regulative, and expressive (Austin, Reference Austin1962; Searle, Reference Searle1969). To make different validity claims (e.g., to claim effectiveness, goodness, or rightness), speech acts have to follow a logic that is inherent to the particular claim. This logic is also valid for the publication of research findings, since it represents the way researchers communicate with their scientific community. Thereby, every conversation requires that participants accept that different speech acts hold different validity claims. They have certain inevitable presumptions in common about how humans communicate, as well as how they exchange arguments (Habermas, Reference Habermas1984, Reference Habermas1987, Reference Habermas and Fløistad1994). This involves basic assumptions on the structure of a valid argument. Without these rules, it is not possible to make a claim comprehensible to the discourse counterpart; for instance, a sarcastic statement requires that the recipient of the utterance understands the logic of sarcastic speech. Translating this claim into science practice means that research can only make progress if every involved actor is principally open to accept the structure of arguments on which rules and norms should govern science. The rules for leading an honest discourse, as well as doing science, require that old arguments are replaced with new arguments if the new ones are superior according to logical, stringent, and empirical grounding. Considering the utterance situation in speech, the theory of communicative action provides a framework concerning preconditions for making the utterance, if the discourse should produce a solution to a problem. These preconditions are collectively termed the “discourse rules of the ideal speech situation” (Gilbert et al., Reference Gilbert, Rasche and Arnold2007: 192). The theory of communicative action shows that in an ideal speech situation, the result is a consensus about the norms. The regulatory force that brings unconstrained participants to a valid and generalisable conclusion is the so-called “force of the better argument” (Habermas, Reference Habermas1990). It builds the counterforce to power or dogmas. The rational argument that unfolds its power to reach an agreement requires the fulfilment of the ideal speech situation. This includes a requirement that utterances must rest on particular validity claims and allow all participants in a discourse to either challenge or defend the content of the utterance (Pellizzoni, Reference Pellizzoni2001).
The logic in an ideal speech situation (e.g., no contradiction of utterances, generality of a claim, and consistency of the used terms) strongly aligns with the requirements of a scientific theory (i.e., non-contradiction, generality, and consistency). The theory of communicative action claims that nobody who leads a serious discourse can deny being in favour of only having rational and comprehensible arguments going into this discourse. This means that no actor in the scientific system can contradict the openness and truthfulness of the scientific discourse based on well-formulated arguments as a central norm of science without contradicting their self-concept as a scientist. The dialectical level of this enterprise is in terms of facts or possible states of affairs concerned with truthfulness about the material world, and the personal world of feelings and intentions. This precondition ensures feelings like disappointment—which naturally accompany one’s disproof—should not guide the evaluation of an argument (Collier, 1997).
On the rhetorical level, the preconditions of an ideal speech situation secure the openness of the discourse. It is claimed that no one who is interested in finding the best solution to a problem can have the self-interest to exclude someone from the debate because this particular individual might hold the desired argument. With respect to scientific findings, we argue that a researcher who is interested in producing the best possible rules and norms will appreciate every true argument in favour of or against such rules and norms. In this sense, personal openness in dealing with data is in the interests of both the scientific community and the researcher, as long as the researcher is guided not only by strategic rationale.
Thus, common understanding in an ideal sense is seen by the theory of communicative action as the result of discourse in which an actor is not coordinated via egocentric calculations of success, but through acts of consensus. Therefore, these rules ensure participants are not primarily oriented towards their own success; instead, they pursue their individual goals under the condition they can coordinate their actions on the basis of shared practices (Risse, Reference Risse2004).
Discourse Principles, Moral Norms, and the Structure of an Academic Discipline
Habermas’ theory is concerned with two types of ethical questions, which he calls ‘pragmatic’ and ‘moral’ questions (Habermas, Reference Habermas and Habermas1993). It is a pragmatic question about the best way to achieve a goal. One may consider non-ethical means, consciously or subconsciously, based on the established script of one’s community to achieve a goal. For Habermas, moral questions have a different nature and are concerned with the moral desirability of the end itself. He explicitly addresses the self-understanding of a person or a community by linking the accepted ends of an individual or of a group to the question: what kind of person/group am/are I/we? Hence, moral questions are concerned with self-chosen values that a community accepts as morally just. Qualifying misconduct in academia is linked to the self-concept of the scientific community itself. The question of how to report findings, treat data, or execute research projects refers to concerns at the community level and addressing the question of whether the people involved subscribe to the self-concept of the scientific community. Specifically, it highlights that the end itself is not value-free, since it implicates the means to achieve it and, thus, the practices utilised. In turn, the used practices define the one who utilises them. Therefore, the appropriateness of certain research behaviour is not determinable by the individual researcher applying a particular practice. Instead, the extent to which the community has reasonable arguments against the behaviour in question is the reference point for making a judgment. Thus, the question of where to draw the line has a technical consequence, in the sense that the reliability of research findings fall short of desirability. More importantly, misconduct harms the self-concept of a whole profession and the ideals it cultivates. Habermas’ theory abstracts from the content of the discourse but refers to the discourse process as an institution that ensures the outcome. Therefore, he introduces two principles that must be fulfilled to achieve a consensus. The first one refers to the ideal speech situation:
A norm is valid when the foreseeable consequences and side-effects of its general observance for the interests and value-orientations of each individual could be jointly accepted by all concerned without coercion (Habermas, Reference Habermas1998: 42).
Therewith, it is ensured that no particular concern is overlooked. The second principle—universalisation of a norm—exceeds the justification in the ideal speech act. Discourse ethics requires that a particular action is generalisable among all people concerned; only that kind of behaviour can claim general validity or correctness for the individual researcher alongside all other researchers in the same situation. In other words, if a QRP is justifiable, it should also be the standard for doing research across the discipline. It then becomes embedded in the structure of an academic discipline; it cannot be the sole action of an individual researcher. The academic community has established a rule for a particular way of working with data, and violating this rule would mean crossing the line between A and B. As a principle, the ability to generalise the researcher’s conduct therefore highlights that amenability to consensus is both a necessary and sufficient condition of the validity of a morally acceptable research practice.
NORMATIVE GROUNDS FOR ASSESSING QUESTIONABLE RESEARCH BEHAVIOUR WITHIN AN ACADEMIC DISCIPLINE
Evaluating Research Practices on the Individual Level
It is clear that the altering of a statistical tool could be both an expression of a researcher’s freedom and the production of doubtful results that is an offence against the core values and the culture of an academic field. If we follow discourse ethics, then the capabilities of researchers to reflect on their own research practice generates individual responsibility for the normative quality of their research practices. In considering the normative basis of QRPs and the associated grey area of justifiability, we propose two questions, which allow the line between A and B to be determined and—in other words—provide an answer to whether or not a practice is classified as misconduct in a particular situation. The questions are:
1) Would anybody affected by my academic field have justifiable concerns regarding the rules and norms of dealing with data that are the cornerstone of my practices?
2) Do I envisage my data analysis procedure as the norm in my academic field?
On the deontological premise that underlies individual rational ethics, rather than a given ‘external’ moral principle, these questions collectively prompt a researcher to question their own actions. In answering yes to the first of these questions the researcher assumes practices would not cause justifiable concerns within the scientific community, including its stakeholders. In answering no to the second question, the researcher indicates that practices may not be acceptable in their academic field. We propose that if it is the case that the first question has to be answered with yes and the second question has to be answered with no then indisputably the line between A and B has been crossed as the practice is clearly intended to result in personal benefit. In such cases, we have to assume that a particular research practice is indeed questionable.
The theory of communicative action provides further guidance towards a collective process regarding agreement on norms and structures of a system such as science. We argue here that the question “Do I really want my practices to become the standard in my field?” opens up a useful verification process. Following our prior argument, these two questions provide guidance when researchers face uncertainty about which analysis to run, how to deal with expected or unexpected results, or how the results, or which parts of the results, should be presented. Thereby, both questions in the tradition of deontological ethics underlie the human capacity to provide reasons and understand counterarguments, as well as one’s self-perception as a researcher. Both questions are a response to self-regulation, substantial autonomy, and the researcher’s scientific freedom. While acknowledging that research freedom allows a variety of approaches, methods, and analytical tools, this freedom cannot be conducted without limits.
Further, these questions are largely concerned with the self-concept of every researcher. As practices are defined by the individual as well as by community standards, a researcher must consider that their own practice has the potential to become a standard. If the researcher cannot see that their way of conducting research is already accepted by the majority, or is highly likely to be, then it is not, nor should be, a norm for doing research in an academic field. Consequently, a researcher would violate the rules and norms if they performed research in this way. In such a case, a given practice is clear misconduct that harms one’s self-concept, as well as the whole profession and the ideals it cultivates. The first question echoes the moral quality of an action in relation to others. The second question refers to the fact that someone who is truly interested in scientific progress would not wish to work with false positives or theories with unreproducible claims. Considering potential answers to both of the questions, it is possible to draw a distinct line between space A and space B to indicate acceptable and unacceptable degrees of p-hacking.
Evaluating Research Practices on the Collective Level
Given this reasoning from the point of view of an individual researcher, we also have to consider the relationship between researchers and the norms and rules of the scientific community. Hence, the distinction between strategic action and communicative action does not allow us to draw a conclusion regarding a researcher’s data treatment without considering the scientific community. Instead, QRPs have to be justified in a particular situation with reference to the general desirability of such behaviour for the whole discipline. Desirability refers to two different groups of stakeholders of an academic discipline. The first is the scientific community, consisting of all researchers in a particular field. The discourse about research behaviour here concerns the research practices cultivated in this field. It refers to the way the discipline comprehends its enterprise and how it defines basic concepts such as truth, theory, reliability, and validity, and norms such as freedom or responsibility. The second group of stakeholders is external to science and includes policy makers, enterprises, and the public, although they provide resources such as trust, without which research would not be possible. This trust is not in relation to one individual researcher but to the collective scientific community. From a discursive point of view, arguing for the general acceptance of a particular QRP requires an agreement among both groups. We argue that the line distinguishing acceptable from unacceptable practices is defined by the generally accepted reasons within each scientific community and the external stakeholders.
A further implication of our argument is that even if a practice is explicable as the strategy of a researcher to fulfil requirements within the academic system, it is not excusable with reference to existing pressure, evaluation practice, competition, or a journal’s publishing policy. In this respect, strategic action may happen if salability of results becomes more important than the state of the art of the analytical process and theoretical reasoning. Salability to a journal—which, from a QRP perspective, involves the presentation of selective aspects of the research, disregarding counterevidence, and the omission of variables that are not in line with the story—needs to be justified. Salability, publication pressure, and career models are structures of the system of science. As such, they are the guiding elements of the system and need reflection in the aforementioned sense, but on a collective rather than individual level. The theory of communicative action provides a basic attempt to harmonise action plans and achieve consensus on normative validity claims. In this regard, it ensures the norms and standards of the academic field are not interested in the personal benefit of a particular group of people but in the benefit of the scientific discipline.
DISCOURSE ETHICAL FRAMEWORK AND CURRENT DEBATE ON QUESTIONABLE RESEARCH BEHAVIOUR
Returning to Brady and Dunn (Reference Brady and Dunn1995), our framework provides a stable theoretical basis for applying ethics to science production. As a central element of researchers’ everyday practice, discourse is hardly deniable; publication is discourse and without publishing results science cannot exist. As our article shows, accepting that science production is discourse has some far-reaching implications for how we decide which research practices are acceptable and which are not.
Specifically, we contribute to the discussion on the ethics of science production by providing a framework with three properties. First, we provide a stable ground rooted in the discourse to evaluate concrete research practices. Second, we ensure individual action contributes to the progress of an academic field on the collective level. Third, we are able to address academic freedom in a way that limits a researcher’s action while simultaneously ensuring the individual researcher benefits from this limitation.
Our framework not only appeals to the moral conscience of a researcher but allows them to make decisions in concrete situations. For instance, researchers face situations in which their theorising is not supported by data much more often than when results provide the desired outcome. Such situations are frustrating and the researcher may tweak the analysis approach. P-hacking techniques, like analysing as a subset instead of analysing the whole set, could be an option to achieve relevant results. For example, a researcher interested in an entrepreneurial ecosystem will most likely encounter the fact that national settings influence entrepreneurial ecosystems differently than regional innovation systems (Spigel, Reference Spigel2017) or those in an urban context (Mack & Mayer, Reference Mack and Mayer2016). Hence, selecting a subset that addresses only entrepreneurial ecosystems in cities (Audretsch & Belitski, Reference Audretsch and Belitski2017) may produce results that are significant and meaningful. Indeed, it is in the interests of science to discover these differences. If a researcher is able to theorise further on these subset relations and back up the findings with further investigation, our understanding may benefit from a more detailed view on entrepreneurial activities and the role of urban contexts. In this sense, reference to a subset instead of the whole set may be beneficial for all and hence not questionable. In a similar research setting, the researcher may also try to draw random subsets and run the analysis again and again until the results satisfy basic statistical thresholds. The researcher can choose to remain silent about particular research practices. Being well aware that these findings cannot be reproduced but still publishing them is a breach of scientific norms and standards and would harm scientific progress. In this regard, our framework clearly distinguishes between acceptable and unacceptable forms of the subset analysis.
The second insight our study contributes to the discussion on the ethics of science production is the establishment of a theoretical basis that links individual actions to collective desirability. We ground assessing QRPs in the individual scientist’s capacity for rationality, but with a focus on securing the systems of science on a collective level. Our framework provides guidance in which the freedom of one individual ends with reference to the potential damage to the academic field. Our framework limits action, not by proposing a system of norms and standards that govern or regulate science production, but by grounding these norms in the human capacity to understand arguments. In other words, norms or standards are not the result of a political process to which a researcher has to subscribe but of a collective process that builds and values one’s capacity for rationality. The question “Do I envisage my data-analysis procedure as the norm in my academic field?” puts the researcher in the theoretical position of deciding on the governing standards in a field. However, with this power comes the threat that norms and standards may be biased towards self-interest. To prevent such self-interested implementation and justification of norms and standards, our framework acknowledges that the regulating forces must be found in the community of all those concerned by a particular standard. Norms and standards that would not survive the test towards a general consensus cannot become the practice in a field. Referring to our previous example, we have to assume that researchers do not want to build their research on results that do not reflect reality. Hence, subset analysis for the sake of salable results cannot become the norm or standard in a field. Academic freedom is secured but limited towards contributing to the collective endeavour of producing new knowledge rather than individual gain.
The third contribution is in linking individual benefit and collective benefit. The debate on academic freedom is based on tension between individual and institutional actors such as journals, editors, and review boards (Tierney & Blumberg, Reference Tierney and Blumberg2007). The idea of academic freedom includes allowing researchers to search for truth without being constrained by external interference. Further, researchers do not need to tailor their findings to please or accommodate interest groups. Academic findings are further evaluated regarding the quality of ideas rather than by administrators or legislators for instrumental or ideological reasons (Tierney, Reference Tierney1993; Tierney & Blumberg, Reference Tierney and Blumberg2007). Hence, limiting this freedom must be seen as a serious threat to a higher public good. Against this basic comprehension, we highlight that the rationale of imitating academic freedom is not only grounded externally but also within the rationality of every researcher. Limitation leads not only to advantaging the entire academic field, but since the individual researcher is part of the bigger system of science, such limitation also ensures benefits on the individual level. One’s self-concept as a researcher legitimated to contribute to an ongoing discussion in a field requires one to limit actions. This is not because of external forces but because of one’s capacity to understand one’s own action in relation to the scientific community. Our framework highlights this relation and demonstrates how it allows addressing questionable practices within an academic field.
CONCLUSION AND OUTLOOK
Knowing that there are multiple reasons for and against every practice in the grey area of morally acceptable behaviours, as well as respective norms and counter-norms (Anderson et al., Reference Anderson, Martinson and Vries2007; Anderson et al., Reference Anderson, Ronning, Vries and Martinson2010), we identified two central dimensions formulated in two questions that may help clarify the extent to which a research practice is justifiable. Researchers, journals, and the scientific community as a whole would benefit from this framework, which allows the identification of a specific behaviour becoming misconduct.
However, we note that this is only the first step to achieve more and consistent self-monitoring (Sterba, Reference Sterba2006). To attain a comprehensive platform for such reflection, other players in the science production process, such as editors and reviewers, must be involved. The publishing process requires the researcher to respond to a set of implicit and explicit criteria used by reviewers to evaluate empirical research (Maner, Reference Maner2014; Murayama et al., Reference Murayama, Pekrun and Fiedler2014). This raises the questions of how reviewers should behave when reviewing studies (Eich, Reference Eich2014), which has prompted calls for improved reviewer standards (Maner, Reference Maner2014). Our reasoning here suggests that editors and reviewers have the role of moderating the communicative process. They are the link between those who claim adequate data treatment and those to whom this treatment needs to be justified. Thus, the role of editors and reviewers is to request all information on the scientific process be made subject to a wider discussion. This discussion cannot happen exclusively between editor and author but should happen between author and community.
Journals and editors become the facilitators of discourse. They should perceive themselves as collectives that set the standards for science production at an individual level. Journals and editors should view science as a public good and call for open communication of data treatment and scientific results (David, Reference David2004). Calls for reproducibility of research should be more widespread than they are currently. Once more research is reproduced, it may become clear which research practices potentially lead to questionable results and therefore must not be practised. This could be a valuable addition, over and above the expectation for individuals to be responsible for their acts.
Additionally, academic career seems to be one of the drivers of QRPs that needs to be redefined, especially for early-stage researchers (Allen et al., Reference Allen, Ball and Smith2011). We argue that promotions and career progression provide grounds for individuals’ strategic actions in the communication of scientific results, which have the potential to harm science as it centres on individuals and furthers scholars from the collective view of science. Even though there are programs such as training in responsible conduct of research (Anderson, Reference Anderson2007), the publication pressure associated with academic career progress should not be underestimated. In most cases, recruitment processes are organised by academics themselves, which suggests that academics should become more aware of publication pressures’ potential effect on early career scholars, which may unintentionally harm science.
Finally, we started our discussion with the example of p-hacking, which is a QRP that has received attention within and outside academia. The debate originated mainly in quantitative-based research such as psychology. However, QRPs are not limited to quantitative sciences but occur also in qualitative research. Our general theorising also holds for such research methods. The coding process, the question of which interview to use in the analysis, how to interpret data, or which information to present are all questions that concern the fine line between justifiable research practices and QRPs. In this sense, we believe that our normative assessment of research practises could be applied in principle to all scientific methods and all academic fields. It could be the starting point for a wider discussion on what should be expected from scientists.
Christian Linder is an associate professor of strategy and leadership at ESCP Europe Business School’s London (UK) campus and the academic director of the MBA in International Management programme. He holds two PhDs, one in philosophy (Munich School of Philosophy, Germany) and one in technology management (University of Stuttgart, Germany). Linder’s research interest centers around entrepreneurship and small businesses, with a particular interest on configurations as well as philosophy of science and ethics applied to management and management studies.
Siavash Farahbakhsh is a PhD candidate at the Free University of Bozen-Bolzano. He holds an MSc in management engineering from Politecnico di Milano and a BSc in materials engineering from Tehran Polytechnic. His research is focused on the complexity of social systems and institutional change processes. In doing so, he uses mainly qualitative and computational methods. Qualitatively, Farahbakhsh investigates the underlying decision-making models of different actors within institutions. With a simulation approach, he develops agent-based models to understand how the corresponding institutions are constructed, sustained, and changed.
