Amid a micro-revolution in empirical research designs and experimental methods in our discipline, ^{Footnote 1} this article discusses the distinctive perils in monitoring unethical practices and errors in this new research tradition. Because political science needs to improve its capacity to address ethical dimensions of increasingly common research practices, we propose preventive procedures that should be considered as part of an already active disciplinary discussion on a wide range of ethical standards. ^{Footnote 2} We aim to participate in an already broad conversation about norms and rules of oversight that do not sacrifice scientific freedom and creativity. We emphasize measures that we believe will improve the incentive structure for transparency and regulation in political science.

Why pay special attention to our micro-revolution? A major reason is that our findings have more immediate relevance to a wider audience than was previously the case. A classic study of an earlier era, Moore’s Social Origins of Dictatorship and Democracy (1966), for example, was a landmark contribution to the study of democracy. However, almost nothing would have changed in the world outside of social science if errors were found in one of his analyses. We compare that to a recent paper on democracy that purported to show that transparency in parliamentary debates in partial democracies tends to reduce participation in debates and threatens reelection (Malesky, Schuler, and Tran Reference Malesky, Schuler and Tran2012). In this case, the implications for democracy-promotion programs in the world of international aid are immediate and the costs of error for wider society much higher.

It is not that political science in the past has been politically irrelevant. Rather, the micro-findings with clear causal attributions for well-identified treatments tend to have more immediate and obvious applications. For example, the findings on how best to “get out the vote” are adopted rather quickly by electoral campaigns and the findings on how to address voter fraud in new democracies affect aid-program design. We compare this with the more indirect effects on budgets and strategies when political scientists in the past served as expert witnesses in voting-rights trials, or as advisers and consultants for state and federal campaigns, or in high-level positions in the American foreign-policy establishment.

Immediate relevance increases not only the costs of error but also the incentives to cheat. Several factors are involved. First, the greater the immediate impact of our field on wider society, the greater the incentive to present results that advance one’s political or moral interests. Second, with an increasing number of well-designed studies all attacking the same problems (e.g., conditions favorable to democratic stability) using the same tools, the space for a truly original finding is narrowing. To make a mark on the discipline today is a genuine challenge; falsifying data or misportraying findings can ensure originality. Third, in part because of a diminution of public funding for social science, recent studies are increasingly funded by private donors who have an interest in the findings and who put considerable pressure on researchers (who hope for continued funding) to tailor questions and adjust findings consistent with the funder’s goals.

Despite stronger incentives to cheat—or at least powerful incentives to exaggerate marginal findings and/or hide nonfindings—our discipline has not matured in a parallel manner to curb the unethical behavior that may follow. Ethical research practices are not a core—or even a peripheral—element of graduate training; furthermore, we have weak or nonexistent institutionalized policing mechanisms. Indeed, the American Political Science Association (APSA) lacks the personnel and—until now—the will to budget for disciplinary oversight that is becoming increasingly important. We hope that the widely publicized allegations of data falsification in a 2015 experimental design will spur concerted and proactive efforts for a disciplinary remedy (Singal Reference Singal2015). Political science should not wait for what happened in economics: a popular documentary, Inside Job, exposed unsettling ethical transgressions at the heart of the discipline.

To be sure, several innovations from within our research community have helped us to monitor ourselves. In 1995, Gary King published “Replication, Replication” in this journal in which he advocated, at a minimum, a replication footnote for all data-driven contributions to our scientific journals. This idea at the time was so radical that it was voted down at the annual business meeting of the APSA Comparative Politics Section. However, norms have changed. We consider a recent paper on the use of interaction terms in regression models. The authors requested the raw data from more than 20 scholars and all but two authors complied (Hainmueller, Mummolo, and Xu Reference Hainmueller, Mummolo and Xu2016). This is genuine progress; however, even with this progress, we cannot rely only on replication to identify flawed findings and errors, if only because self-correction works very slowly.

More recently, the Berkeley Initiative for Transparency in the Social Sciences (BITSS) was founded under the leadership of Edward Miguel (University of California, Berkeley) and Kevin Esterling (University of California, Riverside). With strong support from philanthropy, BITSS is now active in a range of transparency and replication initiatives. Most prominently, it is promoting preanalysis plan registration that inter alia would deter common practices of ex-post subgroup analyses that inappropriately permit researchers to attain standard significance levels for their statistical models—or, in more common parlance, to “go fishing” for a compelling result. ^{Footnote 3}

These self-monitoring efforts have two main effects. First, the emphasis on replication and preanalysis plan registration increases the likelihood that p-hacking will be identified, thereby deterring fishing for significant results. Second, these efforts correct well-intentioned but flawed research findings. On this latter dimension, it is perhaps more appropriate to say that replication and preanalysis plans are useful but limited mechanisms for the practice of good science.

We think more should be done. Political science should promote further institutional mechanisms to ensure ethical practices that are incentive-compatible with encouragement to sustain scientific creativity. Applying the evocative metaphor of McCubbins and Schwartz (Reference McCubbins and Schwartz1984), we can assume that scientific police patrols will stifle creativity. In their stead, we need to cultivate effective “fire alarms” (that actually activate responders and need not assume scientific malpractice) to serve as warnings of error or violations of scientific norms. In addition, we must cultivate—early in graduate training—clear norms of good scientific practice that will diminish the likelihood of “fire” in the first place. We need the equivalent of flame retardants in our labs.

This is no easy task. We are humbled by the evaluation of Neuroskeptic (2012), which showed “circles of hell” housing manifold levels of bad science. Moreover, it is no mean task to separate ethical misconduct from intradisciplinary debates about what constitutes best practices. We do not want to provide ammunition to scholars embroiled in disciplinary debates that allows them to discredit work from other research traditions through accusations of unethical practices. However, fears that we might move from disuse to overuse of ethical fire alarms should not deter us from a search for better practices. A few suggestions follow, more to sustain debates in our discipline than to lay out a full-scale plan.