In his essay, Evan Lieberman makes a very compelling argument: research and knowledge accumulation in political science, as in the biomedical sciences, develop in stages. In both fields, different methodologies are applied to tackle important problems and questions. Research designs aimed at descriptive inference, statistical correlations, or experimental causal inference should build upon the findings of prior research in the topic of interest, irrespective of the methodologies used. Lieberman makes the excellent point that while in political science “the bar for what should be trusted as causal evidence has certainly been raised,” (ms. p. 12) it has come at the cost of neglecting or underestimating what the discipline can learn (or continue learning) from observational studies. Lieberman advocates for increased pluralism in the types of research published in our top disciplinary journals.^{Footnote 1} Lieberman then derives a series of recommendations for journal editors, reviewers, and graduate students, including a discussion about the utility of pre-analysis plan registration in different strands of research. I wholeheartedly agree with Lieberman’s plea for increased pluralism in research accepted for publication by our top journals and with his recommendations.

Two points remain, nonetheless, where I would probe Lieberman’s essay. First, I would prefer to conceptualize the research cycle of political science as non-hierarchical. The research cycle, as presented by Lieberman, entails a hierarchical ordering. It is a cycle that “progresses from pure description, through correlational analyses and natural experiments, to phased randomized controlled trials (RCTs).”^{Footnote 2} I would prefer to conceptualize these stages as part of a continuous, non-laddered, research cycle or process of refinement of findings and knowledge accumulation. As I envision it, the research process is a set of professional practices that permits the advancement of our knowledge of the political world, with no hierarchy regarding more or less progressed or causally elucidating methodologies. I would argue that in political science, depending on the topic or the characteristics of the units of analysis, the research process may productively end at the observational stage. In other instances, the experimental stage may feed into (rather than solely be fed by) other stages of research. In other words, the cycle is circular, not teleological.

For instance, the most important and founding topics of our discipline (issues of state consolidation, social revolutions, democratization, peace and war) cannot be randomized, or do not present themselves to us as if randomly distributed. Lieberman argues that we should nevertheless continue to study these phenomena, and I would add that not to do so would risk trivializing our discipline.

Moreover, where the assumption of unit homogeneity does not hold, RCTs are poor tools for knowledge accumulation, and political science research is better served by other methodologies. This is also often true in the biomedical sciences. Take the case of cancer research, for instance. The ontology of what cancer is, thanks to advances in genetics, has radically changed in the last two decades. Since the 2000s, cancer researchers and oncologists are abandoning the RCTs model of research and treatment (which were the gold standard of the 1980s and 1990s), in favor of genetically-informed and individually-tailored cancer treatments.^{Footnote 3} Similarly, in the social sciences, when the assumption of unit homogeneity cannot be sustained, the research process may productively start and end with observational case studies.

In political science, the research process may also lead from RCTs to observational stages of research. Where temporal and spatial contexts interact fundamentally with our treatment, RCT findings may trigger other studies that apply different methodologies to elucidate issues of external validity. In the words of the authors of an experimental study, “laboratory experiments—which undoubtedly have contributed immensely to the understanding of human behavior—strip context away and are limited in their ability to replicate the mutual trust, past experience, shared norms, and group identity that are central for balancing tension between private and public interests.”^{Footnote 4} In such cases, as the authors show, RCTs are not the end but the beginning of a research process that moves towards an observational stage that can causally take context into account.

Second, I would also argue, counter to Lieberman, that size matters. According to Lieberman, to “the extent that researchers develop strong and credible causal research design for testing well-motivated causal claims, we should be less concerned with the extent to which effect sizes are small or large as a criteria for publication or for professional merit more generally” (ms. p. 25). Perhaps Lieberman is right that small effects may sometimes merit publication. However, null or very small substantive effects of RCTs may be symptomatic of the deployment of an ill-fitting tool to study the problems at hand. If causality is primarily interactive or conjunctural, or if historical legacies and other contextual factors matter, would it not be better to start with a research design that tackles complex causality head on and deals with it?^{Footnote 5}

For example, experimental studies on participatory and monitoring institutions have extensively shown small or no effects. As argued elsewhere, this is due to the fact that participatory institutions are treated as exogenous to the policy-making process that leads to their creation.^{Footnote 6} While much can be learned from the isolation of an institution as a cause, if the effects of participatory institutions are conditional on the historic process of their creation and institutionalization, then parachuting this institution as a random treatment, is likely to produce small size effects because of the research methodology employed.^{Footnote 7} As the authors of an experimental research design write: “The simplest explanation for the weak effects [of a community driven development program] on governance outcomes is that existing structures are resilient and that while behavior may change temporarily to meet the conditions of development actors, more fundamental change is not being achieved.”^{Footnote 8} Those resilient structures are precisely the ones subject to change when participatory institutions are endogenously generated via social mobilization. In other words, the RCT methodology is not adequate to study participatory institutions. This is the main reason, I suspect, that explains why treatment effects of randomly imposed participatory or monitoring institutions are consistently small. A mismatch exists between the ontology of the problem to be investigated and the research methodology used.^{Footnote 9}

Notwithstanding the above issues, I believe Lieberman’s essay comes at a much-needed time for our discipline and its future relevance. My hope is that this discussion may dissuade future political scientists from deciding on dissertation projects based on issues of identification strategies, and thus go a long way toward avoiding the trivialization of political science.