Registered Reports are appropriate for research with existing datasets

The very common concern that Registered Reports are only appropriate for certain types of study designs is almost always wrong. Much of the initial development and implementation of Registered Reports did indeed occur in the context of new data gathered via lab experiments (Chambers, Reference Chambers2013), a context that does not reflect the full breadth of research in developmental psychopathology. Rather, our field relies heavily on longitudinal data, analyses of secondary data, and potentially identifying data, all of which are less controlled than lab experiments.

The good news is that this problem has been long known and fully addressed. A recent special issue of Child Development featuring Registered Reports (Syed et al., Reference Syed, Frank and Roisman2023) included articles across nearly the full spectrum of developmental science, including lab studies, interventions, meta-analysis, and national public data. Similarly, a special issue of Journal of Research in Personality featured studies using longitudinal, secondary, and sensitive data (Tackett et al., Reference Tackett, Ritchie, Syed and Weston2021). Two special issues have focused specifically on Registered Reports with secondary data (Davis-Kean et al., Reference Davis-Kean, Ellis and Syed2024; Syed & Donnellan, Reference Syed and Donnellan2020), and there is both a template for preregistering secondary data analyses (https://osf.io/x4gzt/), and an associated tutorial paper (van den Akker et al., Reference Van den Akker, Weston, Campbell, Chopik, Damian, Davis-Kean, Hall, Kosie, Kruse, Olsen, Ritchie, Valentine, Veer and Bakker2021). Guidelines and recommendations have also been developed for Registered Reports with qualitative studies (Demkowicz & Hickman Dunne, Reference Demkowicz and Hickman Dunne2024; Karhulahti et al., Reference Karhulahti, Branney, Siutila and Syed2023). Peer Community in (PCI) Registered Reports – a journal-independent preprint review service – has developed extensive resources and guidelines for preparing Registered Reports that covers a wide variety of study designs (see https://rr.peercommunityin.org/; (PCI Registered Reports, n.d.).

Whereas Registered Reports can be conducted with preexisting data, doing so requires close consideration of researchers’ prior knowledge of the data. The situation is not as simple as thinking dichotomously in terms of new or existing data. With large-scale datasets, researchers may have previously worked with some measures or sub-samples but not others, or may have worked with prior waves in an ongoing longitudinal project. All of these scenarios may be well-suited to conducting a Registered Report. What is important is that researchers are transparent about their experience with the data, and we suggest including a subsection titled “Prior knowledge of the data” to make it clear to readers (see Duncan et al., Reference Duncan, Nordgren, Schmitt and Vandell2024 and Spiegler et al., Reference Spiegler, Zingora and Jugert2024, for examples). PCI Registered Reports have developed a helpful bias-control taxonomy for researchers to understand and declare their prior knowledge of the data, if any (PCI Registered Reports, n.d.).

An additional consideration when conducting Registered Reports with preexisting data is that the review process may be more likely to identify “fatal flaws” in the Stage 1 proposal that cannot be addressed as the data have already been collected, a situation not relevant to Registered Reports based on new data collection. Thus, altogether there are clearly additional constraints when it comes to conducting Registered Reports with preexisting data, but they are nevertheless possible and, indeed, desirable.

Registered Reports allow for exploratory research

Registered Reports are perfectly consistent with conducting exploratory analyses. Researchers are welcome to conduct as many nonregistered exploratory analyses as they wish, but they must be clearly labeled as such. Moreover, the inferences based on these analyses need to be appropriately calibrated (e.g., as having weaker evidentiary value than confirmatory analyses, or as generating hypotheses to be tested in future research), and they cannot be overly highlighted in the manuscript (e.g., the abstract cannot report only the exploratory findings, without mentioning the outcomes of the preregistered confirmatory analyses).

Some researchers claim to conduct only exploratory research, and so the hypothesis-testing focus of Registered Reports is not well-suited. This may be the case and that is fine; exploratory research is crucial for science. If you are truly engaging in an exploratory study, then the Registered Report format might not be a good fit. The argument is not that every study needs to be a Registered Report. However, it is crucial that exploratory research is reported as such, rather than masquerading as confirmatory. As Daniel Nettle (Reference Nettle2021) notes, “People often misunderstand the preregistration revolution as being the requirement to only do confirmatory analyses. But (…) it’s not this at all. It’s the freedom to do confirmatory analyses when these are appropriate, and exploratory ones when these are appropriate, and be clear and unashamed about which it really is: don’t muddle the one with the trappings of the other.”

That said, in our experience, researchers will often claim they are conducting exploratory research, but in reality they have clear hypotheses that they seek to test – sometimes more tentatively referred to as “expectations.” Such an approach is ill-advised; if you have hypotheses, you should state them clearly and test them appropriately, preferably via a Registered Report.

Registered Reports can increase theoretical rigor

A concern expressed about science reform efforts is that they are too focused on procedures – checklists, standards, and rules – at the expense of core concerns such as theory, validity, and inference (e.g., Penders, Reference Penders2022). At first glance, this criticism could be levied at Registered Reports, which seem to emphasize planning and protocol over serendipity and exploration. As we have already discussed, and will elaborate later, these perceptions are ill-founded. In fact, Registered Reports can facilitate theoretically-rich research and are well-suited to improving theory. Psychology has long been plagued by a reliance on vaguely specified verbal theories (Borsboom et al., Reference Borsboom, van der Maas, Dalege, Kievit and Haig2021; Frankenhuis, Panchanathan, et al., Reference Frankenhuis, Panchanathan and Smaldino2023; Fried, Reference Fried2020; Guest & Martin, Reference Guest and Martin2021; Smaldino, Reference Smaldino2020; van Rooij & Baggio, Reference van Rooij and Baggio2021). Despite the fact that theory is central to research on developmental psychopathology in general, and in the pages of Development and Psychopathology in particular, the vast majority of theories in the field are gravely underspecified. That is, there is a lack of clarity about the observations that would be consistent or inconsistent with the theory. Rather, most theories are sufficiently flexible that nearly any pattern of data can be claimed as showing support. For instance, a theory can be re-interpreted more expansively to absorb data outside of its original scope (theory stretching), or more narrowly so the theory appears more precisely aligned with the data (post hoc precision; Frankenhuis, Panchanathan, et al., Reference Frankenhuis, Panchanathan and Smaldino2023). Moreover, there are few constraints on the number and type of tests that a researcher can conduct to find support for a vaguely specified theory. Taken together, these practices slow, rather than contribute to, theory development.

Registered Reports can be helpful in this regard. As completed empirical products, Registered Reports will generally provide reliable results that can be used to build and refine theories. The format can also be used to generate stronger tests of existing theories. When implemented well, they encourage greater specificity about predictions that constitute stronger tests of hypotheses than standard reports. The degree to which they do so, however, depends on those involved. It is incumbent on researchers to work to improve specificity as a path to greater theoretical specification, and it is incumbent on reviewers and editors to demand it. A good starting point is to become familiar with the concepts of theoretical and empirical estimands, which pertain to what specific quantity we seek to estimate and how we will go about doing so (Lundberg et al., Reference Lundberg, Johnson and Stewart2021). With more time and experience, researchers can then begin to use computational modeling and simulations as a way to work out their theoretical expectations (Fraley et al., Reference Fraley, Roisman and Haltigan2013; Frankenhuis, Borsboom, et al., Reference Frankenhuis, Borsboom, Nettle and Roisman2023; Frankenhuis, Panchanathan, et al., Reference Frankenhuis, Panchanathan and Smaldino2023; Grahek et al., Reference Grahek, Schaller and Tackett2021) and even make hypothesis tests machine-readable (Lakens & DeBruine, Reference Lakens and DeBruine2021; Van Lissa, Reference Van Lissa2023). The value of this level of specificity is evident with replication studies, in which there is often disagreement about whether or not a particular replication attempt was successful in replicating the original result. This situation arises because there is not one criterion for determining replication (i.e., not a single estimand), and so researchers can selectively focus on estimands after their values are known. Setting the terms for what would constitute a successful replication before seeing the results is one way to contribute to a theoretically progressive field. At Development and Psychopathology, researchers conducting replications are encouraged to do so as Registered Reports, or even as Registered Replication Reports, which constitute coordination across many independent research labs to each carry out an identical study protocol and then aggregate the results (Simons et al., Reference Simons, Holcombe and Spellman2014).

Registered Reports allow for deviations from the research plan

When journals guarantee publication following the in-principle acceptance, that guarantee is conditional on researchers carrying out the study as approved, and doing so competently. But most studies don’t go quite as planned. If you have received an in-principle acceptance for your project and anticipate some necessary changes to the study, the best course of action is to get in touch with the editor who issued the decision (Henderson & Chambers, Reference Henderson and Chambers2022). They may provide approval for the changes, may determine that the changes need to be passed by the reviewers, or perhaps are grounds for withdrawal of the in-principle acceptance. If changes need to be made on the fly (e.g., during a testing session), authors may contact the editor afterwards and explain why they deviated from their plan. In our experience, editors tend to be sympathetic and supportive; however, we are not aware of empirical research on this topic. In general, though, a nice feature of Registered Reports is that they create a system in which dialog between parties is baked into the process, and authors would be wise to take full advantage. If the manuscript ultimately must be withdrawn as a Registered Report, the authors always have the option to conduct the study anyway and submit it as a traditional manuscript (while having benefited from the Stage 1 reviewer feedback).

Registered Reports do not require embracing “open science”

Open science is a broad term with no singular agreed-upon definition (see Silverstein et al., Reference Silverstein, Elman, Montoya, McGillivray, Pennington, Harrison, Steltenpohl, Röer, Corker, Charron, Elsherif, Malicki, Hayes-Harb, Grinschgl, Neal, Evans, Karhulahti, Krenzer, Belaus and Syed2024). Parsons et al. (Reference Parsons, Azevedo, Elsherif, Guay, Shahim, Govaart, Norris, O’Mahony, Parker, Todorovic, Pennington, Garcia-Pelegrin, Lazić, Robertson, Middleton, Valentini, McCuaig, Baker, Collins and Aczel2022) provided a succinct and inclusive definition: “An umbrella term reflecting the idea that scientific knowledge of all kinds, where appropriate, should be openly accessible, transparent, rigorous, reproducible, replicable, accumulative, and inclusive, all which are considered fundamental features of the scientific endeavor” (p. 312). This definition clearly comprises a vast array of principles and behaviors, some of which individual researchers may value highly, whereas others they may treat with deep skepticism. Some may perceive the heavy emphasis on replicating past studies as misplaced (Feest, Reference Feest2019) or may be dubious of open science because of its perceived lack of consideration for research focused on diversity in psychology (Fuentes et al., Reference Fuentes, Zelaya, Delgado-Romero and Butt2022). Concerns about one aspect of open science, then, could lead to a rejection of all others.

The good news is that open science is not “all or nothing” (Corker, Reference Corker2018), and researchers can and should think about the principles and practices that are most relevant to their research methods and goals (Bergmann, Reference Bergmann2023; Nuijten, Reference Nuijten2019). Publishing Registered Reports does not require researchers to embrace open science generally or identify with open science in any way. Rather, open science offers a buffet of principles and practices, from which we can select those that are relevant for enhancing the quality and impact of our work (Bergmann, Reference Bergmann2023).

Registered Reports offer more benefits than preregistration alone

Preregistration has some overlap with Registered Reports, with a few crucial differences. Registered Reports subsumes preregistration, in that preregistration is automatically a part of the Registered Report publication process – this is the function of the Stage 1 proposal that receives the in-principle acceptance from the journal. Unlike with Registered Reports, preregistrations do not figure heavily into the manuscript review process. Researchers frequently deviate from their preregistration plan, which on its own is not a problem, but these deviations are too often undisclosed in the manuscript (Claesen et al., Reference Claesen, Gomes, Tuerlinckx and Vanpaemel2021; Willroth & Atherton, Reference Willroth and Atherton2024). Moreover, there is evidence that reviewers and editors rarely check the preregistration plans during the review process (Syed, Reference Syed2025). Most centrally, journals do not commit to publishing a study based on a preregistration. Researchers can produce a high-quality preregistration and engage in detailed reporting, but a journal could still reject the paper if the results are not deemed sufficiently novel or not consistent enough to warrant strong conclusions. Thus, preregistration does not sufficiently contribute to a shift away from selection based on findings and toward selection based on conceptualization and design.

Registered Reports offer greater control over timelines

There is a perception that Registered Reports take longer to publish than standard reports. It is not so clear that this is the case, and there are no good data to bring to bear, partially due to the difficulty in identifying the proper comparison (see Syed, Reference Syed2022 for discussion). It is true that the review process for Registered Reports will likely take longer given that the review is split into two stages. However, because of the in-principle acceptance of the Stage 1 proposal, Registered Reports greatly reduce engagement in the submission-rejection cycle in which manuscripts are “shopped around” until they “find a home.” Although we lack the appropriate comparative data, Registered Reports provide researchers with greater control over their timelines and with a sense of comfort that accompanies the certainty that their paper will be accepted at the journal that issued the in-principle acceptance (as illustrated in Figure 2). Additional innovations such as the scheduled review track at PCI Registered Reports, in which reviewers are lined up in advance of a future target submission date, provide researchers with even more predictability and efficiency. The preceding notwithstanding, we are well aware that there is a perception that Registered Reports take longer to publish than standard reports, and absent strong data, those perceptions cannot be empirically corroborated or refuted. Timely review is a concern for all publication formats, across all of the sciences, and as Registered Reports increase in prevalence, so too will data that can speak to this understandable concern.

Figure 2.

A cartoon illustrating the anxiety over needing significant results in the traditional publishing model vs. the peace of mind that comes with Registered Reports. Used with permission from Henderson (Reference Henderson2019) under CC BY 4.0.

Registered Reports afford a more positive and collaborative review process

The purpose of the Stage 1 review process is to ensure that the proposed project will be informative of the research questions regardless of the results. This is done, as publishing generally is, with input from reviewers and an editor. Because the study has not yet been conducted, however, there could be concerns about reviewers having too great an influence over the direction and content of the study. For example, reviewers may ask researchers to answer the questions that they would like to see answered (e.g., “In your planned data collection, it would be great to also measure [reviewer’s preferred topic].”) or may request that authors use their preferred instruments (e.g., “To measure [reviewer’s preferred topic], you can use the scale I have developed”). These are certainly plausible scenarios. However, concerns about reviewers having too much power are certainly not new to Registered Reports. Authors have long complained about the demands of reviewers, so much so that “Reviewer 2” is frequently used as shorthand for the general annoyance of reviewer behavior. This is anecdotal, but in our personal experience the review process for Registered Reports is more positive and more collaborative. Not surprisingly, reframing the job of reviewers from “how do I find fault with this” to “how do I help this be the best project it can be” leads to a more positive experience for all.

There are resources to support early career researchers

Registered Reports reflect a major change in the approach to publishing research, and thus can be unsettling and disruptive to those who are used to, or comfortable with, the existing system. If you have supervisors or collaborators who are skeptical about open science, taking a gradual approach could be productive. Kathawalla et al. (Reference Kathawalla, Silverstein and Syed2021) developed a resource for “easing into” open science, which helps researchers to get started without feeling overwhelmed or experiencing a radical change to their process. Silverstein et al. (Reference Silverstein, Elman, Montoya, McGillivray, Pennington, Harrison, Steltenpohl, Röer, Corker, Charron, Elsherif, Malicki, Hayes-Harb, Grinschgl, Neal, Evans, Karhulahti, Krenzer, Belaus and Syed2024) created a parallel resource for cautious journal editors. The article, “Ten simple rules for writing a Registered Report” by Henderson & Chambers (Reference Henderson and Chambers2022) is required reading for anyone venturing into Registered Reports for the first time (as well as for those with experience). Finally, as noted, PCI Registered Reports has developed a wealth of resources to support interested researchers (PCI Registered Reports, n.d.).

It is also worth noting that Registered Reports very closely resemble how PhD dissertations are handled in the U.S. and several other countries. Students develop a proposal of their plan, which is then reviewed by a committee and ultimately approved, at which point the student can conduct the project. The (un)official contract between the student and the committee is that if they do what they proposed, and do so competently, then they will pass their final exam. That is, the PhD degree is based on strong conceptualization and design, and not contingent on producing a particular set of findings. This is nearly identical to the process of Registered Reports. If this approach is the standard for awarding PhDs, then it should be the standard for publishing research.

Registered Reports help address most malignant behaviors

Registered Reports will not solve all of the problems with how research is conducted or disseminated. Transparency itself is a positive value, but it does not ensure credibility; rather, transparency allows for a stronger assessment of credibility (Vazire, Reference Vazire2017). Researchers can still suppress null or contradictory findings and reframe exploratory findings as if they were preplanned. Journals can still display bias in the types of projects for which they issue in-principle acceptances, and can make unreasonable demands during the Stage 2 review process. And, of course, they do nothing to prevent outright fraud. The reality, however, is that Registered Reports greatly reduce these behaviors, while better aligning the values and incentives in science (Chambers & Tzavella, Reference Chambers and Tzavella2022; Nosek et al., Reference Nosek, Spies and Motyl2012). In our experience, when some of these problems have come up, it was related to editors’ lack of expertise with the format. Knowledgeable editors who are trained in how to handle Registered Reports can ensure that the practice comes as close as possible to the ideal. Fortunately, the editorial team at Development and Psychopathology meets this standard, and so researchers and readers are in good hands.