Positionality statements

Author 1 is a second-language psycholinguist whose training and intellectual home has been firmly rooted in quantitative, experimental approaches to studying the cognitive mechanisms in additional-language learning. This background has shaped her assumptions about what constitutes methodological rigor, with concepts such as reproducibility, generalizability, and design control functioning as central markers of research quality. Until recently, she had limited exposure to qualitative research traditions, and she therefore engaged with them largely from a distance rather than through direct practice. She is excited about the potential of Open Research to transform the way applied linguists do research, believing it can improve access to research materials and findings for people worldwide, promote a collaborative mindset whereby researchers build more directly on each other’s work, and ultimately improve the transparency and quality of research. These commitments also shape her mentoring of Ph.D. students and her editorial work at Language Learning, where she has sought to make Open Research practices more visible and more mainstream in the field. More recently, however, her teaching and mentoring—particularly of Ph.D. candidates pursuing qualitative work, including some advised by her co-author—have prompted her to engage with different methodological worldviews more directly. She enters this dialogue conscious that her quantitative orientation and enthusiasm about Open Research may introduce blind spots and biases. At the same time, she views her status as a relative newcomer to qualitative research traditions as an opportunity to listen closely, learn from new perspectives, and ask foundational questions about openness with both curiosity and humility. This dialogue therefore began from her strong commitment to Open Research, but it also required her to examine more carefully what hidden assumptions may underlie openness and whether openness should ever be tempered by epistemological, relational, or ethical considerations.

Author 2 is a critical applied linguist originally from the Global South whose work in the areas of language identity, ideology, and emotions has been guided by qualitative methods. As a sociopolitically oriented applied linguist who is committed to educational justice, the disciplines from which he primarily draws are sociology and anthropology. Epistemologically, he seeks to better understand how power inequities in language learning emerge as a result of limited access to or a lack of access to resources. Because such resources are ecologically distributed, emergent inequities often occur at the micro-level of the classroom, the meso-level of the school, and the macro-level of society. Ontologically, he subscribes to the view that there is not one singular perspective of reality. Rather, because multiple realities exist—with each reality tied to an individual’s contextually and culturally situated identity position—it is the onus of the researcher to learn about these multiple realities in order to better understand a given phenomenon under investigation. To this end, he has found ethnography to be particularly helpful in trying to unpack the language learning experiences of multilingual students, whose rich, individual lived experiences and encounters with unequal access to material and cultural resources shape their respective experiences. Like his co-author, he has worked with students who are enrolled in their applied linguistics doctoral program. He is a newcomer to Open Research, however, and warmly welcomed the opportunity to start this dialogue with his co-author. Through this dialogue he came to understand how Open Research, when understood beyond replication-oriented logics, might align with qualitative commitments to reflexivity, auditability, relational accountability, and ethical transparency. Having said that, however, he also recognizes that not all Open Research logics, such as the generalizability of findings as a yardstick of research quality adopted in quantitative research, should unproblematically extend to qualitative research.

What does transparency mean to me? Why is it important?

Aline: Transparency is a characteristic of research quality. At its core, transparency is about allowing others to see, inspect, evaluate, scrutinize, and ultimately benefit from what the researcher did at all stages of the research process. Transparency is often used as a shorthand for methodological transparency (Al-Hoorie et al., Reference Al-Hoorie, Cinaglia, Hiver, Huensch, Isbell, Leung and Sudina2024; Marsden, Reference Marsden, McKinley and Rose2020), but that doesn’t mean that transparency is about open materials, open data, and open code only. The need to be transparent as a researcher starts well before designing the materials for a study, when planning and conceptualizing the research, and it continues past the data analysis into the reporting and dissemination stages of the research. Therefore, transparency—as a key value of Open Research and an indicator of research quality—should permeate all stages of the research process.

Let’s say researchers wanted to investigate for whom audiobooks might be especially beneficial for developing second-language (L2) comprehension skills, as Hui and Godfroid did in a study that led me to experience both the strengths and limitations of Open Research firsthand (Hui & Godfroid, Reference Hui and Godfroid2025). We entered the study with the popular assumption—grounded in teaching practice and some empirical evidence—that reading-while-listening leads to better text comprehension than silent reading. Building on this assumption, we set out to test two hypotheses about the cognitive mechanisms that support audiobooks’ presumed advantages for text comprehension: namely, that the combination of written text and audio supports learners’ audio and text decoding and that this, in turn, frees up cognitive resources for higher-order comprehension. For full disclosure, this study was conducted in a research lab on our university campus with international students studying at the university—in other words, a convenience sample of educated college students. This makes the findings relevant primarily to this demographic and naturally constrains their generalizability to other multilingual users (Andringa & Godfroid, Reference Andringa and Godfroid2020; Godfroid & Andringa, Reference Godfroid and Andringa2023).

This audiobook study is an example of confirmatory, hypothesis-testing research: The researchers adopt a scientific method to “improve models by generating hypotheses based on existing observations and testing those hypotheses by obtaining new observations” (Nosek et al., Reference Nosek, Ebersole, DeHaven and Mellor2018, p. 2600). The example highlights that transparency starts well before the design of research instruments. The researchers formulate hypotheses and make predictions a priori. In the case of Hui and Godfroid’s study, we also preregistered the hypotheses and predictions and submitted them as part of a registered report—a special, open-research article type—to a journal for peer review before the data for the study were collected. It took several rounds of review before our registered report received in-principle acceptance. Once it did, however, the journal committed to publishing the study regardless of whether the results were aligned with our predictions. I firmly believe that, without the registered report and its in-principle acceptance, we would have published a different study.

The example, therefore, underscores that transparency does not end once the data have been collected and analyzed. In the reporting phase, being transparent means that researchers present all results—including those that contradict expectations or were not statistically significant (i.e., inconclusive)—so that readers gain a complete picture. In the case of Hui and Godfroid’s study, the basic hypothesis—that reading-while-listening leads to better comprehension—was not supported; if anything, the results pointed in the opposite direction. In cases like that, being transparent becomes especially important. Without the Open Research component, we might have been tempted to shift the aims of the study, for instance, by modifying a research question, omitting variables or introducing new ones, or changing the analysis. As many of us can relate, research is often unpredictable and even “messy” in this way. However, being transparent about the messiness—not trying to hide it—means telling “the whole/real story no matter how imperfect” and resisting the temptation to “craft a good/sanitized methodological story” (Al-Hoorie et al., Reference Al-Hoorie, Cinaglia, Hiver, Huensch, Isbell, Leung and Sudina2024, p. 541). I should note that although this is the ideal we aspire to in principle, our everyday practices sometimes fall short of this ideal. For instance, survey-based research on questionable research practices has shown that a substantial proportion of researchers admit to selectively reporting results and excluding findings that run counter to their own or their colleagues’ previous research (Isbell et al., Reference Isbell, Brown, Chen, Derrick, Ghanem, Arvizu and Plonsky2022; Larsson et al., Reference Larsson, Plonsky, Sterling, Kytö, Yaw and Wood2023). For me, this experience sharpened the value of registered reports: They do not make research less messy, but they make the messiness harder to conceal and easier for readers to evaluate. They also taught me the value of approaching published evidence critically because publication alone does not guarantee that the full research process has been made visible.

Lastly, the commitment to transparency does not end with reporting—it extends into dissemination. Open access publishing allows individuals anywhere in the world to access research findings, which is essential if we want our knowledge to benefit society. For those of us at well-resourced universities with extensive library subscriptions and so-called transformative agreements with big publishing companies, it is easy to forget that most research articles remain locked behind a paywall. Imagine, for instance, a scenario that many multilingual families will recognize: a teacher educator working with teachers of linguistically diverse children in early childhood education. The work raises questions about how best to support multilingual learners in the classroom, the role of the home language, parental involvement, and other areas in which applied linguists have expertise. Yet without institutional access or funding to unlock research articles, the latest research insights remain out of reach for this educational community and the multilingual families who might benefit from them. This is a serious problem—and, as others have argued, an ethical one (e.g., Al-Hoorie et al., Reference Al-Hoorie, Cinaglia, Hiver, Huensch, Isbell, Leung and Sudina2024; Andringa et al., Reference Andringa, Mos, van Beuningen, González, Hornikx and Steinkrauss2024; Plonsky, Reference Plonsky2024). Because much research is funded through public resources, it follows naturally that its products should be publicly available. Open access publishing, therefore, is a keystone of the Open Research movement (Marsden & Morgan-Short, Reference Marsden and Morgan-Short2023). In this way, transparency comes full circle, guiding the research process from initial conceptualization, through materials, data, and analysis, to reporting and final publication.

Peter: I agree with your interpretation of transparency in that I believe that consumers of research should be afforded the opportunity to see, inspect, evaluate, and scrutinize the entire research process, warts and all, from the point of conceptualization of a study through findings dissemination. Ultimately, it is not just them who benefit but the broader field at large, too, because the primary purpose of research should be to build knowledge for the benefit of everyone. In other words, knowledge building needs to be cumulative, and attempts should be made within any given field—SLA included—to develop the infrastructure to allow for the consolidation of knowledge. At the same time, however, I have always felt that research needs to have social utility (Ortega, Reference Ortega2005), and thus have social value.

Lest I contradict myself, however, I would like to emphasize that what is perceived as bearing social utility also needs to be problematized. This is because not everyone might see a piece of investigative research as having value to society. And in some cases, a study may be viewed as being detrimental to the participants in a study and the communities to which they belong. Relatedly, having the entire research process exposed to scrutiny might also bring more harm than good, especially if cultural norms of the community to which the participants belong are seen as having been violated. A case in point would be research involving Indigenous communities that may prioritize compliance with community-defined protocols and consultation with community members before a study is undertaken (Coronel-Molina & McCarty, Reference Coronel-Molina and McCarty2016). To Indigenous communities who have been exposed to extractive processes over generations by Western-based scholars working within a positivist paradigm, a universal and externally imposed understanding of transparency would be perceived as an ongoing form of epistemic injustice and (re)colonization, especially if access to research materials were shared without engaging with them in dialogue and securing their consent. Or even worse, skewing what members of Indigenous communities have to say and thus distorting or overlooking their perspectives altogether. In other words, we need to avoid having a one-size-fits-all, universal approach to transparency that is imposed across all communities at the risk of demonstrating cultural insensitivity and disrespect toward certain vulnerable populations. There are and will be times when we need to accommodate some degree of productive opacity and seek community governance to ensure community accountability and ethical stewardship are observed. Notably, such a perspective is not inconsistent with the UNESCO Recommendation on Open Science (UNESCO, 2021), which calls for open dialogue with other knowledge systems that are mediated by local communities, marginalized scholars, and Indigenous scholars, among others. From my perspective, we also need to remember to do research with our participant collaborators and not on them to avoid engaging in extractive practices. In short, by working closely and in consultation with them through community-based participatory action research projects (e.g., De Costa, Ojha, Lee, & Montgomery, Reference De Costa, Ojha, Lee and Montgomery2024) that are deeply collaborative in nature, I think we can realize a common goal of generating knowledge for the greater good.

At this juncture, I would also like to clarify broad characteristics of qualitative research, namely, that it (1) is generally inductive, descriptive, and interpretive in nature, (2) favors data triangulation (i.e., elicits multiple perspectives and draws on several data sources), (3) is highly context dependent (i.e., takes into account multilevel contextual factors that may impact the phenomenon under investigation), and (4) requires researchers to exercise their reflexivity throughout the research process from conceptualization of the study through the dissemination of their findings (De Costa, Randez, Cinaglia, & Montgomery, Reference De Costa, Randez, Cinaglia, Montgomery, Gurzynski-Weiss and Kim2022), as stated earlier. Because of these aforementioned characteristics—primarily that qualitative research at its core inherently relies on context, subjectivity, and researcher reflexivity (Pratt, Kaplan, & Whittington, Reference Pratt, Kaplan and Whittington2020)—I find that attempts to replicate or “repeat” any study in a new context somewhat misguided because more often than not, language learners find themselves confronting unique sociopolitical circumstances—a point to which I’ll return shortly—that shape their learning experiences in ultimately unique, unreproducible ways.

As noted, the goal of qualitatively oriented SLA researchers is generally to understand the experiences, abilities, perceptions, and performances of language learners (instead of testing hypotheses or looking for cause-effect relationships in language learning). To illustrate, let me give you an example from my own longitudinal work (De Costa, Reference De Costa2015). In this earlier ethnographic case study, which spanned the course of one school year, I sought to understand the language learning experiences of a ninth-grade ESL learner from Indonesia, Daphne, and the broader sociopolitical context in which she was embedded. Daphne had received a scholarship to study in an English-medium secondary school in Singapore, and being a scholarship recipient, she was expected to perform well academically, lest her scholarship be revoked. The fear of losing her scholarship and having to manage the high academic expectations placed on her by her peers, teachers, and broader society because her scholarship was funded by the Singapore government made it extremely stressful for Daphne. As stated, context is key in qualitative research. In Daphne’s case, and also because she was situated in a distinctly Singaporean sociopolitical context, I had to develop a bottom-up (inductive) approach to understand what was going on in Daphne’s school life, an approach that was co-constructed not only with her but also with other research participants who worked closely with her. I collected multiple data sources (e.g., interviews, observations, learner and teacher artifacts) and sought the perspectives of different people (Daphne’s peers and teachers). These data sets were then triangulated to make sense of the micro-level classroom, meso-level school, and macro-level societal factors that impacted her language learning experience. Central to my study was describing this experience and the context in which Daphne was situated. In addition, I had to interpret my findings based on the insights gleaned from spending one year at my school research site.

Importantly, as you pointed out, research is messy. And this aforementioned study certainly turned out to be so. Designing the study, which was part of my conceptualization process, proved problematic insofar as I first had to negotiate access to a public school in Singapore. That is, I needed to find a school that was willing to grant me access to its students. After two failed attempts (I was turned away by two initial schools), I was successful in securing access to my eventual research site. As a former ESL teacher, I had worked with scholarship immigrant students like Daphne. Hence, I brought my own assumptions to the research site, which I described in my positionality statement (for a helpful overview of positionality statements, see King, Reference King2024). You had mentioned that transparency begins well before the design of the research instruments. In my case, I was transparent in that I explained why I chose to focus on the topic of language anxiety involving this specific group of scholarship students. But while I knew that students like Daphne faced intense pressure, I didn’t know at the time what the sources of these anxiety-driven pressures and their intensity were.

I also entered my research site with a theory (identity theory) in mind. But along the way, and based on my preliminary findings, I had to expand that theory to include emotions in order to understand the phenomenon under investigation. Specifically, I initially turned to the notion of Willingness to Communicate (MacIntyre, Reference MacIntyre2007), an emotion-related construct, to help understand Daphne’s challenges with communicating in class. But shortly after, as I collected even more data and analyzed them, I had to supplement that construct with the notion of a social imaginary (Ibrahim, Reference Ibrahim1999) because Willingness to Communicate could not adequately explain the sociopolitically inflected phenomenon I was investigating. In the spirit of ethnography (Hornberger, Reference Hornberger, Chalhoub-Deville, Chapelle and Duff2006), I ended up cycling back and forth between theory and my findings to come up with ways to better understand and explain Daphne’s learning experience.

It’s been 10 years since that study was published (for more details of my larger dissertation study, see De Costa, Reference De Costa2016). In retrospect, if I had to rewrite it for greater transparency, I would certainly have detailed the theoretical choices I had to make along the way. On a methodological level, I did explain why I elected to focus on just Daphne, one of five case participants who were part of my larger study, in my Language Learning article that sought to tell Daphne’s story. But I do concede that data excerpts were selected by me to craft her story. In addition, Daphne had decided which data sources would be made selectively available to me (see De Costa, Reference De Costa2014) because she deliberately wanted some aspects of herself to remain confidential. Some might argue that I engaged in cherry-picking, and hence was not being completely transparent. But to me, that is inevitable in qualitative research. How does one include all the data from a year-long study in a 10,000-word journal manuscript? From my perspective, such strategic selections do not constitute a questionable research practice, just as long as the rationale for making them is explained (which I did).

Finally, I do agree that as a field, we need to do better in making sure that access to research is available to everyone, not just the elite few who work at well-resourced universities with libraries that can afford to overcome the paywall behind which most research exists. This is just one manifestation of how structural inequalities exist. However, Open Research is not altogether free from power asymmetries either because the democratization of knowledge often comes with a high financial cost—a cost that disadvantaged Global South scholars and early-career scholars are often unable to bear. For example, equity-oriented practices such as Gold Open Access (i.e., publications that are free to read online but require payment of article processing charges by authors or their institutions) are generally unattainable to such scholars because the under-resourced institutions at which they work are often unable to cover the costs of such access. Thus, these groups of scholars find themselves in a catch-22 situation.

At the same time, for many qualitative researchers, making their work more accessible means deciding who their target audience would be. For example, for those of us who conduct school-based research, that entails sharing our findings through different dissemination platforms such as newsletters for teachers, if our target audience is practitioners. On a more positive note, I am glad to see that many applied linguists—regardless of methodological orientation—are starting to take active measures to make their work more accessible to wider audiences through a host of dissemination platforms such as TESOLgraphics (https://www.tesolgraphics.com/) and Open Accessible Summaries in Language Studies (OASIS; https://www.oasis-database.org/). Such a convergence of interests does give me hope that there are some key matters that we can agree on.

Why should researchers be methodologically transparent?

Aline: Thank you, Peter, for this illuminating example of how methodological transparency can take shape in ethnographic work and what factors might constrain it. I think we both agree that transparency encompasses the entire research cycle and that alternative forms of sharing research findings—beyond the traditional research article—are becoming increasingly important. I also agree with you that Open Access publishing could inadvertently reinforce existing inequities; fortunately, Diamond Open Access is free for both readers and authors because (unlike Gold) it does not require article processing charges (Andringa et al., Reference Andringa, Mos, van Beuningen, González, Hornikx and Steinkrauss2024). Finally, both of our examples highlight that researchers make countless decisions along the way, and that these decisions—large and small—shape the study outcomes and the final narrative we present.

In psychology and applied linguistics, scholars sometimes refer to these decision points as researcher degrees of freedom (e.g., Maie, Eguchi, & Uchihara, Reference Maie, Eguchi and Uchihara2024; Simmons, Nelson, & Simonsohn, Reference Simmons, Nelson and Simonsohn2011)—a term borrowed from statistical analysis—to describe the flexibility inherent in research. By making this flexibility visible, and thus promoting methodological transparency, we enable others to understand how researchers’ choices might have influenced the results and interpretations. In quantitative traditions, this concern has given rise to practices such as preregistration, open sharing of data and analytical code, and engaging in what is known as a multiverse analysis (Maie et al., Reference Maie, Eguchi and Uchihara2024; Steegen, Tuerlinckx, Gelman, & Vanpaemel, Reference Steegen, Tuerlinckx, Gelman and Vanpaemel2016). The multiverse approach to data analysis essentially tests how robust the findings are to different, reasonable ways of preprocessing and analyzing the data by systematically varying reasonable methodological decisions. The goal is to see whether the results depend on those decisions; ideally, the main conclusions remain stable across reasonable analytic choices. In this sense, transparency is less about exposing imperfections and more about cultivating trust in the cumulative knowledge we build together.

At this point, it is helpful to recall the historical roots of the Open Research movement in the replication crisis and the ensuing credibility revolution in psychology. A landmark study in psychology published in Science by the Open Science Collaboration (2015)—and widely taken up by media around the world—uncovered widespread failures to replicate the findings of high-impact studies in three subdisciplines of psychology, thereby amplifying concerns about a “replication crisis” (Pashler & Wagenmakers, Reference Pashler and Wagenmakers2012). In the Collaboration’s large-scale initiative, fewer than half of the original 100 studies’ findings replicated—a rate of non-replication far higher than researchers would conventionally expect if statistically significant findings reflected true effects in most cases. This suggested a structural imbalance between innovation (novelty) and verification, with current scholarly reward structures tipping toward novelty at the expense of careful verification of existing findings. These landmark findings, therefore, helped catalyze a shift from a replication crisis to a credibility revolution, in which open and transparent research practices were—and continue to be—central to improving the rigor and credibility of scientific research.

I revisit this history not to elevate replicability as the central criterion for all forms of research, but because it provides an important historical explanation for why transparency became so salient in some corners of the social sciences. Importantly, this historical argument for open and transparent research practices—while influential—focuses narrowly on reproducibility and replicability and therefore reflects a predominantly quantitative orientation (see also Liu, Reference Liu2023). I believe that methodological transparency can and should be conceptualized more broadly, including in terms of the trust that society places in research institutions and their outputs (Plonsky, Reference Plonsky2024). If the goal of social science is to understand human behavior in ways that can improve people’s lives, then the trustworthiness and credibility of our results—however defined within different epistemological traditions—matter profoundly. Replicability is only one possible indicator of such trustworthiness, and it applies unevenly across epistemologies (Kim, Reference Kim2025). Starting from the premise that Open Research aims to be inclusive (UNESCO, 2021), and that methodological transparency is a key component to opening the research process, the question for applied linguists then becomes what makes for trustworthy or credible research within their respective paradigms, and how Open Research practices—broadly construed—can help us achieve that. I would very much welcome your thoughts on this, Peter, especially from the perspective of qualitative and critical traditions.

Peter: Thanks, Aline, for highlighting and explaining the key antecedents to open quantitative scholarship. And I do agree with you that researchers, regardless of methodological orientation, need to inspire and win the trust of various stakeholders. Research openness strengthens legitimacy: Gaining public trust is necessary to sustain a field of research, and transparency is key to building public confidence in our expertise through visible accountability.

But before I proceed any further, I would like to clarify a few things about qualitative research in general and what usually would animate applied linguists who conduct qualitative work. I would like to be clear that not all qualitative applied linguists—myself, a critical applied linguist, included—would see themselves as scientists. This distinction is crucial given that the Open Research movement, which you highlighted, has its roots in psychology. I’m in agreement with you that results of any type of research need to be trustworthy, and thus credible. As applied linguists, we owe it to our colleagues, our research funders, society at large, but also participants, a point to which I will return later. At the same time, however, I think it is important for us to remember that applied linguists who draw from various disciplines (other than the sciences) may interpret trustworthiness differently.

I foreground this point because, in contrast to scientists who might seek to test hypotheses (as noted earlier) and therefore work from a predictive stance, qualitative researchers generally work from a postdictive stance. As underscored by Nosek et al. (Reference Nosek, Ebersole, DeHaven and Mellor2018), “In postdiction, analytic decisions are influenced by the observed data creating the forking paths. The researcher is exploring the data to discover what is possible. The data helps generate, not test, new questions and hypotheses” (p. 5). Put differently, postdiction affords qualitative researchers the opportunity to engage in exploratory work, and in the process of investigating a phenomenon, they arrive at a fork and subsequently decide which “forking path,” to which Nosek et al. allude, to take. In the lead-up to this metaphorical fork, and in the spirit of grounded theory (Glaser & Strauss, Reference Glaser and Strauss1967), it is not uncommon for qualitative researchers to engage in constant comparison across and within individual case participants. Such a practice is also consistent with what I described earlier, that is, when the researcher moves back and forth between theory and their findings in ways that would contribute to the trustworthiness of the study.

I would also like to be clear that because of the highly contextual and subjective nature of qualitative research, which is co-constructed by the researcher and their participants—points I raised earlier—I don’t think that replication is possible or desired. In other words, because the qualitative researcher is so deeply interwoven with the research process—to the point where it is impossible to divorce the researcher from their research site—it would be practically impossible to repeat the same study, much less the interpretive process. But as you rightly pointed out a short while ago, replicability is just one possible indicator of such trustworthiness. For qualitative researchers, trustworthiness can be achieved in several ways (Lincoln & Guba, Reference Lincoln and Guba1985). One way is through having several data sources (e.g., interviews, observation field notes, video, and audio recordings) that are triangulated so that different facets of a given phenomenon are presented and analyzed for the reader. At the same time, and in keeping with what I mentioned earlier about transparency, it is crucial that qualitative researchers reflect on their a priori values from the start of the research so that other researchers and consumers of their work know where they are coming from. Put simply, qualitative researchers need to articulate clearly their researcher positionality—through a positionality statement—in terms of topic selection, their relationship to the research site, and their participants. Crucially, they need to be reflexive about how their own biases shaped the entire research process, from conceptualizing their study to disseminating their findings.

During the actual conduct of their study, they are expected to engage in preliminary analyses of the data while collecting it (and not wait until after the study is over to analyze the data) because more often than not, their original plan will need to be modified due to circumstances beyond their control. Another type of flexibility that needs to be exercised by them for their work to remain trustworthy is to move in tandem with the dynamic rhythms of their relationships with participants. For example, it is highly likely that a researcher who is interested in investigating how emotional vulnerability impacts language learning might conduct an interview with a participant who has experienced trauma. Given these circumstances, a high degree of flexibility is needed. Sticking to a scripted set of interview questions would not be appropriate, especially if the participant becomes deeply emotional and decides to narrate their traumatic experiences on a deeply personal level. The researcher would thus also need to engage in relational work by expressing empathy and providing emotional support, if necessary. In that respect, the participant would need to be viewed as a living, embodied being and not merely an entity for data extraction. All these emergent decision-making practices need to be recorded in analytic memos for further examination later. At the same time, before findings are presented, researchers are encouraged to engage in member checking; that is, they are to share their preliminary analyses with their participants and verify with them whether the data have been analyzed accurately. Such member checking is vitally important to upholding the credibility of the researchers’ findings. Circling back to my point about trauma narratives, traumatized participants need to be consulted before the findings of their analyzed narratives are disseminated because a researcher might have misinterpreted the former’s narrative—a major credibility issue that needs to be avoided at all costs.

Also, at the writing stage and as stated earlier, qualitative researchers need to explain why some data sources may have been excluded and/or modified, as well as describe how the order of the data collection (of different data sources) was decided. In particular, they need to explicate their selection decisions, a point that I mentioned earlier when discussing De Costa (Reference De Costa2015). For example, if they worked with five case participants, why were the findings from only two participants included in the manuscript? In short, a clear audit trail on what decisions were made along the way needs to be presented. Also called the confirmability audit, an audit trail, according to Wolf (Reference Wolf2003), provides evidence that recorded raw data have gone through a process of analysis, reduction, and synthesis. Thus, rather than relying on the intuition of the researcher, such a trail requires qualitative researchers to describe as much as possible how they managed and interpreted their data, with the goal of ensuring data trustworthiness. At the end of the day, the ultimate goal is to ensure that others are able to track the research process in a systematic way with a view to enhancing the credibility of the study being undertaken. And if the word limit of the publication prevents including the audit trail in the publication itself, it could be made publicly available on a repository (e.g., Open Science Framework), with a link to the webpage included in the main article. Such sharing of the audit trail would be entirely aligned with Open Research principles of analytic transparency, interpretive accountability, and methodological rigor.

What are some possible ways forward for both quantitative and qualitative researchers seeking to enhance their methodological transparency?

Aline: Thank you, Peter, for explaining how trustworthiness is achieved within qualitative and critical traditions. Through our dialogue, I have been struck by how many of the questions that qualitative researchers grapple with—such as how to justify data exclusions or articulate selection decisions—resonate with parallel conversations in quantitative research. Of course, the standards differ across paradigms: What might look like selective reporting from a quantitative vantage point is, in qualitative research, a legitimate and often necessary analytic decision—as long as the rationale for that choice is articulated and grounded in theory. Yet the convergence I see is that researchers across the methodological spectrum benefit from making their decisions and decision-making processes visible to reviewers, editors, and readers. Some of this visibility will occur in the published article itself, including the supplementary materials, as you described. But I am also encouraged by your idea that qualitative researchers might explore public repositories such as the Open Science Framework for sharing materials such as audit trails, researcher memos, or documentation of analytic decisions.

At the same time, your examples of cases in which openness should be intentionally limited—out of cultural sensitivity, respect for community protocols, or because a research participant requested it—have pushed me to reconsider whether openness should always be treated as an unqualified good. This question is especially salient when we consider the case of open data. Until this point in our dialogue, we have largely discussed open data as data made available to other members of the research community for purposes such as analytic transparency, secondary data analysis, and reproducibility. However, in practice, any actor or system outside the scholarly community can equally access publicly available data and feed it to algorithms and generative AI systems for entirely different purposes. This possibility does not negate the value of openness, but it does underscore the need for more nuanced distinctions between openness, transparency, and unrestricted public availability.

Thank you also for describing in such detail the practices through which qualitative researchers enact transparency and enhance trustworthiness. Your account has helped me see more clearly that the credibility of research can be established not only in relation to reviewers, editors, and readers, but also in relation to research participants themselves. This is an interesting idea that is perhaps not as well represented in some quantitative research, but is one from which quantitative researchers, too, could learn.

As we continue thinking together about how transparency can be achieved across traditions, I am reminded that transparency is not an all-or-nothing proposition (Al-Hoorie et al., Reference Al-Hoorie, Cinaglia, Hiver, Huensch, Isbell, Leung and Sudina2024; Liu et al., Reference Liu, Chong, Marsden, McManus, Morgan-Short, Al-Hoorie and Hui2023; Marsden & Morgan-Short, Reference Marsden and Morgan-Short2023). Incremental steps toward greater openness are both realistic and fair. For many researchers, this will involve making small, sustainable changes to thinking and behaviors linked to specific stages of the research process. The cumulative potential of such changes should not be underestimated. For example, even when ethical or practical concerns prevent researchers from sharing all—or any—of their data, as you and I have previously described, they may still be able to share other components of their study, such as materials, protocols, analysis code or coding schemes, analytic memos, and other documentation of decision-making processes. Likewise, preregistration can take different forms: Researchers may choose to preregister a study independently in a public repository before beginning data collection, or they may pursue peer review of their preregistration first (i.e., hold off on data collection until they obtain in-principle acceptance) through the registered report route. Even the former—a preregistration without peer review—meaningfully enhances transparency. Taken together, these kinds of incremental, context-sensitive steps have the potential to contribute to a profound cultural shift in how we do and communicate about research. The point, then, is not to maximize openness in every case and for every aspect of the research process, but to educate and empower researchers to make principled, transparent decisions about what can be shared, with whom, and for what purposes.

So where and how do we start? Barriers to adopting Open Research practices can be conceptual (e.g., What is preregistration? Why preregister?), practical (e.g., How do I prepare my data so they can be made publicly available?), or emotional (e.g., Why change the status quo? I have enough to do as it is). Importantly, no single group can remove these barriers on its own. Changing our research culture will involve journal editors, funding agencies, learned societies, and graduate programs (Plonsky, Reference Plonsky2024), all of which can shape the incentives, expectations, and norms that support (or hinder) transparency. I would like to focus on graduate education here, however, in part because it is our graduate students (current and past students at our institution) who inspired this cross-paradigm dialogue. In my experience, emerging researchers tend to be enthusiastic about Open Research, but they need training—and an accompanying incentive structure—to help them navigate the many practices associated with Open Research, and to implement these practices confidently (Hui & Huntley, Reference Hui, Huntley and Plonsky2024; Hui, Koh, & Ogawa, Reference Hui, Koh and Ogawa2023).

One way to provide such guidance is to embed transparency-promoting practices within existing research-training structures. A few years ago, our doctoral program consolidated various individual-course-level initiatives into the formal degree structure by incorporating open-research initiatives into the submission process for students’ qualifying research papers (QRPs). (These are empirical studies that students complete before advancing to their doctoral dissertation research.) Of relevance to the points about incrementalism and training which I raised earlier, the QRP submission form was redesigned to integrate a menu of open-research options—much like journal submission portals that invite authors to apply for open-materials, open-data, or preregistration badges. Figure 1 displays the focal portion of the submission form; a downloadable and editable version was made available on the Open Science Framework by Winke (Reference Winke2024) at https://osf.io/h25nj/files/m2fx9.

Figure 1.

Integrating Open Research practices in graduate program degree milestones (Winke, Reference Winke2024).

Figure 1. Long description

The document is a research-paper submission form from Michigan State University Second Language Studies. It contains numbered questions 11 through 19.

Questions 11 and 12 ask for a funding declaration and the target journal name, providing long text boxes.

Questions 13 and 14 ask for the abstract and main manuscript word limits with small text boxes.

Question 15 uses checkboxes to ask if the word limit includes Figures, Tables, Appendices, or References.

Question 16 asks about plans to submit online supplementary files, video abstracts, or translated abstracts.

A section on the Open Science Framework follows, featuring five colorful hexagonal badges: Open Materials (orange), Open Data (blue), Open Code (green), Preregistered (red), and Preregistered plus (dark red).

Question 17 asks which badges the author will apply for (Open Materials, Open Data, or Open Code) and provides text boxes for repository names.

Question 18 asks for a D O I U R L for preregistration.

Question 19 asks if the author will prepare an O A S I S Accessible Summary with a Yes checkbox.

The footer includes a thank you note and contact information for Program Director Paula Winke at winke at m s u dot e d u.

The form raises awareness of the different ways that junior researchers—and researchers more broadly—can contribute to making their research more open. Moreover, the practices represented in the form illustrate the breadth of options available to researchers at different stages of the research cycle: (1) transparency in conceptualization, design, and methodology (preregistration), (2) transparency in materials and procedures (open materials), (3) transparency in data and analysis (open data, open code), (4) transparency in dissemination (video abstracts, translated abstracts), and (5) transparency through accessibility (OASIS summaries; a point that you briefly mentioned earlier, Peter).Footnote ² None of the practices are required, but the visibility of these options—be it in a degree milestone or a journal submission system—may encourage researchers to participate in at least some of them, highlighting the point about incrementality.

Peter: I concur that an incremental but measured approach is the best way forward. And I like the advisory, non-prescriptive approach our doctoral program takes in having our Ph.D. students think across all five stages of the research cycle you described above. To me, graduate education is a great way to nudge both emergent qualitative and quantitative researchers to consider how they can incorporate Open Research practices into their methodological toolkit. On my part, I have to say that our dialogue has truly enriched my understanding of what constitutes transparency in particular and what Open Research means in general to my quantitatively oriented colleagues. In particular, I’m glad that we were able to establish the importance of being transparent about our research practices. Looking ahead, perhaps quantitative researchers, in the spirit of learning mutually from each other, might want to explore the adoption of positionality statements in their work and the creation of an audit trail during their research process.

On my part, I found Haven et al.’s (Reference Haven, Errington, Gleditsch, van Grootel, Jacobs, Kern and Mokkink2020) development of a form, which was designed in consultation with a panel of qualitative researchers, for the preregistration of qualitative studies highly compelling. For clarity, I have outlined this form (for details, see Haven et al., Reference Haven, Errington, Gleditsch, van Grootel, Jacobs, Kern and Mokkink2020, pp. 8–9) below:

1. 1. Study information

   • • Research aims

   • • Research questions

   • • Anticipated duration

2. 2. Design plan

   • • Study design

   • • Sampling and case selection strategy

3. 3. Data collection

   • • Data sources and data types

   • • Data collection methods

   • • Data collection tools, instruments, or plans

   • • Stopping criteria

4. 4. Analysis plan

   • • Data analysis approach (e.g., grounded theory, narrative analysis, discourse analysis)

   • • Data analytic process (e.g., who will be involved in the analysis, software tools used)

   • • Credibility strategies (e.g., member checking, reflexivity, emotionality, bringing in different perspectives, political praxis, dialogues with subjects)

5. 5. Miscellaneous

   • • Reflection on your positionality (optional)

In my opinion, with the notable exception of an optional positionality statement (which should be mandatory), I think having qualitative SLA researchers preregister their work is a positive step forward toward embracing Open Research practices. At the same time, however, we need to be cognizant of how rigid adherence to a preregistration can prevent a qualitative researcher from inductively modifying how they collect their data in response to fluid circumstances that surround a research site. In short, we need to (re)frame preregistration as a flexible planning tool—instead of a rigid yardstick—that is applied in tandem with evolving sociocultural circumstances that characterize the research site and participants.