The evolving discourse of open science in applied linguistics

Open Science Practices (OSPs) represent a broader movement designed to make both the process and outputs of scholarship transparent and widely accessible (Dienlin et al., Reference Dienlin, Johannes, Bowman, Masur, Engesser, Kümpel and de Vreese2021; Lewis, Reference Lewis2019). Central to this movement is the commitment to procedural transparency and social inclusion in scholarship (Rinke & Wuttke, Reference Rinke and Wuttke2021). The OS movement has also been propelled by digital advancements that facilitate data sharing and collaboration, alongside multiple perceived benefits, including increased citation rates and research impact (McKiernan et al., Reference McKiernan, Bourne, Brown, Buck, Kenall, Lin and Yarkoni2016). However, a cohesive global understanding of its implementation and challenges remains elusive across disciplines (UNESCO, 2020). Within applied linguistics, concrete initiatives are indicative of growing but cautious engagement with open science principles (Liu et al., Reference Liu, Chong, Marsden, McManus, Morgan-Short, Al-Hoorie and Hui2023). These initiatives include forming new research networks, dedicated publications, and special issues (Plonsky, Reference Plonsky and Plonsky2024). On the other hand, researcher-driven initiatives like the Postprint Pledge encourage sharing accepted manuscripts (Al-Hoorie & Hiver, Reference Al-Hoorie and Hiver2023). However, recent surveys confirm that qualitative researchers in applied linguistics continue to perceive OS as predominantly quantitative in orientation and potentially exclusionary (Liu & De Cat, Reference Liu, De Cat and Plonsky2024), reinforcing the need for critical adaptation rather than uncritical adoption of transparency practices developed for other research paradigms (Campbell et al., Reference Campbell, Javorka, Engleton, Fishwick, Gregory and Goodman-Williams2023).

The historical context of this movement reveals its origins in addressing reproducibility, a primary concern in quantitative research (Christensen & Miguel, Reference Christensen and Miguel2018; Nelson et al., Reference Nelson, Simmons and Simonsohn2018). To address the reproducibility concern, practices such as preregistration and data sharing have been developed specifically for experimental and correlational research designs (Ahn et al., Reference Ahn, Johnson, Krcmar and Reinecke2021; Center for Open Science, n.d.; Nosek & Lakens, Reference Nosek and Lakens2014). This quantitative heritage has created significant challenges for qualitative researchers, who must navigate ethical considerations around participant confidentiality (Bennett, Reference Bennett2021; Elman et al., Reference Elman, Kapiszewski and Vinuela2010), epistemological concerns about the nature of qualitative data (Bennett, Reference Bennett2021), and practical constraints of qualitative research reporting (Cooper, Reference Cooper2020; Tuval-Mashiach, Reference Tuval-Mashiach2017). Despite these challenges, the movement continues to evolve, with UNESCO advocating for more inclusive approaches that recognize diverse epistemological frameworks (UNESCO, 2021).

Conceptualizing transparency in qualitative research

Openness and transparency have long been recognized as foundational principles of the scientific ethos (Merton, Reference Merton1973). At its core, open science also emphasizes the importance of transparency, urging scholars to elucidate their research processes to facilitate understanding and trustworthiness among diverse audiences (Field, van Ravenzwaaij, Pittelkow, Hoek, & Derksen, Reference Field, van Ravenzwaaij, Pittelkow, Hoek and Derksen2021). As noted by Plonsky (Reference Plonsky and Plonsky2024), transparency is undoubtedly the aspect of open science that has the most direct and immediate connection to this movement. While much of the transparency discussion has traditionally centered on quantitative research, advocates for open science assert that the principle of transparency is applicable to all research methodologies, including qualitative research. This perspective underscores following the same procedures for sharing qualitative data, coding, and analytical processes as those used in quantitative research (Field et al., Reference Field, van Ravenzwaaij, Pittelkow, Hoek and Derksen2021).

From the standpoint of open scholarship, steps toward transparency allow readers to “look under the hood” (Isbell, Reference Isbell and Plonsky2024) or examine the “nuts and bolts” of research studies (Bowman et al., Reference Bowman, Rinke, Lee, Nabi and de Vreese2022). That is, interested readers may derive greater benefit from seeing what has gone on “behind closed doors” than from learning the study’s results (Tuval-Mashiach, 2016). The benefits of transparency in qualitative work are multifaceted and significant. It demystifies research processes often described as “mysterious and elusive” for newcomers to the field (Miles et al., Reference Miles, Huberman and Saldana2014, p. xvii), facilitates more effective research training and pedagogical practices (Poulin, Reference Poulin2007; Tracy, Reference Tracy2010), enhances methodological rigor, credibility, and practical applicability of findings (Bartling & Friesike, Reference Bartling and Friesike2014), and fosters richer scholarly collaboration and cumulative knowledge building. This transparency is achieved through foundational qualitative principles such as “owning one’s perspective” (Elliott, Fischer, & Rennie, Reference Elliott, Fischer and Rennie1999) and engaging in a “knowing practice” (Braun & Clarke, Reference Braun and Clarke2023), where researchers reflexively articulate and justify their methodological and epistemological choices. In the context of OS, this level of principled transparency is the bedrock of methodological accountability and trustworthiness.

However, implementing transparency in qualitative research requires unfolding complex methodological considerations. As Hiles and Cermak (Reference Hiles and Cermak2007) argue, transparency in qualitative research has often been taken for granted rather than deliberately defined and communicated. The inherent flexibility of qualitative approaches (Levitt, Reference Levitt2020; Saldana, Reference Saldana2009) means that researchers must balance comprehensive reporting with the need to maintain narrative coherence and interpretive depth (Alvesson & Kärreman, Reference Alvesson and Kärreman2011; Chenail, Reference Chenail2009). This challenge is particularly acute in L2 research, where qualitative studies often involve multiple data sources and complex analytical processes (Riazi et al., Reference Riazi, Rezvani and Ghanbar2023b). Moreover, as Tsai et al. (Reference Tsai, Kohrt, Matthews, Betancourt, Lee, Papachristos and Dworkin2016) note, the expectation that transparency will lead to identical interpretations across different researchers misunderstands the fundamental nature of qualitative research, where context and researcher positionality necessarily shape analytical outcomes.

Before we proceed, it is important to clarify how we use two key terms. Following Chenail (Reference Chenail2009) and Riazi (Reference Riazi2016), we distinguish between methodology, the overarching theoretical and epistemological framework that guides a study (e.g., grounded theory (GT) and phenomenology), and method, the specific techniques and procedures used to collect and analyze data (e.g., thematic analysis, content analysis). As Braun and Clarke (Reference Braun and Clarke2021) note, thematic analysis (TA) is a method rather than a methodology. Similarly, Nicmanis (Reference Nicmanis2024, p. 1) argues that content analysis (CA) should be treated as “a method for data analysis, not a methodology.” Throughout this paper, we use “methodology” and “method” in this sense.

The transparency-trustworthiness nexus in qualitative analysis

Transparency, understood as the extent to which researchers describe and disclose their procedures, decisions, and judgment calls throughout a study (Aguinis, Ramani, & Alabduljader, Reference Aguinis, Ramani and Alabduljader2018), is closely tied to trustworthiness in qualitative research. It is a prerequisite for readers to evaluate the credibility and rigor of qualitative studies (Denzin & Lincoln, Reference Denzin and Lincoln1994; Lincoln & Guba, Reference Lincoln and Guba1985). This link has long been recognized as a core expectation for qualitative writing and reporting (Hiles, Reference Hiles and Given2008) and is consistent with Merton’s (Reference Merton1973) norm of “communalism,” which holds that scientific claims should be openly available for scrutiny by the broader community.

The use of quality-related terminology in applied linguistics has not always been consistent (Riazi et al., Reference Riazi, Rezvani and Ghanbar2023b). For the purposes of this study, we follow Lincoln and Guba (Reference Lincoln and Guba1985) in understanding trustworthiness as the overarching quality criterion in qualitative inquiry, encompassing credibility (confidence in the truth of findings), transferability (applicability to other contexts), dependability (consistency over time), and confirmability (the degree to which findings reflect participants’ voices rather than researcher bias). Transparency is a prerequisite for each of these dimensions as readers cannot assess these dimensions without a clear account of how the research was conducted.

Concerns about quality and transparency in qualitative research are not new to applied linguistics. Scholars have long called for greater methodological rigor and clearer reporting practices (Richards, Reference Richards2009). As Smagorinsky (Reference Smagorinsky2008) argued, method sections in qualitative research often lack sufficient detail to make the resulting findings trustworthy, noting that “if I don’t know pretty clearly how the researcher is conducting the study, then it doesn’t matter much to me what the results are because I have no idea of how they were produced” (p. 393). This concern has been extended in recent work. Friedman (Reference Friedman2025) provides practical guidance for conducting and reporting qualitative classroom research, emphasizing that transparency allows readers to evaluate research design and rigor while strengthening the credibility of interpretations. Mirhosseini and Pearson (Reference Mirhosseini and Pearson2025) examine how language education researchers attend to quality in qualitative studies. Seibert (Reference Seibert2022) addresses transparency specifically in transcription practices, analyzing reporting practices in TESOL Quarterly and calling for greater transparency in transcription processes. Our study builds on these discussions by providing empirical evidence of current reporting practices, focusing specifically on transparency in data analysis.

The complex, iterative, and multifaceted nature of qualitative data analysis (Gibbs, Reference Gibbs2013; Miles & Huberman, Reference Miles and Huberman1994; Saldana, Reference Saldana2009; Strauss, Reference Strauss1987), characterized by some scholars as necessarily messy and elusive due to its engagement with human complexity (Alvesson & Kärreman, Reference Alvesson and Kärreman2011; Aspers & Corte, Reference Aspers and Corte2019), demands particular attention to transparent reporting practices. Qualitative analysis requires researchers to converge multiple and diverse data sources (Ary, Jacobs, Irvine, & Walker, Reference Ary, Jacobs, Irvine and Walker2018), engaging in what has been described as both transforming and transcending raw data through interpretive processes (Charmaz, Reference Charmaz2006; Wolcott, Reference Wolcott1994). This complexity is especially pronounced in inductive approaches common to qualitative design (Pratt, Reference Pratt2009), making the careful articulation of analytical processes with sufficient precision and detail essential for establishing methodological trustworthiness (Aguinis, Cascio, & Ramani, Reference Aguinis, Cascio and Ramani2017; Aguinis, Ramani, & Alabduljader, Reference Aguinis, Ramani and Alabduljader2018; Nowell, Norris, White, & Moules, Reference Nowell, Norris, White and Moules2017).

Recent methodological debates highlight that transparency in qualitative research is an evolving concept. As Kapiszewski and Karcher (Reference Kapiszewski and Karcher2021) observe, the focus has shifted from whether researchers should be transparent to how they can enact meaningful transparency in practice. This shift reflects what Marsden (Reference Marsden, McKinley and Rose2020) describes as a growing recognition that transparency serves not only accountability but also pedagogical and collaborative functions within the research community. Furthermore, as Byrnes (Reference Byrnes2013) convincingly argues, applied linguistics has now reached a level of disciplinary maturity that requires more rigorous professional examination of methodological practices, particularly in how research processes are documented and communicated.

The current landscape reveals ongoing efforts to develop qualitative-specific transparency frameworks. Scholars have proposed various approaches to adapting open science principles for qualitative research (Arthur & Hearn, Reference Arthur and Hearn2021; Class, de Bruyne, Wuillemin, Donzé, & Claivaz, Reference Class, de Bruyne, Wuillemin, Donzé and Claivaz2021; Steltenpohl et al., Reference Steltenpohl, Lustick, Meyer, Lee, Stegenga, Standiford Reyes and Renbarger2023), with Chenail’s (Reference Chenail2009) triadic framework emerging as particularly relevant for understanding how transparency can be achieved through conceptual, operational, and referential reporting. These developments reflect a growing consensus that qualitative transparency requires specialized approaches that respect the epistemological foundations of interpretive research while still enabling appropriate levels of methodological accountability and scrutiny. As described in the next section, we have used Chenail’s triadic framework to analyze transparency in published qualitative research in top-tier journals of applied linguistics.

Methods and corpus construction

To address the three research questions, we conducted a qualitative content analysis of recent articles published in leading L2 journals. To ensure that our corpus reflected practices in high-quality and influential L2 research, we identified top journals in the field using a dual-index strategy. Journals were included only if they were listed in both SCImago Journal Rank (SJR), based on Elsevier’s Scopus database, and Clarivate’s Journal Citation Reports (JCR) from Web of Science. Using these two sources privileged cross-validation and served as a proxy for journal quality and impact. Journals that did not appear in both indexing systems were excluded.

We began by identifying 20 candidate journals from each index: the “Linguistics and Language” category in SJR and the “Linguistics” category in JCR. We then excluded journals that were not primarily focused on L2 (and foreign languages) learning, teaching, and research (for example, Transactions of the Association for Computational Linguistics and Artificial Intelligence Review). For the remaining journals, we calculated an aggregated rank by summing their SJR and JCR positions, which resulted in several ties. Based on these combined rankings, we initially selected 11 high-ranked L2 journals.

During the article sampling stage, however, we faced a practical limitation: three journals—Studies in Second Language Acquisition, Language Learning, and Language Teaching—had published very few purely qualitative studies within our 5-year time frame, making it impossible to select five articles per journal. These three journals were therefore excluded from the final corpus and replaced by the next highest-ranked journals meeting our inclusion criteria. The final corpus thus consisted of the ten journals listed in Table 1.

Table 1.

Corpus of sampled qualitative articles

Table 1. Long description

The table consists of six columns: Index, Journal name, S J R ranking, J C R ranking, Aggregated ranking, and Number of articles sampled. All journals have 5 articles sampled. The data rows are as follows:

* 1. CALL: S J R 1, J C R 1, Aggregated 2.

* 2. RELC J dot: S J R 4, J C R 2, Aggregated 6.

* 3. A W: S J R 2, J C R 5, Aggregated 7.

* 4. System: S J R 8, J C R 4, Aggregated 12.

* 5. ReCALL: S J R 11, J C R 3, Aggregated 14.

* 6. S S L L T: S J R 12, J C R 6, Aggregated 18.

* 7. J S L W: S J R 10, J C R 10, Aggregated 20.

* 8. M L J: S J R 6, J C R 15, Aggregated 21.

* 9. L T R: S J R 9, J C R 17, Aggregated 26.

* 10. I L L T: S J R 17, J C R 9, Aggregated 26.

Note: Full journal titles are as follows: Computer Assisted Language Learning (CALL); RELC Journal (RELC J.); Assessing Writing (AW); System (System); ReCALL (ReCALL); Studies in Second Language Learning and Teaching (SSLLT); Journal of Second Language Writing (JSLW); Modern Language Journal (MLJ); Language Teaching Research (LTR); Innovation in Language Learning and Teaching (ILLT).

To identify qualitative articles from these ten journals, we first retrieved the table of contents for each issue published between 2021 and 2025. We then screened the title and abstract of every empirical article. Articles were marked as potentially qualitative if the abstract indicated non-numerical data (e.g., interviews, observations, and reflective journals) and an interpretive analytic approach. When the abstract was not clearly indicative of the research method, we read the full methods section before making a final determination. We did not use keyword searching because our goal was to capture the full population of qualitative articles in these journals, rather than a keyword-defined subset. By purely qualitative, we mean studies using non-numerical data with analytic approaches aimed at identifying themes or patterns and using, at most, descriptive statistics rather than inferential analysis in analyzing data and reporting results.

From the identified qualitative articles in each journal, we randomly selected five English-language articles using a stratified random sampling procedure. For each journal, we compiled a chronological list of all qualitative articles meeting our criteria, assigned each article a sequential number, and used a random number generator to select five numbers. If a selected article, upon full-text review, did not meet our qualitative criteria, we replaced it with the next randomly selected article from that journal’s list.

This procedure yielded a total sample of 50 qualitative research articles. The 5-year window was chosen to capture the most recent trends in how qualitative data analysis is reported in high-impact L2 journals. Although a sample of five articles per journal cannot represent the full historical range of practices in each outlet, our goal was specifically to investigate current reporting practices. This focus enables us to identify the most common contemporary practices reflected in recent publications.

It is important to clarify that our sample is not intended to represent the entire field of L2 qualitative research. Rather, the findings should be read as a snapshot of transparency practices in this particular set of journals, not as a definitive statement about the field as a whole. By the same token, the absence of a journal from these bibliometric databases—and thus from our corpus—should not be interpreted as evidence of lower quality; it simply indicates that the journal did not meet our dual-index inclusion criterion for this specific sampling strategy. Despite these limitations, our findings offer empirical substantiation for concerns about qualitative transparency that have largely been raised through impressionistic critique and methodological commentary (e.g., Chenail, Reference Chenail2009; Richards, Reference Richards2009; Smagorinsky, Reference Smagorinsky2008). In this sense, the study serves as an initial, systematically gathered probe—a foundation upon which larger-scale investigations can build—rather than a definitive census of the entire field.

Analytical framework and procedure

This study is informed by Chenail’s (Reference Chenail2009) triadic framework of research reporting. Chenail drew on his extensive work as the editor of The Qualitative Report and identified five key strategies for effectively communicating qualitative research. The most critical of these strategies, and the one that guides our review, is to “make transparency goal one” (p. 105). Chenail argues that in qualitative research, no quality control practice is more essential than ensuring transparent reporting of the steps taken throughout the research process. To achieve this, he recommends employing a triadic framework of reporting that presents methodological procedures in three dimensions:

1. 1) Conceptually (in theory): For example, “In GT, the first step is open coding.”

2. 2) Operationally (in practice): For example, “The first author open coded the data by reviewing each line of the transcripts…”

3. 3) Referentially (in context): For example, “…according to Corbin and Strauss’s (Reference Corbin and Strauss2008) account.” (Chenail, Reference Chenail2009, p. 107)

This framework provides the precise analytical lens for our study. To operationalize this triadic framework for our CA, we expanded it into a detailed codebook that defined specific, observable indicators for each dimension. The codebook guided the systematic identification of key elements of transparency. For conceptual representation, we identified the explicit naming of a methodology, the provision of a definition or rationale for its use, and its linkage to an epistemological or theoretical tradition. For operational description, we coded for the presence of a clear, step-by-step account of the analytical process; mentions of specific tools; a description of who performed the analysis, with special attention to role division in multi-author teams; the temporal sequence of analysis; and the provision of data exemplification. For referential contextualization, we assessed the citation of methodological literature, coding whether citations were used substantively to justify choices or merely ritualistically. In our coding, we treated conceptual specification (naming a methodology or method) and operational description (providing step-by-step procedures) as separate dimensions. An article that named a methodology but provided no procedural detail was coded as having a specified methodology in Table 2 but as lacking operational transparency in Table 4. We did not assume that naming a methodology automatically implies procedural clarity; each dimension was coded independently.

Table 2.

Summary of conceptual representation in the corpus (N = 50)

Table 2. Long description

The table consists of five columns and eleven data rows.

Column headers from left to right are: Journal, Articles with a specified methodological approach, Articles with an unclear or unspecified methodological approach, Articles providing a definition or rationale for the choice of the methodology, and Methodology grounded in epistemology.

Data rows by journal:

* C A L L: 4 specified, 1 unclear, 1 definition, 1 grounded.

* R E L C J dot: 2 specified, 3 unclear, 2 definition, 1 grounded.

* A W: 1 specified, 4 unclear, 0 definition, 0 grounded.

* System: 2 specified, 3 unclear, 2 definition, 0 grounded.

* Re C A L L: 4 specified, 1 unclear, 1 definition, 0 grounded.

* S S L L T: 3 specified, 2 unclear, 0 definition, 0 grounded.

* J S L W: 2 specified, 3 unclear, 0 definition, 1 grounded.

* M L J: 3 specified, 2 unclear, 1 definition, 0 grounded.

* L T R: 3 specified, 2 unclear, 2 definition, 0 grounded.

* I L L T: 4 specified, 1 unclear, 2 definition, 2 grounded.

The final row provides totals:

* Total specified: 28 (56 percent).

* Total unclear: 22 (44 percent).

* Total providing definition: 11 (22 percent).

* Total grounded in epistemology: 5 (10 percent).

To conduct the coding and analysis, we employed a qualitative content analysis approach, focusing exclusively on the manifest content of the sampled articles (Mayring, Reference Mayring2019; Riazi, Ghanbar, & Rezvani, Reference Riazi, Ghanbar and Rezvani2023a; Schreier, Reference Schreier and Flick2014). We approached this study from an interpretive/constructivist stance, treating the published methods sections as texts constructed within the constraints of journal conventions, disciplinary norms, and the rhetorical demands of academic publishing. Our goal was to interpret how authors chose to represent their analytical processes, not to judge whether those processes were “correct” or to uncover what might have happened beyond what was reported. In other words, we analyze published accounts as representations of research processes, not as windows into what actually occurred but went unreported. This epistemological orientation informed our use of qualitative CA: we systematically identified and interpreted authors’ explicit statements, rather than inferring latent meanings (Graneheim & Lundman, Reference Graneheim and Lundman2004). We focused on manifest content for two reasons: first, to capture authors’ own methodological accounts; and second, to avoid speculative inferences and an evaluative stance that a latent analysis would necessitate.

The coding process was iterative. We developed the preliminary coding scheme based on Chenail’s (Reference Chenail2009) triadic framework, which was then refined through the close reading of a subset of articles. To establish consistency, both authors independently coded an initial sample using the codebook. At this stage, we noted some discrepancies in our coding and difficulties in using the initial scheme. Noting these, and through our reflection on the procedures and the coding we had done, both authors collaboratively refined the scheme through discussion. One recurring discrepancy involved the coding of “who performed the analysis.” For example, one article (one of the best in terms of transparent reporting) described its analytical process using either passive structures or “we” throughout, as in “we conducted two cycles of …,” “we conducted reflexive analysis,” but did not specify which author performed which tasks or how collaboration was managed (article 50 in the corpus). The first author coded this as “clear role division” because the consistent use of “we” implied collaborative work. The second author coded it as “unclear” because the article provided no information about whether tasks were divided, how disagreements were resolved, or which author took primary responsibility for coding. Through discussion, we agreed that genuine operational transparency required more than collective pronouns; authors needed to specify who did what, how tasks were distributed, and how interpretive consensus was reached. We refined the codebook to reflect this distinction, coding such instances as “unclear role division” unless explicit details were provided.

Several analytical decisions shaped this refinement. We initially considered evaluating whether authors’ operational steps aligned with their stated conceptual methodology, for example, whether a study claiming to use GT actually followed GT procedures. However, such assessment would have required inferential judgments about methodological fidelity beyond the manifest content we had committed to analyzing. We also considered assigning an overarching transparency rating to each article, combining conceptual, operational, and referential dimensions into a single score. We thought that such a rating would be inherently subjective and would mask the nuanced patterns we sought to capture. We considered evaluating whether studies provided access to raw data or coding schemes as part of Open Science transparency, but given that such access was rarely provided in our corpus, we set aside this dimension. Finally, we decided to include verbatim excerpts from the analyzed articles directly in our codebook, anchoring our coding decisions in the original text and creating an audit trail for our interpretations. These decisions reflect our goal of analyzing manifest content consistently while acknowledging the limitations of what a transparency audit can achieve. Having finalized the scheme, the first author then coded the entire corpus. All coding, both the initial sample and the full corpus, was done manually.

It is critical to state that the aim of this analysis was not to evaluate whether the researchers’ analytical choices were “correct” or whether their findings were “valid”; such judgments lie outside the scope of a transparency audit and are often incompatible with the interpretive nature of qualitative inquiry. Instead, our objective was strictly to assess the clarity, comprehensiveness, and coherence of the methodological reporting, evaluating whether they provided readers with a sufficiently transparent account to understand and evaluate the research process on its own terms. We acknowledge that a study might name a methodology (e.g., grounded theory) but describe procedures that are inconsistent with that methodology. Evaluating such fidelity would require inferential judgments beyond our manifest-content approach. However, we recognize that mislabeling a methodology is itself a transparency issue, as it may mislead readers about the epistemological grounding of the analysis (see “The peril of unexplained methodological eclecticism” for examples of unexplained methodological “mash-ups”).

Analysis of conceptual representation in L2 qualitative research

The data from Table 2 reveal a field that has established a baseline practice of naming methodological approaches, with a majority of studies (n = 28, 56%) specifying either a methodology (e.g., grounded theory, IPA) or a method (e.g., thematic analysis, content analysis) by name. However, this surface-level consensus often masks a relative lack of depth in conceptual justification, creating a transparency gap that limits readers’ ability to assess the trustworthiness of qualitative inquiry in L2 research. As the following subsections demonstrate, this gap represents a departure from the transparency advocated by the Open Science movement. In what follows, TA = Thematic Analysis, CA = Content Analysis, GT = Grounded Theory, DA = Discourse Analysis, and IPA = Interpretative Phenomenological Analysis.

Surface-level agreement masks conceptual vagueness

Looking at Table 2, the most immediate finding is a fundamental split in the sampled journals. While a majority of articles (n = 28, 56%) specified a methodological approach, a substantial number (n = 22, 44%) did not. It is encouraging that the majority followed the basic practice of clearly identifying their analytical approach. In our analysis, we cumulatively reported articles with unclear, unspecified, or generic descriptions. It should be noted that we do not assume generic approaches are inherently problematic; rather, these descriptions often lacked the procedural transparency needed to understand how the analysis was conducted. Like Chenail (Reference Chenail2009), we recognize that generic approaches are legitimate when clearly described. What matters most is not the label itself but whether authors provide a transparent account of their procedures—whether they name a methodology or take a generic approach. As Chenail (Reference Chenail2009) observed, a lack of conceptual clarity directly undermines the trustworthiness of qualitative findings.

Given that journals’ word count constraints often limit how much detail authors can provide in typically lengthier qualitative reports, we argue that specifying and labeling a data-analytic approach is advantageous for readers’ understanding of the data analysis, though it may only go halfway toward transparency. For instance, one article [11] described its analysis only as “the interview data was transcribed, analyzed and interpreted” without ever naming a specific method such as thematic analysis, content analysis, or grounded theory. Such absence of any methodological label does not say much more than that the study is qualitative, something likely already clear to readers, and offers little insight into the actual analytical steps taken.

In our data, articles published in journals such as CALL, ReCALL, and ILLT stood out, with four out of five articles clearly specifying their approach. For instance, one study [44] explicitly named IPA (Interpretative Phenomenological Analysis) as its methodological framework, citing Smith, Flowers, and Larkin (Reference Smith, Flowers and Larkin2009), and explained that this approach was chosen because it “facilitated sensitivity to participants’ experiences.” Here, the authors not only name the methodology but also cite its foundational source and briefly explain why it was appropriate for their research focus.

Table 3 presents the distribution of the specified methodological (e.g., grounded theory or interpretive phenomenological approach) and analytical approaches (e.g., thematic analysis or content analysis). As can be seen, analytical approaches of TA (n = 15) and CA (n = 11) were predominant. These two collectively account for the vast majority (26 out of 31) of specified analytical approaches, a concentration that, while evident in our sampled journals, may not reflect the full diversity of methodological and analytical practices across all subfields of L2 research.

Table 3.

Distribution of specified methodological and analytical approaches across journals

Table 3. Long description

The table consists of 8 columns and 12 rows. The columns are labeled: Journal, T A, C A, G T, D A, I P A, Other, and Total specified.

* C A L L: T A 3, C A 1, G T 0, D A 0, I P A 0, Other 0, Total 4.

* R E L C J dot: T A 0, C A 2, G T 0, D A 0, I P A 0, Other 0, Total 2.

* A W: T A 0, C A 0, G T 0, D A 1, I P A 0, Other 0, Total 1.

* System: T A 1, C A 1, G T 0, D A 0, I P A 0, Other 0, Total 2.

* Re C A L L: T A 2, C A 2, G T 0, D A 0, I P A 0, Other 0, Total 4.

* S S L L T: T A 3, C A 0, G T 0, D A 0, I P A 0, Other 0, Total 3.

* J S L W: T A 1, C A 1, G T 0, D A 0, I P A 0, Other 0, Total 2.

* M L J: T A 1, C A 2, G T 0, D A 0, I P A 0, Other 0, Total 3.

* L T R: T A 2, C A 0, G T 0, D A 0, I P A 1, Other 0, Total 3.

* I L L T: T A 2, C A 2, G T 2, D A 0, I P A 0, Other 1, Total 7.

* Total: T A 15, C A 11, G T 2, D A 1, I P A 1, Other 1, Total 31.

Note: The “Other” category for ILLT includes one unspecified additional method.

* The total number of specified methodological applications (31) is greater than the number of articles with a specified methodology in Table 2, as some articles employed multiple named methods.

Analysis of operational transparency in data analysis

This section presents findings regarding operational transparency, the “in practice” description of data analysis procedures according to Chenail’s (Reference Chenail2009) triadic framework. Table 4 summarizes the outcome of our analysis and shows that the sample studies have established strengths in basic procedural reporting, while there are some deficiencies in detailing collaborative, evidential, and logistical aspects of analysis.

Table 4.

Summary of operational transparency in data analysis procedures

Table 4. Long description

The table consists of three columns: Characteristic, Subcategory, and Frequency and percentage.

* Description of analytical procedures: Specific and sequential is 35 (70 percent); Unclear or Brief is 15 (30 percent).

* Role of analyst(s): Clear roles is 12 (24 percent); Unclear or unspecified roles (multi-author) is 25 (50 percent); Single-authored is 13 (26 percent).

* Data analysis exemplification: Provided example is 8 (16 percent).

* Analytical software: Used and specified is 12 (24 percent); Not used or not mentioned is 38 (76 percent).

* Temporal sequence: Iterative or concurrent is 5 (10 percent); Post hoc (implied) is 35 (70 percent); Not specified is 10 (20 percent).

Analysis of referential contextualization of data analysis methods

The third level of analysis included an analysis of the referential contextualization of data analysis methods. That is, the practice of citing established methodological literature when discussing methodological approaches. Of the 50 articles analyzed, the vast majority (88%) provided citations to methodological literature, while a small minority (12%) did not cite any specific methodological sources in their conceptual framing. Our analysis revealed a landscape defined by a strong central norm with notable variations in depth and sophistication. While the field demonstrates, as far as the sampled studies indicate, a clear consensus on a foundational methodological canon, the depth, specificity, and consistency of engagement with these citations vary, pointing to a lack of standardized practice in epistemologically grounding qualitative analysis. The practice of citing methodological literature was prevalent but not universal across the corpus, as discussed in more detail in the following subsections.

The dominance of a core methodological canon

A closer analysis of the articles that provided citations reveals a clear and overwhelming trend: the persistent dominance of a core methodological literature. The landscape of citations is dominated by a concise list of seminal sources that constitute a foundational canon for qualitative analysis in this field. This is quantitatively underscored by the data presented in Table 5, which shows a heavy concentration of citations among a small group of authoritative works.

Table 5.

Most frequently cited methodological resources in the corpus

Table 5. Long description

The table consists of four columns and seven data rows.

* Row 1: Braun and Clarke. Years: 2006, 2012, 2013, 2022. Primary methodological association: T A. Frequency: 13 (26 percent).

* Row 2: Saldana. Years: 2013, 2016, 2021. Primary methodological association: Coding or qualitative analysis. Frequency: 5 (10 percent).

* Row 3: Strauss and Corbin (Corbin and Strauss). Years: 1998, 2008. Primary methodological association: G T. Frequency: 5 (10 percent).

* Row 4: Creswell (and Poth). Years: 1998, 2016, 2024. Primary methodological association: General Qualitative Research Design. Frequency: 3 (6 percent).

* Row 5: Miles and Huberman (Saldana). Year: 1994. Primary methodological association: Qualitative Data Analysis Framework. Frequency: 3 (6 percent).

* Row 6: Hsieh and Shannon. Year: 2005. Primary methodological association: Qualitative C A. Frequency: 3 (6 percent).

* Row 7: Procedural Exemplars (e.g., Rose and Montakantiwong 2018; Lowe and Kiczkowiak 2016). Primary methodological association: Empirical methodological models. Frequency: 2 (4 percent).

As detailed in Table 5, the most frequently cited works (26%) are led by Braun and Clarke’s (Reference Braun and Clarke2006) seminal article on TA, which serves as the undisputed cornerstone for qualitative data analysis in this sample of L2 research. The citation of their later works (e.g., 2012, 2022) also indicates some engagement with the evolution of their approach. Their influence spans nearly all the sampled journals.

Closely following in prevalence is Saldana’s The Coding Manual for Qualitative Researchers (2013, 2016, 2021), whose successive editions are the primary source for the practical, operational aspects of coding. By contrast, the practice of citing empirical “sister studies” as procedural exemplars is strikingly rare, further highlighting the dominance of this canon. A close reading of the corpus reveals that only two articles (4% of the sample) explicitly grounded their procedural logic in prior empirical models. This near-universal reliance on canonical texts suggests a strategic and pragmatic choice: researchers appear to favor the methodological clarity and credibility associated with textbooks and seminal works over the potential ambiguity of empirical accounts. Consequently, L2 qualitative researchers tend to anchor their work in a well-established, interdisciplinary methodological tradition, and the consistency of these citations points to a shared sense of what constitutes a legitimate methodological foundation. However, we did identify minor lapses in procedural rigor; in two instances, methodological works cited in the text were missing from the reference list.