Introduction
Input is important for second language (L2) learning, but input alone may not result in optimal L2 development (Gass & Mackey, Reference Gass, Mackey, VanPatten, Keating and Wulff2020). According to the Output Hypothesis (Swain, Reference Swain and Hinkel2005), output, conceived as both a product and a process of L2 production, can facilitate L2 learning via four functions: enhancing fluency, triggering noticing, testing hypotheses, and promoting metalinguistic reflection. Specifically, the noticing function of output proposes that being pushed to produce (speak or write) in the L2 can trigger learners to notice the problems in their interlanguage systems, and when relevant input is available subsequently, learners may pay focused attention to the problematic aspects noticed previously (Swain, Reference Swain and Hinkel2005).
As a potential link between output and input, the noticing function of output is particularly relevant to understanding a vital issue in L2 research: how to facilitate the cognitive processes of converting input to intake (i.e., cognitively registered input) to promote L2 learning (Leow et al., Reference Leow, Grey, Marijuan and Moorman2014; Robinson et al., Reference Robinson, Mackey, Gass, Schmidt, Gass and Mackey2012). The Noticing Hypothesis (Schmidt, Reference Schmidt and Robinson2001) is one of the widely investigated proposals to account for the attentional mechanisms underlying input and intake (Robinson et al., Reference Robinson, Mackey, Gass, Schmidt, Gass and Mackey2012). Schmidt (Reference Schmidt, Chan, Chin, Bhatt and Walker2012) defined noticing as “conscious registration of attended specific instances of language” (p. 32), and hypothesized that “input does not become intake for language learning unless it is noticed, that is, consciously registered” (p. 27). The noticing function of output is closely linked to Schmidt’s Noticing Hypothesis (Izumi & Bigelow, Reference Izumi and Bigelow2000), and is concordant with Schmidt and Frota’s (Reference Schmidt, Frota and Day1986) noticing the gap principle that noticing a problem or mismatch in the interlanguage system while comparing relevant input can facilitate L2 development (Izumi et al., Reference Izumi, Bigelow, Fujiwara and Fearnow1999).
L2 research has investigated the noticing function of output but has found mixed results so far (see synthesis below). As suggested by previous studies (Hanaoka, Reference Hanaoka2007; Hanaoka & Izumi, Reference Hanaoka and Izumi2012; Russell, Reference Russell2014; Song & Suh, Reference Song and Suh2008; Uggen, Reference Uggen2012; Zalbidea, Reference Zalbidea2021), the inconsistent findings may result from different operationalizations of noticing (e.g., underlining, note-taking, stimulated recall), whereby noticing itself is regarded as a hybrid construct involving both attention and awareness (e.g., Godfroid et al., Reference Godfroid, Boers and Housen2013; Indrarathne & Kormos, Reference Indrarathne and Kormos2017). In our view, attention and awareness are two sides of the same coin—noticing—they are inseparable from a theoretical perspective. Methodologically, however, no single measure can simultaneously capture both constructs with equal depth and precision; each measure inevitably provides a more comprehensive picture of one side than the other.Footnote 1 Adopting real-time eye-movement registration as a measure of attention, first, we aim to expand L2 research on the noticing function of output by contributing the first set of eye-tracking data. Using quantitative eye-movement and pre/post-test data, we make the first attempt to examine the interrelationships among output, attention, and learning through a mediation analysis. This approach allows us to explore the core prediction of the Output Hypothesis (Swain, Reference Swain and Hinkel2005)—output promotes noticing, which in turn facilitates learning. Our third aim is to investigate noticing at both attention and awareness levels by employing an explanatory sequential design that involves stimulated recall, during which L2 learners recall their thought processes while watching video playback of their eye movements. Focusing on Chinese learners’ processing, perception, and production of the English past counterfactual conditional, we aim to shed light on the roles of attention and awareness in the output-learning relationship.
Synthesizing the noticing function of output: Operationalizations and findings
According to the Output Hypothesis (Swain, Reference Swain and Hinkel2005), noticing can occur at two stages: when producing output, and when relevant input is available subsequently (see also Izumi, Reference Izumi2003). While some L2 studies investigated noticing during production without subsequent input (e.g., Swain & Lapkin, Reference Swain and Lapkin1995), others focused on noticing in instructional sequences involving both output and input and explored how output may promote noticing in the following input (e.g., Izumi et al., Reference Izumi, Bigelow, Fujiwara and Fearnow1999). The output-plus-input studies generally followed two types of research designs: output plus one round of input (e.g., Uggen, Reference Uggen2012), and output plus two rounds of input (e.g., Izumi, Reference Izumi2002). Consequently, empirical studies on the noticing function of output can be organized into three categories based on their research designs: (I) no input, (II) one round of input, and (III) two rounds of input. To synthesize this line of research, we created three summary tables. Table A1 in Online Supplementary Materials A focuses on the research design and summarizes each study’s key methodological features, including group treatments and measures of noticing and learning. Table A2 in Online Supplementary Materials A synthesizes the operationalizations and findings on the noticing function of output, as well as four design features: focus of noticing, modality of input/output, and involvement of a no-output comparison group and L2 target forms. For a more concise and direct reference, Table 1 in the text below reproduces Table A2, excluding details for focus of noticing and modality of input/output.
Operationalizations and findings on the noticing function of output
Table 1. Long description
The table is divided into three primary sections:
1. Noticing during output:
- Think-aloud: Swain and Lapkin (1995) and Qi and Lapkin (2001) show ‘Yes’ for Output-noticing. Griffin (2004) shows ‘Yes’ for Output-noticing and ‘Partly Yes’ for learning relationships.
- Note-taking: Hanaoka (2007) and Hanaoka and Izumi (2012) show ‘Yes’ across all relationship columns.
- Retrospective interview: Sakai (2004) shows ‘Yes’ for Output-noticing but ‘No’ for learning.
- Output product: Includes studies by Izumi and Bigelow (2000), Uggen (2012), Russell (2014), and Izumi (2002). Results vary, with Russell and Izumi showing ‘Yes’ for output-noticing and learning, while others show ‘No’ or ‘Partly Yes’.
2. Noticing during input:
- Think-aloud verified with retrospective interview: Qi and Lapkin (2001) show ‘Yes’ across all relationships.
- Note-taking: Izumi (2002) and Leeser (2008) show ‘No’ or ‘Partly Yes’ for Output-noticing but ‘Yes’ or ‘Partly Yes’ for learning. Hanaoka (2007) and Hanaoka and Izumi (2012) show ‘Yes’ throughout.
- Retrospective interview: Sakai (2004) shows ‘No’ across all relationships.
- Stimulated recall: Uggen (2012) shows ‘Partly Yes’ for noticing and learning.
- Underlining: Multiple phases of Izumi et al. (1999) and Izumi and Bigelow (2000) show ‘No’ for Output-noticing. Russell (2014) shows ‘Partly Yes’ for noticing and ‘Yes’ for learning.
- Eye-tracking plus stimulated recall: ‘Our Study’ is marked with question marks for all relationships.
3. Noticing during combined stages (output plus input):
- Measures including Output product, Reflection questionnaire, Underlining, and End-of-study interview: Nassif (2019) shows ‘No’ for Output-noticing and ‘Partly Yes’ for Output-learning.
- Stimulated recall: Zalbidea (2021) shows ‘Yes’ for Output-noticing and ‘Partly Yes’ for Output-learning.
Note. *Studies that compared output vs. no-output groups and focused on L2 target forms in two rounds of input.
1 The categorization was mainly based on the Results section.
From Table A1, several patterns emerge regarding the development of research designs to examine the noticing function of output, which generally reflects the increasingly refined focus in this line of research. The two early studies that examined output treatments only (Griffin, Reference Griffin2004; Swain & Lapkin, Reference Swain and Lapkin1995) sought to demonstrate the facilitative effects of output on noticing. Later studies mostly built on the tradition of exploring the noticing function of output in L2 writing (see Modality in Table A2) and incorporated input treatments of reading model texts, whose purpose mirrored that of written corrective feedback in L2 writing practice (see Leow, Reference Leow and Manchón2020). These output-plus-input studies further differed in the frequency of input and the presence of a no-output comparison group. Among studies with one round of input, four focused more on global L2 writing development and the role of noticing during this process (see Focus of Noticing in Table A2) and did not include a no-output group or specify L2 target forms (Hanaoka, Reference Hanaoka2007; Hanaoka & Izumi, Reference Hanaoka and Izumi2012; Qi & Lapkin, Reference Qi and Lapkin2001; Sakai, Reference Sakai2004). In the remaining studies with one round of input (e.g., Uggen, Reference Uggen2012) and all of the studies with two rounds of input (e.g., Izumi, Reference Izumi2002), the focus was narrowed down to particular L2 grammatical structures (e.g., Spanish past tense in Leeser, Reference Leeser2008), and noticing and learning of the target grammar was compared between the output and no-output groups.
Moving beyond one round of input (e.g., Nassif, Reference Nassif2019), the use of two rounds of input (e.g., Izumi, Reference Izumi2002) allowed for a more detailed examination of noticing, not only between the output and no-output groups (between-groups comparison) but also before and after output (within-groups comparison). Although each design has its advantages and disadvantages and the various studies offer complementary perspectives on the noticing function of output, in this study, we focused on the English past counterfactual conditional and adopted an output/no-output group design with two rounds of input, allowing for both between- and within-groups comparisons. This design supported us in conducting a more detailed investigation into how output may promote noticing of input and L2 learning.
Building on the synthesis of research designs, Table 1 synthesizes the operationalizations and measures of noticing, and importantly, previous findings on three relationships between: output and noticing (Output ~ Noticing), output and learning (Output ~ Learning), and noticing and learning (Noticing ~ Learning). These three relationships are fundamental to the core prediction of the Output Hypothesis (Swain, Reference Swain and Hinkel2005)—output promotes noticing, which in turn facilitates learning. Table 1 therefore connects previous findings with key methodological considerations, and supports examining the current inconclusive results on the noticing function of output, as well as exploring the core prediction of the Output Hypothesis.
Focusing on the output-noticing relationship, results in Table 1 were mostly affirmative when noticing was measured during output. By contrast, when noticing was measured during input processing following production, results were nearly evenly split between supportive and non-supportive findings. Likewise, inconsistent evidence for noticing was also found in studies that did not differentiate between output and input stages when measuring noticing. Furthermore, when considering these findings together with two key design features (No-Output Group, L2 Target Form), the noticing function of output was supported by almost all studies that did not include a no-output group or specific L2 target forms. Conversely, in studies that included a no-output group and targeted specific L2 forms, supportive and non-supportive results were balanced. When the findings were compared for each noticing measure, results were mostly supportive for think-aloud, note-taking, and stimulated recall, yet mostly negative for underlining, whereas for other measures, findings were less conclusive. These patterns altogether suggest that the current mixed findings on the noticing function of output may result from one or more methodological differences: how noticing was measured, at which stage noticing was measured, and whether the study involved a no-output comparison group and examined noticing of specific L2 target forms.
Regarding the output-learning relationship, results in Table 1 were generally positive, supporting the facilitative role of output for L2 development. As for the noticing-learning relationship, only five studies directly addressed this part of the Output Hypothesis, and their results, as seen in Table 1, were largely affirmative. Still, these findings are lacking in providing a comprehensive picture of how output-induced noticing may affect L2 learning. Moreover, despite the sequential, directional relationships predicted by the Output Hypothesis—output promotes noticing, which in turn facilitates learning—no research to date has captured the interrelationships among output, noticing, and learning simultaneously. We therefore make the first attempt to address this major conceptual gap in the current understanding by conducting a mediation analysis to explore how noticing may mediate the effects of output on learning.
Another research gap we aim to address is the overwhelmingly holistic operationalization of the noticing function of output. While L2 researchers have proposed operationalizing noticing at the level of either attention or awareness to enable more accurate measurement (Godfroid et al., Reference Godfroid, Boers and Housen2013; Indrarathne & Kormos, Reference Indrarathne and Kormos2017), none of the noticing measures in Table 1 were intended to discern between attention and awareness. By adopting eye-tracking technology as an optimal measure of attention (Godfroid, Reference Godfroid2020; Leow et al., Reference Leow, Grey, Marijuan and Moorman2014; Robinson et al., Reference Robinson, Mackey, Gass, Schmidt, Gass and Mackey2012), our study aims to investigate the noticing function of output at the level of attention by contributing the first set of eye-movement data. Meanwhile, the quantitative eye-movement data can be entered into a mediation analysis to explore the mediating role of attention between output and learning. Following up on patterns of heightened attention discerned in the eye-movement data, we also aim to measure the noticing function of output at the level of awareness by conducting stimulated recall on learners’ reasons for paying increased attention.
Taken together, we adopt an explanatory sequential design that consists of two experiments to examine noticing at the input stage within a sequence of written input-output-input-output treatments. By focusing on the English past counterfactual conditional as the L2 target form and including a no-output group for comparison, we account for potential methodological factors that may have contributed to the widely inconsistent findings on noticing of input following output. Ultimately, our goal is to advance the current understanding of the noticing function of output and the Output Hypothesis (Swain, Reference Swain and Hinkel2005).
Eye-tracking as a measure of attention during repeated reading
Research on repeated reading has generally found a text repetition effect, that is, reading a text multiple times facilitates its processing (see Raney, Reference Raney2003 for review), as revealed in reading patterns captured by eye-tracking technology. Although most of this research was conducted in psychology with developing (child) and adult first language (L1) readers, several L2 studies on the noticing function of output have also involved reading the same input twice and, in this sense, the two areas of work share a conceptual connection.
L1 studies adopting various eye-tracking measures (e.g., fixation probability, regressive fixation) have found facilitated processing during the second reading among children (Foster et al., Reference Foster, Ardoin and Binder2013; Rogers, Reference Rogers2014; Zawoyski et al., Reference Zawoyski, Ardoin and Binder2015) and adults (Chamberland et al., Reference Chamberland, Saint-Aubin and Légère2013; Hyönä & Niemi, Reference Hyönä and Niemi1990; Kaakinen & Hyönä, Reference Kaakinen and Hyönä2007; Raney & Rayner, Reference Raney and Rayner1995; Raney et al., Reference Raney, Therriault and Minkoff2000). In L2 contexts, only two unpublished master’s theses (Hansen, Reference Hansen2019; Rich, Reference Rich2019) have investigated eye movements during repeated reading, and their results were similar in suggesting a text repetition effect: compared with the first reading, L2 learners skipped more words and had shorter first-pass reading times and fewer fixations in the second reading.
Generally, eye-tracking measures can be mapped onto different cognitive processes during reading, with early measures tapping into initial processes such as lexical access and word recognition, and late measures into later processes that may be related to processing difficulty (Clifton et al., Reference Clifton, Staub, Rayner and van Gompel2007; see Conklin et al., Reference Conklin, Pellicer-Sánchez and Carrol2018; Godfroid, Reference Godfroid2020 for overviews of early and late measures). Because of this, it is often recommended to employ multiple eye-tracking measures to reveal a fuller picture of cognitive processing while reading (Rayner, Reference Rayner2009). Accordingly, our study adopts a combination of early and late eye-movement measures to compare attention to the L2 target form during two sessions of reading separated by a writing/comprehension task.
This study
Following an explanatory sequential design, we conducted two experiments: Experiment 1 focused on Research Questions (RQs) 1–3 and Experiment 2 focused on RQ4. In Experiment 1, two groups of participants, the output vs. no-output groups, were compared to investigate the effects of output on learning (RQ1) and attention (RQ2), as well as the mediating role of attention between output and learning (RQ3). Experiment 2 was conducted as a follow-up to Experiment 1 to further explore the statistically non-significant mediation of attention found in RQ3, despite the statistically significant results for RQ1 and RQ2 regarding output’s facilitative effects on learning and attention. In Experiment 2, a third group of participants completed a similar treatment as the output group in Experiment 1, and subsequently they engaged in a stimulated recall that used episodes of increased attention from their eye-movement recordings as a basis to elicit the thought processes behind the heightened attention (RQ4). Our RQs are as follows:
RQ1. How does output plus input affect accuracy of L2 grammar production?
RQ2. How does output affect attention to target L2 grammar in subsequent input?
RQ3. To what extent does attention to subsequent input mediate effects of output on L2 grammar production?
RQ4. What are the reasons L2 learners pay increased attention to input following output?
Method
Participants
L1 Chinese learners of English were recruited from a university in the United States. For Experiment 1, 68 participants were randomly assigned to the output/experimental group (EG, n = 34) or the no-output/comparison group (CG, n = 34). Based on statistical analysis of TOEFL scores, the EG (Mean = 79.79, SD = 7.44) and the CG (Mean = 79.69, SD = 8.11) had comparable overall English proficiency at the intermediate level. To assess prior knowledge of the target form, a pretest (see Materials) was conducted. Statistical analysis on pretest scores suggested similar prior knowledge of the target structure between the EG (Mean = 21%, SD = 13%) and the CG (Mean = 20%, SD = 18%). For Experiment 2, 25 participants completed data collection. After data-quality screening, 11 were included in the stimulated-recall group (SR, n = 11). Generally, the SR’s profile was comparable to the EG and the CG, with similar TOEFL scores (Mean = 76.78, SD = 10.15) and pretest scores (Mean = 26%, SD = 13%). Further details about the participants are presented in Online Supplementary Materials B.
Materials
Target grammatical form and input essay
The target grammatical structure was the English past counterfactual conditional. For example, If I had finished the project early, I would have taken a trip to Paris. This form is widely observed to pose difficulty for L2 learners of English, particularly L1 Chinese speakers (see Online Supplementary Materials C for detailed discussion). The pretest scores showed that our L1 Chinese participants had some knowledge but not full mastery of this structure (see Participants), which afforded opportunities to examine how output may affect learning this complex form.
The input essay was one of the model texts used in Uggen (Reference Uggen2012). This essay discussed four job offers and provided input flood of the target form in 6 examples out of 12 sentences. Details of this essay and its presentation in the eye-tracking experiments are provided in Online Supplementary Materials C.
Pretest and posttest
Both the pretest and posttest consisted of two production tasks: the first task was from Uggen (Reference Uggen2012), and the second was from Suh (Reference Suh2010) with added story backgrounds. For every item in each task, participants were asked to write a sentence based on two picture prompts, with one presenting an option and the other depicting a possible consequence. For test scoring, Uggen’s (Reference Uggen2012) scoring scheme was followed, and 5 points were assigned to a correct sentence/item, with 45 points for each test. The full tests and scoring details are provided in Online Supplementary Materials D.
Two authors scored the pretest and posttest independently. Intraclass correlation coefficient (ICC) was calculated for the scores, with .987 (p < .001) for the pretest and .996 (p < .001) for the posttest. The ICCs exceeded the .70 recommended benchmark and indicated high inter-rater reliability (Field, Reference Field2018). The discrepant scores were then discussed and resolved completely by the two authors.
Procedures
Figure 1 outlines the data collection procedures of the two experiments. An author or a research assistant (RA) met with each participant individually to collect data.
Data collection procedures.
Figure 1. Long description
The flowchart is divided into two main sections.
Experiment 1, on the left, is color-coded in blue and splits into two groups: E G (Output Group) and C G (No-Output Group). Both groups follow a shared sequence:
1. Background Questionnaire.
2. Pretest.
3. 1st Essay Reading.
4. The path diverges: E G performs a 1st Reconstruction Task while C G performs a 1st Comprehension Task.
5. Both groups perform a 2nd Essay Reading.
6. The path diverges again: E G performs a 2nd Reconstruction Task while C G performs a 2nd Comprehension Task.
7. Both groups conclude with a Posttest.
Experiment 2, on the right, is color-coded in green and focuses on the S R (Stimulated Recall Group). The linear sequence is:
1. Pretest.
2. 1st Essay Reading.
3. 1st Reconstruction Task.
4. 2nd Essay Reading.
5. 2nd Reconstruction Task.
6. Background Questionnaire.
7. Stimulated Recall (2nd Reading).
In Experiment 1, both the EG and the CG started with a background questionnaire and the pretest, and then read the model essay for the first time with their eye movements being recorded. The EG continued to the first reconstruction task, whose instruction was to “reconstruct the essay as accurately as possible in both content and grammar.” Differently, the CG worked on comprehension questions about the model essay, which were mainly extension questions that did not draw attention to the targeted form (Izumi et al., Reference Izumi, Bigelow, Fujiwara and Fearnow1999). Subsequently, both groups read the model essay a second time with their eye movements being recorded. Then the EG completed a second round of essay reconstruction, whereas the CG answered another set of comprehension questions. Finally, both groups finished the posttest. Notably, before each round of essay reading, no information was given about the post-reading tasks, so as to avoid undue effects of task expectation on attention (Song & Suh, Reference Song and Suh2008). Eye movements were recorded by EyeLink 1000 (SR Research) with a sampling rate of 1000 Hz. The reconstruction and comprehension tasks are provided in Online Supplementary Materials E.
In Experiment 2, the SR went through the EG’s procedure with two modifications. First, the background questionnaire was moved to follow the second reconstruction task. Second, a stimulated-recall session was added after the background questionnaire and replaced the posttest. The SR did not complete a posttest because the act of commenting on the target form during stimulated recall may itself promote learners’ attention to the target form and facilitate learning. We therefore removed the posttest as a precaution to avoid potential reactivity from stimulated recall (see also Shintani & Ellis, Reference Shintani and Ellis2013). Additionally, inserting the background questionnaire before stimulated recall would allow the researcher more time to preview eye-movement recordings and consider where to pause and ask questions.
During stimulated recall, eye-movement recordings from the second round of essay reading were played at 50% speed to allow easier observation of eye-movement patterns. As the noticing function of output was more prominent in the late measures (see Results: RQ2, RQ3), the researcher paused the video mainly at instances of regressive eye movements, followed by inquiries such as “You were doing X here. What were you thinking?” Participants were also encouraged to initiate the pause and share their thoughts. The stimulated-recall session was conducted in L2 English and was audio- and video-recorded, with the latter also capturing a replay of the eye movements in question. Details of the eye-tracking experiments and the stimulated recall protocol are provided in Online Supplementary Materials F.
Areas of interest (AOIs) and eye-tracking measures
Thirteen AOIs were created for the components in the if- and the main clauses of the target form (see Table 2). For each AOI, four eye-tracking measures were employed for each participant during each round of essay reading. Specifically, (a) fixation probability (fixating an AOI or not) and (b) first-pass reading time (the sum of all fixations in a multiword AOI during the first time of reading) afforded two early measures, whereas (c) re-reading time (the sum of all fixations in an AOI after the first time of reading) provided a late measure (Conklin et al., Reference Conklin, Pellicer-Sánchez and Carrol2018; Godfroid, Reference Godfroid2020). Both early and late measures, (d) regression path duration indexed the sum of all fixations after the eyes first entered into an AOI until they left it to the right, including fixations following regression to earlier parts of the sentence (i.e., fixations in other AOIs to the left of the current AOI) (Conklin et al., Reference Conklin, Pellicer-Sánchez and Carrol2018; Godfroid, Reference Godfroid2020). To further explore later cognitive processes, (e) regressive transitions were counted for the number of times the eyes moved from a latter AOI to a former AOI in the same sentence. Table 2 summarizes 7 regressive transitions between the 13 AOIs. Figure 2 presents a fictional example to show how the five eye-tracking measures were calculated. Before statistical analysis, data cleaning was performed by merging or deleting fixations below 50 milliseconds and by deleting fixations over 800 milliseconds (Godfroid, Reference Godfroid2020).
Summary of AOIs and regressive transitions between AOIs
Table 2. Long description
The table consists of three columns: Screen, A O I in the same sentence, and Regressive transition.
Screen 1 contains three entries:
1. A O I s 1 had been and 2 would have considered, with a regressive transition of 2 to 1.
2. A O I s 3 had taken and 4 would have had, with a regressive transition of 4 to 3.
3. A O I s 5 had accepted and 6 would have been disappointed, with a regressive transition of 6 to 5.
Screen 2 contains three entries:
1. A O I s 7 had been and 8 would have followed, with a regressive transition of 8 to 7.
2. A O I s 9 would have been, 10 had already paid, and 11 had saved, with regressive transitions of 10 to 9 and 11 to 9.
3. A O I s 12 had been and 13 would have chosen, with a regressive transition of 13 to 12.
A fictional example for calculating the five eye-tracking measures.
Figure 2. Long description
At the top, a sentence reads ‘If I had finished the project early, I would have taken a trip to Paris.’ Two pink boxes define Areas of Interest. A O I number 1 covers ‘had finished’ and A O I number 2 covers ‘would have taken’. A blue scanpath connects eight numbered circles representing fixations. The path starts at 1, moves to 2 in A O I number 1, then to 3, then to 4 in A O I number 2. It then regresses back to 5 in A O I number 1, moves forward to 6, then to 7 in A O I number 2, and finally to 8 at the end of the sentence. Below this, a list defines five measures. 1. Fixation probability: A O I number 2 equals 1. 2. First-pass reading time: A O I number 2 equals fixation 4. 3. Re-reading time: A O I number 2 equals fixation 7. 4. Regression path duration: A O I number 2 equals fixations 5 plus 6 plus 7. 5. Regressive transition: A O I number 21 equals 1, representing the jump from fixation 4 to fixation 5.
Data analysis
For RQ1, bootstrapped descriptive statistics were calculated for the pretest and posttest scores. Mixed effects modeling was conducted with item-level scores of the pretest and posttest as the outcome variable. The fixed effects were treatment groups (CG vs. EG), test types (pretest vs. posttest), and their interaction. The full mixed effects model could be written as: Score ~ Group + Test + Group:Test + (1+Test | Participant) + (1+Group | Item).
To answer RQ2, bootstrapped descriptive statistics were calculated for the five eye-tracking measures. Mixed effects modeling was conducted for each measure by entering eye-movement data at the transition (regressive transitions) or the AOI (the other measures) level as the item-level outcome variable separately. The same fixed effects were hypothesized for the five mixed effects models: treatment groups (CG vs. EG), input sessions (Time 1 vs. Time 2), and their interaction. The full mixed effects models could be written as: Eye-movement Data ~ Group + Time + Group:Time + (1+Time | Participant) + (1+Group+Time+Group:Time | AOI/Transition).
For each mixed effects model in RQ1 and RQ2, we calculated marginal and conditional R2 values as effect size and created effects plots to facilitate interpretation. Model diagnostics were also conducted for each model. Given space limits, here we focus on results of fixed effects in each mixed effects model. Detailed analyses and complete results for RQ1 and RQ2 are reported in Online Supplementary Materials G.
Regarding RQ3, mediation analysis was conducted to test the core prediction by the Output Hypothesis: producing output (predictor) promotes attention to subsequent input (mediator), which in turn, facilitates L2 learning (outcome variable). Figure 3 presents a conceptual mediation model for output, attention, and learning, with a, b, c, and c′ representing model coefficients. Specifically, c denotes the total effect of output on learning, which can be partitioned into the direct effect of output on learning as indexed by c′, and the indirect effect of output on learning via attention as denoted by the product ab. Generally, mediation effects are assessed by testing statistical significance of the indirect effects ab.
Conceptual mediation model for output, attention, and learning.
Data of each eye-tracking measure were entered separately into the mediation model together with pretest and posttest scores, which generated five sets of results. Due to space limits, here we focus on the total (c), direct (c′), and indirect (ab) effects of treatment groups on posttest scores in each mediation model. Detailed analyses and complete results of RQ3 are presented in Online Supplementary Materials G.
For RQ4, we first generated transcriptions of the audio recordings using AI-based software, and checked and edited the transcriptions manually. We then added exact timestamps to the transcripts, which enabled us to link each utterance with its occurrence in the video recordings. Using an iterative approach, we developed a coding schemeFootnote 2 based on two authors’ and one RA’s independent coding of data from three participants (27% of the data). Employing the coding scheme, an author and an RA coded the remaining data separately. Inter-coder reliability was Cohen’s kappa .865, indicating good reliability (McHugh, Reference McHugh2012). Discrepant coding was discussed and resolved until we reached 100% agreement. We summarized the quantitative results of our coding by reporting the number of instances/participants contributing to each theme in the coding scheme. To complement these results, we also selected representative excerpts for all coding themes. Given space limits, here we report excerpts for target-form related themes. Detailed analyses and complete results for RQ4 are provided in Online Supplementary Materials G.
Results
RQ1: Effects of output plus input on accuracy of grammar production
Table G3 in Online Supplementary Materials G reports results of the mixed effects model for pretest and posttest scores. The interaction between treatment groups and test types (Group:Test) was statistically significant (Estimate = 0.22, p = .016), suggesting joint effects of treatment groups and test types on test scores. Figure 4 shows an effects plot with model-predicted test scores. Compared with the CG’s ignorable score change, the EG’s scores increased noticeably from pretest to posttest. These results suggested positive effects of output plus input on accuracy of L2 grammar production.
Effects plot for test scores.
Figure 4. Long description
The graph consists of two side by side panels with a shared vertical axis labeled Predicted values of test scores, ranging from 0.50 to 1.25. The horizontal axis for both panels lists Pretest and Posttest.
Left panel titled C G No Output. A red horizontal line remains nearly flat, starting at approximately 0.76 at Pretest and ending at 0.75 at Posttest. A light red shaded confidence interval band surrounds the line, spanning from roughly 0.55 to 1.05.
Right panel titled E G Output. A blue line shows a linear increase, starting at approximately 0.80 at Pretest and rising to nearly 1.00 at Posttest. A light blue shaded confidence interval band surrounds the line, widening as it moves right, spanning from roughly 0.58 to 1.10 at Pretest and 0.68 to 1.35 at Posttest.
RQ2: Effects of output on attention to target grammar in subsequent input
Table G5 in Online Supplementary Materials G reports results of the mixed effects model for fixation probability. Treatment groups (Group) were a significant predictor of fixation probability: compared with the CG, the EG was more likely to fixate on the target form (Estimate = 1.31, p = .014). Figure 5(a) presents an effects plot for the model. From the first to the second time of input, the CG’s fixation probability decreased sizably, whereas the EG’s stayed almost the same. These results suggested that output facilitated curbing the tendency to skip the target form.
Effects plots for five eye-tracking measures.
Figure 5. Long description
A legend at the top left identifies red lines as C G (No output) and blue lines as E G (Output). Each panel contains two sub-graphs comparing these groups from Time 1 to Time 2.
Panel a, Fixation probability (Early). The C G group shows a linear decrease from approximately 99 percent to 96 percent. The E G group remains nearly flat at approximately 99.5 percent.
Panel b, First-pass reading time (Early). Both groups show a linear decrease. C G drops from approximately 510 to 420. E G drops from approximately 560 to 510.
Panel c, Re-reading time (Late). C G shows a linear decrease from approximately 800 to 650. E G shows a linear increase from approximately 750 to 920.
Panel d, Regression path duration (Early forward slash Late). C G shows a linear decrease from approximately 6.7 to 6.45. E G shows a slight linear increase from approximately 6.75 to 6.8.
Panel e, Regressive transition (Late). C G shows a slight linear decrease from approximately 0.34 to 0.28. E G shows a linear increase from approximately 0.24 to 0.43.
All plots include shaded error bands representing confidence intervals.
Table G6 in Online Supplementary Materials G reports results of the mixed effects model for first-pass reading time. Treatment groups (Group) and input times (Time) were significant predictors. The EG had overall longer first-pass reading time than the CG: Estimate = 0.14, p = .030. First-pass reading time was shorter during the first than the second time of input: Estimate = −0.15, p < .001. Figure 5(b) presents an effects plot for the model. Between the two times of input, both groups’ first-pass reading time reduced almost parallelly. These results suggested little effects of output on first-pass reading of the target form.
Table G8 in Online Supplementary Materials G reports results of the mixed effects model for re-reading time. The interaction between treatment groups and input times (Group:Time) was significant in both the binary (Estimate = −1.19, p < .001) and the positive parts (Estimate = 0.45, p = .001). This indicated that treatment groups and input times jointly affected re-reading time in two respects: the probability of re-reading (binary part), and the duration of re-reading when it happened (positive part). Figure 5(c) presents an effects plot for the model. From the first to the second time of input, re-reading time decreased for the CG but increased for the EG. These results suggested that output promoted re-reading the target form.
Table G9 in Online Supplementary Materials G reports results of the mixed effects models for regression path duration. Statistical significance was found for all predictors: treatment groups (Group; Estimate = 0.18, p = .032), input times (Time; Estimate = −0.10, p = .019), and their interaction (Group:Time; Estimate = 0.33, p < .001). These indicated that treatment groups and input times affected regression path duration both independently and jointly. Figure 5(d) presents an effects plot for the model. Between two times of input, regression path duration followed a downward trend for the CG but an upward trend for the EG. These results suggested that output extended regression path duration for the target form.
Table G10 in Online Supplementary Materials G reports results of the mixed effects models for regressive transitions. The interaction between treatment groups and input times (Group:Time) was statistically significant: Estimate = 0.76, p = .001. Figure 5(e) presents an effects plot for the model. From the first to the second time of input, regressive transitions decreased for the CG but increased for the EG. These results suggested that output promoted returning to the former component from the latter component of the target form.
Comparing the statistical results (Tables G5–6, G8–10) and the effects plots (Figure 5) of the five eye-tracking measures, the following pattern emerges regarding the early and later cognitive processes associated with these measures. The interaction between treatment groups and input times was significant for the three measures that mainly tapped into later cognitive processes: re-reading time, regression path duration, and regressive transitions showed a downward trend for the CG but an upward trend for the EG. Contrastively, this interaction was not significant, with non-opposite trends for the two measures reflecting early cognitive processes, fixation probability and first-pass reading time. Particularly, first-pass reading time decreased almost in parallel for the two groups. These results suggested that the effects of output on attention to subsequent input were more prominent at later than earlier stages of cognitive processing.
RQ3: Attention to input as a mediator between output and grammar production
Table 3 reports results for the total, direct, and indirect effects of treatment groups on posttest scores in five mediation models, with fixation probability, first-pass reading time, re-reading time, regression path duration, and regressive transitions as the mediator, respectively. Figure 6 plots 95% CIs of these effects to facilitate result interpretation. First, the total effect (path c in Figure 3) of treatment groups on posttest scores was statistically significant and positive in all models, with 95% CIs above zero. Because the total effect represents the difference between the group means, these results suggested that the output group’s posttest performance was better than the no-output group’s, echoing RQ1’s finding that output plus input impacted L2 grammar production positively.
Total, direct, and indirect effects in five mediation models
Table 3. Long description
The table columns represent five eye-tracking metrics: Fixation probability, First-pass reading time, Re-reading time, Regression path duration, and Regressive transitions. The rows are divided into three sections.
1. Total effect c: All metrics show significant B values ranging from 2.19 to 2.66 with p-values less than .05. Standard errors S E are approximately 1.00. The 95 percent C I lower bounds are all positive, ranging from 0.25 to 0.75.
2. Direct effect c prime: Only Fixation probability remains significant with a B of 2.37 and p-value of .023. Other metrics show non-significant p-values ranging from .051 to .186, with 95 percent C I lower bounds crossing into negative values (e.g., minus 0.01 to minus 0.67).
3. Indirect effect a b: This section reports bootstrapped values. B values range from minus 0.15 for Fixation probability to 1.21 for Regression path duration. All 95 percent bootstrapped C I intervals include zero, with lower bounds ranging from minus 0.69 to minus 0.21 and upper bounds from 0.47 to 2.39, indicating no significant indirect effects across the models.
Note. *p < .05. SEbt = Bootstrapped SE. 95% CIbt = Bootstrapped 95% CI.
95% CIs of the total, direct, and indirect effects in five mediation models.
Figure 6. Long description
A multi-panel figure with a legend on the top left and three data panels.
Legend:
* Olive: Fixation Probability
* Red: First-Pass Reading Time
* Green: Re-reading Time
* Blue: Regression Path Duration
* Purple: Regressive Transition
Panel a, Total Effect c:
The y-axis ranges from 0 to 4. All five measures show positive effects with confidence intervals entirely above zero. The means for all measures cluster between 2 and 3.
Panel b, Direct Effect c prime:
The y-axis ranges from 0 to 4 with a dashed horizontal line at 0. Fixation Probability and First-Pass Reading Time show positive effects with intervals above zero. Re-reading Time, Regression Path Duration, and Regressive Transition have lower means between 1 and 2, with their lower error bars extending near or slightly below the zero line.
Panel c, Indirect Effect a b:
The y-axis ranges from 0 to 2 with a dashed horizontal line at 0. Fixation Probability is slightly below zero. First-Pass Reading Time is slightly above zero. Re-reading Time and Regression Path Duration show the highest indirect effects with means above 1, while Regressive Transition is approximately 0.6. The error bars for all measures in this panel cross the zero line except for Re-reading Time and Regression Path Duration.
Second, the direct effect (path c′ in Figure 3) of treatment groups on posttest scores was statistically significant in the model with fixation probability, marginally significant in the model with first-pass reading time, and non-significant in the models with late eye-movement measures as the mediators. Because the direct effect denotes group differences that are independent of the mediator’s effects, these results indicated that the output group’s superior posttest performance remained significant or near significant even when accounting for the mediating effects of the two early measures, fixation probability and first-pass reading time. Conversely, when the effects from the three late measures were partialed out, the group difference in posttest performance was no longer significant. These results suggested that the effects of output on L2 grammar production may be independent of attention during early but not later stages of cognitive processing.
Third, the indirect effect (path ab in Figure 3) of treatment groups on posttest scores via attention to subsequent input was statistically not significant in any of the models, with 95% CIs crossing zero. Notably, when considering the locations of the 95% CIs, re-reading time, regression path duration, and regressive transitions had large proportions of their 95% CIs above zero, indicating a tendency for these late measures to approach statistical significance. Also, the 95% CIs were much larger for the three late measures than the two early measures, suggesting large individual differences in later-stage cognitive processing among our participants. Further comparison of the B values for the direct and indirect effects showed that in the two models with the early measures as the mediators, the indirect effects were much smaller than the direct effects, suggesting early-stage cognitive processing may not mediate the output advantage. Conversely, in the models with the late measures as the mediators, the indirect effects were proportionally closer to the direct effects. These results suggested that the indirect effect of output on L2 grammar production via attention to subsequent input might be stronger during later than early stages of cognitive processing.
Overall, our results did not achieve statistical significance for a mediating role of attention between output and L2 grammar learning, although we observed a tendency toward significant mediation in the measures reflecting later cognitive processes (re-reading time, regression path duration, regressive transitions), which deserves further investigation. One possibility is that the large individual differences in these late measures (as revealed by their large 95% CIs) may have obscured any group-level mediation effects. From a theoretical perspective, our findings suggested a need to further consider attention and its co-occurrence with awareness to explore the mediating effect of noticing between output and learning.
RQ4: Reasons for heightened attention to input following output
Table 4 presents the coding scheme together with the quantitative results for each theme. These themes emerged from coding participants’ stimulated recall for reasons for their heightened attention (regressive eye movements) to input after producing output. These themes were categorized at three levels reflecting both progression and depth in qualitative coding.
Coding scheme with quantitative results.
Table 4. Long description
The table consists of four columns: Category and Sub-category, Instances, Unique I D s, and Unique I D s percentage.
1. Main Category: Positive experiences of the S L A.
- Sub-category: Fun and enjoyment. 11 Instances, 7 Unique I D s, 64 percent.
- Sub-category: Ease of use. 10 Instances, 7 Unique I D s, 64 percent.
- Sub-category: Usefulness. 10 Instances, 6 Unique I D s, 55 percent.
2. Main Category: Negative experiences of the S L A.
- Sub-category: Technical issues. 11 Instances, 6 Unique I D s, 55 percent.
- Sub-category: Difficulty of use. 6 Instances, 4 Unique I D s, 36 percent.
- Sub-category: Lack of usefulness. 1 Instance, 1 Unique I D, 9 percent.
3. Main Category: Suggestions for improvement.
- Sub-category: Features and functionality. 11 Instances, 7 Unique I D s, 64 percent.
Note: Total number of participants equals 11.
Note. Instances = Number of instances. Unique IDs = Number of participants. Unique IDs (%) = Percentage of participants. Total number of participants = 11.
At Level 1, the first stage of coding, the simulated-recall data were assigned one of three themes: (1a) Lapse in attention, when participants forgot their thoughts; (1b) Reading input, when they mentioned the reading content only; or (1c) Task (reading and reconstruction), when they referred to the reconstruction task on top of the reading content. If (1a) was assigned, coding finished at this level, whereas for (1b) or (1c), coding continued to Level 2.
At Level 2, one or more of five themes were assigned depending on the content of participants’ stimulated recall. For (2a) Connection with real-world knowledge/own experience, participants connected the reading content with their personal life situations. Differently, (2b) Content applied when the focus was the reading content itself. (2c) Vocabulary was assigned to reasons for heightened attention to L2 words. Two themes were grammar-related: (2d) Grammar: Other forms for comments about L2 grammatical forms other than the target structure, or (2e) Grammar: Target form for the English past counterfactual conditional. Coding either finished at this level, or moved to the final level when (2e) Grammar: Target form applied.
At Level 3, one or more of five themes were assigned to comments about the target form. (3a) Connections with existing knowledge referred to situations where participants compared the target form to other L1, L2, or L3 forms in their multilingual repertoires (see Excerpt-1 below for an example). When discussing the target form, if participants used metalinguistic terminology, (3b) Form-level noticing + metalinguistic terminology was assigned; otherwise, we assigned (3c) Form-level noticing – metalinguistic terminology. Excerpt-2 and Excerpt-3 provide an example for (3b) and (3c), respectively. For (3d) Meaning-level noticing, the grammatical meaning of the target form (counterfactuality) was commented on (see an example in Excerpt-1). Lastly, (3e) Noticing the gap applied when participants connected their attempted use of the target form during text reconstruction with what they noticed about the target form during the second reading (example see Excerpt-2).
Excerpt-1.
Text in focus: For the last job offer, I would have been able to accept the suggestion if I had already paid all my bills.
Participant: I just want to recheck what time is it and if that real happened or not. (3d. Meaning-level noticing)
Researcher: Okay, yeah, so did it really happen or are they just talking about if it happened?
Participant: Yeah, because in Chinese, it will … I will get the information directly from the characters. (3a. Connections with existing knowledge)
Excerpt-2.
Text in focus: If I had taken the first option, I would have had to worry about paying my bills.
Participant: Because I really forgot how to write a sentence in subjunctive mood, so I always write “if I took” or “if I would like to take,” so I go back here. (3b. Form-level noticing + metalinguistic terminology)
Researcher: So you’re going back to look at the words here? Can you explain more?
Participant: How … how to … how to say that? The structure or the formula of the grammar, “had taken” or “have taken,” or … because I could not tell the difference in here, whether I should use “have” or “had.” And I realized I need to use “had” here, so I need to remember it for a while. (3e. Noticing the gap)
Excerpt-3.
Text in focus: If I had been in this situation, I would have considered two main points: my present financial needs, and my future professional goals.
Researcher: Yeah. You read the sentence without stopping, and then I think you got to this comma, and you went back to here. Do you remember what you were thinking?
Participant: Because this is the second time I read this passage, so I noticed there’s been consistently using “if I had been,” “I would have done something.” That’s my first time noticed that. Then I … when I finished the sentence, I jumped back to the “I would have.” You may see I would jump back across the lines later. I keep checking if my assumption was correct. (3c. Form-level noticing – metalinguistic terminology)
We further break down these results by individual participants in Table G16 of Online Supplementary Materials G to facilitate checking for participant overlap between themes. At Level 1, over half of the participants reported at least one (1a) Lapse in attention where they could not recall reasons for their regressive eye movements. Further comparison of participant lists for (1b) Reading input and (1c) Task (reading and reconstruction) showed that all participants referred to input in their reasons, and that almost three-quarters of them went further to reference output (text reconstruction) as well.
At Level 2, regarding the reading (input) or the reconstruction task (output), half or more of the participants noticed not only the text meaning (2b. Content) but also the language aspects (2c. Vocabulary; 2e. Grammar: Target form). The fewest participants made a (2a) Connection with real-world knowledge/own experience or commented on (2d) Grammar: Other forms. Participants who mentioned other grammatical forms (2d) also discussed the target form (2e), suggesting they were strong learners. Grammar (2e) also enjoyed more noticing than vocabulary (2c), possibly due to input flooding.
At Level 3, the target structure elicited form-level noticing most often, both with and without metalinguistic terminology (3b/3c. Form-level noticing +/– metalinguistic terminology). Whereas form-level categories (3b) and (3c) showed some overlap in terms of participants, those engaged in (3d) Meaning-level noticing overlapped entirely with either (3b) or (3c), indicating that when meaning-level noticing occurred, form-level noticing also happened. A relatively smaller number of participants made (3a) Connections with existing knowledge or (3e) Noticed the gap between their text reconstruction and the reading input.
To summarize, our results suggested that heightened attention to input after producing output overwhelmingly resulted from noticing (i.e., attention with awareness) of reading input or written output, although lapses in attention did occur occasionally and prevented the eye movement and the stimulated-recall data from being linked in these cases. Noticing of input/output took place not only at the content (meaning) level, but also at the linguistic level involving noticing of L2 vocabulary and grammar. For the target structure, learners noticed the formal aspects of the English past counterfactual conditional more often than its grammatical meaning, and in fact, such form-level noticing may have been a precondition for meaning-level noticing to occur. Noticing the gap and connecting the target structure with existing L1/L2/L3 knowledge also accounted for heightened attention to the target form.
Discussion
Accuracy of grammar production and attention to target grammar in subsequent input
We adopted an explanatory sequential design that consisted of two experiments to examine L1 Chinese learners’ reading and production of an L2 target form, the English past counterfactual conditional. We measured noticing at the level of attention and awareness, respectively, by combining eye-tracking technology and stimulated recall. Particularly, RQ1 focused on the effects of output plus input on accuracy of L2 grammar production. Results showed that two rounds of essay-reading (input) plus text-reconstruction (output) led to improved production of the target structure from pretest to posttest compared to the no-output group. Our study extends the long line of L2 research that employed a similar within- and between-groups design (Izumi, Reference Izumi2002; Izumi & Bigelow, Reference Izumi and Bigelow2000, Phase 2; Izumi et al., Reference Izumi, Bigelow, Fujiwara and Fearnow1999, Phase 1; Leeser, Reference Leeser2008; Russell, Reference Russell2014; Song & Suh, Reference Song and Suh2008; see studies with * in Table 1) and enriches evidence for the positive effects of output on L2 learning, which were generally found in the preceding studies (except Izumi & Bigelow, Reference Izumi and Bigelow2000, Phase 2).
RQ2 explored the effects of output on attention to the L2 target form in subsequent input—the noticing function of output at the attention level. Results showed that the no-output participants’ attention to the target structure decreased from the first to the second time of reading (reduction in all five eye-movement measures), similar to previous L2 research findings (Hansen, Reference Hansen2019; Rich, Reference Rich2019). This finding likely reflects a text repetition effect, whereby reading a text a second time leads to facilitated, or less effortful processing (Chamberland et al., Reference Chamberland, Saint-Aubin and Légère2013; Foster et al., Reference Foster, Ardoin and Binder2013; Hyönä & Niemi, Reference Hyönä and Niemi1990; Kaakinen & Hyönä, Reference Kaakinen and Hyönä2007; Raney & Rayner, Reference Raney and Rayner1995; Raney et al., Reference Raney, Therriault and Minkoff2000; Rogers, Reference Rogers2014; Zawoyski et al., Reference Zawoyski, Ardoin and Binder2015). The no-output group’s reduced attention provided a baseline for comparison with the output group, which enabled us to assess the noticing function of output at the attention level.
Specifically, the output group’s first-pass reading time displayed a decreasing trend between the two times of input, almost parallel to that of the no-output group. The output group’s first-pass reading time, then, similarly showed a text repetition effect. In contrast, fixation probability of the output group remained almost unchanged between the two times of input, whereas the no-output group showed a notable drop. During the first reading, the output group looked at 98% of the target structures, and this remained high, at 97%, even during the second reading. The noticing function of output therefore canceled out the expected text repetition effect (i.e., reduction) in the output group’s fixation probability. Going one step further still, for re-reading time, regression path duration, and regressive transitions, which are associated with later cognitive processes, the output group showed a significant increase in attention between the two times of input, reversing the no-output group’s reduced attention. The noticing function of output, therefore, was most pronounced at later stages of input processing and, indeed, was substantial enough to override the text repetition effect.
Results of RQ2 together support the noticing function of output at the attention level: output promotes attention to target L2 grammar in subsequent input. For the output group who engaged in text reconstruction, the grammatical encoding and monitoring processes during production may have highlighted the problems in their knowledge of the target structure and prompted them to search for solutions (Izumi, Reference Izumi2003). When the model essay was available for a second time, the repeated exposure aided lexical access and word recognition, characteristic of early cognitive processes, as reflected in similar reductions in first-pass reading time for both the output and no-output groups. Importantly, for the output group, the problems highlighted by output may have driven them to deliberately analyze the second round of input: instead of skipping the target form like the no-output participants did, they spent more time re-reading it and connecting it to former text through regressive eye movements. In this way, output may have acted as an “internal priming device” (Izumi, Reference Izumi2003, p. 184) to promote attention to and facilitate deeper processing of target L2 grammar in subsequent input.
We operationalized noticing at the attention level using eye-tracking technology, whereas previous L2 studies employing a similar design (see studies with * in Table 1) measured noticing with note-taking (Izumi, Reference Izumi2002; Leeser, Reference Leeser2008) or underlining (Izumi & Bigelow, Reference Izumi and Bigelow2000, Phase 2; Izumi et al., Reference Izumi, Bigelow, Fujiwara and Fearnow1999, Phase 1; Russell, Reference Russell2014; Song & Suh, Reference Song and Suh2008). Findings from these studies generally did not support the noticing function of output (except partial support from Leeser, Reference Leeser2008; Russell, Reference Russell2014), in contrast with the positive results we report for RQ2. The different findings may be explained by two methodological improvements for RQ2: operationalizing noticing at the attention level using eye-tracking to enable more accurate measurement (Godfroid et al., Reference Godfroid, Boers and Housen2013; Indrarathne & Kormos, Reference Indrarathne and Kormos2017), and adopting both early and late eye-movement measures to allow selective examination of different cognitive processes (Maie & Godfroid, Reference Maie and Godfroid2022). Thanks to these improvements, our study has paved the way for understanding the effects of output on attention to subsequent input as a dynamic, temporally staged process: the effects grew stronger over time. Indeed, at early stages of input processing, the effects of output on attention may not manifest themselves, but instead, they may arise and become progressively stronger during later stages when learners engage with the input more deliberately through re-reading and/or re-analysis.
Mediating role of attention and reasons for heightened attention inferred from awareness data
We tested the core prediction of the Output Hypothesis (Swain, Reference Swain and Hinkel2005)—output promotes noticing, which in turn facilitates learning—by conducting the first mediation analysis linking all three components simultaneously. Specifically, RQ3 examined the mediating role of noticing operationalized at the attention level using eye-tracking data. While the mediation of attention did not achieve statistical significance, the patterns differed for early (fixation probability, first-pass reading time) versus late eye-movement measures (re-reading time, regression path duration, regressive transitions). Table 3 suggests that attention at later (vs. early) stages of input processing may contribute more to the total beneficial effects of output on learning. This interpretation is supported by the large individual differences in the late eye-movement measures (see large 95% CIs in Figure 6), and it aligns with the finding for RQ2 that the effects of output on attention emerged more clearly in late measures.
Taken together, results from RQ2 and RQ3 suggest that output can promote increasingly heightened attention to subsequent input, and such attention in turn might facilitate learning for some individuals. However, the absence of a statistically significant mediating effect—combined with pronounced individual differences in late eye-movement measures—raises the possibility that awareness may be implicated in the mediating relationship between output and learning, particularly when attention and awareness of the target structure co-occur. This possibility provided the rationale for RQ4.
Specifically, for RQ4, we adopted stimulated recall to investigate why L2 learners paid increased attention to input following output, thereby examining the noticing function of output at both the attention and awareness levels. Consistent with the view that attention can mediate learning, more than half of the participants attributed their heightened attention to noticing the L2 target form, the English past counterfactual conditional (i.e., attention and awareness) (see Table 4). They reported noticing the gap between how they used the target form in their writing and how it was used in the model essay, and described linking their knowledge of the target form with existing knowledge (e.g., verb tense) in their multilingual repertoires. Most noticing episodes of the target form concerned its formal features, sometimes accompanied by its grammatical meaning of counterfactuality. These findings suggest that attention at later stages of input processing (when coinciding with awareness) may mediate the benefits of output for grammar learning.
At the same time, our stimulated-recall data revealed substantial individual variation in learners’ reasons for paying increased attention to some parts of the text.Footnote 3 Noticing was generally balanced between the target structure and the essay content (32 vs. 36 instances; see Table 4). At the individual level, all participants reported noticing the essay content and over half also reported noticing L2 vocabulary. Lastly, few participants reported noticing L2 grammar other than the target structure, which, while infrequent, supports the notion from early output research that producing output can benefit L2 learning in general (Griffin, Reference Griffin2004; Hanaoka, Reference Hanaoka2007; Hanaoka & Izumi, Reference Hanaoka and Izumi2012; Qi & Lapkin, Reference Qi and Lapkin2001; but see Sakai, Reference Sakai2004). Of note, more than half of the participants reported in one or two cases that they did not remember why they made regressive eye movements. It is possible that they truly forgot, that the regressions were unconscious (e.g., to correct a motor error in the preceding eye movement and move the eye to a more optimal location in the text), or that processing itself was unconscious.
Overall, the qualitative findings of awareness (RQ4) complement the quantitative results for attention (RQ3) by illuminating the complex interrelationships among output, attention, awareness, and learning. Heightened attention to input following output can arise for many reasons—some related to target-form noticing, others related to content or vocabulary, and some possibly implicit. As Winke (Reference Winke2013) noted, eye-movement data alone cannot explain the why of increased attention; our results help fill this gap by showing that in the subset of cases where increased attention coincided with awareness of the target structure and therefore signaled target-structure noticing, attention facilitated (mediated) learning.
Our study has provided the first quantitative and qualitative evidence for how attention as part of noticing may mediate the effects of output on learning—the crux of the Output Hypothesis (Swain, Reference Swain and Hinkel2005). Although previous studies on the noticing function of output occasionally examined the relationship between noticing and learning (Griffin, Reference Griffin2004; Hanaoka, Reference Hanaoka2007; Hanaoka & Izumi, Reference Hanaoka and Izumi2012; Qi & Lapkin, Reference Qi and Lapkin2001; Sakai, Reference Sakai2004), none of their statistical analyses captured output, noticing, and learning simultaneously—this long-standing gap has been addressed by our study. Furthermore, while prior research has characterized noticing of input following output as either facilitative (Hanaoka, Reference Hanaoka2007; Hanaoka & Izumi, Reference Hanaoka and Izumi2012; Qi & Lapkin, Reference Qi and Lapkin2001) or non-facilitative (Sakai, Reference Sakai2004), our study calls for a more nuanced interpretation of how such noticing may mediate the relationship between output and learning. In light of these results, we propose a conceptual framework for advancing research on the Output Hypothesis (see Figure 7).
Conceptual framework for output, noticing (attention, awareness), and learning.
This frameworkFootnote 4 underscores that producing output can improve L2 grammar learning and promote attention to subsequent input. Heightened attention with awareness (i.e., noticing) of target L2 grammar during input processing may facilitate interlanguage restructuring and consequently lead to improved L2 performance. At the same time, the contributions of attention and awareness, jointly and individually, remain underexplored in the output-learning relationship. Notably, our focus on the stage of input processing following output parallels work on written corrective feedback, and our framework is similar to Leow’s (Reference Leow and Manchón2020) Feedback Processing Framework in emphasizing attention and awareness for this stage. Nonetheless, in Leow’s (Reference Leow and Manchón2020) framework, the key factors contributing to the learning process are awareness and depth of processing; although attention plays a crucial role for initial input processing and provides a precondition for explicit learning, there is no major attempt to measure it or analyze its contribution further. Grounded in eye-tracking research on reading, we conceptualize attention as a continuous, quantifiable, and temporally unfolding process. By measuring attention concurrently through eye-movement registration, we can examine it at varying levels of granularity and distinguish early versus late stages of processing, which map onto automatic vs. controlled cognitive processing (e.g., Maie & Godfroid, Reference Maie and Godfroid2022), and may relate to learning differently. Critically, by analyzing attention with this temporal and spatial resolution, our framework provides the empirical scaffolding needed to investigate the co-occurrence of awareness with specific attentional states—a key condition for understanding how noticing emerges and how it may mediate the effects of output on learning.
Limitations and future directions
While our study design addressed some of the methodological concerns that may have contributed to the mixed findings in previous research, we did not include a delayed posttest to assess the long-term effects of output on L2 grammar learning. Several studies on the noticing function of output have adopted delayed posttests (Hanaoka, Reference Hanaoka2007; Nassif, Reference Nassif2019; Uggen, Reference Uggen2012; Zalbidea, Reference Zalbidea2021), and it is recommended that researchers continue this optimal practice if feasible. Due to space limits, we did not analyze the essays produced by the output group during the reconstruction tasks either. Future research can quantify the output group’s attempted use of the target form (see, e.g., Izumi et al., Reference Izumi, Bigelow, Fujiwara and Fearnow1999, for an example), thereby operationalizing the output variable as continuous (i.e., number of attempts) rather than categorical (output vs. no-output). Lastly, although we aimed for a larger sample size for the stimulated-recall group (n = 25), we were able to include only 11 participants for analysis due to variability in the data collection procedure, which made it difficult to identify the text in focus and/or the focused component of the target structure in some participants’ stimulated-recall data (see Online Supplementary Materials B for details). Beyond additional training, it is recommended to preview the stimulated-recall data while data collection is ongoing to identify and address potential quality issues early on.
Future research could build on our study by incorporating awareness data into a mediation analysis to produce a more stringent test of its causal role in learning (Figure 7, green arrow). It will also be important to examine learning outcomes in cases where there is heightened attention but no reported awareness (as in the nine instances in our study) or when learners process the target structure fluently (without regressing in the text), which we did not examine here. Notably, Godfroid and Schmidtke (Reference Godfroid, Schmidtke, Bergsleithner, Frota and Yoshioka2013) observed that some vocabulary learning also happened even when the levels of both attention and awareness were low. Ultimately, further clarifying the respective roles of attention and awareness will be key to developing an increasingly nuanced understanding of why output may benefit learning, and to support the pedagogical practice of writing-to-learn in L2 instruction.
Conclusion
We adopted an explanatory sequential design to investigate the noticing function of output at both the attention and awareness levels, using eye-tracking technology plus stimulated recall. Our results support the effects of written output on improving accuracy of written L2 grammar production and promoting attention to target L2 grammar in subsequent readings, and further demonstrated how the effects of output on attention dynamically grew stronger during later-stage input processing and critically involved learners’ conscious engagement with the target structure. Importantly, our study has provided the first quantitative and qualitative evidence identifying the conditions under which attention as part of noticing may mediate the effects of output on learning, thus capturing the core prediction of the Output Hypothesis (Swain, Reference Swain and Hinkel2005)—output promotes noticing, which in turn facilitates learning.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0272263126101843.
Acknowledgments
We are thankful to Dr. Jared Kubokawa, Yusuke Kuroki, and Yuko Nakanishi for their help with data collection and data coding. Our heartfelt appreciation also goes to Dr. Patti Spinner, Prof. Susan Gass, and Prof. Shawn Loewen for their feedback during the initial stages of research idea development. Lastly, we express our sincere gratitude to our handling editor, Prof. Andrea Révész, for her strong support throughout the review process.
Data availability statement
The experiment in this article earned Open Materials badge for transparent practices. The materials are available at https://osf.io/8xpdy/overview.
Competing interests
The authors declare none.
Open access
