INTRODUCTION
The process of attention has long occupied a central place in SLA theory as a mechanism that may guide and enable the second-language (L2) acquisition process (e.g., Indrarathne & Kormos, Reference Indrarathne and Kormos2017; Issa & Morgan-Short, Reference Issa and Morgan-Short2019; Robinson, Reference Robinson1995, Reference Robinson, Doughty and Long2003; Robinson et al., Reference Robinson, Mackey, Gass, Schmidt, Gass and Mackey2012; Schmidt, Reference Schmidt1990, Reference Schmidt and Robinson2001). Focal attention, as a manifestation of noticing, appears to be conducive and necessary for the acquisition of grammatical forms (Schmidt, Reference Schmidt1990). Attentional resources are also required to detect forms in the input (Tomlin & Villa, Reference Tomlin and Villa1994), even if such registration takes place unconsciously, without awareness. One line of research that is of particular relevance here has investigated the effectiveness of instructions for drawing L2 learners’ attention to linguistic properties of forms in meaning-focused contexts (e.g., Doughty & Williams, Reference Doughty and Williams1998; Lightbown & Spada, Reference Lightbown and Spada1990; Long, Reference Long, DeBot, Ginsberg and Kramsch1991).
Given that L2 learners’ capacity to process information is limited, they may not be able to fully utilize their attentional resources; hence, when they attend to form (i.e., pay conscious attention to a particular form), they may not attend to meaning (i.e., processing for meaning; that is, comprehension) (e.g., VanPatten, Reference VanPatten, Larson, Judd and Messerschmitt1985). In response to Schmidt’s (Reference Schmidt1988) argument that second language is acquired through attention to grammatical forms, VanPatten (Reference VanPatten1990) questioned whether L2 learners can pay conscious attention to grammatical forms while processing for meaning of the input text. Since VanPatten’s pioneering Reference VanPatten1990 study, he and other researchers have conducted a series of interconnected studies to explore whether L2 learners allocate available attentional resources to the processing of form, processing of meaning, or both (Greenslade et al., Reference Greenslade, Bouden and Sanz1999; Leow et al., Reference Leow, Hsieh and Moreno2008; Morgan-Short et al., Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012; Morgan-Short et al., Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018; VanPatten, Reference VanPatten1990; Wong, Reference Wong2001). These studies have produced mixed results with regard to L2 learners’ ability to comprehend the meaning of input while simultaneously attending to form, further complicating the ongoing debate on how L2 learners distribute their attention to the form versus the meaning of input.
The foundational argument of VanPatten’s (Reference VanPatten1990) work is his “primacy of meaning” principle. It posits that L2 learners have limited attentional resources and therefore process meaning before form. Based on this view, VanPatten suggested a new model of input processing and its pedagogical corollary, the approach of processing instruction. The replication studies that followed have extended the investigation of attention from the auditory to the written modality, while also adding concurrent measurements of the extent to which L2 learners attend to target forms. These studies have tried to address both theoretical and methodological issues related to the role of attention in processing form and meaning but have reported mixed results.
As previous researchers have noted (Moreno, Reference Moreno, Han and Rast2014; Morgan-Short et al., Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012; Morgan-Short et al., Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018), more replication studies with new methodological approaches are necessary to clarify conflicting findings and advance the strand of attentional research in SLA. Outcome-based measurements of L2 learners’ attention such as marking are insufficient to show whether and to what extent L2 learners pay attention to form versus meaning. Innovating the measurement of attention, Leow et al. (Reference Leow, Hsieh and Moreno2008) and Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012) adopted think-aloud protocols as an online (real-time) technique to explore these matters; at the same time, think-alouds cannot provide a full account of attentional processes, as learners cannot verbalize every aspect of their attentional behavior (on nonveridicality, see Bowles, Reference Bowles2010; Godfroid, Reference Godfroid2020; Rebuschat, Reference Rebuschat2013; Shanks & St. John, Reference Shanks and St. John1994). Think-alouds also have the in-built assumption that processing form and meaning, and the form-meaning relationship, is a conscious process (or else it won’t be reflected in the think-aloud protocol). Further research with other concurrent measures of attention that provide a more complete record of attention and are not as tightly linked to awareness is therefore still needed to augment our understanding of the degree to which L2 learners can pay attention to form and meaning.
To fill this gap, the current study adopts materials from previous studies (Leow et al., Reference Leow, Hsieh and Moreno2008; Morgan-Short et al., Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012, Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) and investigates whether L2 Spanish learners can attend to form and meaning simultaneously. The methodological innovation of the present study is that, in place of think-aloud protocols, we employ an eye-tracking paradigm to provide reliable information about L2 learners’ attention (e.g., Godfroid et al., Reference Godfroid, Boers and Housen2013; Indrarathne & Kormos, Reference Indrarathne and Kormos2017; Issa & Morgan-Short, Reference Issa and Morgan-Short2019) and to reflect their online comprehension processes (e.g., Rayner et al., Reference Rayner, Chace, Slattery and Ashby2006). The unobtrusive methodological paradigm of eye-tracking (Conklin et al., Reference Conklin, Pellicer-Sánchez and Carrol2018; Godfroid, Reference Godfroid2020) will provide concurrent evidence for whether or not L2 learners’ attention is allocated to the target form, the meaning of the text, or both, as well as on the extent (or depth) with which L2 learners process form, meaning, or both (i.e., depth of processing; Godfroid, Reference Godfroid and Leow2019). In addition, because eye-tracking can capture L2 learners’ ongoing reading behaviors throughout an entire reading passage, it will show whether attention to form leads to different text reading patterns. Lastly, differing from previous studies, we controlled for proficiency effects by means of an independent proficiency measure, namely an elicited imitation test (EIT; Bowden, Reference Bowden2016; Ortega et al., Reference Ortega, Iwashita, Norris and Rabie2002), which resulted in a more accurate (confound-free) picture of learners’ simultaneous attention. The study thus revisits the questions of whether simultaneous attention to form and meaning is possible, how it would be manifested in real-time reading behavior, and whether attention to form in addition to meaning leads to differences in comprehension compared to attention to meaning alone.
LITERATURE REVIEW
SIMULTANEOUS ATTENTION TO FORM AND MEANING
VanPatten (Reference VanPatten1994, Reference VanPatten1996, Reference VanPatten and VanPatten2004) proposed input processing in L2 acquisition as a theoretical account of learners’ attention to the linguistic form of L2 input. VanPatten (Reference VanPatten and VanPatten2004) defined input processing as an online procedure in which L2 learners make form–meaning/function connections during their comprehension of input. Drawing on principles of cognitive psychology (e.g., McLaughlin, Reference McLaughlin1987), VanPatten posited that L2 learners have limited processing resources and limited working memory capacity, and therefore they process the meaning (the message or communicative intent of the input) before they process the form of L2 input; this is the primacy of meaning principle.
If L2 learners’ limited attentional resources make it difficult for them to attend to form and meaning simultaneously, then it is expected that conscious attention to form will have adverse impacts on comprehension. In VanPatten’s initial Reference VanPatten1990 study, a total of 202 participants at three different Spanish proficiency levels listened to a passage under one of four conditions: attention-to-form for one of three different target forms, or attention-to-meaning (content-only). The three target forms were the lexical item inflación, the definite article la, and the verb suffix - n (the verb inflection to mark third-person plural). To measure their conscious attention to specific target forms, the participants in the three experimental groups were required to put a check mark on a blank piece of paper whenever they heard the target form they had been assigned. The fourth group, who were told to focus on content, were the control participants. Each target form was included 11 or 12 times in the 3-minute listening passage. Participants had to record at least 60% of the occurrences of their assigned target form (indicating adequate attention to form) for them to be included in the data analysis. After the listening task, participants were asked to conduct a free written recall of the listening passage in English as a measure of their processing for meaning (comprehension). The analysis of the participants’ written recall indicated that attention to the grammatical forms (i.e., la and -n) negatively influenced comprehension, while attention to the lexical form (i.e., inflación) did not affect comprehension. VanPatten (Reference VanPatten1990) also examined proficiency differences. He reported that beginning L2 learners had more difficulty in attending to both form and meaning, suggesting that their capacity to process the L2 was more limited than that of higher proficiency learners. Based on these results, VanPatten concluded that simultaneous attention to form and meaning is difficult for L2 learners, particularly at an early stage.
Subsequently, Greenslade et al. (Reference Greenslade, Bouden and Sanz1999) replicated VanPatten’s (Reference VanPatten1990) study in the written modality, using the same passage as a written text. The participants were 62 third-semester L2 Spanish students. Although Greenslade et al. did not control the students’ rereading of the passage, they did control the exposure time to the passage (i.e., 2.5 minutes). The participants in the experimental groups were required to underline, circle, or check the assigned form. The authors also administered a written recall task to check comprehension of the reading passage. Following VanPatten’s procedure, only data from learners who marked at least 60% of the target items were included in the data analysis. The experimental groups’ recall scores were lower than the scores of the control group. Greenslade et al. attributed the more limited text comprehension by the form-attention groups to the competition between form and meaning for the learners’ attentional resources. Extending VanPatten’s findings to the written modality, Greenslade et al. similarly concluded that attention to form impedes beginning L2 learners’ attention to meaning.
Wong (Reference Wong2001) investigated simultaneous attention to form and meaning in both aural and written modalities with 79 L1 French lower-intermediate ESL learners. Wong translated the passages used by VanPatten (Reference VanPatten1990) and Greenslade et al. (Reference Greenslade, Bouden and Sanz1999) into English. Therefore, the target forms were the and inflation; the verb suffix -n was not used because it has no corresponding form in English. As in the previous studies, only the participants who marked at least 60% of the target forms were included in the data analysis. In the aural mode, the recall results were consistent with VanPatten’s in that the (content-focused) control group performed significantly better than the the group, but at a similar level as the inflation group. In the written mode, however, no significant differences were found across the groups. Wong’s findings highlight the importance of modality in studying attention to form and meaning. The results from the aural input supported VanPatten’s conclusions. Different from Greenslade et al. (Reference Greenslade, Bouden and Sanz1999), the results from the written input, however, suggest that simultaneous attention to form and meaning may be possible during L2 processing, at least in the written modality.
In a critical reappraisal of VanPatten’s primacy of meaning principle, Leow et al. (Reference Leow, Hsieh and Moreno2008, p. 673) addressed some methodological weaknesses inherent in the previous studies, particularly the lack of concurrent data, which would be required to “establish” rather than “assume” that L2 learners process for meaning while attending to form. In addition to circling specific forms, Leow et al. asked L2 learners to speak their thoughts aloud while reading, which allowed the researchers to assess the extent to which the learners were processing the forms consciously (i.e., depth of processing; operationalized as simply circling the target form vs. saying it out loud vs. interpreting or translating the target form) as well as whether the learners made efforts to process the meaning of the input.
In addition, Leow et al. (Reference Leow, Hsieh and Moreno2008) tried to improve upon the methodology of previous studies by using a different reading passage in which target forms were evenly distributed, and by controlling the target forms for length. They replaced inflación, which has an English cognate, with sol “sun,” which does not. Using monosyllabic sol also meant the target forms had similar levels of saliency in all experimental conditions. Leow et al. also added the clitic lo “it,” which carries grammatical information and meaning at the same time, as another target form. In this way, the four target forms vary in their semantic and grammatical properties. Another difference from previous studies was that Leow et al. used a multiple-choice test instead of written recall to test comprehension of the content. With these changes, the experimental groups in the study did not have significantly lower scores on the comprehension check questions than the control group, indicating that the L2 learners were able to pay attention to form and meaning simultaneously. The results were consistent with the results for written input in Wong’s (Reference Wong2001) study.
Although the sample size was relatively small (a total of 72 participants in five groups), Leow et al. (Reference Leow, Hsieh and Moreno2008) revealed a more precise picture of attention and depth of processing for the target forms while reading, providing the first evidence of how L2 learners juggle these concurrent task demands in real time. Using the data from the think-aloud protocols, the study found no evidence that deeper processing of form influenced the L2 learners’ processing for meaning negatively, as VanPatten’s primacy of meaning principle would predict. Considering most of the L2 learners in their study showed a minimal level of processing of the forms (by merely circling the form or circling and briefly mentioning the form out loud), Leow et al. (Reference Leow, Hsieh and Moreno2008) speculated that they did not find a trade-off between attention to form and attention to meaning due to the learners’ minimal processing depth. A trade-off might be observed more readily, then, when learners expend more effort processing forms while attending to the meaning of the text (Leow et al., Reference Leow, Hsieh and Moreno2008; VanPatten, Reference VanPatten1990). In terms of attention to text meaning, participants were considered to process meaning, and therefore included in the study, if they showed an effort to read the passage for meaning. Beyond this use of the meaning data as an inclusion criterion, further analyses of participants’ think-aloud protocols exclusively focused on the L2 learners’ conscious processing of the target forms and the degree (or depth) of processing. Concurrent data of learners’ processing for the meaning of the text or of the amount of processing for meaning while they are asked to attend to form are still needed.
Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012) built on the research by Leow et al. (Reference Leow, Hsieh and Moreno2008), using the same materials and procedures, but with a larger sample size. They also included a non-think-aloud group to examine the reactivity of the think-aloud protocols, which might compromise results about simultaneous attention to form and meaning. The results revealed that fewer participants in the think-aloud group met the criterion of attending to 60% of the target forms in the input compared to those who read the text silently; that is, they found a statistically significant association between the think-aloud method and participant attrition. Thinking aloud further reduced the comprehension of those who were included in the analysis, albeit with a small effect size (
$ {\eta}_p^2 $
= .01), which the authors interpreted as “minimal” (p. 659). At a theoretical level, the findings of this study and Leow et al. (Reference Leow, Hsieh and Moreno2008) were similar: L2 learners instructed to attend to form did not show weaker comprehension. The results from the think-aloud protocols indicated a positive correlation between depth of processing for form and comprehension scores; in other words, participants who processed the target forms more deeply (e.g., by translating them to English) had better text-comprehension scores.
Recently, Morgan-Short et al. (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) conducted a large-scale multisite replication study with a total of 704 L2 Spanish learners from seven different sites. They adopted the experimental materials used by Leow et al. (Reference Leow, Hsieh and Moreno2008), but used rapid serial visual presentation (RSVP), in which each word is presented sequentially. Unlike untimed paper-based reading, RSVP does not allow backtracking, which makes it more similar to listening (see also Kim & Godfroid, Reference Kim and Godfroid2019). The researchers compared the comprehension of the experimental groups who paid attention to the target forms (sol, la, and -n) to that of a control group in both aural and written modes. Participants at four sites (two schools in the United States, one in the United Kingdom, and one in Poland) were tested with the aural paradigm, and participants at three sites (all but the one in Poland) were tested with the written paradigm. The study found mixed results across the sites. In the aural mode, results from the two US schools did not show differences between the experimental and the control groups, whereas the results from the schools in Poland and the United Kingdom showed that attending to -n negatively affected comprehension scores compared to the control. In the written mode, the participants in the experimental groups had similar comprehension scores compared to the control, except for the participants at one of the US sites, where the group that attended to sol had lower comprehension scores. Although the findings were not identical across the sites, the authors argued that their overall findings were consistent with those of Leow et al. (Reference Leow, Hsieh and Moreno2008) and Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012), and that they gained validity from the use of multiple sites and different approaches to analyzing the data (see Appendix S1 in the supplementary material).
In all, this series of interconnected studies also coincided with methodological developments in the measurement of attention: circling or underlining the target forms (Greenslade et al., Reference Greenslade, Bouden and Sanz1999; VanPatten, Reference VanPatten1990; Wong, Reference Wong2001), circling the target forms + think-aloud protocols (Leow et al., Reference Leow, Hsieh and Moreno2008; Morgan-Short et al., Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012), and clicking the forms during RSVP (Morgan-Short et al., Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018). Each of these newer methods of assessing learners’ attention to target forms addressed a weakness of previous studies. In what follows, we introduce eye-tracking, or eye-movement registration, as yet another method to assess attention.
EYE-TRACKING IN ATTENTION AND READING RESEARCH
Eye Movements and Attention
Eye movements are an overt response that signals the orienting of attention; humans align their eye gaze with their attention to specific objects (Godfroid, Reference Godfroid2020; Tomlin & Villa, Reference Tomlin and Villa1994; Wright & Ward, Reference Wright, Ward, Nobre and Kasner2008). Because of this “eye–mind link” (Holmqvist et al., Reference Holmqvist, Nyström, Andersson, Dewhurst, Jarodzka and Van de Weijer2011; Just & Carpenter, Reference Just and Carpenter1980; Reichle et al., Reference Reichle, Pollatsek and Rayner2006), eye movements have been examined as reliable indicators of participants’ attention in SLA (e.g., Godfroid et al., Reference Godfroid, Boers and Housen2013; Indrarathne & Kormos, Reference Indrarathne and Kormos2017; Issa & Morgan-Short, Reference Issa and Morgan-Short2019; Koval, Reference Koval2019).
As discussed, previous empirical studies of simultaneous attention have asked participants to mark their noticing of a particular form; this method can check the extent to which they complete the task’s requirements. Although VanPatten (Reference VanPatten1990) argued that a learner’s completion of the task indicates his/her conscious processing for linguistic form, indeed, the locus and the amount of his/her attention must be assumed based on his/her marking (see also Godfroid et al., Reference Godfroid, Boers and Housen2013; Leow et al., Reference Leow, Hsieh and Moreno2008). The eye-tracking paradigm provides this precise information, as eye movement reflects not only participants’ concurrent attentional distribution (spatial information) but also the amount of their attention (temporal information) (Godfroid, Reference Godfroid and Leow2019; Radach & Kennedy, Reference Radach and Kennedy2004). To elaborate, during complex tasks such as reading, a fixation on a specific part of the linguistic input generally signals orienting of attention to that part, and the fixation is sustained for a certain duration in which visual and cognitive processing occur. Hence, other things being equal, the longer the fixation, the more deeply the input is processed. Eye movements thus provide a quantitative measure of attention. This measure (as reflected in eye fixation durations on a linguistic form in the input) will increase as processing time increases. Hence, there is evidence for depth of processing in the eye-movement record (Godfroid, Reference Godfroid and Leow2019).Footnote 1
Eye movements do not provide direct evidence about the nature of processing that takes place during the fixation (e.g., whether a participant attempted to translate the form); in other words, eye movements do not distinguish processing for form and processing for linguistic features within the form. Verbal data do provide insight into the nature of processing, as long as the processing is conscious (participants cannot verbalize that of which they are not conscious). As far as we can tell, it is an open question whether the processing of linguistic features encoded in spoken or written forms must happen consciously for form-meaning mapping to occur. Hence, we view eye-movement data and verbal data as complementary in terms of their strengths, affordances, and the information that they provide (also see Godfroid et al., Reference Godfroid, Winke and Conklin2020). Here, we demonstrate what eye-tracking can contribute to research on simultaneous attention.
Eye Movements and Comprehension
There is a long tradition of eye-tracking in reading research that examines readers’ moment-to-moment comprehension processes as reflected in their eye-movement records (Fletcher, Reference Fletcher2006; Rayner, Reference Rayner1997, Reference Rayner1998, Reference Rayner2009; Rayner et al., Reference Rayner, Chace, Slattery and Ashby2006). While reading, readers may pause (i.e., make an eye fixation) to identify low-frequency vocabulary or parse complex syntactic structures; in addition, they may also go back in the text (i.e., regress) to reread something when they require more time to process the information (e.g., Foster et al., Reference Foster, Ardoin and Binder2018). Thus, compared to overall reading times, eye-movement data can reveal much more of what readers do when they read for comprehension purposes (e.g., Cook & Myers, Reference Cook and Myers2004; Garrod et al., Reference Garrod, O’Brien, Morris and Rayner1990).
As an exemplar study to explore readers’ global understanding reflected in eye movements, Rayner et al. (Reference Rayner, Chace, Slattery and Ashby2006) tracked participants’ eye movements while they were reading passages. After they finished their reading, the participants were instructed to rate the difficulty of the passages on a 10-point scale and answer multiple-choice comprehension questions. The eye-movement data (i.e., mean fixation duration, mean number of fixations, and total reading time) were highly correlated with the participants’ ratings of difficulty. In addition, readers had longer and more fixations and longer overall reading times for more difficult passages, and these were negatively correlated with their comprehension accuracy scores; that is, the participants who had limited comprehension of passages required longer fixations, more fixations, and longer reading times.
In addition, different aspects of reading processes have been explained by eye-movement data (e.g., Cop et al., Reference Cop, Drieghe and Duyck2015; Titone et al., Reference Titone, Libben, Mercier, Whitford and Pivneva2011). For instance, in a study on the interaction of global reading processes and word-level processing, Taylor and Perfetti (Reference Taylor and Perfetti2016) examined 35 native English speakers’ reading behaviors, focusing on the effects of individual differences in reading. In one of their two experiments, they demonstrated how individuals’ reading experience and lexical factors influence reading comprehension. For example, careless readers with extensive reading experience tended to read the text quickly at the expense of accuracy. Recently, Jung and Révész (Reference Jung and Révész2018) investigated 38 Korean EFL learners’ English reading processes with eye movements and stimulated recalls. They manipulated the characteristics of L2 reading activities to create a more careful and a less careful reading condition (operationalized by the number of subsections the participants had to place in the correct order in the text). They then assessed both global text reading processes and participants’ processing of the target feature (English unaccusative form). The eye-movement data provided evidence that different reading activities elicited different reading processes at both global and local levels. The participants in the more careful reading condition showed longer eye fixations on the text, made more forward saccades, and made more regressive eye movements. In addition, the more careful reading condition seemed to trigger more attention to and awareness of the unaccusative verbs than the less careful reading condition. Taken together, this demonstrated that the characteristics of reading activities seem to affect reading behaviors, which in turn may affect the L2 readers’ cognitive learning processes. In this study, we will examine the effect of attentional conditions on L2 reading comprehension, not by manipulating the text presentation features as in Jung and Révész’s (Reference Jung and Révész2018) study but manipulating the orienting of attention by means of task instructions. The overarching aim, then, is to examine whether and to what extent different attentional conditions affect how the L2 learners attend to form versus meaning in a reading activity.
THE CURRENT STUDY
While the debates over simultaneous attention to L2 form and meaning continue, there is, as pointed out, a paucity of concurrent data on how learners attend to and process input. Earlier methods of probing attention (e.g., having learners mark a particular form) did not sufficiently capture the role of attention; we still know little about whether and how the amount of attention might hamper or facilitate the processing of L2 input, particularly online processing while reading (see also Godfroid et al., Reference Godfroid, Boers and Housen2013; Leow et al., Reference Leow, Hsieh and Moreno2008).
Leow et al. (Reference Leow, Hsieh and Moreno2008) was the first study that employed think-aloud protocols in an attempt to explore how participants in experimental groups reacted to task manipulations and whether deeper processing of form had adverse impacts on the comprehension of meaning. However, think-aloud protocols may be “too general to provide a full account of attention paid to discrete items in the data” (Leow et al., Reference Leow, Grey, Marijuan and Moorman2014, p. 115) and their possible reactivity makes this method controversial in attention research. Eye-movement registration provides an alternative to capturing attention that can record participants’ visual attention with high spatial and temporal resolution (Godfroid, Reference Godfroid and Leow2019; Godfroid et al., Reference Godfroid, Boers and Housen2013) and reflects both conscious and unconscious processing (Godfroid et al., Reference Godfroid, Winke and Conklin2020). Furthermore, eye-movement methodology is exquisitely suited to capture L2 learners’ online processing during reading, which may add further insight into the participants’ processing of the meaning of the text.
To expand and enhance previous research on simultaneous attention to form and meaning, this study uses eye-tracking, which allows us to infer participants’ processing of the target forms and the text through their eye movements and fixation points, based on the established link between eye movements and attention (Godfroid, Reference Godfroid2020; Wright & Ward, Reference Wright, Ward, Nobre and Kasner2008). Eye-tracking allows for measuring attention to form and depth of processing of the form (Godfroid et al., Reference Godfroid, Boers and Housen2013; Godfroid, Reference Godfroid and Leow2019), in addition to capturing learners’ online processing of the meaning of the text. The methodology therefore sheds light on different aspects of attention, according to type of target form and instruction to pay attention to meaning versus meaning and form.
RESEARCH QUESTIONS
-
1. To what extent do Spanish L2 learners attend to form and meaning of written input simultaneously?
-
2. Does Spanish L2 learners’ comprehension differ depending on how they allocate their attention?
-
3. To what extent do L2 learners’ general reading behaviors predict the L2 learners’ comprehension scores?
METHOD
PARTICIPANTS
The participants were 84 L2 Spanish students enrolled in a third-semester Spanish course at a midwestern university in the United States from the end of 2016 to early 2017 (age: M = 18.91, SD = 1.26). All the participants spoke English as a native language. They were randomly assigned to one of five groups: four experimental groups, which were instructed to pay attention to different target forms in a text as they read the text for comprehension, and one control group, which read the same text for comprehension only. Although the L2 Spanish students in Leow et al. (Reference Leow, Hsieh and Moreno2008) were in their second semester, this study’s participants were in the third semester, because the second-semester students at the site of our study had not yet learned the Spanish preterite and imperfect verb forms used in the reading passages (Morgan-Short et al., Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012), and they had low comprehension scores in a pilot study. As in previous studies, the participants had prior knowledge of the target forms as a result of classroom instruction. The descriptive statistics of the EIT scores are shown in Table 1. The EIT demonstrated that the learners in the five groups were comparable in terms of proficiency (F(4,70) = .21, p = .93,
$ {\eta}^2=.01 $
).
Descriptive statistics of the EIT scores
MATERIALS
Reading Passages
The main reading passage used in the experiment was the one developed by Leow et al. (Reference Leow, Hsieh and Moreno2008). The passage had 354 words, and it was a modified version of an authentic online article related to the Aztecs. The passage included four target forms: one lexical item (sol), the clitic pronoun (lo), and two grammatical items (la, -n). Each form was embedded within the passage 10 times and distributed evenly across the passage. As previously discussed, the lexical item sol and the other target items were all monosyllabic to control for saliency. The four target forms vary in their semantic and grammatical features. Unlike la and -n, lo “it” is difficult to characterize as either a lexical item or a grammatical item, as it “carries both meaning and grammatical function” (p. 674). As such, the four target forms spanned the range from grammar to lexicon.
Greenslade et al.’s (Reference Greenslade, Bouden and Sanz1999) reading passage, which is a written version of VanPatten’s (Reference VanPatten1990) aural passage, was selected as a practice passage to help participants become familiar with the experimental procedure and the task of paying attention to a designated target form. This reading passage contained one lexical item (inflación) and two grammatical items (la, -n), which were evenly distributed throughout the passage, as in the main reading passage. Participants received the same instructions as for the main reading passage but practiced with a different target form.
Comprehension Questions
Leow’s (Reference Leow, Hsieh and Moreno2008) comprehension questions were utilized to check participants’ understanding of the reading passage. The 10 multiple-choice questions were in English, the participants’ L1, and the main focus of the questions was the content of the passage rather than specific grammatical items in the passage. The current study found the reliability of the test to be low (Cronbach’s α = .293), at a similar level as that reported by Morgan-Short et al. (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) for the same test (α = .197).
Elicited Imitation Test
To assess and control for the students’ general Spanish proficiency, a Spanish EIT was administered (Bowden, Reference Bowden2016).Footnote 2 This test was composed of 30 Spanish sentence items. Each item required the test-taker to listen to a sentence, wait 2 seconds, and repeat the sentence. The learners’ oral responses were recorded. One Spanish native speaker and one Spanish heritage speaker rated all speech samples by scoring each item on a scale of 0–4 with 120 points in total. First, the two raters rated a small portion of the EIT data (N = 7) together while discussing the rubric and disagreements. Then, the raters rated all the remaining EIT data individually (exact agreement rate: 78%, adjacent agreement rate: 99%, total score interrater reliability: r = .96). The two raters discussed and reached consensus on cases of discrepancies of 2 or more points. Then, another Spanish native speaker as a third rater was invited to resolve the adjacent disagreements (discrepancies of 1 point) to determine the final scores for the responses that were not in exact agreement.
PROCEDURES
Individual participants completed the experiment in the Michigan State University Second Language Studies eye-tracking lab with the help of a researcher. An Eyelink 1000 (SR Research, Ltd.) was used to track participants’ eye movements while they read the passages on a computer monitor. The eye-tracker required the participants to place their forehead and chin on a desk-mounted rest while reading the passage. The height of the chair was adjusted for each participant before the experiment. The distance between the computer monitor and the eye-tracker was approximately 65 cm to optimize the accuracy of the eye-movement data (Conklin et al., Reference Conklin, Pellicer-Sánchez and Carrol2018; Godfroid, Reference Godfroid2020). The eye-tracker was calibrated using a 9-point fixation point procedure.
All participants were instructed that they would read a short passage on the computer monitor and then answer some comprehension questions to check their understanding of the content. They were also informed that the reading would be two or three pages long so they would need to press a designated button in a response pad (i.e., a green button) to move to the next page. The participants in the experimental groups were given further instructions. They were told that whenever they saw their assigned form (i.e., either sol, lo, la, or -n in the main passage; either inflación, la, or -n in the practice passage), they should press a different designated button in the response pad (i.e., a blue button) and look at the form, keeping their eyes on it for 2 seconds. There was no timer or automatic system to force participants to move to the next lexical item; the participants in the experimental groups gauged the 2 seconds themselves. We included this additional instruction to preempt participants’ question of how long they should look at the form. Recall that the participants were participating in an eye-tracking experiment and so they knew that everything they looked at was being recorded. The 2 seconds were intended to provide them with a time during which they would process the targeted forms. The target forms in the practice passage were assigned to each participant so as to avoid any effects of repetition on the results. For instance, participants in the la group were asked to pay attention to either inflación or -n, but not la, in the practice session.
The participants first read the practice passage to get used to the experimental setting. Then, the eye-tracker was recalibrated, and the participants read the main passage. After reading the passage, the 10 multiple-choice comprehension questions were shown on the monitor, and the participants chose their answers by using the left mouse button. They then took the Spanish EIT. The entire experiment took about 30 minutes. The participants were given $10 as compensation after the experiment.
DATA ANALYSIS
In the analysis, nine participants’ data were excluded due to unsuccessful calibration (N = 3) or lack of attention to the assigned form (N = 6). Following the procedures of Leow et al. (Reference Leow, Hsieh and Moreno2008) and Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012), we analyzed only the data from participants who paid attention to their assigned form at least six times (out of 10), based on the number of button presses. Individuals’ scores on the comprehension questions were calculated by counting the number of accurate answers. Thus, 75 participants’ data (15 participants per group) were used for the analysis.
RESULTS
We operationalized attention to form as eye fixations on the assigned forms, as well as button presses in response to seeing the form. We refer to this as local processing. Global processing, however, refers to participants’ general reading behavior of the text. We operationalize global text processing in terms of participants’ eye movements throughout the entire reading paragraph.
SIMULTANEOUS ATTENTION TO FORM AND MEANING (RQ 1)
First, the button press data confirmed that the experimental groups paid attention to target forms as instructed (see Table 2 and Table 3). Depending on type of form, button press accuracy differed (e.g., la group: 82%; sol group: 96%). In a generalized linear mixed model, the dependent variable was accuracy of button press (correct, incorrect), with group (la, lo, -n, sol) as a fixed effect, participants and item as random intercepts, and standardized scores on the EIT as a covariate. The sol group, which had the highest accuracy (96%), served as the reference group.Footnote 3 The sol group had significantly higher button press accuracy than the la group (β = –1.80, SE = 0.71, z = –2.53, p = .01) and the -n group (β = –1.54, SE = 0.71, z = –2.16, p = .03), but not than the lo group (β = –1.03, SE = 0.73, z = –1.40, p = .16). The lo group could not be distinguished statistically from the la group (β = –.76, SE = 0.61, z = –1.26, p = .21) or the -n group (β = –.51, SE = 0.62, z = –.81, p = .42) in terms of their button press accuracy. More proficient learners missed (i.e., failed to press the button for) more target forms than less proficient learners during reading (β = –.30, SE = .15, z = –1.96, p = .05).
Descriptive statistics for accuracy of button press
Results of generalized mixed model for accuracy of button press
Next, the experimental groups’ total fixation duration on target items was compared to the control group’s to examine the extent to which participants attended to form while also processing the meaning of the text. For this analysis, only target items with correct responses (button presses) were included. The total fixation duration on target forms of the experimental groups as a whole (M = 1803 ms, SD = 643) differed from that of the control group (M = 269 ms, SD = 280) (F(1,73) = 84.39, p < .001; see Table 4 and Figure 1). Moreover, although the experimental groups were given identical instructions for attention to forms, each group had a different total fixation duration on its assigned form (e.g., the la group: M = 2107 ms, SD = 967 vs. the -n group: M = 1479 ms, SD = 788) (see Table 4). In a linear mixed model, the dependent variable was total fixation duration on each target form, with group (la, lo, -n, sol, control) as a fixed effect, participants and item as random intercepts, and standardized scores on EIT as a covariate. The control group served as the reference group and is represented by the intercept in Table 5. The experimental groups fixated on the assigned forms significantly longer than did the control group (β = 269.79, SE = 129.46, t = 2.08, p = .04). Participants’ Spanish proficiency did not significantly affect their fixation duration on target forms (β = –34.85, SE = 60.56, t = –.58, p = .57). In addition, follow-up analyses indicated that the -n group’s fixation duration was significantly lower than the sol group’s (β = 541.53, SE = 193.35, t = 2.80, p < .001) and the la group’s (β = 741.94, SE = 193.48, t = 3.84, p <.001), but not than the lo group (β = 233.63, SE = 193.14, t = 1.21, p = .23). Fixation durations on sol and la could not be distinguished statistically (β = –200.42, SE = 193.42, t = –1.04, p = .30).
Descriptive statistics of local processing of assigned target forms
Attention to designated target forms in the four experimental groups and the control group.
Results of linear mixed model for local processing of assigned target forms
Taken together, the experimental groups paid more attention to the target forms than did the control group while reading the passage. Moreover, button press accuracy and fixation duration on target forms differed for the experimental groups in a pairwise fashion, suggesting that the sol and la groups allocated more attention to their assigned forms than the lo and -n groups. The participants’ EIT scores impacted their attention to form only on the offline decision. In particular, the participants who had lower EIT scores showed greater button press accuracy; however, their fixation duration on the assigned target forms was not influenced by their EIT scores.
To obtain a more general picture of participants’ meaning-focused processing, we also analyzed how participants in the different groups read the entire text. We performed the textual analysis at the lexical level, for all the words in the text (minus the target forms), and at the global level, for the entire text passage (minus the target forms). Together, the various word- and text-level reading metrics (total fixation duration, number of fixations, and numbers of regressions on words; total text reading time) paint a picture of participants’ reading behavior throughout the entire passage (see Table 6 and Figure 2).
Descriptive statistics of global text processing: Lexical level (ms) and text level (sec)
Global text processing by experimental group: total fixation duration, number of fixations, number of regressions/word in the text, and total text reading time.
Although the experimental groups were instructed to pay attention to assigned forms, the task instruction did not lead to significant differences in their overall reading of the text, compared to the control group (e.g., the la group’s total fixation duration: M = 453 ms, SD = 422 vs. the control group’s: M = 455 ms, SD = 518). The results obtained from a series of linear and generalized linear mixed models for total fixation duration, number of fixations, and number of regressions and a linear regression analysis for total text reading time showed no group differences and no effects of proficiency on the text-level eye-movement data (see Table 7). This finding indicated that all of the groups had similar reading patterns for the whole text, including the groups that had been instructed to attend to form. In other words, the instruction for the experimental groups did not affect their global reading processes.Footnote 4 In addition, the participants’ scores on EIT did not predict their global processing for meaning, possibly due to a high degree of variability inherent in the present sample of intermediate learners’ reading behavior.
Statistical results for global text processing as a function of experimental condition
TEXT COMPREHENSION (RQ 2)
Overall, participants in the control group (M = 5.07, SD = 1.71) showed slightly higher scores on the comprehension test than the experimental groups (M = 3.95, SD = 1.31) (see Table 8 and Figure 3). In particular, the la and -n groups received lower scores. To determine the extent of these differences after adjusting for the participants’ scores on EIT, an ANCOVA was performed, with group as an independent variable, scores on the comprehension test as a dependent variable, and scores on EIT as a covariate. Following Morgan-Short et al.’s (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) procedure, we used Cohen’s d to calculate the effect size. An effect size calculator was utilized to find Cohen’s d (Wilson, Reference Wilson2001). The results indicated that the difference in comprehension scores between the groups approached statistical significance with a medium effect size (F(5, 69) = 2.56, p = .054, d = .80, 95% CI for d = .22, 1.38), whereas the EIT scores did not affect the comprehension scores (F(1, 69) = 2.42, p = .12, d = .05, 95% CI for d = –.50, .08). Tukey post-hoc analysis was performed to compare each group’s comprehension score. Compared to the control group, the la group (t = –2.69, p = .067, d = .69, 95% CI for d = –.04, 1.43) and the -n group (t = –2.82, p = .047, d = .75, 95% CI for d = .02, 1.50) had lower comprehension scores with statistical significance and a medium effect size. No statistically significant difference was found between the control group and the lo group (t = –1.57, p = .52, d = .23, 95% CI for d = –.48, .95) or the sol group (t = –1.76, p = .41, d = .30, 95% CI for d = –.41, 1.02).
Descriptive statistics of scores on comprehension test
Comprehension test scores across the groups.
Compared to the meta analytic effect sizes reported in Morgan-Short et al. (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018), the effect sizes in this study were similar for sol (Morgan-Short et al.: d = –.34, 95% CI for d = –1.12, .44; this study: d = .30, 95% CI for d = –.41, 1.02) and larger for la (Morgan-Short et al.: d = –.28, 95% CI for d = –.81, .25; this study: d = .69, 95% CI for d = –.04, 1.43) and -n (Morgan-Short et al.: d = –.13, 95% CI for d = –.80, .53; this study: d = .75, 95% CI for d = .02, 1.50) Morgan-Short et al. (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) did not include lo as a target form; however, a comparison with Leow et al. (Reference Leow, Hsieh and Moreno2008) and Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012) revealed similar effect sizes for lo in Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012) and this study (Morgan-Short et al.: d = .22, 95% CI for d = –.33, .77 in think-aloud group and d = .17, 95% CI for d = –.27, .61 in non-think-aloud groups; this study: d = .23, 95% CI for d = –.48, .95) and a somewhat lower effect size in Leow et al. (Reference Leow, Hsieh and Moreno2008) (d = –.01, 95% CI for d = –.69, .67). Overall, as the effect sizes show, simultaneous attention to form had a larger impact on text comprehension in this study than in previous research, especially in the groups that attended to la and -n, respectively.
TEXT COMPREHENSION, TASK INSTRUCTIONS, AND GENERAL READING BEHAVIOR (RQ 3)
Lastly, we explored whether the participants’ overall text processing predicted their comprehension scores above and beyond their group assignment. Eye-movement data averaged for each word in the text (i.e., total fixation duration, number of fixations, number of regressions) and total text reading time were entered as a predictor, along with experimental condition and scores on EIT, in a series of four different linear regressions (see Table 9).Footnote 5
Results of a series of linear regressions for text comprehension as a function of global text
The results for the four linear regressions were very similar: total fixation duration/word in the text (adjusted R 2 = .18, F(6, 68) = 3.74, p = .003), number of fixations/word in the text (adjusted R 2 = .13, F(6, 68) = 2.84, p = .016), number of regressions/word in the text (adjusted R 2= .12, F(6, 68) = 2.61, p = .024), and total text reading time (adjusted R 2 = .19, F(6, 68) = 3.83, p = .002) each significantly predicted participants’ comprehension scores (see Table 9 and Figure 4). Slower, more careful readers manifested statistically better text comprehension regardless of their attentional condition (i.e., regardless of whether they were in an experimental group or the control group) and Spanish proficiency level (as a covariate). The results also strengthened earlier findings (RQ2) that attention to form impacted the participants’ text-comprehension scores negatively. Specifically, in this more comprehensive model, the la group and -n group now displayed significantly lower text comprehension than the meaning-focused control group. A similar trend toward lower comprehension was observed for participants in the lo and the sol group in three of the four regression models. In sum, the current analyses supported a joint role for individual reading patterns and attention to form versus meaning (as imposed externally, by task instructions) in explaining a participant’s text comprehension.
Relationship of global text-processing measures and comprehension.
DISCUSSION
The purpose of this study was to delve into the long-standing debate on L2 simultaneous attention to form and meaning by providing concurrent evidence of attention with the use of the eye-tracking paradigm. We investigated whether L2 Spanish learners can pay attention to the form and the meaning of L2 written input simultaneously, and the extent to which attention to form affects the L2 learners’ text comprehension and reading behavior. To this end, we compared learners who attended to particular forms while processing the meaning of the input to those who only attended to meaning. These groups were compared in two ways: (a) by their scores on a comprehension test and (b) by their eye movements on selected target forms and throughout the entire text. Fine-grained analyses of the eye-movement data showed that intermediate L2 Spanish learners can attend to designated target forms during reading and that this will not alter their global text reading behavior (beyond these target forms) in any detectable way. However, the instruction to attend to form and meaning simultaneously had a negative impact on the L2 readers’ overall text comprehension, which demonstrated a trade-off effect between attention to form and attention to the meaning of the text. Individual differences in reading behavior could compensate for this form-meaning trade-off. Slower, more careful readers displayed better text comprehension than faster readers regardless of whether they attended to form and meaning simultaneously or to meaning only.
SIMULTANEOUS ATTENTION TO FORM AND MEANING DURING REAL-TIME PROCESSING (RQ 1)
The experimental groups pressed the designated button and fixated on the assigned forms, as they were instructed to do. Depending on the type of target form, the different groups showed different button press accuracy, that is, sol was attended more accurately than la and -n. The experimental groups spent significantly longer times looking at their target forms than the control group did at the same forms. Moreover, by comparing fixation durations between the participant groups, we were able to establish that depth of processing for form differed depending on the type of target form. In particular, different groups allocated different amounts of attention to the assigned forms: sol and la appeared to have benefited from more attention than lo and -n.
The participants’ Spanish proficiency measured by EIT affected their attention to form to some extent. Less proficient learners showed greater button press accuracy than more proficient learners. One possible explanation for this phenomenon is that the more proficient participants possessed more automatized reading skills (DeKeyser, Reference DeKeyser1997; Segalowitz & Segalowitz, Reference Segalowitz and Segalowitz1993). The requirement to focus on a linguistic form would therefore have conflicted with their automatic reading behavior and may have caused them to miss some of the nonsalient forms because they were processing the forms more automatically. Compared to the less proficient learners, it was more difficult for the more proficient learners to interrupt their more automatic reading processes to make a conscious decision and press the button. The eye-movement data, however, do not have a conscious decision component and consequently more and less proficient learners could not be distinguished based on their eye-movement behaviors.
The researchers who initially replicated VanPatten’s Reference VanPatten1990 study attempted to measure their participants’ conscious attention to specific forms by requiring participants to mark the forms (e.g., circling or underlining them) while either listening to or reading L2 input. Marking the target form to measure online processing for linguistic information has been the default approach since VanPatten (Reference VanPatten1990) (also see Wong, Reference Wong2001). Although this approach may reflect participants’ attention to the target form, merely counting the number of markings without concurrent evidence of attentional allocation provides limited information on the amount of attention or depth of processing that ensued (also see Leow et al., Reference Leow, Hsieh and Moreno2008; Morgan-Short et al., Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012). In this study, we were able to measure both the locus of attention and the amount of attention or depth of processing by combining button presses with eye-movement registration. We found that the experimental groups did indeed attend to their assigned form consciously and paid more attention to these target forms than the control group, which was instructed to focus on meaning.
Within the experimental groups, although they were given identical instructions to attend to the form, the extent to which they processed the assigned form differed depending on the type of the form. The findings concur with previous studies (Jung & Révész, Reference Jung and Révész2018; Taylor & Perfetti, Reference Taylor and Perfetti2016), suggesting that different experimental conditions or different types of target form lead to differences in participants’ attentional allocation. Deeper processing of sol, compared to lo or -n, may be due to the lexical status of sol, the fact that it is a free morpheme, and the longest of all the target forms (longer words tend to attract longer eye fixations during reading). Conversely, as a short, bound, and purely grammatical morpheme, -n may be difficult to attend to overtly, even when readers are instructed to do so. It is less obvious why la and lo, which are both free, two-letter morphemes, differed significantly in total fixation duration, with la processed more deeply than lo. Given the more complex grammatical and semantic meaning of lo, and the fact that lo occurs in an unexpected syntactic position (preverbally), one might have predicted the opposite pattern of results (i.e., longer fixations on lo than la). One possibility, which would need to be explored in future research, is that the current intermediate-level L2 Spanish participants had not fully acquired the complex meaning of lo yet and hence processed its form-meaning connection more superficially. The addition of lo as a target form, first introduced in Leow et al. (Reference Leow, Hsieh and Moreno2008), has proven a valuable expansion of the current paradigm to disentangle the different factors at play in simultaneous attention.
An innovative feature of this eye-tracking study is that we combined both local and global analyses of eye-tracking data. Controlling for participants’ Spanish proficiency, we analyzed the eye-movement data reflecting global processing—total fixation duration, total number of fixations, and total number of regressions on every single word in the text (minus the target forms), and total text reading time—in an attempt to capture L2 readers’ general text-comprehension processes (e.g., Cop et al., Reference Cop, Drieghe and Duyck2015; Rayner et al., Reference Rayner, Chace, Slattery and Ashby2006; Titone et al., Reference Titone, Libben, Mercier, Whitford and Pivneva2011). Previous research has shown that if readers encounter difficulty in reading a passage, they will make more fixations and regressions while reading. In this regard, the eye-movement data from the control group may be considered to show the “default” manner in which L2 Spanish learners in the third semester read to comprehend content, whereas the experimental groups’ eye movements lend themselves to examining potential differences in global text processing due to their additional task to attend to form while reading. In spite of the additional tasks, however, the experimental groups’ eye-movement data on words other than the target items in the text showed no significant differences from the control group. At a global level, experimental participants had similar reading patterns as the control group, even though they were given an additional task. Any effects of simultaneous attention to form and meaning, therefore, were highly localized on the specific target forms in the text, and apparently did not add to the cognitive load of reading the entire text and did not manifest themselves in participants’ online reading behavior beyond the reading of these target words. In this way, the present study adduced the first evidence that it is possible for L2 learners to attend simultaneously to form and meaning during real-time processing, as recorded by an eye-tracking camera filming participants’ eye movements.
TRADE-OFFS BETWEEN ATTENTION TO FORM AND (OFFLINE) TEXT COMPREHENSION (RQ 2)
Different from other recent studies with written materials (Leow et al., Reference Leow, Hsieh and Moreno2008; Morgan-Short et al., Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012; Morgan-Short et al., Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018), we found that simultaneous attention to form and meaning came at a cost of reduced text comprehension on the comprehension posttest. In other words, there was a trade-off between attention to form and attention to meaning, particularly in the la and -n groups. We used Cohen’s d effect sizes to quantify group differences and compare the effect sizes to those in previous studies. Similar to Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012, Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018), we obtained small effect sizes for sol and lo. Participants who attended to the lexical and lexico-grammatical forms (i.e., sol and lo) during reading incurred a small comprehension cost compared to those who read the text for comprehension only. Interestingly, for the purely grammatical forms la and -n, the comprehension costs reached a large effect size and exceeded the meta analytic effect sizes reported by Morgan-Short and her colleagues (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018). For instance, taking comprehension in the la group as an example, the comprehension deficit was (d = .66) in this study and exceeded the comprehension deficit reported by Morgan-Short and colleagues (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) by about one third of a standard deviation (d = –.28).
Using different materials and a different comprehension test, Greenslade et al. (Reference Greenslade, Bouden and Sanz1999) also observed that attending to grammatical forms (i.e., la, -n) negatively affected comprehension in the written modality. Wong (Reference Wong2001), also with a different test and different materials, found that the control group showed higher scores in comprehension than the experimental groups in the written modality, although the difference was not statistically significant. The present finding is also similar to that of one of the sites (US timed written Site 2) in Morgan-Short et al.’s (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) multisite study, who also showed that attending to form negatively impacted comprehension. Participants in Morgan-Short et al. (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) read the same text from a computer screen, as in the present study, albeit one word at a time, in RSVP. We will return to the potentially important role of the reading medium in the next paragraph. Together, these studies, along with the current study, seem to support the existence of a trade-off in the la and -n groups. The fact that these effects held true after we controlled for participants’ Spanish proficiency strengthens the case for a trade-off between attention to form and text comprehension.
The participants in this study read the text from a computer screen (to allow for the recording of their eye movements) rather than paper, as they did in Leow et al. (Reference Leow, Hsieh and Moreno2008) and Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012). Research suggests that reading from a computer screen yields lower comprehension than reading on paper (e.g., Delgado et al., Reference Delgado, Vargas, Ackerman and Salmerón2018; Singer & Alexander, Reference Singer and Alexander2017). It is possible that the additional instruction to attend to form interacted with the reading medium: those who read digitally experienced a greater cognitive load, even in the control condition, than previous participants in paper-based studies and, hence, the additional instruction to attend to form while reading may have compounded the load and impacted their comprehension negatively. L2 speakers who are not proficient in a language yet, such as the current intermediate-level learners, would seem particularly susceptible to additional load effects (Singer & Alexander, Reference Singer and Alexander2017). This issue of the reading medium, together with questions of task instruction and concurrent data collection method, raise interesting questions that can be addressed in further research studies.
The current study’s methodological difference from previous studies such as Leow et al.’s (Reference Leow, Hsieh and Moreno2008) could be another factor for why this study observed a trade-off between attention to form and attention to meaning, where many of the recent studies did not. Leow et al. (Reference Leow, Hsieh and Moreno2008) suggested that a trade-off between attention to form and meaning is most likely when L2 learners process the linguistic forms deeply (also see VanPatten, Reference VanPatten1990). Although we did not design our experiment with this goal in mind (i.e., to increase depth of processing), it could be that the particular instruction to attend to forms for 2 seconds enhanced the participants’ depth of processing of the form and this, in turn, resulted in lower text comprehension. This is an interesting possibility. The jury is still out on the role of depth of processing in meaning comprehension, yet the results of this study, together with Morgan-Short et al.’s (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012), indicate that deep processing might in fact help, rather than hinder, comprehension. In this light, better comprehension in the experimental groups might have been the predicted pattern of results, yet we found that the experimental groups comprehended the text less well. A simple account of the present results is that VanPatten’s primacy of meaning principle holds true, also in the written modality, but it has not been observed consistently in recent studies due to the particular instrument that was used to assess comprehension, including in this study. Further research, using a more reliable assessment of comprehension that does not attenuate potential effects of simultaneous attention, is warranted to advance our understanding of this issue (Morgan-Short et al., Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018).
In sum, future research will be necessary to unveil the source of these diverging findings in recent work and the extent to which methodological differences in the measurement of attention played a role. Initial results from the current study suggest that the use of a nonintrusive, concurrent measure of attention such as eye-tracking may lend itself well to the study of simultaneous attention to form and text meaning and may reveal larger deficits in comprehension than have been assumed in recent years.
TEXT COMPREHENSION, TASK INSTRUCTIONS, AND GENERAL READING BEHAVIOR (RQ 3)
In addition to task instructions, individual differences in global text processing (as revealed by participants’ eye-movement data) also predicted the participants’ text-comprehension scores on the posttest. Longer and more fixations on each lexical item, longer reading times, and a greater number of regressions on each lexical item in the reading passage (excluding the target forms) predicted higher scores on the comprehension questions. Slower, more careful L2 readers exhibited better text comprehension, and this result held true after we accounted for participants’ Spanish proficiency level. A slower, more careful reading style may thus compensate for the requirement to attend to form and meaning simultaneously.
These findings may be aligned with Morgan-Short et al.’s (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012) finding of a positive correlation between participants’ level of processing and comprehension scores. In Morgan-Short et al.’s study, experimental groups who showed deeper processing of target forms, as evidenced by a metalinguistic comment or a translation, also had higher scores on the comprehension test. This result raises the possibility that depth of processing for form coincides with a deeper processing of the meaning of the text. The fact that the comprehension questions were carefully designed to address the part of the text in which the target items were located may have helped to bring this relationship to light.Footnote 6 One of the contributions of the present eye-movement study was that it afforded a detailed view into participants’ real-time processing of the meaning of the text.
Because we controlled for L2 proficiency effects, differences in eye-movement metrics likely reflect differences in depth of processing (Godfroid, Reference Godfroid and Leow2019), rather than reading difficulties attributable to varying levels of L2 proficiency. In this regard, our results align with those of Taylor and Perfetti (Reference Taylor and Perfetti2016) for accuracy as well as with Jung and Révész’s (Reference Jung and Révész2018) results. Taylor and Perfetti demonstrated that a focus on accuracy was one of the factors in reading in their study; participants who prioritized accuracy in test performance had longer total viewing times, with more refixating and rereading. In addition, Jung and Révész (Reference Jung and Révész2018) showed that L2 readers’ sum of fixation durations increased more in the more careful reading condition than the less careful reading condition. Under the more careful reading condition, the participants were more likely to attend to and be aware of the target forms, in addition to processing the text intensively. Crucially, in our study, these results cannot be attributed to language proficiency (because we controlled for it), which may otherwise be a factor in studies (e.g., Rayner et al., Reference Rayner, Chace, Slattery and Ashby2006) that report a link between increased reading behaviors and perceived text difficulty.
There may thus be different reasons for why readers make longer and/or more frequent eye fixations; the most relevant here is that some of the previous studies show that slower reading enables more accurate and careful reading, which in turn yields better comprehension. In line with these studies, we observed positive effects of slower reading on comprehension. Intermediate L2 Spanish learners who read the text carefully and slowly demonstrated better comprehension and this held true, regardless of their L2 proficiency level and instruction to focus on meaning or meaning and form.
LIMITATIONS AND FUTURE DIRECTIONS
The study had some limitations that need to be considered when interpreting the findings. As Morgan-Short et al. (Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018) noted, comprehension questions might not be ideal for measuring the effects of attending to form. The reliability of the comprehension check questions was low in both that study (α = .197 across all experimental sites) and the current one (α = .293). This could be due to the test or the participants’ performance in the specific setting of the present study. In the current study, we triangulated the data from the comprehension posttests with online reading data recorded for the entire reading passage with eye-tracking. Although this yielded a more comprehensive view of participants’ text-comprehension processes, future research would benefit from having a more robust comprehension measure after the reading task.
To make sure of participants’ attention to form, we instructed the experimental groups to attend to their assigned target forms for a predetermined amount of time (i.e., 2 seconds). As suggested by VanPatten (Reference VanPatten1990), a certain amount of time may be necessary for L2 learners to pay attention to the form. Even so, due to this methodological difference, we acknowledge that the experimental design was not perfectly aligned with previous studies, such as Leow et al. (Reference Leow, Hsieh and Moreno2008) and Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012), in which there was no additional instruction for the amount of time to attend to form. Despite this explicit instruction, however, the participants in the experimental groups showed similar global reading behavior of the text as the control group, suggesting that the additional instruction did not hinder their processing of the passage.
Future researchers should further explore simultaneous attention to the form and meaning of L2 input by using different experimental instructions, for instance by replicating the present eye-tracking study with no explicit instruction regarding time on the assigned forms, using a combination of different grammatical forms and L2 proficiency levels, and triangulating eye-movement records and verbal data to gain better insight into this issue (see also Godfroid et al., Reference Godfroid, Winke and Conklin2020; Godfroid & Spino, Reference Godfroid and Spino2015).
CONCLUSION
We revisited the long-standing question of whether simultaneous attention to form and meaning impacts text comprehension negatively (VanPatten, Reference VanPatten1990). Adding to a growing series of interconnected studies (Leow et al., Reference Leow, Hsieh and Moreno2008; Morgan-Short et al., Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012, Reference Morgan-Short, Marsden, Heil, Issa, Leow, Mikhaylova, Mikolajczak, Moreno, Slabakova and Szudarski2018), we introduced a much-needed eye-tracking perspective to the debate surrounding VanPatten’s (Reference VanPatten1990) primacy of meaning principle. We found that attention to form hampered L2 learners’ comprehension of the L2 written input and that these effects were large for participants who had been instructed to attend to purely grammatical (as opposed to lexical or lexico-grammatical) forms. Participants who read the texts more slowly and carefully could offset these comprehension costs and increase their comprehension.
The present results therefore may provide support to VanPatten’s (Reference VanPatten1990) primacy of meaning principle, while also highlighting a role for individual differences. Faster L2 readers may not be able to allocate their attention to form and meaning simultaneously without experiencing a cost in comprehension. Slower and careful L2 readers, however, process the input more deeply, either by their disposition or by training. Such readers seem to be able to handle competing demands on their attention just fine, which in turn, enables them to comprehend context as well as attend to form.
SUPPLEMENTARY MATERIALS
To view supplementary material for this article, please visit http://doi.org/10.1017/S0272263121000565.
COMPETING INTERESTS
The author(s) declare none.
