INTRODUCTION
VanPatten’s (Reference VanPatten1996) Processing Instruction (PI) is an input-based pedagogical type of grammar instruction based on the SLA theoretical model of Input Processing. According to VanPatten’s model of Input Processing, second language (L2) learners use a number of internal in-built strategies when they attempt to comprehend and process L2 input. These internal strategies may prevent L2 learners from accurately making form-meaning connections and therefore might delay or hinder the acquisition of target linguistic features. VanPatten (Reference VanPatten1996) states that PI helps L2 learners acquire specific grammatical structures by altering their processing strategies and encouraging better form-meaning connections in the input they receive than traditional drill-oriented practice.
Since VanPatten and Cadierno’s first published study in 1993, PI has been broadly researched on a wide range of languages, populations and contexts (Benati and Lee, Reference Benati and Lee2015). The results have consistently shown L2 learners significantly improve their scores in both interpretation and production tasks after receiving PI (VanPatten and Cadierno, Reference VanPatten and Cadierno1993; Cadierno, Reference Cadierno1995; Benati, Reference Benati2001; Cheng, Reference Cheng and VanPatten2004; VanPatten and Wong, Reference VanPatten and Wong2004; Lee and Benati, Reference Lee and Benati2007b). Research has shown the effectiveness of PI on L2 learners’ ability to process input by altering their processing strategies. The large database of research findings has also demonstrated that it is a better instructional intervention than other pedagogical interventions, such as the output-based practice known as Traditional Instruction (TI) (Benati, Reference Benati2005), meaning-based output instruction (Farley, Reference Farley2001, Reference Farley and VanPatten2004; Benati, Reference Benati2005), input-based intervention (Lee and Benati, Reference Lee and Benati2007a) and dictogloss (Uludag and VanPatten, Reference Uludag and VanPatten2012), at improving learners’ rate of processing and at altering L2 learners’ processing problems (e.g. First Noun Principle; see VanPatten, Reference VanPatten, VanPatten and Williams2015b). However, most assessment tasks used to evaluate the effects of PI have been offline, relying on pen and paper tasks. Therefore, the processing of input by L2 learners has not been measured during real-time comprehension. The present study uses the online measurement of eye-tracking to observe L2 learners’ eye movement patterns in real time input processing and investigate whether PI changes the way L2 learners process the target form, namely the French imperfect.
This study has two primary goals. The first is to assess the impact of PI on both online and offline measures by presenting the results of a partial replication experimental study investigating the processing of pastness in French and the acquisition of the French imperfect using eye-tracking methodology as an online measure. This is done to examine L2 learners’ cognitive processing behaviours before and after they receive PI and TI and establish whether modifications in cognitive processing behaviours during the processing of the linguistic input occur after PI treatment. The effects of instruction are also addressed by analysing changes in pre-test and post-test scores in order to link the results to those of other PI studies in general but particularly on the French imperfective aspect (Laval, Reference Laval, Benati and Lee2008, Reference Laval, Lee and Benati2013).
The second goal is to offer reflections on methodological issues with the use of eye-tracking in experimental approaches in SLA research and to address these methodological issues by drawing on the research methodology used in the design of this experimental study. This is done to highlight the need for methodological considerations when using sensitive online research methods such as eye-tracking.
BACKGROUND AND MOTIVATION
Input Processing and Processing Instruction
In his Input Processing model, VanPatten (Reference VanPatten1996, Reference VanPatten2004b, Reference VanPatten, VanPatten and Williams2007, Reference VanPatten, Benati and Lee2015a) identified two main processing principles that affect L2 processing.
The first principle is known as the Primacy of Meaning Principle, whereas the second is the First Noun Principle. In the current study, we focus on the first principle, with its six sub-principles (the Primacy for Content Words Principle, the Lexical Preference Principle, the Preference for Non-Redundancy Principle, the Meaning-Before-Nonmeaning Principle, the Availability of Resources Principle and the Sentence Location principle). This first principle addresses the fact that when learners are engaged in communicative, meaningful exchanges, they are primarily concerned with meaning. According to VanPatten’s Primacy of Meaning Principle ‘L2 learners process input for meaning before they process it for form’ (VanPatten, Reference VanPatten, VanPatten and Williams2007:116). In other words, L2 learners process input for meaning first and by doing so have a tendency to disregard grammatical features.
VanPatten (Reference VanPatten1996) created a pedagogical intervention, the input-based practice PI, to change these processing strategies by engaging L2 learners in efficient parsing and processing of forms and meaning. Subsequent studies (Cadierno, Reference Cadierno1995; Marsden, Reference Marsden2006; Toth, Reference Toth2006; Laval, Reference Laval, Benati and Lee2008, Reference Laval, Lee and Benati2013) have shown PI to be effective at teaching L2 learners a target-language-appropriate processing strategy as described by VanPatten (Reference VanPatten1996) insofar as they have been shown to severely reduce, or substitute, their dependence on the Primacy of Meaning Principle and its sub-principles in a wide range of languages (e.g. Spanish, Italian, French). Research on the effects of PI on the acquisition of the French imperfect, with a focus on non-past/past distinction, also shows the positive effects of PI in drawing L2 learners’ attention to the verbal morphology and in accuracy scores. Laval (Reference Laval, Benati and Lee2008) investigated the effects of PI on the acquisition of the French past tense imperfective aspect as measured by interpretation and production tests using a pre-test/post-test design.
PI treatment was compared to TI treatment. The PI group’s scores significantly improved from pre-test to post-test, and their performance was greater and statistically better than the TI group. The PI group gained about 58% from pre-test to post-test scores in their ability to interpret the French imperfect. The control group’s score did not improve. The results of this study on the primary effects of PI have shown that PI is a better approach to grammar instruction than TI at improving learners’ rate of processing linguistic features affected by the Primacy for Meaning Principle, such as the French imperfect. The positive effects of PI are also demonstrated by the increasing L2 learners’ accuracy in production. The PI group gained 140% from pre-test to post-test scores in the production task.
Laval (Reference Laval, Lee and Benati2013) conducted a follow-up investigation to focus on individual differences in PI and to further investigate the age factor. Like in Laval (Reference Laval, Benati and Lee2008), the study investigated the effects of PI on the acquisition of the French past tense imperfective aspect as measured by interpretation and production tests using a pre-test/post-test design. The participants were 9–10-year-old school-aged native English speakers learning French as an L2. PI treatment was compared to a control group. The PI group gained 64% from pre-test to post-test scores in their ability to interpret the French imperfect. The control group decreased by 8% from pre-test to post-test scores. The PI group gained 159% from pre-test to post-test scores in the production task whereas the control group made no improvements from pre-test to post-test scores. The results are consistent with Laval (Reference Laval, Benati and Lee2008) and demonstrate that PI is an effective approach to grammar instruction in that it never failed to yield significant improvement in learners’ performance on either interpretation or production tasks. A new research area in PI is now expanding the scope of the methodology to not only compare accuracy scores as in previous studies but also to measure processing behaviours using eye-tracking. Only two studies thus far (Wong and Ito, Reference Wong and Ito2017; Lee and Doherty, Reference Lee and Doherty2019) have considered the use of eye-tracking to measure the effectiveness of PI; both investigate a different processing principle (i.e. the First Noun Principle) than the one under investigation in this study.
Lee and Doherty (Reference Lee and Doherty2019) investigated the effects of PI on native and non-native processing of Spanish active and passive sentences using eye-tracking as a measurement of processing behaviours. A visual world paradigm was used to measure accuracy and response time. After PI treatment the non-native speakers showed no significant differences with native speakers in accuracy and response time. These results can be attributed to the positive effects of PI.
Wong and Ito (Reference Wong and Ito2017) explored the changes in processing patterns between two groups of learners, one group receiving PI and the other receiving TI on the acquisition of the causative construction in French. The results provide evidence for the FNP in eye-movement data, along with beneficial results from PI in the accuracy of picture selection post-instruction, in line with previous offline studies (VanPatten and Wong, Reference VanPatten and Wong2004).
These are the first published studies to investigate the effects of PI with the online measure of eye-tracking and have provided evidence to support that PI ‘does change the way learners process the target structure’ (Wong and Ito, Reference Wong and Ito2017: 26). To reinforce this claim and further develop the pool of data, more studies need to use eye-tracking to measure the effects of PI on other target items affected by different processing principles. This will only be possible if there is a clear understanding of methodological issues which need addressing when using eye-tracking in experimental approaches SLA.
What eye-tracking brings to PI research and methodological issues to consider
Eye-tracking is being increasingly used in SLA research to determine how people process input. It is used to detect and measure eye movements (saccades) and stops (fixations) while reading or attending to visual scenes. This online technique provides insights that are not accessible when offline studies are used (e.g. pen and paper tests, reaction tests, questionnaires), gathering data on specific words or phrases in a sentence as it is perceived in moment-by-moment processing. This real-time measure is suggested to ‘tap participant’s implicit knowledge of language’ (Keating and Jegerski, Reference Keating and Jegerski2015: 2) through two assumptions (Conklin and Pellicer-Sánchez, Reference Conklin and Pellicer-Sánchez2016). The first is that overt attention, as shown through where and when the eyes move during a task, reflects covert attention and the cognitive effort required to process a target item (Godfroid, Reference Godfroid and Robinson2012). The second assumption suggests that the item being fixated on in eye-movements is what is being processed cognitively.
Under these two assumptions, eye-tracking provides data demonstrating which sentential aspect L2 learners are processing, which are being skipped and which require more conscious effort. This type of information can greatly advance the understanding of L2 input processing and provide further details regarding the beneficial effects of PI treatment. VanPatten (Reference VanPatten, VanPatten and Williams2015b: 92) defines input processing as ‘the moment-by-moment computation of sentence structure’ and the scope of PI is to ensure that L2 learners make form-meaning connections during real-time comprehension. Eye-tracking provides researchers with the tools to measure this real-time comprehension and connections of form and meaning during input processing. More specifically, with eye-tracking, researchers are able to determine whether learners are focused on the target item of the sentence before and after receiving PI treatment. This data shows the impact of PI on learners’ implicit cognitive processing, which provides opportunity for deeper analysis of processing mechanisms and a more direct measurement on implicit knowledge than previously used offline measures.
However, the positives of eye-tracking can be hindered by the simplest of design flaws, unintentionally inviting the use of explicit knowledge or strategic processing such as unusually slow or careful reading (Keating and Jegerski, Reference Keating and Jegerski2015). These flaws, along with various lexical variables, which have been shown to influence fixation times (Godfroid, Boers, and Housen, Reference Godfroid, Boers and Housen2013), can potentially render the moment-by-moment processing data invalid. Spinner et al. (Reference Spinner, Gass and Behney2013) suggest that research in SLA should become more sophisticated in the development of eye-tracking studies and materials. Without clear and detailed guidelines for setting up eye-tracking experiments, researchers risk producing invalid data or studies difficult or impossible to replicate. These guidelines will also promote development of valid and replicable research materials which can be shared for future studies such as the IRIS database (Marsden, Mackey, and Plonsky, Reference Marsden, Mackey, Plonsky, Mackey and Marsden2016).
There are two levels to consider when designing a reading-based interpretation study. The first high-level considerations concern the language itself. When conducting pen and paper studies, participants may be provided unlimited time and can therefore focus on less frequent vocabulary at no cost to the researcher. In eye-tracking, however, time-sensitive data are collected and used in analysis. It becomes essential to ensure the language used to test the phenomenon under study is designed appropriately to avoid natural eye movement data which is invalid due to design. This includes controlling for word frequency, the length of the items, including word length and overall sentential length, influenced by syntactic form, the position of the target item in the sentence and the position of the sentence on the screen.
Word frequency has a significant impact on the data produced during eye-tracking (Rayner, 1998 as cited in Tiffin-Richards and Schroeder, Reference Tiffin-Richards and Schroeder2015). More frequently occurring words take less time to process and consequently receive fewer fixations than frequent ones (Rayner and Duffy, Reference Rayner and Duffy1986; Williams and Morris, Reference Williams and Morris2004; Juhaz and Rayner, Reference Juhaz and Rayner2006). They are also more likely to be skipped while infrequently encountered words may receive multiple fixations (ibid.). This is due to the higher quality of representations in the mental lexicon which facilitates the recognition of more frequent items in both written and spoken language. Frequency can be determined through corpora or pre-exposed vocabulary through syllabus-analysis.
The context and plausibility of sentences may also have effects on participants’ eye movement patterns. Similar to low-frequency words, participants may produce unexpected or unrelated eye movements when parsing an ambiguous or unexpected sentence. Avoiding (morpho)syntactic anomalies can be ensured through sentence norming studies, which check the plausibility of the intended target items (Keating and Jegerski, Reference Keating and Jegerski2015), reducing unwanted changes in eye movement behaviour. Keeping the context consistent can also reduce unrelated eye movement data, providing learners with an expectation for the sentences presented.
Word length can directly influence eye movement during reading (Tiffin-Richards and Schroeder, Reference Tiffin-Richards and Schroeder2015) as longer words receive more fixations than short words, and short (i.e. two- to three-letter) words are usually skipped over or included within the parafoveal gaze when fixating on an adjacent word (Frenck-Mestre, Reference Frenck-Mestre2005). The target linguistic feature should be kept consistent, where possible, across all items or within a difference of a few characters to avoid effects on fixation patterns. Keating and Jegerski (Reference Keating and Jegerski2015) suggests that if it is not possible to control for character length, syllable count may be a viable alternative. Word length of the other lexis coming directly before and after the target feature can also influence fixations on the target feature, known as spillover effect (Rayner and Duffy, Reference Rayner and Duffy1986), which occurs when processing of the target aspect in a sentence continues onto the words immediately after the critical region.
In maintaining word length, sentence length is also kept consistent. This is also a result of keeping the syntactic frames as similar as possible across all items. The use of the same syntactic frame will maintain the position of the target linguistic aspect on each item. If the target feature moves position, it can cause eye movement data to change, as it is generally seen that reading speed decreases as a reader progresses through a text (Conklin and Pellicer-Sánchez, Reference Conklin and Pellicer-Sánchez2016), which would suggest that words at the end of the sentence are read more slowly. Conklin and Pellicer-Sánchez (Reference Conklin and Pellicer-Sánchez2016), on the other hand, suggest a counterbalance of the location of target items to avoid fixations due to natural reading sequences. If target feature x appears on the top left of the screen, while feature y appears in the bottom right, x will always be fixated first. Additionally, participants may potentially skip the last word of the sentence if this aspect is redundant for processing. The position of the target aspect should therefore be considered in relation to the task and items presented, but further research is needed to support this, especially in reading studies in languages, such as German, where the final position is not redundant.
The second level of considerations for eye-tracking design refers to the physical properties of the materials (Frenck-Mestre, Reference Frenck-Mestre2005) such as font style and size, screen layout and colours used. All of these aspects vary greatly between studies so far, some of which do not mention these at all (Spinner et al., Reference Spinner, Gass and Behney2013: 393). Font style and size have been examined in several studies specifically relating to language acquisition and online reading. Beymer et al.’s (Reference Beymer, Orton, Russell, Baranauskas, Palanque, Abascal and Barbosa2007) overview of literature on the most effective font for online reading was inconclusive and failed to mention the screen sizes or resolutions; so, the actual font sizes were not compared. Beymer et al. (Reference Beymer, Orton, Russell, Baranauskas, Palanque, Abascal and Barbosa2007) suggested a slight benefit in reading speed for larger fonts but this was not statistically significant. However, a general one-size-fits-all approach is not appropriate in such sensitive methodologies such as eye-tracking.
Similarly, Beymer et al.’s (Reference Beymer, Orton, Russell, Baranauskas, Palanque, Abascal and Barbosa2007) results yielded no significant conclusions when comparing the effects of sans-serif and serif font on eye-tracking measures. However, with the current everyday use of online reading, the comprehension of sans- and serif fonts for speakers of a roman orthography is no longer the concern. Rather, with the precise measures from the eye-tracker, the width of the letters in a word should be the main consideration when choosing font styles. A four-letter word containing relatively ‘thin’ and ‘thick’ letters on Times New Roman ‘mime’ is considerably different to the same size font with a monospaced style, such as Lucida Console. As word position, word length and layout are such important considerations when using eye-tracking, researchers’ choices of font style should avoid inducing further problems.
Although background colour is not included in the explanation of research design in many SLA eye-tracking studies, research into the psychological effect of colour, particularly red and blue, during tasks has been widely investigated (Elliot and Maier, Reference Elliot and Maier2014). As eye-tracking is suggested to tap into learners’ cognitive processors, it is important to consider the effects that colour may have when participants are processing linguistic data. Anuardi et al. (Reference Anuardi, Shinohara and Yamazaki2016) investigated the effects of screen background colours on the brain during tasks on a tablet computer. The participants counted the number of circles on five different background colours: white, blue, yellow, red and green. The data showed the participants obtained higher scores on a coloured background compared to a white background. This result suggests that white might not encourage best performance ability and attention, therefore it is avoided in the present study.
Although many papers mention the importance of promoting ‘natural’ reading during eye-tracking studies through the use of ‘normal’ fonts and colour, Spinner et al. (Reference Spinner, Gass and Behney2013) investigated the effects of ‘natural’ ecologically valid experiment design versus a specially designed format. This format included a larger font (44pt), presented over three separate lines, to measure precise eye movements on the articles in each target item. To avoid the top left favoured position, adverbs of varying length were used at the beginning of the sentence so the target item would not appear in this position. The ecologically valid design used a standard sized font with text which was more closely aligned to normal reading material, with the whole sentence shown over one line at the top of the screen. The two designs yielded different results in reading times and on regressions, suggesting that methodology and presentation of items matter (Spinner et al., Reference Spinner, Gass and Behney2013: 409).
More recent publications have also highlighted the above methodological aspects key in the design of studies using online reading in SLA research and emphasized the need for future studies to consider key design principles for psycholinguistic in research design (VanPatten and Jegerski, Reference VanPatten and Jegerski2014; Keating and Jegerski, Reference Keating and Jegerski2015; Conklin and Pellicer-Sánchez, Reference Conklin and Pellicer-Sánchez2016). This is essential when conducting quasi-replication studies with online measures, such as the current study. Thus, the aforementioned methodological issues are taken into consideration in the present study, as discussed in the Method section below.
THE PRESENT STUDY
Primary target item: The French imperfect past tense of regular verbs
In many languages, including French and all Romance languages, a crucial dichotomy dominates the tense aspectual system: perfective vs. imperfective (Giorgi and Pianesi, Reference Giorgi and Pianesi1997; Salabeery and Ayoun, Reference Salabeery, Ayoun, Ayoun and Salaberry2005). For French in particular, these two aspects are generally encoded with two common verb forms: the passé composé and imparfait. Understanding the distinction between the passé composé and imparfait poses quite a few obstacles for L2 learners and in most SLA studies on the French imperfect, there is a focus on the opposition between the imparfait-passé composé (Izquierdo and Collins, Reference Izquierdo and Collins2008; Izquierdo, Reference Izquierdo2009). However, it is important to note that this is not the case in the present study. Instead, as input processing predicts that L2 learners will interpret the present ‘chante’ and the imperfect ‘chantait’ in the same way, without attention to the verbal morphology, the focus is on pastness with an opposition between the past (French imperfect) and the non-past (present). PI aims to draw L2 learners’ attention to this verbal morphology and the fact that it encodes the non-past/past distinction.
This study being a partial replication study of Laval (Reference Laval, Benati and Lee2008, Reference Laval, Lee and Benati2013), the same primary target; namely, the French imperfective is under investigation. There are multiple functions of French imperfective (Ayoun, Reference Ayoun, Ayoun and Salaberry2005); however, only one, the habitual imperfect, which is communicatively important, is tested in this study. As stated in Laval (Reference Laval, Benati and Lee2008, Reference Laval, Lee and Benati2013), the French imperfect past tense was chosen for a number of reasons.
First, the French imperfect past tense is affected by the Primacy of Meaning Principle and three of its six sub-principles are also at play in the processing of the French imperfect.
The first sub-principle which affects the processing of the French imperfect is the Primacy of Content Words Principle, which states that ‘L2 learners process content words in the input before anything else’ (VanPatten, Reference VanPatten, VanPatten and Williams2007: 117). Content words, primarily nouns, adjectives and the base form of the verb, are relied on by learners as cues to extract meaning as opposed to determiners, partitives or inflections. In the case of the French imperfect past tense, the verb inflection would therefore not be processed to extract the meaning of pastness.
The second sub-principle of the Primacy of Meaning Principle considered to be affecting the processing of the French imperfect past tense is the Lexical Preference Principle, which claims that ‘L2 learners will tend to rely on lexical items as opposed to grammatical form to get meaning when both encode the same semantic information’ (VanPatten, Reference VanPatten, VanPatten and Williams2007:118). According to this processing principle, L2 learners are hypothesized to disregard verbal morphology; therefore, they will not make natural connections between the French imperfect past tense form marker and the concept that the action took place in the past. In other words, L2 learners may not attend to the verbal inflections of the French imperfect past tense in the input if they were co-referenced with lexical temporal adverbials. L2 learners would process the lexical items over the grammatical forms since they both encode the same information.
Finally, the third sub-principle of the Primacy of Meaning Principle under consideration is the Sentence Location Principle which asserts that the position of a form in a sentence matters as ‘L2 learners tend to process items in sentence initial position before those in final position and those in medial position’ (VanPatten, Reference VanPatten and VanPatten2004a: 14). In French, like in all SVO-languages, the verbal syntagm is generally in sentence medial position. Therefore, the imperfect past tense, like most other verb forms in French, generally occurs in sentence medial position. According to the Sentence Location Principle (VanPatten, Reference VanPatten and VanPatten2004a, Reference VanPatten, VanPatten and Williams2007, Reference VanPatten, VanPatten and Williams2015b), this is the least salient processing position and therefore an unfavourable processing position as L2 learners are less likely to detect it.
Another reason for the selection of the French imperfect past tense for investigation is that the offline measure of the impact of PI has previously been assessed for the French imperfect (Laval, Reference Laval, Benati and Lee2008, Reference Laval, Lee and Benati2013). Therefore, expanding the research line by using an online measure to establish the impact of PI on this particular linguistic feature will contribute to the generalizability of the effects of PI on the past imperfect on both online and offline measures.
RESEARCH QUESTIONS
As mentioned above, the experiment investigates the L2 acquisition of the French imperfect using eye-tracking methodology to compare the accuracy of scores and L2 learners’ cognitive processing behaviours before and after they receive PI. The present study compares two types of instruction, the input-based practice PI and the output-based practice TI. Accuracy scores and eye movements on the pre-test and post-test interpretation tasks are examined to investigate any cognitive changes in processing strategy in the interpretation of the past tense in French after instruction. The experiment is a conceptual replication of Laval (Reference Laval, Benati and Lee2008, Reference Laval, Lee and Benati2013) and uses materials, procedures and assessments specifically designed for eye-tracking methodology. Importantly, it addresses what further evidence eye-tracking data can provide in answering questions regarding SLA. The three research questions below guided the study.
RQ1. Before treatment, is the Primacy of Meaning Principle (specifically the Primacy of Content Word Principle) displayed in both groups’ performance, as measured by accuracy of response and eye-tracking movement patterns?
RQ2. Does PI treatment on the French imperfect improve L2 learners’ interpretation of the French imperfect as measured by accuracy of response?
RQ3. Does receiving PI affect L2 learners’ processing behaviours used to process the French imperfect as measured by eye movements while reading the linguistic input?
METHOD
Participants
A total of 22 lower intermediate undergraduate students studying French at university in the UK participated in the study. The participants were recruited from a second-year university-level French course as the curriculum had not yet included the target linguistic feature, the French imperfect. This was also assessed in the pre-test interpretation task resulting in six participants being excluded from the study: two participants were excluded due to calibration issues with the eye tracker and four participants were excluded because they scored over 60% in the pre-test, following VanPatten and Cadierno (Reference VanPatten and Cadierno1993). This resulted in a final pool of 16 participants. They were all volunteers and received a gift voucher for participating. The 16 participants were randomly assigned to the PI or TI group. None of the participants declared any visual impairments, an important point to address prior to an eye-tracking experiment as it would affect eye-tracking data.
Instructional materials
The instructional materials were the same as in Laval (Reference Laval, Benati and Lee2008). The material in the PI treatment encourages L2 learners to focus their attention on the French imperfect form in the input. The relation between form and meaning is always in focus in the presentation of the target item. L2 learners received explicit information on the French imperfect and on the processing problems (the Primacy of Meaning Principle and its three sub-principles as described above), and Structured Input (SI) activities to help them modify these processing strategies. These SI activities required L2 learners to attend to both meaning and form to successfully complete the activities and the learners were never required to produce the target forms. The SI activities consisted of both referential and affective activities. Referential activities are those meaning-based activities with right or wrong answers (Lee and VanPatten, Reference Lee and VanPatten2003). These activities required L2 learners to listen to a series of sentences which had Zinédine Zidane as the grammatical subject. Learners ticked boxes to indicate whether the statement they heard was referring to Zinédine Zidane’s past life as a professional football player or his current life as a retired football player. The only way to correctly decide to which part of his life the sentences referred was to process the (either past or present) verbal inflections and use this information. This type of SI practice was designed so that learners would no longer strictly rely on the Primacy of Meaning Principle and the three sub-principles. There were no lexical temporal adverbials in the sentences and the activities were designed so the first item L2 learners encountered in a sentence was the verb in the present tense or in the Imperfect past tense (an example of such activity in which sentences started with verbs can be seen in Figure 1 below).
Structured input activities: sample referential activity used in the material for processing instruction treatment.

In affective activities, French L2 learners offered a personal reaction to a statement; for example, by indicating whether or not it was true for them or some other reference group with which they are familiar. In Figure 2, for example, L2 learners read a series of statements using the target form about teenagers’ actions and they were asked to tick boxes to indicate whether a parent, a relative or their instructor would have carried out any of the statements they read. By doing so, meaning is kept in focus as L2 learners are relating information to the people they know. An additional layer of meaning is included in this activity in that learners get to find out if they were accurate about their instructor’s teenage years. Again, the processing strategies affecting the acquisition of the French imperfect tense are kept in mind and none of the sentences contain a lexical adverbial to cue tense or aspect.
Structured input activities – sample of affective activity used in the material for processing instruction treatment (adapted from Farley, Reference Farley and VanPatten2004).

Feedback during the instructional treatment was restricted. On the referential activities, the instructor informed L2 learners whether their interpretations were correct or not but did not offer any further information on the item nor offered further explanations. In both sets of activities, the subject noun or pronoun was removed, placing the target form (verb) in initial position, the most favoured processing position. This was done to help L2 learners make better form-meaning connections.
Like in Laval (Reference Laval, Benati and Lee2008, Reference Laval, Lee and Benati2013), in the TI treatment, explicit information on the French imperfect past tense was provided but no information on the processing principles was given. This was followed by practice involving mechanical production of the correct verbal inflection (oral and written production). Activities included fill-in-the-blank tasks, sentence completion tasks and traditional substitution drills (see Figure 3).
Sample of mechanical output practice activity used in the material for traditional instruction treatment.

Pre-test/post-test and methodological issues addressed
The pre-test and post-test interpretation tasks for the target item included 20 written sentences, 10 of which included the present tense as distractor items (see Figure 4), while 10 included the French imparfait. These items were adapted from Laval (Reference Laval, Benati and Lee2008). The items were shown individually on the computer screen with two buttons below which stated ‘past’ and ‘present’. This task required participants to determine whether the sentence was in the past or present. For example, participants read: Il jouait au tennis et football and then chose either ‘present’ or ‘past’ by clicking on the corresponding button which took them to the following sentence (see Figure 5). There was no time limit on each sentence nor any feedback after each sentence. Each sentence included one verb, varying between six to eight characters, which was followed by a preposition, an article or, in one case, an adverb, varying between two and four characters, and two content words, which had no temporal indication. The length of the words was controlled to avoid spillover effects. The personal pronoun before the verb was kept to a two-letter French pronoun. This was to minimize the effects on the data collection as words containing two letters are often skipped (Frenck-Mestre, Reference Frenck-Mestre2005) and to maintain the position of the target feature (Conklin and Pellicer-Sánchez, Reference Conklin and Pellicer-Sánchez2016). The syntactic frames of both the target and distractor sentences were kept consistent to avoid significant changes in reading speed. The vocabulary included was derived from the syllabus that the students had been following and should not have been new to the participants. This was to reduce any unwanted design effects on the data collected, as previously noted.
Example of distractor item presented on eye-tracker.

Example of target item presented on eye-tracker.

On the 23” screen with a resolution of 1920x1080 pixels, the sentences were centred and 1.5cm below the position of the fixation cross shown between each item. This allowed data to be collected from the first fixation on the sentence, rather than from the fixation point. The current study used a grey background colour (#f2f2f2), to avoid associated motivational effects with colour, and 24pt font size with a sans-serif font style (Lucida Console) consistently across the study, which is monospaced font, minimizing design effects on fixation patterns. With the screen resolution provided, the letters were around 8.33mm on the screen, a size designed to allow participants to sit comfortably at 60cm distance from the screen. At this distance, the visual angle of each letter was 0.7954°, which meant the entirety of each letter could be formed on the viewer’s fovea (highest-resolution area of the retina), giving a 0.14° precision from the binocular eye movement data. These design decisions were influenced by the previously mentioned methodological considerations and by the results of various pilot studies to ensure the data collected from the eye-tracker reflects the effect of the instructional treatments rather than the design.
PROCEDURE AND METHODOLOGICAL ISSUES ADDRESSED
Eye-tracking data gathering
Participants completed the pre-test interpretation task in the eye-tracking lab two weeks before receiving instruction. Instruction for both groups lasted 1 hour 30 minutes. After completing the consent form and background questionnaire, participants were calibrated on the Tobii TX300 eye-tracker using a 9-point calibration sequence. Participants were provided with instructions, which told participants to choose whether the sentence was referring to the past or present, and an example item before starting the test to check calibration and understanding of the task. Then, participants completed the 20-item interpretation task with the sentences presented in a pseudorandomized order. Participants’ eye-movements were collected continuously throughout the interpretation tasks, sampled at 300Hz by the Tobii Studio system. Two weeks after the pre-test, participants received classroom-based instruction, either PI or TI treatment, during their usual class time with an informed instructor. Immediately after, participants completed the 20-item post-test interpretation task following the same steps as the pre-test. Neither test phase of the study was timed with participants varying between 5 to 10 minutes to complete the task. Participants generally completed the post-test interpretation task faster than the pre-test.
Eye-tracking data processing
Eye-movement recording quality was determined by removing all target sentence slides which had lower than 90% of the recorded eye-movement data (Doherty, Reference Doherty2012). Areas of Interest (AOIs) were drawn around each item of the target sentence to isolate eye-movement data for the specific spatial-temporal regions. These AOIs were created manually but kept consistent, with a border of 10 pixels, allowing the inclusion of eye-movement data surrounding the target feature. All data were then exported to SPSS for statistical analysis.
ANALYSES AND RESULTS
Mean accuracy scores
The effects of time and treatment on the correct interpretation of the French imperfect were analysed using a repeated-measures analysis of variance, in line with previous studies which measure the effects of PI over TI (Laval, Reference Laval, Benati and Lee2008, Reference Laval, Lee and Benati2013). A one-way ANOVA was run on the scores to establish consistency between the two groups, which saw no difference in pre-test scores (F(2,15) = .370, p = .553). This suggests any change in scores is a result of instruction. From the means and standard deviations shown in Table 1, an increase can be seen for the PI group over the TI group.
Mean scores and standard deviations for the two treatment groups on the interpretation tasks: PI and TI

A repeated-measures ANOVA to analyse the effects of Time, Treatment and the Interaction between Time and Treatment saw a significant effect for Time (F(1,15) = 15.836, p = .001); Treatment (F(2,15) = 8.045, p = .013) and the interaction between Time and Treatment (F(2,15) = 6.591, p = .022). Further post-hoc analysis was run through the use of two-samples t-tests. This demonstrated that the PI statistically significantly outperformed the TI group on the post-test interpretation task (p = .002) and the TI group made no statistically significant increase between pre-test and post-test (p = .230).
Eye-movement patterns: total fixation duration
As the accuracy of scores demonstrated a robust effect for instruction, the eye-movement data collected during the interpretation of the target items were expected to demonstrate an effect for instruction, and more specifically, a change in processing behaviour for those receiving PI.
The Total Fixation Duration data were collected from the AOI around the verb (content word 1) and content words two and three (non-verbs). Total Fixation Duration data were compared with Time to First Mouse Click, which is when the participant chose their answer and therefore assumed to demonstrate the total time spent processing the sentence. The Total Fixation Duration and Time to First Mouse Click data were calculated as the proportion of time fixated on each aspect for each item. Any fixations less than 80ms were not considered for analysis along with any fixation which exceeded 2.5 standard deviations from the participant mean (Keating and Jegerski, Reference Keating and Jegerski2015: 24). The proportion data were plotted onto a graph demonstrating the mean fixation duration of each group on each part of the stimuli. In Figure 6, the ‘other’ category includes fixations on ‘small words’ such as prepositions or pronouns, or on other areas of the screen, outside an AOI. The top panels of the graph display the proportion data for the post-test and the bottom panels display the proportion data for the pre-test.
French imperfect fixation duration data: PI vs. TI.

As seen in Figure 6, both groups show similar strategies in the pre-test, fixating on most items for a similar amount of time. A one-way ANOVA on the pre-test proportion data saw consistency across both groups for the verb (F(2,138) = 1.941, p = .166), the two other content words (content word 1: F(2,138) = .062, p = .803; content word 2: F(2,138) = .169, p = .682), and both past (F(2,138) = .005, p = .942) and present (F(2,138) = .000, p = .986) buttons. This suggests the changes in proportion data are a result of instruction.
The upper panels demonstrate that the PI group fixated longer on the first content word, the verb, in the post-test, simultaneously, spending less time fixating on the two content words with no temporal meaning. Content word 3 received the least fixation time in the PI group post-test data but also in the pre-test for both groups. This could be due to the fact that many of the third content words were at in the final position of the sentence. The PI group’s fixation duration on the ‘present’ button also decreased in the post-test, while an increase in fixations on the ‘past’ button was seen. The ‘other’ category shows a decrease between pre-test and post-test which suggests further focus for the PI group. In comparison, the TI group did not produce such significant changes in Total Fixation Duration across all categories of AOI data. The only observable significant change is the decrease in fixations on the third content word, but as previously mentioned, this could be due to reading strategies employed by the participants. A slight increase can be seen for the second content word, but this is minor.
These data suggest changes in the processing strategies L2 learners use after receiving PI treatment. For statistical analysis, a repeated-measures ANOVA was run on the pre-test and post-test proportion data for the two groups for each AOI on the sentences. Time was used as the within-subjects variable and Treatment as the between-subjects variable. The results from the ANOVA on the data for the verb demonstrated a significant effect for Treatment (F(1, 137) = 14.811, p = .000); Time (F(2, 137) = 22.069, p = .000) and the interaction, Time*Treatment (F(2, 137) = 4.745, p = .031). This demonstrates that the PI group spent significantly longer time fixating on the verb (content word 1) in the post-test than the TI group. The ANOVA for the second content word in the sentence demonstrated a significant effect for Treatment (F(1, 137) = 7.869, p = .006) and the interaction between Time and Treatment (F(2, 137) = 6.666, p = .011), but not for Time (F(2, 137) = 2.712, p = .102). This demonstrates that the TI continued to rely on the content words with no temporal aspect to interpret the meaning of the sentence. For the third word, the ANOVA demonstrated no difference between Treatment (F(1, 137) = .157, p = .693) but a significance for Time (F(2, 137) = 32.225, p = .000) which suggests both groups declined in fixation duration on the third content word. This statistical analysis confirms observations that the PI group changed their processing strategies after receiving treatment, while the TI group relied on the same strategies to interpret the target items.
DISCUSSION AND CONCLUSION
Overall the results of the present study reveal that after receiving PI on the French imperfect, L2 learners become more accurate and quicker in their responses and allocate less visual attention to the words in sentences which do not carry temporal meaning.
In the pre-test, both groups demonstrated a reliance on content words to process the meaning of the sentence. No correlation was made between reliance on one specific content word (with or without temporal aspect) and the accuracy of the answers. Therefore, the results of the present study provide a clear answer to research question one and demonstrate that VanPatten’s (Reference VanPatten1996) Primacy of Meaning Principle and one of its sub-principle, the Primacy of Content Word Principle, are displayed in both groups’ performance, as measured by accuracy of response and eye-tracking movement patterns. In other words, L2 learners rely on content words (big words) in the input to interpret meaning rather than the inflections on the verb.
In terms of accuracy scores, before treatment, no significant differences were seen between the PI and TI groups on the interpretation of the French imperfect. After treatment, accuracy scores were reliably higher for the PI group with a statistically significant gain in accuracy scores from pre-test to post-test. PI treatment significantly improved L2 learners’ interpretation of the target imperfect structure as compared to the TI training. Therefore, these results answer research question two and demonstrate that PI treatment on the French imperfect improve L2 learners’ interpretation of the French imperfect as measured by accuracy of response. The results are consistent with previous PI offline studies (Lee et al., Reference Lee, Benati, Aguilar-Sanchez, McNulty, Lee and Benati2007; Benati et al., Reference Benati, Lee, Laval, Benati and Lee2008; Laval, Reference Laval, Benati and Lee2008, Reference Laval and Benati2011, Reference Laval, Lee and Benati2013) and are now supported by online moment-by-moment data from eye-tracking, providing a new dimension of data to further support such conclusions.
Also, the improvement in accuracy scores for the PI group is supported by noticeable changes in eye movements from the real-time processing of the target sentences. There is a statistically significant increase in time fixating on the verb after receiving instruction and less reliance on the content words with no temporal indication and other features of the sentence to interpret the meaning. This suggests a change in processing behaviours in the PI group. A closer examination of fixation data from the eye-tracker suggests that PI altered participants’ processing mechanisms while TI instruction did not lead to any changes in processing strategy. A very slight improvement can be seen in accuracy scores for interpretation of the target feature for the TI group (pre-test: 3.8 mean, post-test: 4.8 mean); however, no change in cognitive processing as measured via eye movement data is shown after instruction. These results answer research question three by showing that receiving PI affects L2 learners’ processing behaviours used to process the French imperfect as measured by eye movements while reading linguistic input. The additional data provided by the use of eye-tracking allows conclusions to be drawn which may not have previously been noted in offline studies. PI altered the Primacy of Content Words Principle and encouraged interpretation of meaning through the processing of the grammatical structure, as seen in the eye-tracking data. This confirms that PI alters L2 learners’ processing to facilitate parsing and form-meaning connections (VanPatten, Reference VanPatten, VanPatten and Williams2015b).
Data from eye-tracking provided a new insight into the cognitive processing of L2 learners of French. Although a comparison of accuracy scores demonstrated the effects of PI were more beneficial than TI in the interpretation of the French imperfect structure, the eye-tracking data demonstrated cognitive changes in L2 learners before and after receiving treatment. This confirms the beneficial cognitive effects of PI in the interpretation of the French imperfect over TI. PI positively affected L2 learners’ processing behaviours with respect to the French imperfect as measured by eye movements while reading linguistic input. This was shown in the differences in Total Fixation Duration on each sentential aspect by the PI group and TI group. This online methodology brings a new dimension of data to PI and SLA research in general, providing rich data on the implicit processes relied on by learners when processing L2 input. Previous studies using eye-tracking as a means to investigate the effects of PI have only looked so far at changes in processing strategies on visual world paradigms (Wong and Ito, Reference Wong and Ito2017; Lee and Doherty, Reference Lee and Doherty2019).
As seen in previous studies (Laval, Reference Laval, Benati and Lee2008, Reference Laval and Benati2011), the PI training in this study was sufficient to improve L2 learners’ interpretation of the French Imperfect tense. PI treatment led to more efficient and accurate processing of the target linguistic feature as compare to TI treatment, which was not potent enough to help L2 learners improve their interpretation of the French imperfect tense. The present study also shows a strong tendency toward the Primacy of Meaning Principle and its sub-principles in L2 processing of the French imperfect tense as measured by eye-tracking while reading linguistic input. The study demonstrates a change in the online responses to the linguistic input. L2 learners start showing less reliance on the content words with no temporal indication and other features of the sentence but instead more reliance on the verb and its inflection to process meaning. The replication of the positive effects of PI over TI as measured by accuracy of response is clear, like in previous PI studies. The use of the online measure of eye-tracking allows for such conclusions which have been previously hypothesised and subsequently promotes the use of such research methodology in future studies. To further develop from previous research, this study opens questions regarding the impact of PI on the development of learners’ processing strategies and calls for future studies to include delayed post-tests to measure possible longitudinal cognitive benefits of PI and to investigate the range of processing principles as well as the potential transfer-of-training effects on eye-movement data and individual differences.
The field of PI started using online methods to capture how learners process L2 during real-time processing and to measure L2 learners’ processing behaviours. This study contributes to this new field of PI research and adds to the conclusions drawn by Wong and Ito (Reference Wong and Ito2017) and Lee and Doherty (Reference Lee and Doherty2019), suggesting that PI changes the way learners process the target linguistic structure. To our knowledge, the present study is the first study within the PI framework to use eye-tracking to investigate sentence processing of the French imperfect and the Primacy of Meaning Principle.
This study, together with the growing number of studies using eye-tracking in SLA research, highlights the increasing importance of this research tool in the field. However, due to the sensitivity of this online research measure, it is important to provide reflections on methodological issues and to highlight the need to have clear and detailed methodological guidelines for setting up eye-tracking experiments, in order to avoid producing invalid data or studies difficult or impossible to replicate.
First, to ensure the validity of the design and to reduce any design effects on the data collected, it is essential to pay particular attention to the language used to test the phenomenon under study. This includes controlling word frequency, the length of the items (word length and overall sentential length) influenced by syntactic form, the position of the target item in the sentence and the position of the sentence.
The second methodological aspect to consider concerns the physical properties of the materials (Frenck-Mestre, Reference Frenck-Mestre2005; Spinner et al., Reference Spinner, Gass and Behney2013). Indeed, a general one-size-fits-all approach with regards to font style and size, screen layout and colours used is not appropriate in such sensitive methodologies such as eye-tracking. For example, considering the precise measures from the eye-tracker, the width of the letters in a word should be the main consideration when choosing font styles. As mentioned above, word position, layout and length are important considerations and font style should be chosen carefully. As for the colours used, research findings (Anuardi et al., Reference Anuardi, Shinohara and Yamazaki2016) suggest a coloured background should be favoured compared to a white background.
When using eye-tracking, the methodology and presentation of target linguistic items matter immensely. The present study emphasizes the need for future studies to consider these methodological reflections and key design principles. This is especially essential when conducting quasi-replication studies with online measures, such as the current study.







