Introduction
Understanding the ways in which prior language knowledge and experience can influence additional language learning constitutes a major line of SLA research (Odlin, Reference Odlin1989; White, Reference White2003; MacWhinney, Reference MacWhinney, Kroll and De Groot2005; Ellis, Reference Ellis2006; Jarvis & Pavlenko, Reference Jarvis and Pavlenko2008). One reason for this is that before learning an additional language all L2 learners possess some type of knowledge of one or more languages. Investigating the extent to which this prior language knowledge and experience influences additional language learning, both positively and negatively, is therefore a defining feature of SLA research. To date, this research program has led to major theoretical and empirical insights into the nature and extent of cross-language relationships in L2 learning (MacWhinney, Reference MacWhinney, Kroll and De Groot2005; Ellis, Reference Ellis2006; Kroll, Van Hell, Tokowicz, & Green, Reference Kroll, Van Hell, Tokowicz and Green2010), developing broad agreement that understandings about prior language knowledge and experience can provide important explanations for the routes and rates of L2 learning.
A major contribution to this body of knowledge has involved investigations of specific first language (L1) properties and the extent to which they appear to positively or negatively influence L2 learning (e.g., Collins, Reference Collins2002; McManus, Reference McManus2013, Reference McManus2015; Roberts & Liszka, Reference Roberts and Liszka2013; Alonso, Reference Alonso2016). Building on this, studies have recently investigated the extent to which applying crosslinguistic SLA research findings can improve L2 learning (e.g., Tolentino & Tokowicz, Reference Tolentino and Tokowicz2014; Zhao & MacWhinney, Reference Zhao and MacWhinney2018). For example, McManus and Marsden (Reference McManus and Marsden2017, Reference McManus and Marsden2018, Reference McManus and Marsden2019a, Reference McManus and Marsden2019b) sought to address well-documented L2 learning difficulties resulting from L1-L2 differences in English speakers’ use of the French Imparfait (Howard, Reference Howard, Labeau and Larrivée2005; McManus, Reference McManus2013, Reference McManus2015; Kihlstedt, Reference Kihlstedt2015) by drawing on the accumulated body of SLA evidence informed by the Unified Competition Model (UCM, MacWhinney, Reference MacWhinney, Kroll and De Groot2005), which predicts L2 learning difficulties when the same cues (or form-meaning mappings) in L1 and L2 index different meanings. English speakers received explicit information about English and French form-meaning mappings for ongoingness and habituality to raise awareness of L1-L2 differences, followed by extensive comprehension practice of L1 and L2 sentences that expressed these meanings. Findings showed that raising learners’ awareness to these L1-L2 differences, followed by extensive practice, improved the accuracy, speed, and stability of online and offline L2 processing and oral production, both immediately after instruction and also six weeks later at delayed posttest (see also Spada, Lightbown, & White, Reference Spada, Lightbown, White and Housen2005; Horst, White, & Bell, Reference Horst, White and Bell2010; Lucas & Takeuchi, Reference Lucas and Takeuchi2019; McManus, Reference McManus2019, Reference McManus2020). This growing line of research indicates that explicit instruction tailored to address the nature of the crosslinguistic learning problem (defined based on SLA research instead of linguistic description) can improve L2 learning. At the same time, however, this line of inquiry is relatively novel and under-researched.
The aim of this paper, therefore, is to suggest further research which might advance knowledge and understanding about the ways in which explicit instruction can facilitate L2 learning of crosslinguistically difficult target features, as informed by SLA research. To this end, I lay the ground for a series of replications of two studies designed to understand the role of prior language knowledge and experience in L2 learning: Ellis and Sagarra (Reference Ellis and Sagarra2011) and Tolentino and Tokowicz (Reference Tolentino and Tokowicz2014). These studies were selected for replication because they are grounded in empirical SLA research and SLA theory (Associative-Cognitive CREED, Ellis, Reference Ellis2006; Unified Competition Model, MacWhinney, Reference MacWhinney, Kroll and De Groot2005) and have made early contributions that continue to influence the outlined research program. In addition, the selected studies facilitate replication because they have either (a) made their data collection materials available in digital repositories (e.g., IRIS, www.iris-database.org) or (b) have robustly described their design and provided the stimuli used. For each study, I begin with a brief overview of the study followed by a series of approaches to close replication (see Porte & McManus, Reference Porte and McManus2019).
The original study: Ellis and Sagarra (Reference Ellis and Sagarra2011), experiment 1
Ellis and Sagarra (Reference Ellis and Sagarra2011) reported two experiments that ‘replicate and extend’ Ellis and Sagarra (Reference Ellis and Sagarra2010),Footnote 1 which investigated ‘short- and long-term learned attention in the acquisition of temporal reference in Latin’ (p. 589). The 2011 study investigated the predictions of associative learning theory (Kruschke, Reference Kruschke2001) in initial L2 learning: to what extent does associating one set of linguistic cues with temporal reference (e.g., adverbs) block subsequent association of a different set of cues with the same meaning (e.g., verbal inflections). This study is important for at least two reasons: (1) it advances our understanding of the ways in which prior language knowledge and experience can influence initial L2 learning; (2) it helps us to understand how instruction can be tailored to the nature of the learning problem. While the original study presented two experiments, my focus here is on experiment 1 only.
Participants were randomly allocated to one of three experimental groups: Adverb pretraining group (n = 16), Verb pretraining group (n = 17), or Control group (n = 17). Training lasted less than one hour and comprised four phases. Phase one provided pretraining to the Adverb pretraining and the Verb pretraining groups only. The Adverb pretraining group saw either hodie (‘today’) or heri (‘yesterday’) on the computer screen and had to select its English translation (or ‘meaning’) from two options, ‘today’, ‘yesterday’. The Verb pretraining group saw one of six verb inflections (cogito ‘I think’, cogitas ‘you think’, cogitat ‘X thinks’, cogitavi ‘I thought’, cogitavisti ‘you thought’, cogitavit ‘X thought’) and had to select its English translation from two options, ‘X think(s)’, ‘X thought’. Corrective feedback was provided following correct and incorrect answers.
In phase two, all groups saw short sentences that appropriately combined the adverb with the verb (e.g., hodie cogito, cogito hodie, ‘today I think’). Learners had to select whether each sentence referred to the ‘present’, ‘past’, or ‘future’. Corrective feedback was provided on incorrect answers only.
Phases three and four tested learners’ receptive and production knowledge, respectively. For receptive knowledge, all single item words (e.g., hodie, cogito) were provided in isolation and also in double item combinations (e.g., hodie cogito, cogito hodie) that included both appropriate combinations (e.g., heri cogitavi, ‘yesterday I thought’) and ‘strange combinations’ (e.g., heri cogito, ‘yesterday I think’). Participants first read each item and then judged its meaning on a five-point scale: 1 = past, 2 = between present and past, 3 = present, 4 = between present and future, 5 = future. No feedback was provided. The rationale for this design was to assess which cues (adverbial or verbal) were weighted heavier in learners’ judgements when the adverb and verb conflicted (e.g., in ‘yesterday I think’, past adverb + present tense). Lastly, in phase four, participants read on screen an English temporal item (e.g., X think(s), yesterday) for the words they had seen in phase two and typed the Latin translation.
The results indicated that the three groups responded differently to adverb and verbal cues, with a patterning of results consistent with the type of pretraining received. First, the adverb pretraining group was very sensitive to adverb cues for time reference, which determined 99% of their ratings, but not verb cues (=0% of ratings). Second, control group learners, who received no pretraining, were also heavily reliant on adverb cues (=86% of their ratings) with little sensitivity to verb cues (=3%). Lastly, the verb pretraining group was more sensitive to verbal cues (=37%), but, consistent with the other groups, also showed dominant sensitivity to adverbial cues (=58%). Taken together, these results indicate that without pretraining English speakers were most sensitive to adverb cues (as evidenced by performance in the control condition) over verb cues for interpreting temporal reference. However, short-term training did appear to play a role: cue-based training on either adverbs or verbs increased sensitivity to that cue. That is, adverb training increased attention to adverb cues and verb training increased attention to verb cues. Importantly, however, verb pretraining appeared to counteract the existing processing strategy, to some extent, to use adverb cues for processing temporal information.
The main take-away is that making verb cues salient through pretraining increased learners’ sensitivity to verb cues in utterances that contained both adverbials and verbal cues for temporal reference. Furthermore, as evidenced by control group performance, exposure alone was not sufficient to change learners’ existing processing strategy. In other words, intensive and short-term training designed to increase the saliency of new cues appeared able to contribute to the development of new processing strategies.
Approaches to close replication of Ellis and Sagarra (Reference Ellis and Sagarra2011)
The approach to close replication used here is described in detail in Porte and McManus (Reference Porte and McManus2019). Briefly, a close replication (as opposed to an approximate or a conceptual replication) involves repeating the original study as closely as possible except for one major variable modification. This approach aims to understand to what extent modifying that one variable contributes to the study's outcomes in order to better understand the original study's findings.
In approaching a close replication of Ellis and Sagarra (Reference Ellis and Sagarra2011), as with any replication study, there must first be a rationale for replication. Replicating a study without a clear purpose does not advance our goal of better understanding that original study. Doing so therefore requires a critical reading of that original study (see Porte & McManus, Reference Porte and McManus2019). I suggest two ways to conduct a close replication of Ellis and Sagarra (Reference Ellis and Sagarra2011), beginning first with a motivation for a specific variable modification followed by ways in which that variable could be modified. In addition to the variable modifications outlined in the following text, another good candidate variable for modification includes L1 background to understand how the different trainings contribute to initial L2 learning with cue weightings that differ from English. Although Ellis and Sagarra (Reference Ellis and Sagarra2011) explored this idea in experiment 2, learners in that experiment only received the control training. Providing the full training suite to learners of a different L1 background would therefore help advance our understanding of the original study's findings.
Stimuli in the pretraining conditions
As noted, Ellis and Sagarra (Reference Ellis and Sagarra2011) is a replication and extension of Ellis and Sagarra (Reference Ellis and Sagarra2010). The motivation for replication was that the verb pretraining condition in the 2010 study included ‘just one verb form (first-person singular) for each tense, a selection that artificially balanced the complexity and salience of the adverbial and inflectional cues’ (p. 593). To address this, Ellis and Sagarra (Reference Ellis and Sagarra2011) replicated ‘the basic design [of the 2010 study] but with a more typical, complete, and complicated verbal paradigm’ (p. 593). This involved increasing the number of different cues in the verb pretraining condition to six, but no changes were made to the adverb pretraining condition: learners receiving adverb pretraining attended to two different cues (heri, hodie), while learners in the verb pretraining attended to six different cues (cogito, cogitas, cogitat, cogitavi, cogitavisti, cogitavit). Furthermore, the two adverb cues were each seen 18 times, while the six verb cues were each seen six times. Given that frequency can be an important contributor to cue learnability (MacWhinney, Reference MacWhinney, Kroll and De Groot2005; Ellis, Reference Ellis2006; Divjak, Reference Divjak2019), these different amounts of exposure may have influenced the findings in unintended ways.Footnote 2
One way to tease apart this potential confound could be to modify the adverb pretraining condition to more closely match the verb pretraining condition. Increasing the number of adverb cues to six could be one approach (e.g., adding two adverb cues for past time meaning and two adverb cues for present time meaning). Alternatively, the exposure frequency of the two adverb cues could be reduced so that exposure to each cue is balanced across pretraining conditions (e.g., participants are exposed to each cue six times). All other design and methodology aspects of Ellis and Sagarra (Reference Ellis and Sagarra2011) would remain the same.
The potential impact of this variable modification could help us to further understand the original study's finding, especially in terms of training effects on cue learnability. This modification has the potential to advance understanding about the extent to which greater exposure to a smaller number of cues can explain the larger learning effects in the adverb pretraining group. In other words, did this finding appear to suggest a frequency of exposure effect or a cue learnability effect?
Adding a delayed posttest
Ellis and Sagarra's (Reference Ellis and Sagarra2011) training lasted ‘less than one hour’, with four phases: two exposure phases, one receptive testing phase, and one production testing phase. The testing phases served to determine ‘if the acquisition of one set of cues is impaired if another is already known as a reliable indicator of event time’ (p. 594). Importantly, the results ‘demonstrate that salient adverbs were better learned than less salient verb inflections’ (p. 607). One important consideration is the extent to which this design provided a sufficiently robust conclusion on ‘learning’. In other words, did exposure to novel words and the testing of those words within one hour provide enough evidence that these words were ‘learned’. Adding a delayed posttest one-to-two weeks after the main study that included testing phases three and four has the potential to strengthen this conclusion and better understand claims that these cues were ‘learned’ and the role of the short-term training in that process. Indeed, previous research has indicated that the type of knowledge developed immediately following instruction might be qualitatively different from that at a delayed posttest. For example, Shintani (Reference Shintani2015) suggests that immediate posttests may be a reflection of what learners have ‘remembered’ from the instruction and knowledge may have deteriorated or changed at a delayed posttest, depending on its type (see also DeKeyser & Prieto Botana, Reference DeKeyser and Prieto Botana2015; Leow, Reference Leow2015a). Furthermore, a delayed posttest permits a better understanding about the durability of potential learning effects resulting from instruction, which is desirable for making claims about instructional effectiveness.
The original study: Tolentino and Tokowicz (Reference Tolentino and Tokowicz2014)
Tolentino and Tokowicz (Reference Tolentino and Tokowicz2014) investigated the extent to which explicit instruction can promote the learning of L2 target features that are (1) similar to the L1, (2) dissimilar to the L2, and (3) unique to the L2. English speakers with no knowledge of Swedish were assigned to one of three instructional treatments that exposed learners to the same Swedish sentences. The Salience group (n = 11) received sentences that contrasted with respect to the given morphosyntactic feature (e.g., definiteness) and ‘included bold blue highlighting of the morphemes at the point of agreement (e.g., En pojke springer [‘A boy runs’] vs. Pojken springer [‘The boy runs’])’ (p. 287). Participants were not told about the function of the highlighting. The Rule and Salience group (n = 11) received the same highlighted sentences plus additional explicit information of the rules underlying the L2 features. The Control group (n = 12) was exposed to sentences only (i.e., no contrasting, no highlighting, no explicit information). All sentences were presented aurally (via computer speakers) and visually on a computer screen. Participants were instructed to ‘pay attention to any grammatical patterns’ and had to repeat each sentence pair aloud once after hearing it’ (p. 292). In sum, this study investigated the extent to which drawing learners’ attention to aspects of the L2 exhibiting L1-L2 differences improved L2 grammatical learning.
The study tested learners’ knowledge of L2 Swedish grammar at Pretest using a grammaticality judgement test (GJT). Instruction was then provided at three points (day 1, day 3, day 18) and learning was assessed at three posttests (day 3, day 16, day 18). Following the UCM, Tolentino and Tokowicz predicted no between-group differences for L2 features similar to the L1, due to positive transfer. The Salience Group was expected to improve the most for L2 features dissimilar to the L1 because directing attention to forms through input enhancement would help learners overcome negative transfer and competition. Further, the absence of explicit information (EI) was predicted to prevent the potentially costly online maintenance of two conflicting grammatical rules. The Rule & Salience Group was expected to exhibit the highest GJT scores for features unique to the L2. EI (and enhanced input) were predicted to assist learning in overcoming feature novelty. Therefore, EI with enhanced input was predicted to support the development of features unique to the L2, but enhanced input only was predicted to support the development of L2 features dissimilar to the L1.
Results indicated that these predictions were broadly met. L2 features similar to the L1 were associated with higher accuracy overall and were less impacted by instruction method. Instruction that emphasized enhanced input (Salience Group) resulted in higher GJT scores for L2 features dissimilar to the L1. Instruction that provided grammatical rule explanations (Rule & Salience Group) resulted in higher GJT scores early in the learning of unique L2 features (posttest 1) and, to a lesser extent, in the learning of dissimilar features in later stages. Overall, dissimilar features were associated with relatively lower scores, particularly in the Control Group. Taken together, these results suggest that instruction that draws learners’ attention to L2 features that are dissimilar to the L1 can facilitate learning, and that these instructional methods offer few benefits for target features exhibiting L1-L2 similarities, largely due to positive transfer.
Approaches to close replication of Tolentino and Tokowicz (Reference Tolentino and Tokowicz2014)
Close replications of Tolentino and Tokowicz focus on two variable modifications: the GJT outcome measure and the training. Any replication would additionally seek to increase the sample size given that group numbers are small, which is particularly important for understanding instructional effectiveness (see Aberson, Reference Aberson2019). Other variable modifications could include changing the proficiency level of the L2 learners, especially since both studies investigated initial L2 learning. In addition, changing the L1 background of the learners would advance knowledge and understanding about the role of instruction in facilitating learning of target features that are dissimilar to the L1 and unique to the L2 (e.g., L1 speakers of inflectionally rich languages, including Russian, French, Spanish, Russian).
Modifying the GJT
The GJTs played an important role in this study because they were the only measure of L2 grammatical development. Claims about the effectiveness of each instructional treatment were therefore based on GJT performance by comparing performance at pretest and the three posttests. To this end, two aspects of the GJTs are important to note. First, pretest and posttest versions of GJT were different. The pretest version asked participants to judge stimuli on a four-point scale, but the posttest version asked participants to judge the stimuli as either grammatical or ungrammatical. In addition, pretest GJT stimuli were presented one sentence at a time, but posttest GJT sentence stimuli were presented word-by-word. While methodological variations are recommended to prevent test familiarity effects (see Mackey & Gass, Reference Mackey and Gass2016), these should be at the level of the stimuli rather than in terms of the test itself. Presenting the stimuli in different ways (full sentence presentation vs. word-by-word presentation) may have impacted the nature of learners’ judgements and confounded judgements on stimuli grammaticality with other attention and memory factors. Furthermore, the use of different judgement scales (i.e., four-point rating scale vs. grammatical/ungrammatical) makes comparisons between pretest and posttests difficult (see Spinner & Gass, Reference Spinner and Gass2019). Second, and arguably most importantly, the posttest GJTs provided corrective feedback (correct, incorrect) following participant responses, but no feedback was provided in the pretest GJT. This difference complicates the pretest-posttest comparisons and makes it difficult to draw conclusions about the role of the training in developing L2 learners’ grammatical accuracy. This is because no feedback was provided during the training and the only way learners could determine if they were improving was during the posttest GJTs. It is therefore likely that the posttest GJTs contributed to learners’ development.
One way to address these limitations would be to systematize the GJTs, for example, choosing either the pretest or posttest design and using that one design at each outcome phase. As in the original study, the stimuli in each test would still be varied to reduce test familiarity effects. It is also recommended to remove the corrective feedback from the posttest GJTs so that the effects of the training can be more reliably evaluated. As noted, the provision of feedback in the posttest GJTs makes it difficult to determine the contribution of the different instructional treatments on L2 development. In terms of how to approach these two limitations (i.e., GJT design and provision of feedback), it would be advisable to carry out two separate close replications. Changing both GJT design and feedback within the same study would make it difficult to draw comparisons with the original study: for example, it would be difficult to know if any differences with the original were due to changing the GJT design or removing the feedback (see Porte & McManus, Reference Porte and McManus2019).
Modifying the training
The training provided to all groups included the provision of Swedish sentences. Textual enhancement with and without EI in the Salience and Salience & Rule groups was additionally provided to enhance the saliency of L2 features different to the L1 and unique to the L2. As noted, participants saw and heard the Swedish sentences on screen, were instructed to look for patterns in the sentence, and then had to repeat aloud the sentences. Participants in the Salience and Salience & Rule groups were ‘not explicitly informed of the function of blue highlighted morphemes’ (p. 293). That is, learners had to work out the significance of the textual enhancement for themselves.
Tolentino and Tokowicz note in their discussion that the training ‘emphasized form over meaning processing, with the identification of morphosyntactic patterns likely being associated with higher performance, regardless of overall comprehension’ (p. 303). In other words, learners had to identify patterns, and ‘learning’ in this study was operationalized as the ability to detect patterns. Since no feedback was provided during the training, it is unlikely that learners’ pattern detection skills improved as a function of the training (but this is a testable claim). Furthermore, learners had no prior knowledge of Swedish and the input was Swedish sentences only (i.e., no images, no L1 translations) suggesting that learners maybe developed little-to-no understanding of the sentences. Even though this design contrasts with many contemporary approaches to L2 instruction that emphasize the development of form-meaning connections (e.g., VanPatten, Reference VanPatten2004; Ellis & Sagarra, Reference Ellis and Sagarra2011; Tyler, Reference Tyler2012; McManus & Marsden, Reference McManus and Marsden2017), a close replication could modify the training to develop comprehension abilities of the Swedish target features.
For example, learners were not informed about the function of the textual enhancement. A close replication study could inform learners about the function of the highlighted text (e.g., to indicate similarities/differences between L1 and L2). It could also be possible to involve some active processing of the Swedish sentences by requiring learners to attend to the meaning of the highlighted text. For instance, providing the sentence pair en pojke springer (‘A boy runs’) vs. Pojken springer (‘The boy runs’) and asking learners to indicate which sentence refers to ‘the boy’. Corrective feedback could be provided based on participant responses. Lastly, Swedish sentences could be contrasted with L1 sentences and highlighting to explicitly draw learners’ attention to grammatical features that are similar/dissimilar between languages (e.g., Pojken springer vs. The boy runs). These designs have the potential to encourage learners to attend to the meanings expressed by Swedish grammatical features, rather than attending to patterns only.
Conclusion
In this paper I have proposed a series of close replications that might help advance knowledge and understanding about the ways in which instruction designed to reduce the effects of crosslinguistic influence can improve L2 grammar learning. I focused on two important contributions to this evidence base and suggested ways in which changes to those study designs could enhance our understanding of the original findings. In addition to specific methodological modifications, I suggested that future research should re-run these studies with learners from different L1 backgrounds in order to draw meaningful comparisons between learners of different L1 backgrounds. In addition, making these changes could also motivate a series of conceptual replications to better understand the role of instruction in L2 learning. For example, making methodological changes to the training (as suggested for a close replication of Tolentino & Tokowicz, Reference Tolentino and Tokowicz2014) by drawing learners’ attention to the meaning implications of sentences could help advance knowledge and understanding about how different types of instruction (meaning-based vs. form-based) can address learnability problems resulting from crosslinguistic influence.
Kevin McManus is Associate Professor and Gil Watz Early Career Professor in the Department of Applied Linguistics at The Pennsylvania State University, USA. His research specializations include second language learning, psycholinguistics, and research methodology, especially replication research.
