INTRODUCTION
Second language (L2) learning outcomes vary considerably. The source of this variation appears to be multifaceted, stemming not only from individual differences (IDs) in cognitive abilities (e.g., Dörnyei, Reference Dörnyei2006) but also from differences in affective factors like personality and motivation (e.g., Grey, Williams, & Rebuschat, Reference Grey, Williams and Rebuschat2015; Masgoret & Gardner, Reference Masgoret and Gardner2003), language exposure conditions (e.g., Godfroid, Reference Godfroid2016; Granena, Reference Granena2013, Reference Granena2015; Morgan-Short, Sanz, Steinhauer, & Ullman, Reference Morgan-Short, Sanz, Steinhauer and Ullman2010; Rebuschat, Reference Rebuschat2008; Tagarelli, Borges Mota, & Rebuschat, Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015), and linguistic complexity (e.g., DeKeyser, Reference DeKeyser2005; Hawkins, Reference Hawkins2004). Although examining these factors in isolation can inform the nature of L2 acquisition, we can learn even more by investigating how multiple factors interact to mediate learning processes and outcomes. This article aims to contribute to our understanding of the sources of variation in L2 success by focusing on three major factors that have been shown to influence it—IDs in cognitive abilities, exposure conditions, and linguistic complexity—and the interactions among them.
Individual Differences
According to Skehan (Reference Skehan1998), IDs in aptitude could have major relevance for language development, inasmuch as this development requires a certain capacity to process and restructure input. Indeed, IDs such as aptitude, working memory (WM), personality, attitude, motivation, learning styles, and learning strategies have been shown to influence L2 processing and achievement (Dörnyei, Reference Dörnyei2006; Juffs & Harrington, Reference Juffs and Harrington2011).
While IDs may predict L2 success, the ways in which cognitive abilities interact with instructional methods cannot be ignored. In L2 research, language exposure is often manipulated on a continuum from explicit conditions, in which learners are provided with metalinguistic information (e.g., pedagogical rules) or instructed to look for rules or patterns, to implicit conditions, in which such information or instruction is not provided (DeKeyser, Reference DeKeyser1995). It is important to note, however, that although these conditions are often designed to respectively promote the development of explicit (conscious) or implicit (unconscious) knowledge, it is not necessarily the case that such knowledge is acquired. In fact, several studies have shown that implicit conditions can lead to the development of explicit knowledge, and vice versa (Godfroid, Reference Godfroid2016; Granena, Reference Granena2013; Grey, Williams, & Rebuschat, Reference Grey, Williams and Rebuschat2014; Rebuschat, Reference Rebuschat2008; Rebuschat & Williams, Reference Rebuschat and Williams2012; Rebuschat, Hamrick, Riestenberg, Sachs, & Ziegler, Reference Rebuschat, Hamrick, Riestenberg, Sachs and Ziegler2015; Tagarelli et al., Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015). There has been some disagreement as to whether more structured, explicit training conditions should amplify (Krashen, Reference Krashen1981) or level out (Skehan, Reference Skehan1989) the effects of IDs on learning, but there is emerging evidence that individual learners might not benefit equally from the same method of instruction (de Graaff, Reference De Graaff1997; Erlam, Reference Erlam2005; Robinson, Reference Robinson2002, Reference Robinson2005; Tagarelli et al., Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015).
This apparent interaction between IDs and exposure conditions could stem from differences between explicit and implicit processes. Traditionally, cognitive psychology suggests that implicit processes should be less susceptible to IDs than explicit processes (Reber, Reference Reber1993; Reber, Walkenfeld, & Hernstadt, Reference Reber, Walkenfeld and Hernstadt1991). However, more recent work has demonstrated IDs in implicit learning abilities (Granena, Reference Granena2013; Kaufman et al., Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010), and several studies have found relationships between IDs and learning outcomes in implicit exposure conditions (Grey et al., Reference Grey, Williams and Rebuschat2015; Hamrick, Reference Hamrick2015; Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter, & Wong, Reference Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter and Wong2014; Robinson, Reference Robinson2002; Williams, Reference Williams, van Hout, Hulk, Kuiken and Towell2003; Williams & Lovatt, Reference Williams and Lovatt2003). These inconsistencies likely arise from the fact that certain cognitive abilities may be associated with implicit or explicit processes and therefore have differential effects on learning under these conditions. Two cognitive domains that are particularly relevant to this issue are WM and procedural memory.
Working memory.
Working memory is the system of temporary storage and manipulation of information during complex cognitive activities such as language comprehension and learning (Baddeley, Reference Baddeley2010). There are several models of WM, but here we assume the multicomponent model proposed by Baddeley and colleagues (Baddeley, Reference Baddeley2010; Baddeley & Hitch, Reference Baddeley, Hitch and Bower1974). This consists of domain-specific storage components for auditory and visuospatial information, an episodic buffer that links these storage components to long-term memory, and the central executive, a domain-general, limited-capacity construct that is responsible for the allocation of attention.
Working memory is widely thought to play an important role in L2 learning (Hummel, Reference Hummel2009; Juffs & Harrington, Reference Juffs and Harrington2011; Linck, Osthus, Koeth, & Bunting, Reference Linck, Osthus, Koeth and Bunting2014; Martin & N. C. Ellis, Reference Martin and Ellis2012; Williams, Reference Williams, Gass and Mackey2012). According to Baars (Reference Baars1988), WM allows explicit deduction, hypothesis formation, analogical reasoning, prioritization, control, and decision making. Roehr (Reference Roehr2008) suggests a possible link between WM and explicit language exposure conditions, particularly when L2 learners need to retain metalinguistic information while comprehending and producing language (see also Clark & Squire, Reference Clark and Squire1998). Working memory has been shown to impact L2 classroom learning, which is often associated with explicit processes (Linck & Weiss, Reference Linck, Weiss, Sanz and Leow2011). Some research suggests that WM is strongly related to controlled, explicit language processing but also plays a moderate role in identifying tasks that can be delegated to more implicit, automatic processing, including less monitored speaking and listening (Erçetin & Alptekin, Reference Erçetin and Alptekin2013). Overall, it seems that WM may underlie both implicit and explicit language processes but is more closely tied to explicit ones.
Procedural memory.
The procedural memory system is one of several long-term memory systems in the brain (Squire & Knowlton, Reference Squire, Knowlton and Gazzaniga2000). It is likely involved in the acquisition, storage, and use of information that requires coordination of actions or sequences, like driving a car. In L1 and highly proficient L2 speakers, procedural memory is thought to underlie the learning, storage, and processing of grammar (Ullman, Reference Ullman, VanPatten and Williams2015). Because it relies on an implicit memory system, procedural learning ability may reliably predict L2 learning in implicit conditions, at least after learners have high levels of proficiency and/or exposure to the language (Morgan-Short et al., Reference Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter and Wong2014).
Linguistic Complexity
The relationship between IDs and training conditions may be mediated by a third factor: the complexity of the structures being learned. This article focuses on linguistic complexity, and more specifically structure complexity, or “the individual linguistic items, structures, or rules that make up the learner’s L2 system” (Bulté & Housen, Reference Bulté, Housen, Housen, Kuiken and Vedder2012, p. 25). Early research suggested that the complexity of linguistic structures interacts with the type of instruction. For instance, explicit instruction may be beneficial when learners are exposed to simple patterns (e.g., Krashen, Reference Krashen and Ellis1994), whereas patterns that are sufficiently complex are more likely to be acquired by an implicit learning mechanism (e.g., Reber, Reference Reber1993). Despite some evidence to support these predictions, the findings are inconsistent, with some suggesting that explicit conditions are more conducive to learning complex rules (see Housen, Reference Housen and Chapelle2014; Spada & Tomita, Reference Spada and Tomita2010). One plausible reason for this is the lack of a consensus in operationalizing rule complexity (Bulté & Housen, Reference Bulté, Housen, Housen, Kuiken and Vedder2012). Another possibility is that the learnability of simple and complex rules in implicit and explicit conditions may be mediated by a third variable, IDs. The current study systematically manipulates these three factors with the goal of understanding their individual and combined roles in L2 learning and processing.
The Current Study
In this study, participants were trained on an artificial language in either an incidental or instructed exposure condition. Importantly, this study did not assume that these learning contexts would lead to implicit and explicit learning or knowledge, respectively. The primary outcome measure in this study was performance on an aural untimed grammaticality judgment task (GJT), which is often categorized as an explicit knowledge measure (see R. Ellis, Reference Ellis2005). In order to obtain a more fine-grained measure of whether our exposure conditions resulted in conscious or unconscious knowledge, online and offline measures of awareness were included in the experimental design (Dienes & Scott, Reference Dienes and Scott2005; see Rebuschat, Reference Rebuschat2013, for a review). Participants also completed cognitive tasks designed to assess WM and procedural learning ability.
Artificial Language.
An artificial language is a model linguistic system composed of a small, novel lexicon and a few grammatical rules that are consistent with natural language rules (e.g., de Graaff, Reference De Graaff1997; DeKeyser, Reference DeKeyser1995; Yang & Givón, Reference Yang and Givón1997). Like natural languages, artificial languages can be spoken and understood, but their small size and novelty allow them to be learned relatively quickly, and in highly controlled settings. Semi-artificial languages, in contrast, often combine lexical information from the participants’ L1 and grammatical information from another language (e.g., Alanen, Reference Alanen and Schmidt1995; Grey, Williams, & Rebuschat, Reference Grey, Williams and Rebuschat2014; Hamrick, Reference Hamrick2014; Leung & Williams, Reference Leung and Williams2012; Rogers, Révész, & Rebuschat, Reference Rogers, Révész and Rebuschat2015; Williams, Reference Williams2005). This facilitates learning even more, as participants do not need to learn new vocabulary.
The stimuli for this experiment were generated from a semi-artificial language consisting of an English lexicon and German syntax (see also Rebuschat, Reference Rebuschat2008; Rebuschat & Williams, Reference Rebuschat and Williams2012; Tagarelli et al., Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015). The linguistic focus was on three verb-placement rules. These rules state that, depending on the type of clause (main vs. subordinate) and clause sequence (main-subordinate vs. subordinate-main), finite verbs have to be placed in either first, second, or final position. Each rule (1–3) is associated with a specific syntactic pattern, as illustrated subsequently. Example sentences are shown in Table 1.
-
1. Simple: The finite verb is placed in the second phrasal position of main clauses that are not preceded by a subordinate clause (V2 pattern).
-
2. Complex 1: The finite verb is placed in the final position of all subordinate clauses (V2-VF pattern).
-
3. Complex 2: When a subordinate clause precedes a main clause, the finite verb is placed in the first position of the main clause and the final position of the subordinate clause (VF-V1 pattern).
Examples of grammatical and ungrammatical (*) sentence patterns
Note. The sentences above are examples taken from the testing set. Ungrammatical sentences are matched to grammatical sentences based on features of verb placement, not complexity. This is why there are single-clause ungrammatical structures in the Complex 1 and Complex 2 groups.
These three rules and associated patterns were designed to vary according to complexity. One way to assess structural complexity is through T-units, which are defined as “one main clause with all subordinate clauses attached to it” (Hunt, Reference Hunt1965, p. 20). Linguistic complexity can thus be defined as the number of clauses per T-unit (e.g., Larsen-Freeman, Reference Larsen-Freeman, Seliger and Long1983). Based on this definition, the V2 pattern is the least complex construction, as it only contains one clause. The V2-VF and the VF-V1 patterns are more complex because they each contain two clauses. This assessment of complexity is also upheld in terms of cognitive complexity, or difficulty. Single-clause sentences are easier to process than those with multiple clauses (Bygate, Reference Bygate1999; Lord, Reference Lord, Bybee and Noonan2002), and both L1 and L2 development are characterized by an increased production of sentences with subordinate clauses (Brandt, Diessel, & Tomasello, Reference Brandt, Diessel and Tomasello2008; Scott & Tucker, Reference Scott and Tucker1974; Gaies, Reference Gaies1976; Larsen-Freeman, Reference Larsen-Freeman1978), suggesting that clause length is related to difficulty. Moreover, although the V2-VF and VF-V1 patterns are similar in terms of linguistic T-unit complexity, it could be argued that the VF-V1 pattern is cognitively more complex or difficult because the subordinate clause precedes the main clause, which may further increase the processing load (Jarvella & Herman, Reference Jarvella and Herman1972).
Research Questions and Hypotheses.
This study aimed to investigate whether WM and procedural learning ability would predict L2 learning outcomes, and to determine whether their predictive ability would vary according to exposure condition and structural complexity. We set out to address three research questions:
-
1. What is the relationship between exposure condition and syntactic complexity? In other words, does exposure condition (incidental vs. instructed) differentially affect the learnability of syntactic structure?
-
2. What is the relationship between the cognitive factors under investigation (WM and procedural learning) and learning outcomes in the incidental and instructed exposure conditions?
-
3. Is there an interaction between cognitive abilities, exposure conditions, and syntactic complexity in predicting L2 outcomes?
Regarding Research Question 1, we expected overall performance to be best on the simplest structure (V2), followed by the complex structures (V2-VF and VF-V1). We might expect poorer performance on Complex 2 (VF-V1) relative to Complex 1 (V2-VF) sentences, because the Complex 2 structure may be more cognitively demanding, as outlined previously. Considering exposure conditions, we hypothesized that the learning of simple rules would be better in the instructed condition and the learning of complex rules would be better in the incidental condition (Reber, Reference Reber1993).
Regarding Research Question 2, we hypothesized that WM span would correlate with performance in the instructed condition, but not in the incidental condition, given previous research suggesting that it is more strongly linked to explicit language processes (Baars, Reference Baars1988; Clark & Squire, Reference Clark and Squire1998; Erçetin & Alptekin, Reference Erçetin and Alptekin2013). We expected the opposite effect for implicit/procedural learning abilities (Morgan-Short et al., Reference Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter and Wong2014).
Our third research question is largely exploratory, and thus we had no specific predictions regarding the three-way interaction between IDs, exposure conditions, and complexity.
METHOD
Participants
Fifty-one native speakers of English with no background in German (or any other V2 language) were randomly assigned to an incidental exposure condition (n = 25, 19 female, M age = 19.30) or to an instructed exposure condition (n = 25, 20 female, M age = 19.32). One additional participant was excluded for failing to follow task instructions. All participants were exposed to the same artificial language stimuli. However, participants in the incidental group were not informed of the linguistic target, nor that there would be a testing phase. In contrast, participants in the instructed group were explicitly taught the target rule system before being exposed to the language. Data were collected in the United States and in the U.K. (see Section S1 in the Supplementary Materials for detailed information about participant groups). Participants received either course credit or monetary compensation.
Materials and Procedure
Participants first completed a language learning session, followed by cognitive tests on a separate day. Each session lasted approximately one hour.
Session 1: Artificial Language Training and Testing.
The language session consisted of training and a GJT. The tasks and materials were exactly the same for both conditions, except that the instructed group was provided with rules before the actual training phase.
Training set.
The training set consisted of 120 sentences, with 40 sentences for each syntactic pattern. For each pattern, half the sentences were semantically plausible (see Table 1), and the other half semantically implausible (e.g., “Rose abandoned in the evening her cats on planet Venus”). Plausible and implausible items were designed so that participants would have to process the entire auditory string before judging its plausibility.
Testing set.
The testing set consisted of 60 new sentences, 30 grammatical and 30 ungrammatical. All sentences were semantically plausible. There were six ungrammatical sentence types; these were matched to specific grammatical patterns for the purposes of d-prime analyses. The Simple V2 pattern was matched to *V3 and *V4, two ungrammatical sentence types that also only contained a single clause. The Complex 1 V2-VF pattern was matched to the *VF pattern, which had a verb in the final position in the main clause, and to the *V1-VF pattern, which had correct verb placement in the subordinate clause, but not in the main clause. Finally, the Complex 2 VF-V1 pattern was matched to the *V1 pattern, which had a verb in the initial position in the main clause, and to the *VF-V2 pattern, which again had a correct subordinate clause but incorrect main clause (see Table 1).
Training task.
Participants in the incidental group were told that they were being tested on their ability to understand the meaning of sentences with scrambled word order, and they were not told that there would be a test on word order. For each trial, they were asked to listen to a sentence, judge the plausibility of the sentence, and then repeat each sentence after a delayed prompt (1,500 ms). The focus of their task was therefore primarily on meaning. There were four practice trials and 120 randomly presented training sentences.
Participants in the instructed group were told that there was a complex rule system that determined the word order of the artificial language. They were then presented with each of the verb-placement rules and were asked to write down two sentences that followed each rule. After this, they engaged in the same meaning-focused task as the incidental group (plausibility judgments, followed by sentence repetitions), but they were instructed to think about the verb-placement rules as they completed the task. The focus of their task was therefore on both form and meaning.
Grammaticality judgment task.
After training, participants completed an untimed GJT. Before the test, participants in the incidental group were told (and participants in the instructed group were reminded) that the word order of the sentences presented during training followed a rule system, and that they should judge the new sentences based on the sentences they had heard in training. After four practice trials, all participants listened to the 60 test sentences, which were presented randomly. For each sentence, participants were asked to judge the sentence’s grammaticality, indicate how confident they were in the accuracy of their judgment, and report the source of their judgment. These confidence ratings and source attributions served as subjective measures of awareness (Dienes & Scott, Reference Dienes and Scott2005; Rebuschat, Reference Rebuschat2013).
Participants indicated their levels of confidence by selecting one of four response options: not at all confident, somewhat confident, quite confident, and extremely confident. This was explained to the participants as a continuum of confidence from not at all to extremely/100 percent. There were also four response options for source attributions: guess, intuition, memory, and rule (see Section S2 for detailed definitions). All responses were entered on a serial response pad.
After completing the test phase, participants filled out a debriefing questionnaire, providing information on their levels of awareness regarding the rule system. The questionnaire also included biographical data, such as age, gender, occupation, major field of study, and language background.
Session 2: Cognitive Tasks.
During the second session, participants completed the RSpan and either the SRT or the ASRT task. Footnote 1 Task order was randomized for each participant.
Reading span task.
The reading span task (RSpan; Daneman & Carpenter, Reference Daneman and Carpenter1980) is a complex WM span task, meaning that it measures both the storage and processing components of WM. The RSpan task used in this study was adapted from Unsworth, Heitz, Schrock, and Engle (Reference Unsworth, Heitz, Schrock and Engle2005) and was presented in Microsoft PowerPoint. In this task, participants saw a sentence appear on the screen. They were asked to read the sentence out loud, judge its plausibility, and then read a capitalized letter out loud (speaking out loud was intended to prevent subvocal rehearsal of previous letters in a set). Between two and five sentences were presented in a set, and set sizes were intermixed to prevent strategies (e.g., creating mnemonics of specific lengths based on set size). At the end of a set, participants were prompted to write down all the letters in the sequence in which they had been presented, leaving a blank space for letters they could not remember. There were 3 practice sets and 12 test sets (3 per set size).
Following Unsworth et al. (Reference Unsworth, Heitz, Schrock and Engle2005), the RSpan score was calculated as the sum of all set sizes that were perfectly recalled. The highest possible score was 42. All participants responded correctly to at least 80% of the plausibility judgments, suggesting that they were both storing letters and processing sentences.
(Alternating) serial reaction time task.
Procedural learning ability was measured by serial reaction time (SRT) and alternating serial reaction time (ASRT) tasks. Prior research suggests that SRT tasks provide the best measure of implicit/procedural learning ability (Howard & Howard, Reference Howard and Howard1997; Kaufman et al., Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010; Shanks, Reference Shanks, Lamberts and Goldstone2005). Explicit learning is minimized in these tasks because participants are not given instructions that direct their attention to underlying rules, and there is no separate testing phase, so participants are never informed that rules exist (e.g., Kaufman et al., Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010).
In both tasks, participants saw four squares or circles arranged horizontally on a screen. In each trial, an object appeared in one of the locations, and the participant pressed a button corresponding to the location of the object on a serial response pad. Once the participant pressed the button, the object disappeared and then reappeared in a new position. The SRT task (Nissen & Bullemer, Reference Nissen and Bullemer1987) consisted of six blocks with 60 trials per block. In the first and sixth blocks, the location of the object was random. In the other blocks, the location followed a 10-trial sequenced pattern. In this task, decreased reaction time over the course of the sequence blocks, followed by an increase in the final block, is taken as evidence that participants have learned the sequence and are not simply getting faster at the button-pressing task. The ID score is the difference in average reaction time between the fifth and sixth blocks. A larger difference indicates a greater learning effect. The SRT task was run on E-Prime Version 2.0 on a PC computer.
In the ASRT task, random and sequence trials are interspersed within blocks, rather than allocated to separate blocks; every other trial follows a pattern, and the alternating trials are random (see Howard & Howard, Reference Howard and Howard1997). The task consisted of 20 blocks of 85 trials each. It was run using E-Prime Version 1.2 on a PC computer. Following Kaufman et al. (Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010), participants were awarded one point for every block in which a learning effect was observed (i.e., faster reaction time for sequence than for random trials). The total score for the participant is the sum of scores across the last 18 blocks, for a maximum score of 18.
The ASRT task has several advantages over the SRT task; these are addressed in detail by Howard and Howard (Reference Howard and Howard1997). We were unable to run the ASRT in the UK for technical reasons. However, although perhaps not ideal, SRT tasks have been successfully used to measure implicit learning (e.g., Kaufman et al., Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010) and serve as an acceptable alternative.
RESULTS
This study aimed to examine the interaction among exposure conditions, syntactic complexity, and individual variation in cognitive abilities. We address each of these in turn, but first we need to establish (a) whether exposure to the artificial language resulted in learning, (b) whether the knowledge acquired was implicit and/or explicit, and (c) whether these outcomes varied according to exposure condition (incidental vs. instructed). This will allow us to place the results of this study within the framework of SLA broadly, and more specifically in the discussion of implicit and explicit processes in L2 acquisition.
Performance on the Grammaticality Judgment Task
Learning was assessed by accuracy and d-prime (d') scores on the GJT. Unlike raw accuracy measures, d' takes response bias into account in tasks with binary response options. It is calculated based on the hit and false alarm rates and provides an index of how sensitive a particular subject is to the difference between grammatical and ungrammatical items.
Accuracy and d' scores for both groups are shown in Table 2. D' is plotted in Figure 1. Although d' scores were generally low, both the incidental group, t(24) = 2.951, p = .007, and the instructed group, t(24) = 4.918, p < .001, performed significantly above chance, indicating a clear learning effect in both exposure conditions. An independent-samples t test showed that the instructed group significantly outperformed the incidental group in terms of overall d' scores, t(48) = 3.07, p = .004.
Accuracy (%) and d' scores on the GJT for each group overall and according to complexity
Significance from chance: + p = .07, *p < .05, **p < .01, ***p < .005, and ****p < .001.
Boxplots of d-prime performance for all sentences, and for each sentence group. Error bars represent standard deviation. Dark grey boxes = incidental condition; light grey boxes = instructed condition.
Subjective Measures of Awareness
For reasons of space, only the results from the source attributions are discussed here (see Section S3 for tables and additional information about the subjective measures of awareness). Table S3.1 summarizes overall accuracy and proportions across the four source attribution categories for both groups. Participants in the incidental group performed significantly above chance only when basing grammaticality judgments on intuition. When attributing grammaticality decisions to memory or rule knowledge, their performance was indistinguishable from chance. Interestingly, the opposite pattern was observed in the instructed group. Here, participants performed significantly above chance only when basing grammaticality judgments on rule knowledge. A chi-square analysis revealed a highly significant association between training condition and source attribution, χ2(1) = 143.51, p < .001, in that the incidental group relied more on the implicit categories (guess and intuition) than the instructed group, and the instructed group relied more on the explicit categories (memory and rule knowledge) than the incidental group.
Exposure Conditions and Linguistic Complexity
To answer Research Question 1, we looked at performance in the two exposure conditions across the three syntactic patterns (see Table 2 and Figure 1). Again, d' scores are relatively low, with the maximum score being 1.00 (corresponding to about 75% accuracy) for Simple sentences in the instructed condition. Nevertheless, there are still evident differences in accuracy according to syntactic complexity. Participants in both conditions performed above chance on the Simple structures, and those in the instructed condition also performed above chance on both types of complex sentences. The d' scores for participants in the incidental group were close to zero for the Complex 1 sentences but approached significance for the Complex 2 sentences. Both groups show a trend of performing best on Simple structures, followed by Complex 2 and then Complex 1 sentences. The difference in d' scores between Simple and Complex 1 sentences was significant for both groups (p < .03).
A 2 × 3 mixed-effects repeated-measures ANOVA with Condition (Incidental vs. Instructed) and Complexity (Simple, Complex 1, and Complex 2) showed that there were main effects of Condition, F(1, 48) = 8.592, p = .005, partial η2 = .152, observed power = .819, and of Complexity, F(2, 96) = 6.310, p = .003, partial η2 = .116, observed power = .889. There was no Complexity × Condition interaction, F < 1, which corroborates the similar trends in both groups. Bonferroni corrected post-hoc comparisons showed significantly better d' scores on Simple sentences than Complex 1 sentences, p < .001, but no other significant differences among sentence types. Learners in the instructed condition performed significantly better on Simple structures than those in the incidental condition, p = .007, and marginally better on Complex 1 sentences, p = .039 (corrected α = .017). There were no differences between conditions on the Complex 2 sentences, but note the very high variance in both conditions.
Cognitive Measures
To answer Research Question 2, we examined performance on the cognitive tests, as well as the relationship between these variables and learning outcomes in the two exposure conditions. Scores from the ASRT and SRT tasks were converted into z scores (SRTz) and were combined into one variable. Average performance on the RSpan and (A)SRT tasks (and the results of between-groups comparisons) is shown in Table S4.1.
Pearson’s correlations were performed separately by group for each of the cognitive measures and d' scores on the GJT. There was a strong significant negative correlation between the SRTz and d' scores for the incidental group, r = −.586, p = .003, and no correlation between RSpan and d' scores. There were no correlations between the cognitive variables and overall GJT performance for the instructed group (see Table S4.2).
Condition, Linguistic Complexity, and Cognitive Abilities
Research Question 3 addressed the three-way relationship among exposure condition, linguistic complexity, and cognitive abilities in modulating learning outcomes. To examine this, we first ran correlations between performance on the ID tests and on the GJT for each sentence type, separately for each exposure condition. This revealed that the correlation between the SRT task and the GJT in the incidental condition was mostly driven by the Complex 2 sentences, r = −.543, p = .007. There was also a medium-to-large positive correlation between the RSpan and d' scores in this condition, r = .431, p = .035. Notably, the RSpan and SRTz scores in the incidental condition were strongly negatively correlated, r = −.501, p = .018. These effect sizes are all rather large, though none withstand the correction for multiple comparisons, α = .003. There were no significant correlations between RSpan or SRTz and d' scores for any sentence groups in the instructed condition.
To determine whether exposure condition, WM, or procedural learning ability predicted performance on the GJT for each sentence group, we entered these factors into a stepwise regression (see Table S5.1). Based on the results reported previously and other research showing a relationship between the explicitness of training conditions and learning outcomes (Godfroid, Reference Godfroid2016; Granena, Reference Granena2013, Reference Granena2015; Morgan-Short, Finger, Grey, & Ullman, Reference Morgan-Short, Finger, Grey and Ullman2012; Morgan-Short et al., Reference Morgan-Short, Sanz, Steinhauer and Ullman2010; Morgan-Short, Steinhauer, Sanz, & Ullman, Reference Morgan-Short, Finger, Grey and Ullman2012; Rebuschat, Reference Rebuschat2008; Tagarelli et al., Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015), we entered exposure condition in the first step of the regression, followed by the cognitive variables in the second step.
In the first model, exposure condition accounted for 14.3% and 9.9% of the variance for Simple and Complex 1 sentences, respectively, and was a significant predictor for both, p < .05. It only accounted for 4.1% of the variance for Complex 2 sentences and was not a significant predictor, p = .181. The addition of the cognitive variables for Simple and Complex 1 accounted for slightly more variance, but neither was a significant predictor. For the Complex 2 sentences, the addition of the cognitive variables accounted for an additional 10% of the variance and significantly improved the model. Specifically, performance on the SRT task was a significant predictor, p < .05. All effect sizes were quite small.
DISCUSSION
This study confirmed that adult learners can acquire L2 syntax under both instructed and incidental exposure conditions, while processing sentences for meaning, and without the benefit of feedback, thus replicating findings from Rebuschat and colleagues (Rebuschat, Reference Rebuschat2008, Experiment 3; Rebuschat & Williams, Experiment 2; Tagarelli et al., Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015). We also demonstrated that the addition of metalinguistic rule presentation prior to meaning-oriented exposure results in a greater learning effect, consistent with previous studies (e.g., de Graaff, Reference De Graaff1997; DeKeyser, Reference DeKeyser1995; Norris & Ortega, Reference Norris and Ortega2000; Spada & Tomita, Reference Spada and Tomita2010). Our study also sheds light on the effect of different explicit exposure conditions on L2 acquisition. A comparison between the current study and Tagarelli et al. (Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015) demonstrates that providing a brief metalinguistic treatment before meaning-oriented exposure resulted in similar gains to asking participants to consciously focus on form (discovering word-order rules) and disregard meaning (learning outcomes were statistically comparable, t < 1).
Our analysis of the subjective measures of awareness demonstrated that our exposure conditions resulted in the kind of knowledge they were designed to promote. Learners in the incidental condition relied more on the implicit categories (guess and intuition) and performed best when basing their judgments on intuition, and learners in the instructed condition relied more on the explicit categories (memory and rule) and performed best when basing their judgments on rule knowledge. In other words, the incidental group appears to have mainly acquired implicit knowledge, whereas the instructed group mainly acquired explicit knowledge. These results contrast with previous research showing that subjects often acquire both types of knowledge, with the relative amount varying according to learning context (e.g., Godfroid, Reference Godfroid2016; Granena, Reference Granena2013; Grey et al., Reference Grey, Williams and Rebuschat2014; Rebuschat, Reference Rebuschat2008; Rebuschat et al., Reference Rebuschat, Hamrick, Riestenberg, Sachs and Ziegler2015; Reference Rogers, Révész and RebuschatRogers et al., in press; Tagarelli et al., Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015).
Research Question 1: Exposure Conditions and Syntactic Complexity
We found no relationship between exposure condition and syntactic complexity. Although the instructed group performed better overall, this was true for all sentence types, and the pattern of performance according to complexity was the same for both groups. Therefore, our hypothesis that linguistically simple rules would be easier to learn in the instructed condition and that complex rules would be easier to learn in the incidental condition was not supported—simple rules were learned better in both conditions. This finding contradicts previous research suggesting that explicit instruction may be more beneficial for learning simple structures, given that these may be more available to conscious processing (DeKeyser, Reference DeKeyser1995; Reber, Reference Reber1993), as well as other research showing an advantage of explicit instruction on the learning of complex structures (de Graaff, Reference De Graaff1997; R. Ellis, Reference Ellis and Mackey2007; Housen, Pierrard, & Van Daele, Reference Housen, Pierrard, Van Daele, Housen and Pierrard2005). There is some evidence, however, that explicit instruction may result in similar outcomes for simple and complex structures (de Graaff, Reference De Graaff1997). In addition, a significant difference between the Simple and Complex 1 sentences in the incidental condition contrasts with previous research suggesting that complexity may not be an important factor in determining learning outcomes in more implicit conditions (Krashen, Reference Krashen and Ellis1994; Reber, Reference Reber1993).
Further, contrary to our predictions, there was no significant difference in performance between the two complex sentence types, even though the Complex 2 pattern might be more cognitively taxing than the Complex 1 pattern. In fact, performance was slightly, though not significantly, better on Complex 2 sentences. Notably, responses on the debriefing questionnaire suggest that the VF-V1 pattern was particularly salient for learners, so benefits due to saliency (see van den Bos & Poletiek, Reference Van den Bos and Poletiek2008) may have overcome any limitations on learning due to the additional cognitive complexity of the VF-V1 pattern.
Research Question 2: Cognitive Abilities in Incidental and Instructed Learning
Although we found correlations between cognitive abilities and learning outcomes, they accounted for little of the variance in overall performance. This is surprising, as previous research has shown a link between both WM (hypothesized to be more important in explicit conditions; e.g., Linck et al., Reference Linck, Osthus, Koeth and Bunting2014) and procedural learning (believed to play a more important role in implicit conditions; e.g., Morgan-Short et al., Reference Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter and Wong2014) in L2 acquisition, and an interaction with implicit and explicit exposure conditions (Erlam, Reference Erlam2005; Robinson, Reference Robinson2002; Tagarelli et al., Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015). Although there were some between-group differences, they were in unexpected directions. The only significant correlation over all sentence types was between performance on the procedural learning task and on the GJT in the incidental group, and this was a negative correlation. Whereas some previous research suggests that cognitive IDs should be apparent under explicit, but not implicit, exposure conditions (de Graaff, Reference De Graaff1997; Gebauer & Mackintosh, Reference Gebauer and Mackintosh2007; Reber et al., Reference Reber, Walkenfeld and Hernstadt1991), it is not particularly surprising that there were no correlations between learning outcomes and cognitive IDs in our explicit condition. Other studies have found no relationship between overall performance and IDs in explicit conditions (Grey et al., Reference Grey, Williams and Rebuschat2015), even when the ID in question was WM, which has been linked to explicit processes (e.g., Tagarelli et al., Reference Tagarelli, Borges Mota, Rebuschat, Zhisheng, Borges Mota and McNeill2015). The provision of metalinguistic information may actually be particularly effective in leveling out any influence of cognitive IDs (e.g., Erlam, Reference Erlam2005). The direction of the correlation between GJT performance and SRT scores in the incidental group is somewhat surprising, given that our measures of awareness demonstrate that these learners were relying on implicit knowledge in the GJT, and that the SRT is a procedural/implicit learning task. We therefore expected these to be positively correlated, as procedural learning abilities have been shown to be positively related to L2 outcomes in implicit conditions, but only after more exposure and at a higher proficiency (Morgan-Short et al., Reference Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter and Wong2014).
It is worth noting that the measure of learning in this study is an untimed GJT. Although oral untimed GJTs have been used to measure implicit knowledge (e.g., Bialystok, Reference Bialystok1982), R. Ellis (Reference Ellis2005) suggests that these tasks may indirectly tap explicit knowledge, as the untimed nature of the task allows learners to (a) attend to the form of the sentence, (b) notice if there was a mistake, and (c) reflect on the mistake. Performance on this task might therefore be less related to procedural learning abilities than expected, even if participants were exposed in an implicit condition. Additionally, the use of a semi-artificial language in the present study may have shifted language processing toward more explicit mechanisms (cf. Godfroid, Reference Godfroid2016). Participants in the present study were learning a foreign syntax with an L1 lexicon, which likely made processing for meaning somewhat automatic, thus freeing up resources to direct more attention to form.
Research Question 3: Exposure Conditions, Complexity, and Individual Differences
Our third research question was largely exploratory, as little is known about the three-way relationship between exposure condition, complexity, and cognitive IDs. Our findings suggest that we still have a lot to learn. Exposure condition was a significant predictor in regression models for both Simple and Complex 1 sentences and accounted for more of the variance for Simple sentences. This suggests that performance on both simple and complex structures is influenced by the type of exposure, but that this influence is greater for simple structures. These were, indeed, the only structures for which learners in the instructed condition significantly outperformed those in the incidental condition. Nevertheless, exposure condition explained very little of the variance (10%–14%). When WM and procedural learning ability were added to the regression model, more variance was accounted for, but the difference was only marginal, and exposure condition was still the only significant (or trending) predictor in these models. This suggests that IDs had very little, if any, influence on the learning of the Simple and Complex 1 patterns.
Interestingly, exposure condition was not a significant predictor in the regression model for the Complex 2 sentences, accounting for only about 4% of the variance. However, adding IDs to this model, particularly the procedural learning measure (SRT), accounted for significantly more variance, though it was still low overall (14%). This suggests that for this particular sentence pattern, the type of training that learners received was not important, but procedural learning abilities were. Correlational analyses showed that procedural learning abilities did not influence learning in the instructed group, but in the incidental group, learners with better procedural memory abilities performed worse. Though not a significant predictor in the regression model, RSpan scores were significantly positively correlated with d' scores on these structures in the incidental condition.
At first glance, these results are quite surprising, as both implicit learning and performance on complex sentences might be expected to be positively related to procedural learning abilities, and not necessarily related to WM abilities (but see Erçetin & Alptekin, Reference Erçetin and Alptekin2013). We begin by discussing the findings related to WM. Results from the debriefing questionnaires suggest that the Complex 2 grammatical sentences, VF-V1, were very salient to learners, as they were able to provide more examples of these sentences than V2-VF sentences, and almost the same number as V2 sentences. Good WM might help participants learn more salient rules in a less structured condition, while less salient rules go by unnoticed and are therefore more difficult to acquire (N. C. Ellis, Reference Ellis2006). This ability to attend to salient rules may be a driving force in incidental learning (van den Bos & Poletiek, Reference Van den Bos and Poletiek2008) and may be more important when L2 learning is more naturalistic, as opposed to classroom based (DeKeyser, Reference DeKeyser2005), which could have implications for our incidental condition. Indeed, knowledge of such salient rules is more likely to be explicit in nature, and so this finding is consistent with literature suggesting that high WM span is beneficial for explicit language processes.
Regarding the relationship between the SRT and learning outcomes for Complex 2 sentences in the incidental condition, we can refer back to our more general discussion of this relationship across all sentence types. The nature of the untimed GJT (R. Ellis, Reference Ellis2005), and perhaps of the semi-artificial language, as well as the saliency of the Complex 2 forms, may have favored more explicit processing of this pattern. Perhaps learners with good procedural memory abilities used this system to extract more complex structural information about these forms (see Ullman, Reference Ullman, VanPatten and Williams2015), whereas the learners with poorer procedural memory abilities instead relied on the more salient, explicit cues, which was ultimately more successful. It should be noted, though, that analyses of source attributions according to sentence type suggest that learners in the incidental condition were highly accurate when relying on implicit knowledge for the Complex 2 sentences (see Table S4.2). These results therefore may not be so clear and may also call into question the reliability of self-reported awareness measures.
Finally, although we observed effects of all three variables in our analyses, regression models demonstrate that exposure condition and IDs only predict a very small amount of the variance in learning outcomes for all sentence types. This once more suggests that there is much more to the puzzle that was not investigated in this study.
LIMITATIONS
This study is not without limitations. First, there was a significant correlation between RSpan and SRTz scores in the incidental group, but not in the instructed group. The variance inflation factors for the regression analyses are all close to 1, indicating that the estimation of the regression coefficients is not overestimated due to multicollinearity. Still, this finding reveals a difference between the groups, so results pertaining to IDs should be interpreted with caution. Although we have a reasonable number of participants in each group, correlational analyses, particularly regressions, would benefit from larger sample sizes, in which between-group differences such as these should not arise by chance.
Another potential limitation of this study is our operationalization of linguistic complexity. First, we defined structural complexity in terms of T-units, which is only one of many possible ways to do so, and perhaps not the most sensitive. Second, we used cognitive complexity, or difficulty, to distinguish the Complex 1 and Complex 2 sentences. Bulté and Housen (Reference Bulté, Housen, Housen, Kuiken and Vedder2012) highlight the subjectivity involved in assessing the difficulty of a particular language feature, as cognitive complexity is relative, or learner dependent. Results from our debriefing questionnaires indeed suggested that the Complex 2 sentences were quite salient to learners and possibly were not as difficult as we anticipated.
Some of our results call into question the reliability of using subjective self-reported awareness measures when trying to assess the nature of L2 knowledge (see also Rebuschat, Reference Rebuschat2013). For example, source attributions indicated that learners in the incidental condition were relying on implicit knowledge during the GJT, but the relationship between cognitive IDs and learning outcomes in this group suggests that explicit processes were at work.
Finally, using accuracy on the GJT as a measure of learning outcomes may have biased our results toward explicit processes. Although it was beyond the scope of the current article, we also collected data—including elicited imitations, accuracy and reaction time on the plausibility judgment task, and reaction time on the GJT—that may speak more to implicit processes during training and testing. Further analyses of these data should help tease apart the role of implicit and explicit processes in L2 learning.
CONCLUSIONS
Overall, this study revealed an interesting relationship among exposure condition, IDs, and syntactic complexity. Although learning effects were observed in both conditions, our findings clearly demonstrate the advantage of providing metalinguistic instruction prior to meaning-focused exposure. Subjective measures of awareness demonstrated that learners in the incidental group seemed to rely on implicit knowledge, whereas learners in the instructed group relied on explicit knowledge. Performance across three syntactic patterns of varying complexity showed that whereas explicit instruction led to overall better performance on all structures, both groups showed similar patterns regarding the structures on which they performed best (Simple, followed by Complex 2, and then Complex 1). Finally, analyses of IDs demonstrated the importance of considering multiple measures of cognitive abilities when assessing the relationship between IDs and exposure conditions, as abilities may relate differently to learning within and across conditions. This study goes a step further by examining the extent to which this relationship is mediated by linguistic complexity. There are limitations that necessitate a cautious interpretation of these results, but overall this study serves as a timely reminder of the multiplicity of factors that affect the process of L2 learning.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S0272263116000036.
