Implicit learning— succinctly defined as acquisition of knowledge without the intention to learn and without the awareness of what has been learned— is fundamental to human cognition (e.g., Berry & Dienes, Reference Berry and Dienes1993; Cleeremans, Destrebecqz, & Boyer, Reference Cleeremans, Destrebecqz and Boyer1998; Perruchet, Reference Perruchet, Byrne and Roediger2008; Perruchet & Pacton, Reference Perruchet and Pacton2006; Reber, Reference Reber1967, Reference Reber1993; Shanks, Reference Shanks, Lambert and Goldstone2005). A prime example of implicit learning is early child language acquisition. Before starting formal school education (at approx. age six), children acquire complex knowledge of the structure of their language unintentionally and implicitly without being able to describe that knowledge (see Hulstijn, Reference Hulstijn and Rebuschat2015 for discussion; but see Lichtman, Reference Lichtman2016 for a different view). Adult acquisition of second language (L2) is a different matter and usually involves processes of explicit learning (see, e.g., N. C. Ellis, Reference Ellis2005, Reference Ellis, Sanz and Leow2011). Explicit learning proceeds intentionally, with conscious awareness of what is being learned and involves hypothesis-driven search for structure. L2 acquisition research has a long-standing interest in investigating the role of attention and awareness in L2 acquisition (e.g., Robinson, Mackey, Gass, & Schmidt, Reference Robinson, Mackey, Gass, Schmidt, Gass and Mackey2012) and, relatedly, the role of implicit and explicit learning/knowledge in L2 acquisition (e.g., DeKeyser, Reference DeKeyser, Doughty and Long2003; Ellis, Reference Ellis1994; Hulstijn & Ellis, Reference Hulstijn2005; Krashen, Reference Krashen1981; Schmidt, Reference Schmidt1990; Williams, Reference Williams2005, Reference Williams, Ritchie and Bhatia2009; see also Andringa & Rebuschat, 2015b, for a recent special issue on this topic; Rebuschat, Reference Rebuschat2015, for a recent edited volume). This interest can be traced back to Krashen’s views on L2 acquisition, in particular his distinction between language acquisition, an incidental process that leads to tacit (implicit) linguistic knowledge, and language learning, a conscious and intentional process that leads to conscious, metalinguistic (explicit) knowledge with no interface between the two (cf. Krashen, Reference Krashen1981 and elsewhere for the “acquisition-learning hypothesis” as a part of his “monitor model”). Krashen claimed that L2 development is primarily contingent on implicit learning that takes place because of processing input for meaning, in which explicit knowledge (metalinguistic awareness) plays an insignificant role. This led Krashen to conclude that language pedagogy should focus on allowing L2 learners to acquire language implicitly, as opposed to traditional explicit language learning. Although Krashen’s Monitor Model has been criticized for the lack of precision in the definition of its central constructs, rendering the model’s hypotheses pertaining to these constructs untestable (cf., e.g., Gregg, Reference Gregg1984; Hulstijn & Hulstijn, Reference Hulstijn and Hulstijn1984; McLaughlin, Reference McLaughlin1987), it has also been responsible for the increased interest in the role of implicit and explicit learning/knowledge in L2 acquisition.
Over the course of last three decades, L2 acquisition research has amassed an impressive body of work on implicit and explicit learning/knowledge that has centered around three issues (see Andringa & Rebuschat Reference Andringa and Rebuschat2015a for a recent overview). The first issue concerns the implicit/explicit interface, that is the question whether and to what extent explicit knowledge can enhance the development of implicit L2 knowledge (for an overview of several theoretical positions on this issue, cf. Ellis, Reference Ellis, Sanz and Leow2011). Focusing on practical implications of the interface issue, a great number of studies have investigated the effects of explicit and implicit teaching conditions on the acquisition of L2 knowledge (cf. Norris & Ortega, Reference Norris and Ortega2000; Spada & Tomita, Reference Spada and Tomita2010 for meta analyses). The second issue pertains to the role of awareness in L2 acquisition and the question of whether and to what extent it is possible to acquire L2 knowledge in the absence of awareness (cf., e.g., Hama & Leow, Reference Hama and Leow2010; Leow, Reference Leow1997; Leung & Williams, Reference Leung and Williams2011, Reference Leung and Williams2012; Paciorek & Williams, Reference Paciorek and Williams2015a, Reference Paciorek and Williams2015b; Rebuschat & Williams, Reference Rebuschat and Williams2012; Schmidt, Reference Schmidt1990, Reference Schmidt and Schmidt1995 and elsewhere; Williams, Reference Williams2005). The third issue concerns the measurement of awareness through the employments of concurrent and nonconcurrent data elicitation procedures (R. Ellis, Reference Ellis2005; Grey, Williams, & Rebuschat, Reference Grey, Williams and Rebuschat2014; Leow, Reference Leow1997, Reference Leow2000; Rebuschat, Reference Rebuschat2013; Rebuschat, Hamrick, Sachs, Riestenberg, & Ziegler, Reference Rebuschat, Hamrick, Sachs, Riestenberg and Ziegler2015).
As the present paper seeks to contribute to the research on the second issue, in what follows we briefly review the literature addressing the question of whether adult L2 learning without awareness is possible. As a critical reaction to Krashen’s proposals, Schmidt (Reference Schmidt1990, Reference Schmidt and Schmidt1995, and elsewhere) put forward the “Noticing Hypothesis,” which states that input does not become intake for language learning unless it is noticed, that is, consciously registered. This hypothesis is based on the distinction between two types of awareness: (a) awareness at the level of noticing, which involves the conscious registration of surface forms of the input and (b) awareness at the level of understanding, that is, a higher level of awareness that includes knowledge of rules and metalinguistic awareness. An example of the former would be registering instances of the suffix “-s” in words like girls, boys, cats, or dogs in written input, while an example of the latter would be recognizing the rule that “-s” signals plurality. Schmidt argued that noticing is a prerequisite for learning to take place, whereas awareness at the level of understanding can facilitate learning but is not a necessary condition for learning to occur. According to Schmidt, noticing typically involves at least some degree of awareness. This would suggest that there is no learning that is entirely implicit. Schmidt’s proposals have had a major impact on the theoretical debate (Ellis, Reference Ellis and Rebuschat2015; Robinson, Reference Robinson1995, Reference Robinson, Doughty and Long2003; Tomlin & Villa, Reference Tomlin and Villa1994) and have triggered a substantial body of empirical research (cf., e.g., Faretta-Stutenberg & Morgan-Short, Reference Faretta-Stutenberg, Morgan-Short, Granena, Koeth, Lee-Ellis, Lukyanchenko, Prieto Botana and Rhoades2011; Hama & Leow, Reference Hama and Leow2010; Leow, Reference Leow1997, Reference Leow2000; Leung & Williams, Reference Leung and Williams2011, Reference Leung and Williams2012; Paciorek & Williams, Reference Paciorek and Williams2015a, Reference Paciorek and Williams2015b; Rosa & Leow, Reference Rosa and Leow2004; Rosa & O’Neill, Reference Rosa and O’Neill1999; Rebuschat & Williams, Reference Rebuschat and Williams2012; Williams, Reference Williams2005). A number of studies have provided empirical support for the facilitative effects of awareness on adult L2 learning, demonstrating that (a) awareness both at the level of noticing and understanding led to a significant increase in L2 learners’ ability to acquire the targeted form or structure (Leow, Reference Leow1997, Reference Leow2000; Rosa & Leow, Reference Rosa and Leow2004; Rosa & O’Neill, Reference Rosa and O’Neill1999); (b) awareness at the level of understanding led to significantly greater learning effects relative to awareness at the level of noticing only (Leow, Reference Leow1997; Rosa & Leow, Reference Rosa and Leow2004; Rosa & O’Neill, Reference Rosa and O’Neill1999); (c) there is a correlation between awareness at the level of understanding and reliance on hypothesis-driven learning/rule formation (Leow, Reference Leow1997, Reference Leow2000; Rosa & Leow, Reference Rosa and Leow2004; Rosa & O’Neill, Reference Rosa and O’Neill1999); and that (d) there is a correlation between level of awareness and formal instruction to identify a rule (Rosa & O’Neill, Reference Rosa and O’Neill1999).
While this prior work has demonstrated the facilitative effects of awareness on adult L2 learning, it did not address whether learning without awareness is possible at all. The first study to challenge Schmidt’s Noticing Hypothesis by providing empirical evidence that learning is possible in the absence of awareness was Williams (Reference Williams2005). This seminal study was a novel and important contribution to understanding the role of implicit learning in L2 acquisition. In contrast to much previous research on implicit learning, which has employed artificial grammar learning paradigms, where the focus is on the learning of the combinatorial aspects of language in the absence of meaning (i.e., on the learning of form-based sequential regularities over letters and syllables), Williams (Reference Williams2005) used a semiartificial language learning paradigm, which involved an aspect of natural language adapted for experimental purposes, namely the acquisition of form-meaning mapping. Footnote 1
In Williams (Reference Williams2005), participants were exposed to a semiartificial language consisting of four artificial determiners (go, ro, ul, and ne), which were embedded in their L1 (English). The artificial determiners encoded both distance (near vs. far) and animacy (animate vs. inanimate). Participants were told that these determiners functioned like English determiners, except that they also encoded distance: Gi and ro were used for near objects, whereas ul and ne were used for far objects. What participants were not told was that the determiners also encoded animacy: Gi and ul were used for with animate objects, whereas ro and ne were used with inanimate objects. Their attention was thus directed to the formal marking of a semantic contrast that concerned the distance between two objects and the role of animacy in determiner usage served as the hidden regularity targeted in the experiment. The study was the first attempt to investigate the acquisition of form-meaning mappings—in this case the hidden regularity of animacy—under incidental conditions. The participants were trained on the distinction between far and near objects on the basis of auditorily presented sentences (e.g., The children threw sticks at ul monkey in the tree or I spent an hour polishing ro table before the dinner party). They were instructed to indicate whether the novel determiner used in the sentences meant near or far, to repeat the sentence verbatim, and form mental images of the situations described. The testing phase comprised of two parts: In the first part, all participants were administered a two-alternative forced choice (2AFC) task, in which they had to use the new determiners in previously unseen sentential contexts. Retrospective verbal reports (RVRs), in which participants were asked what criteria they had used to make choices, served as a measure of awareness. Participants were classified as unaware if they did not provide any references to the targeted regularity, that is, “any references to living or nonliving, moves or does-not-move, and so forth” (Williams Reference Williams2005, p. 283). In the second part, only those participants that have been classified as unaware were given the same test sentences, but this time they were instructed to search for an underlying rule that governed determiner usage and were then interviewed again to assess the status of the acquired knowledge. After the first part of the testing phase, the proportion of self-reported unaware participants was 80%. This number dropped to 50% after the second part. In both cases the participants’ performance in the 2AFC task was statistically significantly above chance (part 1: 61% correct; part 2: 58% correct). These findings were interpreted as evidence that participants can acquire form-meaning mappings without becoming aware of the targeted regularity.
Further support for learning without awareness comes from several replications and extensions of the original study (Chen et al., Reference Chen, Guo, Tang, Zhu, Yang and Dienes2011; Leung & Williams Reference Leung and Williams2011, Reference Leung and Williams2012, Reference Leung and Williams2014; Rebuschat et al., Reference Rebuschat, Hamrick, Sachs, Riestenberg and Ziegler2015) as well as from studies investigating other aspects of L2 learning, such as the acquisition of L2 morphosyntax (Rogers, Revesz, & Rebuschat, Reference Rogers, Revesz, Rebuschat and Rebuschat2015), word order (Rebuschat & Williams, Reference Rebuschat and Williams2012; Williams & Kuribara, Reference Williams and Kuribara2008), and semantic preferences of verbs (Paciorek & Williams, Reference Paciorek and Williams2015a, Reference Paciorek and Williams2015b). Other subsequent extensions of Williams (Reference Williams2005), however, could not replicate the finding that learning without awareness is possible (Faretta-Stutenberg & Morgan-Short, Reference Faretta-Stutenberg, Morgan-Short, Granena, Koeth, Lee-Ellis, Lukyanchenko, Prieto Botana and Rhoades2011; Hama & Leow, Reference Hama and Leow2010). In the first extension, Hama and Leow (Reference Hama and Leow2010) employed not only an offline (nonconcurrent) measure of awareness (RVRs), but also added an online (concurrent) measure of awareness (think aloud protocols). Further modifications included the addition of a production task and the change from a 2AFC task to a 4AFC task, and the use of the auditory modality in the training and the testing phase, whereas Williams (Reference Williams2005) presented the test-items visually. It has been proposed that the failure to replicate the implicit learning effect may be due to the radical procedural changes outlined in the preceding text regarding the original Williams (Reference Williams2005) study, in particular the change to an auditory presentation of the test items and the employment of a 4AFC task, which are likely to increase task demands (Paciorek & Williams, Reference Paciorek and Williams2015b). In the second extension, Faretta-Stutenberg and Morgan-Short (Reference Faretta-Stutenberg, Morgan-Short, Granena, Koeth, Lee-Ellis, Lukyanchenko, Prieto Botana and Rhoades2011) introduced no methodological changes from Williams (Reference Williams2005) and still found no implicit learning effect, suggesting that the conflicting results cannot be explained by differences in experimental design. They attributed the diverging results to systematic differences between the participants of the two studies and the populations from which they were recruited.
In sum, the review of literature on implicit learning in L2 acquisition presented in this article showed that the question of whether adult learners can acquire form-functions mappings implicitly, that is, without awareness, remains empirically unresolved. Furthermore, as recently pointed out by Paciorek and Williams (Reference Paciorek and Williams2015a), most previous studies in this area have typically involved tasks where participants’ attention was directed to the relevant items of the input to guarantee that they were noticed. The assumption here was that directing attention to the relevant forms and meanings would facilitate encoding in memory without necessarily disclosing the targeted regularity/generalization. Thus, in such studies all that participants who were subsequently classified as unaware were not aware of was the hidden regularity connecting forms to meanings that falls under what Schmidt (Reference Schmidt and Robinson2001) termed “awareness at the level of understanding.” These studies have not addressed the role of conscious registration of a surface form that indicates that regularity that falls under what Schmidt called “awareness at the level of noticing” (cf. Loschky & Harrington, Reference Loschky, Harrington, Bergsleithner, Frota and Yoshioka2013 for a recent discussion). The exception is the series of studies conducted by Leow and colleagues (Leow, Reference Leow1997, Reference Leow2000; Rosa & Leow, Reference Rosa and Leow2004), in which awareness at the level of noticing is operationalized using concurrent verbal reports (think aloud protocols) asking participants to verbalize their thoughts while completing the task. More specifically, in these studies, noticing is operationally defined as “some form of subjective awareness of new targeted linguistic forms in L2 data as revealed in learners’ think aloud protocols” (Leow, Reference Leow1997, p. 474). While this attempt to operationally define this construct is commendable, there have been concerns that concurrent measures may be reactive (cf. Bowles, Reference Bowles2010; Rebuschat et al., Reference Rebuschat, Hamrick, Sachs, Riestenberg and Ziegler2015). This leaves open the question as to whether the noticing account is testable and how to operationalize awareness at the level of noticing in a way that in principle disentangles its effects from the effects of awareness at the level of understanding to provide unambiguous and conclusive evidence for its role in L2 learning (cf. Loschky & Harrington, Reference Loschky, Harrington, Bergsleithner, Frota and Yoshioka2013, p. 292).
THE PRESENT STUDY
The purpose of this study is to contribute to the existing body of knowledge regarding the relationship between awareness and L2 acquisition. Up until now, studies on implicit learning have mainly been conducted in laboratory settings under highly controlled conditions with university students as participants (but see Paciorek & Williams, Reference Paciorek and Williams2015b for an online experiment using crowdsourcing). The general goal of the present study is to investigate whether and to what extent the results obtained in such settings can be extrapolated to the general population. The following two central research questions are addressed: (a) to what extent can adult learners acquire L2 knowledge implicitly, that is, without intention to learn and in the absence of awareness of what has been learned and (b) what are the effects of awareness at the level of noticing and at the level of understanding on the acquisition of L2 knowledge?
Our study builds on Williams’s (Reference Williams2005) seminal study on the acquisition of L2 form-meaning mappings but departs from it and its subsequent replications and extensions (see Faretta-Stutenberg & Morgan-Short, Reference Faretta-Stutenberg, Morgan-Short, Granena, Koeth, Lee-Ellis, Lukyanchenko, Prieto Botana and Rhoades2011; Hama & Leow, Reference Hama and Leow2010; Leung & Williams Reference Leung and Williams2011, Reference Leung and Williams2012, Reference Leung and Williams2014; Rebuschat et al., Reference Rebuschat, Hamrick, Sachs, Riestenberg and Ziegler2015) in several important ways: First, instead of encoding the two semantic contrasts (+/- animacy, +/- distance) with four distinct artificial determiners (gi, ul, ro, ne), we decided to use two pairs of artificial determiners (rel/relt and hul/hult) where the hidden semantic contrast (animacy) was encoded by the presence or absence of a suffix and to explicitly introduce participants only to one member of each pair, while the other member appeared in the input during the exposure phase without being explicitly introduced (see below for the description of procedure). This change served two purposes. First, the decision to encode the hidden regularity with a suffix was motivated by the empirical evidence that L2 learners tend to have more difficulty to implicitly learn grammatical forms of low perceptual salience, even though they are highly frequent in the input (cf. Goldschneider & DeKeyser, Reference Goldschneider and DeKeyser2001; see also Ellis, Reference Ellis2006 for an associative learning theory account of this finding). More importantly, however, it allowed us to introduce a written production task that served as a measure of awareness at the level of noticing. Second, our training phase did not include any additional tasks that would increase the amount of rehearsal in short-term memory, such as repeating training items or forming a mental image of the general situation described by the sentence, and the novel determiners were not introduced in isolated sentences but were embedded in short story. Finally, and most importantly, our study was conducted using a web-based crowdsourcing platform, which allowed us to increase statistical power by testing a larger number of participants (see Crump, McDonnell, & Gureckis, Reference Crump, McDonnell and Gureckis2013; Mason & Suri, Reference Mason and Suri2012 for a recent discussion of the use of crowdsourcing services in experimental behavioral research). Another advantage of using a crowdsourcing platform in the context of the present study is that participants come from a more diverse background than the participants in the original study and in its extensions, which were mostly students, in some cases with linguistic backgrounds (cf. Leow & Hama, Reference Leow and Hama2013, p. 546 on the issue of external validity).
METHOD
Experiment 1
Participants
A total of 183 participants were recruited through CrowdFlower (www.crowdflower.com), an online crowdsourcing platform. Data from 71 participants who had not appropriately followed the procedure were discarded. All participants were naive to the purpose of the experiment and reported to be native speakers of English (71 from the United States, 36 from the United Kingdom, and 5 from Australia). The age of the 112 remaining participants (68 females/44 males) fell into four groups: Age 19–28 (N = 29, 26%), Age 29–48 (N = 54, 48%), Age 49–68 (N = 25, 22%), and Age 68+ (N = 4, 4%). Sixty-three participants (53%) had completed college (N = 63, 56%), whereas 46 (41%) had completed high school and 3 (3%) had not completed high school. Eighty-six participants (77%) had no knowledge of an L2.
Materials
Following Williams (Reference Williams2005), we used a semiartificial language paradigm where four artificial determiners are embedded into the carrier language English. The artificial determiners encoded both distance (near vs. far) and animacy (animate vs. inanimate). The role of animacy in determiner usage served as a hidden regularity. Unlike Williams (Reference Williams2005) who used four completely distinct forms (gi, ul, ro, ne), we decided to use two pairs of artificial determiners: (a) hul/rel, which encode the difference in distance but are both inanimate, and (b) hult/relt, which also encode the difference in distance but are both animate (cf. Table 1). The reason for this decision is that of making the animacy distinction less salient, as in contrast to distance where it is marked by two completely distinct forms, the animacy is only marked by the addition of “-t.”
Miniature determiner system
The stimulus material consisted of 34 artificial determiner-noun combinations (see Table 2 for an overview). Sixteen such combinations appeared in a short narrative (462 words long), which was presented during the exposure and pretesting phases. During the testing phase, the remaining 16 combinations were presented in isolated sentences.
Artificial determiner-noun combinations used in Experiment 1
Procedure
The experiment consisted of (a) a familiarization phase, (b) an exposure phase, (c) a pretesting phase, (d) a testing phase, and (e) postexperiment debriefing. The schematics of the procedure is given in Figure 1.
Schematic illustration of experimental procedure.
Familiarization phase.
Participants were introduced to two artificial determiners hul and rel, that is, the inanimate variants, integrated in two sentences. The usage conditions of the two forms regarding distance were explained and illustrated with two pictures. Participants were not told that there were two additional variants of the artificial determiners (hult and relt), which in this miniature artificial determiner system would be used only in combination with nouns referring to animate objects, that is, living things. All text was presented in a black font (Verdana 16) against a white background.
Exposure phase.
In contrast to Williams (Reference Williams2005), participants were not exposed to a set of isolated sentences containing artificial determiner-noun combinations, but were exposed to the first part of a short narrative (266 words long) that contained eight such combinations. Each of the four artificial determiners occurred twice in the narrative. Participants were asked to read the text closely and were informed that they would be presented with content-related questions about the text in the step to follow. During this phase, participants were not asked to perform additional tasks other than read the story for content. Having read the story, participants performed a comprehension task that comprised three comprehension questions. Those participants who did not correctly answer all three questions were sent back to the story presentation screen and asked to reread the text.
Pretesting phase.
In the next step, participants performed a cloze task (CT) based on the second part of the story (196 words long). The story contained eight gaps in determiner positions and participants were asked to fill in the correct form. The task was designed to contain two positions for each of the four determiners. Rather than serving as a measure of learning, this task served the function of identifying participants who had become aware of the unfamiliar forms early in the experiment: Any use of “t” forms implied that the participant had noticed the existence of “t” forms in the exposure phrase. No feedback was provided concerning the performance on this task.
Testing phase.
At this step, participants were informed about the fact that the story they had just read contained two variants of the artificial determiners they had not been introduced to. They were further informed that there was a rule that determined which of the alternatives was correct. Participants performed a 2AFC task where they were presented with 16 sentences with a gap in a determiner position. For each test item, they were asked to choose between two artificial determiners that had the same distance meaning but differed in their animacy meaning, that is, rel vs. relt or hul vs. hult. In addition to assessing awareness using the CT described in the preceding text and the use of RVRs (see the following text), we decided to employ a subjective measure of awareness, namely confidence ratings. Participants were asked to indicate how confident they were in each decision. Confidence was reported on a 5-point Likert scale ranging from “completely guessing” to “completely sure.” The presentation of trials was randomized for each participant. Participants received no feedback on their performance.
Postexperiment debriefing (RVR).
In this step, the participants were asked to report whether they had identified the hidden rule and were asked to describe what they had noticed. They were asked to also indicate if they had not noticed anything. Once they completed the 2AFC task, participants were asked to provide insight into the criteria they used for their choices and to report whether they had identified the hidden rule.
Data Analysis
Participants were classified as “aware” or “unaware” at the level of noticing on the basis of their responses in CT and they were classified as aware or unaware at the level of understanding on the basis of their responses in the RVR. This coding of awareness resulted in six categories (cf. Table 3): Group 1 (no use of unfamiliar form/no rule), Group 2 (use of unfamiliar form/no rule), Group 3 (no use of unfamiliar form/wrong rule), Group 4 (use of unfamiliar form/wrong rule), Group 5 (no use of unfamiliar form/correct rule), and Group 6 (use of unfamiliar form/correct rule). In the present study, only those participants who did not make use of an unfamiliar form in CT and made no reference to living or nonliving as categories were classified as “unaware” (participants in Group 1, see Table 3). We also used a subjective measure of awareness—confidence ratings—to establish whether the acquired knowledge is implicit in nature (cf. Rebuschat, Reference Rebuschat2013 for a review on the use of subjective measures in SLA research). Footnote 2 According to the “zero-correlation criterion” (Dienes, Altmann, Kwan, & Goode, Reference Dienes, Altmann, Kwan and Goode1995), knowledge can be considered implicit/unconscious, if there is no correlation between level of confidence and performance.
Classification of participants
Performance on the 2AFC task served as the measure of learning. We used one sample t-tests to determine if the performance of the participants in the six groups, in particular those in Group 1 was above chance level. To estimate the effects of awareness at the level of noticing and at the level of understanding, mixed logit models Footnote 3 were fitted to the 2AFC performance scores. The models included fixed effects of Response Type in the CT and in the RVR as well as the control variables Age, Education, and L2 Background. Response Type in the CT was modeled as a two-level factor distinguishing between “no use of unfamiliar form” and “use of unfamiliar form,” whereas Response Type in the RVR was modeled as a three-level factor distinguishing between “no rule” responses, “wrong rule” responses, and “correct” responses (cf. Table 3). Random intercepts and random slopes for participants and items were included to account for between-participant and between-item variation. We used the maximal random-effect structure justified by our design (Barr, Levy, Scheepers, & Tily, Reference Barr, Levy, Scheepers and Tily2013). In case the model did not converge, we excluded random slopes with lowest variance. The R implementation of mixed logit models used here—lmer (Bates, Maechler, Bolker, & Walker, Reference Bates, Maechler, Bolker and Walker2014) for R (R version 3.2.0, R Development Core Team, 2015)—maximizes penalized quasi-log-likelihood, which reduces the chance of overfitting (Bates et al., Reference Bates, Maechler, Bolker and Walker2014). The reported p-values were calculated by the lmerTest package (Kuznetsova, Brockhoff, & Christensen, Reference Kuznetsova, Brockhoff and Christensen2015).
RESULTS
Five out of six possible combinations of response types in the CT and the RVR were attested, corresponding to Group 1 (N = 61), Group 3 (N = 23), Group 4 (N = 7), Group 5 (N = 15), and Group 6 (N = 6). Group 2 characterized using unfamiliar form at the CT and no report of a rule at the RVR was unattested. One sample t-tests revealed that all groups performed above chance: Group 1: t(60) = 1.99, p < 0.05; Group 3: t(22) = 2.15, p < 0.05; Group 4: t(6) = 2.44, p < 0.05; Group 5: t(14) = 10.92, p < 0.0001; and Group 6: t(5) = 15.18, p < 0.0001. Group 6 reached overall accuracy of 91% (SD = 0.06), Group 5 of 88% (SD = 0.13), Group 4 of 68% (SD = 0.20), Group 3 of 56% (SD = 0.12), and Group 1 of 53% (SD = 0.13) (see Figure 2).
Proportion of correct responses (with standard errors) in 2AFC task across five groups. Group 1 (no use of unfamiliar form and no rule); Group 3 (no use of unfamiliar form and wrong rule); Group 4 (use of unfamiliar and wrong rule); Group 5 (no use of unfamiliar form and correct rule); and Group 6 (use of unfamiliar and correct rule).
We found significant effects of awareness not only at the level of understanding but also the level of noticing: Model comparisons using likelihood ratio tests showed that a model with both predictors—Response Type in the CT and Response Type in the RVR— fitted the data better than any simpler model (cf., Table A1 in the appendix for an overview of the model comparisons). The results of the final mixed logit model are presented in Table 4. Participants who reported the correct rule performed significantly better than participants who did not report any rule (β = 1.91, z = 8.8, p < 0.001). The difference in performance between participants who reported a wrong rule and those whose reported no rule was not statistically significant (β = 0.08, z = 0.53, p = 0.596). Participants who used the unfamiliar form performed significantly better than those who did not (β = 0.53, z = 2.19, p = 0.028). None of the control variables were significant and no significant interactions were found.
Effects in mixed-logit model fitted to 2AFC performance scores
Note: * p < 0.05; ** p < 0.01; *** p < 0.001.
The analysis of confidence ratings focused on the results of Group 1: The average confidence rating was 2.8 (SD = 1.36). We found that participants were most accurate when reporting to be very confident in their choices. When participants reported to be guessing, performance was at chance level, indicating that the “guessing criterion” was not satisfied (cf. Dienes et al., Reference Dienes, Altmann, Kwan and Goode1995).
Table 5 presents the accuracy across confidence ratings pooled across participants. Footnote 4 To establish whether learning in Group 1 was implicit by the “zero-correlation” criterion (Dienes & Scott, Reference Dienes and Scott2005), we fitted a mixed logit model to the 2AFC performance scores with a fixed effect of confidence—treated as a five-level continuous variable—and random effects for participants and items (random intercepts for participants and items and random slopes for participants). There was no statistically significant effect of confidence (β = 0.09, z = 1.8, p = 0.065), indicating that the zero-correlation criterion was satisfied.
Accuracy and proportion of responses across confidence ratings (pooled across participants)
Note: Significance from chance: * p < .05.
To assess learning over trials during the test phase, a mixed logit model was fitted to the performance scores with Group Membership (Groups 1–6) and Trial Number, that is, the position of a test item in the presentation order and their interaction as fixed effects. The maximal random-effect structure justified by our design included random intercepts for subject and item and random slopes for participants. We found a significant interaction effect between Group Membership and Trial Number, indicating that participants in Group 5, who identified the correct rule but did not use the unfamiliar form in the CT, showed a significant increase in performance with increasing numbers of test items (β = 0.2, z = 3.54, p < 0.001). Furthermore, while there were fluctuations in accuracy in the judgments of the participants in Groups 3, 4, and 6, the judgment accuracy of participants in Group 1 remained constant with increasing number of test items (see Figure 3).
Changes in accuracy over trials during the test phase. The scatter on the top and bottom of each graph represents, respectively, correct and incorrect choices (with random noise added). The solid line is a smoother obtained with local polynomial regression (LOESS).
Experiment 2
The design was identical to that of Experiment 1, except for the following modifications. First, participants were explicitly introduced to the “t”-forms (hult/relt) and were first presented with the other two variants (hul/rel) in the exposure phase. This means that hult/relt were now associated with inanimate objects. Second, to counter potential effects of particular determiner-noun combinations, we reversed the mapping of forms and meanings. That is, while in Experiment 1 the forms hul and hult were used to encode “near” and rel and relt encoded “far,” in Experiment 2 hul and hult were used to encode “far” and rel and relt encoded “near.” And third, to test for effects of increased exposure to training items, we included two additional control variables: We recorded (a) the number of times participants were sent back to the story screen due to their failure to correctly answer the story comprehension questions (Exposure Cycle) and (b) the total amount of time (in seconds) that participants spent on the story presentation screen (Exposure Time).
Participants
A total of 143 participants were recruited through CrowdFlower. The results from 63 participants were excluded from the analysis because they failed to follow the instructions. All participants were naive to the purpose of the experiment and reported to be native speakers of English (64 from the United States, 10 from the United Kingdom, and 6 from Australia). The age of the remaining 80 participants (43 females/37 males) fell into three groups: Age 19–28 (N = 27, 34%), Age 29–48 (N = 38, 48%), and Age 49–68 (N = 15, 19%). Fifty-one participants (64%) had completed college (N = 63, 56%), whereas 26 (33%) had completed high school and 3 (3%) had not completed high school. Fifty-six participants (70%) had no knowledge of L2.
Materials
The same stimulus material was used as in Experiment 1.
Procedure
The general procedure was identical to that of Experiment 1.
Results
In contrast to Experiment 1, in Experiment 2 all six groups were attested: Group 1 (N = 34), Group 2 (N = 9), Group 3 (N = 18), Group 4 (N = 1), Group 5 (N = 13), and Group 6 (N = 5). As in Experiment 1, participants in Group 1 formed the largest group (42%). One sample t-tests revealed that Groups 1, 5, and 6 performed above chance: Group 1: t(34) = 5.43, p < 0.001; Group 2: t(9) = 1.7, p < 0.1; Group 3: t(18) = -1, p > 0.3; Group 4: NA (only 1 participant); Group 5: t(13) = 9.26, p < 0.001; and Group 6: t(5) = 23.59, p < 0.001. Group 6 reached overall accuracy of 95% (SD = 0.11), Group 4 of 75% (SD = NA), Group 5 of 72% (SD = 0.2), Group 1 of 57% (SD = 0.12), Group 2 of 56% (SD = 0.23), and Group 3 of 47% (SD = 0.11) (see Figure 4).
Proportion of correct responses in 2AFC task across five groups: Group 1 (no use of unfamiliar form and no rule); Group 2 (use of unfamiliar form and no rule); Group 3 (no use of unfamiliar form and wrong rule); Group 5 (no use of unfamiliar form and correct rule); and Group 6 (use of unfamiliar form and correct rule).
As in Experiment 1, to investigate how the performance of unaware learners (Group 1) compared to the performance of learners at different levels of awareness (Groups 2–6), a mixed logit model was fitted to the performance data from the 2AFC task using the same model structure as described in the preceding text with two additional control variables—Exposure Cycle and Exposure Time—entered as fixed effects. Model comparison using likelihood ratio tests showed that a model with both awareness-related predictors fitted the data better than any simpler model (cf. Table A1 in the appendix for an overview of the model comparisons). The results of the final mixed logit model indicated that Response Type in the CT and Response Type in the RVR were significant predictors of performance in the 2AFC task. There was a significant main effect of Response Type in the RVR: Participants who reported the correct rule performed significantly better than participants who reported no rule (β = 0.77, z = 3.18, p < 0.001), whereas participants who reported a wrong rule performed significantly worse than participants who reported no rule at all (β = -0.44, z = -2.19, p = 0.028). With regard to noticing, there was a significant interaction effect of Response Type in the RVR and Response Type in the CT: The performance of participants in Group 6 who reported the correct rule and noticed the absence of the “t”-suffix early in the experiment was significantly better than that of participants in Group 5 who did identify the correct rule but showed no indication of early noticing of the absence of the “t” suffix (β = 1.91, z = 2.86, p = 0.004). The only control variable found to be significant was L2 background, which had a negative effect on task performance (β = 0.41, z = -2.23, p = 0.025). The additional control variables—Exposure Time and Exposure Cycle—were uncorrelated with the response, suggesting that these variables were not confounding the results of Experiment 1. The results of the mixed logit model are presented in Table 6.
Effects in mixed-logit model fitted to 2AFC performance scores
Note: * p < 0.05; ** p < 0.01; *** p < 0.001.
The relationship of task performance and confidence ratings was analyzed in the same way as in Experiment 1. The average confidence rating of participants in Group 1 was 2.8 (SD = 1.38).
Like in Experiment 1, we found that participants were most accurate when reporting to be very confident in their choices. When participants reported to be guessing, performance was at chance level, indicating that the guessing criterion was not satisfied. Footnote 5 The results of the mixed logit model revealed that there was no statistically significant effect of confidence (β = 0.09, z = 1.38, p = 0.165), indicating that the zero-correlation criterion was satisfied. As in Experiment 1, we assessed learning over trials during the test phase using a mixed logit model. The maximal random-effect structure justified by our design included random intercepts for subject and item and random slopes for participants. However, in contrast to Experiment 1, there was no evidence of learning over trials for any of the groups. Table 7 presents the accuracy across confidence ratings pooled across participants.
Accuracy and proportion of responses across confidence ratings
Note: Significance from chance: * p < .05.
GENERAL DISCUSSION
The current experiments set out to investigate (a) the extent to which adult learners can acquire L2 knowledge implicitly, that is, without intention to learn and in the absence of awareness of what has been learned and (b) the effects of awareness at the level of noticing and at the level of understanding on the acquisition of L2 knowledge.
In response to the first research question, the findings reported in this study provide additional evidence in support of learning without awareness. Consistent with the findings reported by several previous studies (cf., e.g., Leung & Williams, Reference Leung and Williams2011, Reference Leung and Williams2012, Reference Leung and Williams2014; Paciorek & Williams, Reference Paciorek and Williams2015a, Reference Paciorek and Williams2015b; Rebuschat & Williams, Reference Rebuschat and Williams2012; Rebuschat et al., Reference Rebuschat, Hamrick, Sachs, Riestenberg, Ziegler, Bergsleithner, Frota and Yoshioka2013; Rogers et al., Reference Rogers, Revesz, Rebuschat and Rebuschat2015; Williams & Kuribara, Reference Williams and Kuribara2008), we found evidence in both experiments that adult L2 learners can acquire form-meaning mappings without the intention to learn and without awareness: Participants in Group 1 who did not demonstrate any awareness of the hidden regularity—the animacy rule—and did not show any indication of having noticed the relevant surface form nevertheless selected articles in compliance with the animacy rule at significantly above chance levels in both experiments (average proportion of correct responses was 53% in Experiment 1 and 57% in Experiment 2). In both experiments, there was no significant relationship between confidence and accuracy, indicating that the acquired knowledge was implicit by the zero-correlation criterion. These participants represented the largest group at 54% (Experiment 1) and at 42% (Experiment 2). When the performance data in the 2AFC task from the two experiments were pooled, the analysis revealed that the mean accuracy of Group 1 was 55% (t(94) = 3.49, p < 0.001). Like many other studies of L2 acquisition of form-meaning mappings involving a minimal exposure period to a semiartificial system (cf. Rogers et al., Reference Rogers, Revesz, Rebuschat and Rebuschat2015 for a discussion), we found that the implicit learning effect was rather weak. However, we found the effect with a design that did not include any additional tasks during the exposure phase that would have increased the amount of rehearsal in short-term memory, such as the verbatim repetition of sentences or asking participants to form mental images of the situations described. Furthermore, our results demonstrate that implicit learning can take place even after exposure to substantially fewer items (16 items used in our study compared to 144 items used in Williams, Reference Williams2005). And the testing material did not include any previously encountered items but consisted exclusively of new items (i.e., generalization items), allowing us to assess the learners’ ability to generalize across items and apply their knowledge productively and excluding the possibility of participants recalling memorized items. However, we introduced two methodological changes that may have facilitated the learning process. First, in the present study, the stimulus material used during the exposure phase consisted of a short narrative rather than a set of isolated sentences (cf. references in Willems, Frank, Nijhof, Hagoort, & van den Bosch, Reference Willems, Frank, Nijhof, Hagoort and van den Bosch2015 for a growing body of research on language processing using more naturalistic stimuli). The multisentence context may have had a facilitatory effect on the acquisition of the targeted knowledge. Support for this possibility comes from evidence that knowledge derived from context-free stimuli does not always generalize to more natural situations (cf. Ferreira & Patson, Reference Ferreira and Patson2007; Willems et al., Reference Willems, Frank, Nijhof, Hagoort and van den Bosch2015). Second, contrary to Williams (Reference Williams2005), who used auditory presentation of training material and visual presentation of testing material, in the present study the modality of presentation between the exposure and testing phases remained constant (exposure items and test items were presented visually), reducing the memory demands of the task. A potential effect of cross-modality has also been addressed in a study by Hama and Leow (Reference Hama and Leow2010), a conceptual replication of Williams (Reference Williams2005), in which training and test items were presented in the same modality (for more details see above). However, as recognized by the authors, their failure to find the implicit learning effect could have been due to their decision to present all stimulus material auditorily, making it a more challenging task (cf. Paciorek & Williams, Reference Paciorek, Williams and Rebuschat2015c for a recent discussion of this point; see also Leow, Reference Leow1995; Wong, Reference Wong2001 on the role of modality on L2 learning and processing). Evidence for the more demanding nature of auditory presentation has also been reported in other areas of adult language processing. For example, in a study examining the extent to which multiword units are stored and processed holistically, Tremblay, Derwing, Libben, and Westbury (Reference Tremblay, Derwing, Libben and Westbury2011) found that word and sentence recall rates were inferior when stimuli were presented auditorily rather than visually.
Moving on to the second research question, the findings of the present study indicated that awareness at both levels appears to have facilitative effects on learning: With regard to awareness at the level of understanding, we found a significant main effect of Response Type in the RVR in both experiments, meaning that participants who developed awareness at this level—that is, those who reported the correct rule of the animacy regularity—performed significantly better than those who did not develop it. Recall that in the models reported in the preceding text the variable Response Type in the RVR was entered as a three-level factor (no rule, wrong rule, correct rule). This differs from previous studies mentioned in the preceding text in which awareness at the level of understanding was measured as a binary variable, distinguishing correct rule responses from all others. Our decision to distinguish between no rule and wrong rule responses was based on the intuition that participants who reported a wrong rule might have employed different learning strategies. Support for this possibility comes from the comparison of “learning curves” illustrated in Figure 3. While the judgment accuracy of the “no rule” participants remained constant with increasing number of test items, there were fluctuations in accuracy in the judgments of the “wrong rule.” As the order of presentation of test items was randomized for each participant, such fluctuations cannot be ascribed to any properties of the test items. They can, however, be interpreted as indications of explicit learning, where the participant makes and tests hypotheses in a search for a rule or pattern. Excluding the “wrong rule” participants from the unaware group was intended to make the identification of unaware participants more conservative. However, as reported in the preceding text, the difference in performance between participants who reported a wrong rule and those whose reported no rule was not statistically significant in Experiment 1; in Experiment 2, “wrong rule” participants even performed significantly worse than the “no rule” participants. For this reason and for purposes of comparability with other studies, we also fitted models where Response Type in the RVR was entered as a binary variable by collapsing “no rule” and “wrong rule” responses. The results of these models did not change the overall conclusions of the present study (see Table A1 in the appendix for the estimates and z-values of the mixed-effects models). Taken together, these findings are consistent with several studies based on semiartificial language learning designs (e.g., Leung & Williams, Reference Leung and Williams2011; Rebuschat & Williams, Reference Rebuschat and Williams2012; Williams, Reference Williams2005) and with studies based on natural language learning designs (cf. Leow, Reference Leow2000; Rosa & Leow, Reference Rosa and Leow2004) showing superior task performance in participants who developed higher-order explicit knowledge at the level of understanding. It has been suggested that such higher-order metalinguistic knowledge helps a learner, among other things, to narrow the number of potential rules or patterns that could be at work, that is narrow the learner’s hypothesis space (cf. Ellis, Reference Ellis2002; MacWhinney, Reference MacWhinney1987).
With respect to awareness at the level of noticing, we found significant effects of the Response Type in the CT in both experiments. The CT served to identify participants who had consciously registered the unfamiliar forms early in the experiment, that is before they were informed about the existence of these forms at the beginning of the testing phase. In Experiment 1, we found a significant main effect of an early conscious registration of the unfamiliar forms. In Experiment 2, we found that participants who reported the correct rule and consciously registered the unfamiliar form early in the experiment (Group 6) performed significantly better than participants who also reported the correct rule but did not use the unfamiliar forms early (Group 5). These findings are in line with previous studies demonstrating the facilitative effects of awareness at the level of noticing (e.g., Leow, Reference Leow2000; Rosa & Leow, Reference Rosa and Leow2004; Schmidt, Reference Schmidt1994) and can be explained by the “implicit tallying hypothesis” (cf. N.C. Ellis, Reference Ellis2002, Reference Ellis2005, Reference Ellis2006). This hypothesis states that (a) noticing is necessary only for new elements with certain properties that make low-attentional learning unlikely, such as elements with low perceptual salience, but not for all aspects of language to be acquired and that (b) noticing is necessary only for the initial registration of such elements to make an initial representation in long-term memory possible, but not for subsequent encounters. According to this hypothesis, once a new element has been noticed and first representation of it established in long-term memory, it “need never be noticed again; yet as long as it is attended to for use in the processing of future input for meaning, its strength will be incremented and its associations will be tallied and implicitly cataloged” (Ellis, Reference Ellis2002, p. 174). Applied to the context of the present study, the noticing effect can be explained as follows: In the context of the present study this could mean that the early conscious registration of the unfamiliar forms of the artificial determiners led to an earlier establishment of their representations, allowing for an earlier tracking of their associations with features of the accompanying nouns.
LIMITATIONS AND DIRECTIONS OF FUTURE RESEARCH
The findings reported here should also be considered in light of its limitations. One limitation of the present study is that we investigated performance on one type of form-meaning mapping and on one type of learning assessment, that is an untimed 2AFC task. We demonstrated that it is possible to acquire implicit knowledge about correlations between the surface form of article-like elements and the animacy feature inherent in the meaning of an adjacent noun even after a considerably reduced amount of exposure compared to other related studies. Future research should address to what extent this finding generalizes to more complex form-function mapping. As recently pointed out by Paciorek and Williams (Reference Paciorek and Williams2015a), the findings reported by previous studies conducted in laboratory settings suggest that not all possible correlations are learnable in the absence of awareness. Relatedly, future research should also address to what extent the facilitative effects of awareness at the level of noticing and at the level of understanding found in this study can be replicated for more complex regularities (cf. Arciuli, Torkildsen, Stevens, & Simpson, Reference Arciuli, Torkildsen, Stevens and Simpson2014 for a discussion of the effectiveness of implicit and explicit learning in the acquisition of simple and more complex statistical regularities). The exclusive use of one type of learning assessment in this study—a 2AFC task—brings up the question of whether the learning effect was modulated by task-specific demands: The untimed nature of the task may have allowed participants to attend to and reflect on specific properties of a sentence biasing them toward explicit learning strategies (cf. R. Ellis, Reference Ellis2005 on the validation on the use of measures of implicit and explicit knowledge). While the employment of a timed variant of the task may have led to an increase in the number of unaware participants and hence increased the statistical power available to address our first research question, it would by the same token have reduced the statistical power available to address our second research question. Future work should explore the role of task type in designs similar to the one employed in this study.
A further limitation is the use of a CT as a measure of noticing. Participants were classified as “aware” or “unaware” at the level of noticing on the basis of their responses in CT. The task served the function of identifying participants who had noticed the variation in surface forms early in the experiment. Any use of the unfamiliar forms in this task implied that the participant had become aware of the forms. By contrast, as one reviewer correctly pointed out, the nonuse of the unfamiliar forms may be due to several reasons, including the possibility that participants thought that only explicitly introduced forms were the only permissible options in the task. However, the task instruction “Please fill in the correct form” was intended to be unspecific to not disclose the existence of the unfamiliar forms. Our findings clearly show that the CT classification relates to performance on the 2AFC task, suggesting that the CT can serve as a valid measure of noticing.
Another limitation of the present study is that, like most existing studies of adult L2 learning without awareness, we have reported only group-averaged effects. As pointed out by Frost, Armstrong, Siegelman, and Christiansen (Reference Frost, Armstrong, Siegelman and Christiansen2015), measure of central tendencies can be misleading, as it is often the case that approximately one third of the sample or more is not performing the task above chance level. If we look at the distribution of accuracy scores of unaware participants for both experiments (see Figure 5), we see two peaks: Hartigan’s dip tests (Maechler, Reference Maechler2015) revealed that these distributions were not unimodal but at least bimodal (Experiment 1; D = 0.11, p < 0.0001; Experiment 2: D = 0.13, p < 0.0001; simulated p-values based on 2000 replicates).
Distribution of accuracy scores in the 2AFC task for unaware participants in both experiments.
Traditionally, individual differences in implicit learning are seen to be minimal relative to individual differences in explicit learning (Reber, Walkenfeld, & Hernstadt, Reference Reber, Walkenfeld and Hernstadt1991). However, more recently, research in the area of implicit statistical learning has demonstrated that implicit learning ability is also subject to individual variation and that this variation is related to several cognitive abilities and personality traits (cf., e.g., Kaufman et al., Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010; Misyak & Christiansen, Reference Misyak and Christiansen2012; Siegelman & Frost, Reference Siegelman and Frost2015). In the area of implicit learning in L2 acquisition, research on individual differences is still in its infancy (but see Grey, Williams, & Rebuschat, Reference Grey, Williams and Rebuschat2015; Tagarelli, Ruiz, Moreno-Vega, & Rebuschat, Reference Tagarelli, Ruiz, Morena-Vega and Rebuschat2016 for exceptions). An interesting avenue for future research would therefore be to investigate which learner-internal and learner-external factors are most strongly related to the ability to acquire L2 knowledge without awareness.
CONCLUSION
The present study reported the results of two experiments that investigated whether L2 knowledge can be acquired without awareness and examined the effects of awareness on learning outcomes separately for the level of noticing (i.e., the early conscious registration of surface forms in the input) and understanding. Using crowdsourcing, the study is the first to demonstrate the implicit learning effect “outside the lab” in a more varied sample of participants, suggesting that the results obtained in laboratory settings can be extrapolated to the general population. By introducing a novel experimental design, the study made an important step toward disentangling the effects of awareness at the level of noticing and understanding. The findings reported here challenge the assumption that noticing is a necessary condition for learning. The findings also provide evidence that both awareness levels have a facilitative effect on learning outcomes. Progress in some of the key issues in SLA theory hinges on advances in our understanding of the role of awareness in the processing and learning of L2 input. Despite its limitations, the present study makes a significant contribution to the growing body of literature on implicit and explicit L2 learning in L2 acquisition.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/S027226311700002X
APPENDIX
Model comparisons from Experiment 1 (top) and Experiment 2 (bottom)
