INTRODUCTION
Lexical development and knowledge are essential to language acquisition, and are defined both in terms of breadth (or size) and depth (or level of knowledge). In L1 studies of vocabulary size, it is suggested that 17,000 to 20,000 word families are known by educated native speakers (D’Anna, Zechmeister, & Hall, 1991; Goulden, Nation, & Read, Reference Goulden, Nation and Read1990). A common definition of word family is the base lemma plus related forms; the family of the base fish would include the words fished, fishing, and fisherman (see Nation, Reference Nation2006). Estimates of L2 vocabulary breadth necessary for academic performance vary by L1, from receptive knowledge of 11,000 words for Dutch university students (Hazenberg & Hulstijn, Reference Hazenberg and Hulstijn1996) to receptive knowledge of 9,000+ word families for academic reading proficiency in English—far more than 11,000 individual lexical items, depending on the nature of lexical representation (Nation, Reference Nation2006; Nation & Chung, Reference Nation, Chung, Long and Doughty2009).
This vast amount of necessary vocabulary/lexical items poses a challenge. Because vocabulary learning is foundational to L2 acquisition, researchers and instructors have been deeply interested in efficient ways to facilitate vocabulary learning. Studies comparing intentional and incidental learning of vocabulary (Laufer & Nation, Reference Laufer, Nation, Gass and Mackey2012; Nation & Chung, Reference Nation, Chung, Long and Doughty2009; Sonbul & Schmitt, Reference Sonbul and Schmitt2010) have resulted in claims that intentional study results in more learning, at least among adult learners on immediate posttests. However, with recent research suggesting that ESL/EFL classroom instructors focus only on the most frequent 3,000 word families (Schmitt & Schmitt, Reference Schmitt and Schmitt2012), the 8,000 to 9,000 figure presents a problem and an empirical question.
The sizable gap between words explicitly taught and learned in the classroom and those necessary for L2 academic competence implies that that many less frequent items are learned in the context of other activities, such as reading or listening. Learners may develop knowledge of many more than 3,000 word families intentionally; in fact, thoughtful learners undoubtedly employ a variety of strategies in intentional learning settings, adding many new words. However, the size of the gap between 3,000 and 8,000 to 9,000 word families in English suggests that some learning needs to happen without intention to do so, or incidentally. Multiple strands of L2 research have sought to isolate learning of vocabulary from various input types and controlled methods to test learning outcomes.
Incidental vocabulary learning (IVL) studies have exposed learners to new words without explicit instructions to learn them, with varying results. Three strands of research most relevant to the present study are (a) frequency of exposure, (b) multiple simultaneous modes of input, and (c) individual differences in working memory (WM). While the argument continues for which type of learning focus (incidental alone vs. intentional alone vs. a combination of the two) is best for learning goals/outcomes (see Cobb, Reference Cobb2016; McQuillan, Reference McQuillan2016), that discussion is outside my scope. There is also a growing body of studies examining similar effects in multiword units (see Boers & Lindstromberg, Reference Boers and Lindstromberg2012; Durrant & Schmitt, Reference Durrant and Schmitt2010); however, the present study focused on single-word lexical items only, with exposure frequency, aural enhancement (AE), and WM as moderating variables.
BACKGROUND
INCIDENTAL LEARNING
Incidental learning has generally been defined as learning without intention to do so (see Bruton, Garcia López, & Esquiliche Mesa, 2011). Hulstijn (Reference Hulstijn, Doughty and Long2003) and Dörnyei (Reference Dörnyei2009) defined incidental learning more narrowly in SLA as no forewarning of an upcoming posttest of learning, without reference to any preceding process. IVL has most typically been operationalized in the latter way (see also Godfroid, Boers, & Housen, Reference Godfroid, Boers and Housen2013). Bruton et al. (Reference Bruton, García López and Esquiliche Mesa2011) argued that it is much easier to ensure that learning is a by-product of another activity—being unaware of a forthcoming test—compared with assessing learner intent.
Both definitions of incidental are distinguished here from the terms explicit and implicit learning and knowledge. Rather than focusing on the role of consciousness (see Ellis, Reference Ellis and Ellis1994; Ellis, Reference Ellis2007), incidental and intentional are centered on the role of learner intention (Bruton et al., Reference Bruton, García López and Esquiliche Mesa2011; Hulstijn, Reference Hulstijn, Doughty and Long2003). IVL studies have primarily examined how much vocabulary is learned in contexts where learners are not intentionally focused on learning vocabulary, whether processing or resulting knowledge are conscious (Webb, Reference Webb2007). However, recent research has begun to examine both outcome and process, as Pellicer-Sánchez (Reference Pellicer-Sánchez2016) used both online and offline measures to examine learning and attention in IVL. Further online processing studies should help to shed light on how truly incidental this type of learning is, and the role of attention therein.
Most L2 studies have operationalized IVL when reading in the manner of Hulstijn (Reference Hulstijn, Doughty and Long2003) and Dörnyei (Reference Dörnyei2009). In both laboratory and classroom studies (Chen & Truscott, Reference Chen and Truscott2010; Horst, Cobb, & Meara, Reference Horst, Cobb and Meara1998; Hulstijn, Hollander, & Greidanys, Reference Hulstijn, Hollander and Greidanus1996; Rott, Reference Rott1999; Waring & Takaki, Reference Waring and Takaki2003), a text was given with no control exerted over the duration of exposure. It is very difficult in this context to ensure that learners do not become intentionally focused on learning vocabulary, even when unaware of an upcoming test. Thus, claims of a generalizable impact of frequency of exposure—which each study claims at some level—are questionable.
Bruton et al. (Reference Bruton, García López and Esquiliche Mesa2011) critiqued Horst et al.’s (1998) use of the term incidental, arguing that it is impossible to ascertain exactly what the process of learning vocabulary might be in assigning participants to read a text; participants may not have been focused on comprehending meaning at all in many classroom-based studies (Bruton et al., Reference Bruton, García López and Esquiliche Mesa2011, p. 760). Participants who were very familiar with the text or more proficient could have intentionally deciphered both form and meaning of unfamiliar words. Webb (Reference Webb2007) was one of the only L2 IVL studies in which duration of exposure was controlled, by presenting target words in one to two sentence contexts and restricting exposure time. Varying results and claims regarding L2 IVL learning from reading could (in part) be an artifact of disagreements over how incidental is operationalized. Bruton et al. (Reference Bruton, García López and Esquiliche Mesa2011) therefore argued against using the term incidental at all in classroom studies.
Some classroom studies have sought to avoid the problem of time and attention by having teachers/researchers read the text aloud (Tekmen & Daloğlu, Reference Tekmen and Daloğlu2006; Webb & Chang, Reference Webb and Chang2015; Zahar, Cobb, & Spada, Reference Zahar, Cobb and Spada2001). This may keep participants on task, and avoids additional exposures from rereading. However, including a listening component presents complications. Kelly (Reference Kelly1992) argued that words learned by an L1 reading-while-listening group were more durable than those from a reading-only group (see Bisson, van Heuven, Conklin, & Tunney, Reference Bisson, van Heuven, Conklin and Tunney2013, 2014; Brown, Waring, & Donkaewbua, 2008 for L2 examples). Multiple simultaneous input modalities introduce another potential moderating variable.
FREQUENCY OF EXPOSURE
L1 IVL research has suggested that acquiring new words from context alone is often slow, incremental, and iterative (Jenkins, Stein, & Wysocki, Reference Jenkins, Stein and Wysocki1984; Nagy, Herman, & Anderson, Reference Nagy, Herman and Anderson1985). L2 studies, primarily conducted in naturalistic classroom contexts, have indicated similarly slow processes, with varying claims of effects in L2 learning from reading. Table 1 presents a summary of ten studies that controlled for exposure frequency, with exposure frequency/type, learner proficiency, learning outcome, word type, and findings highlighted.
Summary of L2 IVL studies controlling exposure frequency
MC = multiple choice; POS = part of speech
General findings indicate strong effects, with differences in the reported number of exposures necessary for learning. Hulstijn et al. (Reference Hulstijn, Hollander and Greidanus1996), Webb (Reference Webb2007), and Chen and Truscott (Reference Chen and Truscott2010) each found some learning at the initial exposure to target words. Horst et al. (Reference Horst, Cobb and Meara1998) argued for eight, Rott (Reference Rott1999) for two, and Waring and Takaki (Reference Waring and Takaki2003) more than 20. Recent reviews from Nation (Reference Nation2014) and Laufer (Reference Laufer2017) argued for 12 as a “moderately safe goal” (Nation, Reference Nation2014, p. 3).
Methodological differences make comparisons difficult. Some studies used experimental passages, while others used modified or unmodified graded readers. Learner proficiency and measure type could also contribute to conflicting findings. Four studies were conducted with advanced L2 learners, three with low-intermediate and intermediate groups, and three with variable proficiency levels. The four examining advanced learners found initial word learning from as little as a single exposure to a target word. Hulstijn et al. (Reference Hulstijn, Hollander and Greidanus1996) reported word learning at a single exposure in their control (nonenhancement) condition, albeit with modest gains (16.3%), with frequency effects in receptive knowledge between one and three exposures. Rott (Reference Rott1999) also found modest (20.2%) form-meaning gains in a two-exposure group, with six exposures resulting in greater learning than two or four. Webb (Reference Webb2007) used a battery of ten measures, with learning of orthographic form detected at the first exposure, along with frequency effects for all aspects of word knowledge. Pellicer-Sánchez (Reference Pellicer-Sánchez2016) found durable gains in receptive word knowledge (86% in form; 75% in meaning) when learners were tested after eight exposures, and reported from an online eye-tracking paradigm that learners processed new words similarly to familiar words after three or four exposures.
The three studies of intermediate L2 learners (Chen & Truscott, Reference Chen and Truscott2010; Horst et al., Reference Horst, Cobb and Meara1998; Webb & Chang, Reference Webb and Chang2015) reported mixed results. Horst et al. (Reference Horst, Cobb and Meara1998) suggested eight, while Chen and Truscott (Reference Chen and Truscott2010) reported receptive knowledge of orthography after one exposure. Webb and Chang (Reference Webb and Chang2015) claimed frequency effects without reference to a specific number of exposures. In studies with variable proficiency levels, Zahar et al. (Reference Zahar, Cobb and Spada2001) reported learning from a single exposure; conversely, Waring and Takaki (Reference Waring and Takaki2003) argued that meaningful learning did not occur until 20 exposures. Tekmen and Daloğlu (Reference Tekmen and Daloğlu2006) concurred in their findings with Zahar et al. (Reference Zahar, Cobb and Spada2001), reporting learning from the initial exposure.
These findings are, to a degree, the result of testing measurements of varying sensitivity. Studies reporting learning at early frequency levels generally included more sensitive measures of word learning (e.g., Chen & Truscott, Reference Chen and Truscott2010; Hulstijn et al., Reference Hulstijn, Hollander and Greidanus1996; Webb, Reference Webb2007). The exception to these findings was Waring and Takaki (Reference Waring and Takaki2003), who reported minimal learning in the single-exposure group even in form recognition.
Thus, findings on frequency are often related to the way learning is defined. If IVL is both slow and iterative, as Nagy et al. (Reference Nagy, Herman and Anderson1985) suggest, a clear theoretical definition of learning is essential. As Table 1 demonstrates, the variety of learning measures used in exposure frequency studies reflects a broader disagreement over L2 acquisition theory. Some researchers include simple form or orthographic recognition, whereas others use form-meaning links as the baseline. Interestingly, Waring and Takaki (2003) argued that learners may need to encounter a word more than 20 times, but their definition included a time delay (delayed posttests), and was operationalized as translation.
In sum, the research on exposure-frequency effects on IVL remains difficult to interpret, due to methodological and theoretical differences. Crucial questions of how much can be learned at initial exposures, along with the type of learning (form recognition, form-meaning connections, or receptive/productive translation), remain open. According to Chaudron (Reference Chaudron1985), L2 acquisition progresses through a continuum of increasingly complex processing, with form recognition preceding form-meaning mapping. Wesche and Paribakht (Reference Wesche and Paribakht1996) argued for form recognition as the first of three stages of vocabulary knowledge, followed by form-meaning mapping, then production in context. Barcroft’s (2009) Type of Processing-Resources Allocation (TOPRA) model asserted that form and meaning of new information are separate mental components, governed by separate processing mechanisms. The processing of word form in visual word learning would require less mental effort than that of word meaning, making it easier and faster to learn and recall. This would particularly apply to IVL contexts, where task focus is on meaning comprehension.
(MULTIPLE) SIMULTANEOUS MODALITIES
The influence of multiple input modalities on L2 acquisition has been investigated both in L1 and in L2 research through studies designed to disentangle effects of each mode on learning. In L1 research, Rasinski (Reference Rasinski1990) found no differences in reading fluency in reading versus reading-while-listening L1 groups. However, Shefelbine (Reference Shefelbine1990) argued that listening while reading could facilitate decoding strategies among poor L1 readers. Zahar et al. (Reference Zahar, Cobb and Spada2001) acknowledged Shefelbine’s (1990) argument, applied it to L2 groups, but did not infer that it could also affect learning of new vocabulary apart from exposure frequency. Chang (Reference Chang2009) investigated the issue from the opposite angle, arguing that reading while listening facilitates general listening skill development.
Relatively few L2 vocabulary studies have controlled for multimodal input. Kelly (Reference Kelly1992) examined effects of reading-while-listening on intentional word learning. Even using a very demanding measure of learning (translation), results indicated that including listening was beneficial. Bisson and colleagues (2013, 2014) examined the effects of multiple modes of input (reading, listening, and viewing pictorial information) on IVL, utilizing a complex task of receptive meaning translation, and reported learning effects from two exposures (Bisson et al., Reference Bisson, van Heuven, Conklin and Tunney2014). They argued that multiple modes of input made form-meaning mapping much easier for low-proficiency learners. However, they had no posttest for the incidental condition, and did not separate pictorial from aural effects. Also, picture headings were presented in the L2, while aural information was in the L1.
One L2 IVL study has attempted to examine effects of reading while listening: Brown et al. (Reference Brown, Waring and Donkaewbua2008). They hypothesized that listening while reading could reduce the mental resources necessary for phonological processing in novel words, thereby freeing them for semantic processing instead. Results indicated frequency effects across reading and reading-while-listening groups, but no significant differences in learning between groups. However, participants were told beforehand that they would be participating in a vocabulary learning program (p. 141). Some (if not all) participants were undoubtedly intentionally focused on learning vocabulary. Brown et al.’s (2008) claims both for IVL frequency-of-exposure effects and against multimodal effects are jeopardized. Crucially, if focus was drawn to unfamiliar vocabulary prior to the study, effects of listening while reading could easily have been negated by attention to form and meaning of new words during the treatment. This study’s claims for incidental learning were compromised.
Few studies of IVL using multimodal conditions have given a detailed theoretical account of why aural and/or visual information could facilitate learning. Among these, Bisson et al. (Reference Bisson, van Heuven, Conklin and Tunney2013, Reference Bisson, van Heuven, Conklin and Tunney2014) argued within Paivio’s Dual Coding Theory (Paivio & Csapo, Reference Paivio and Csapo1973), where verbal and nonverbal information are encoded simultaneously when both modalities are presented. Lewandowski and Kobus (Reference Lewandowski and Kobus1993) explicated the Redundant Signals Effect (Kinchla, Reference Kinchla1974), which states that processing of new information (e.g., lights/tones, letters/sonar targets) is faster and more accurate when presented in more than one modality simultaneously.
Lewandowski and Kobus (Reference Lewandowski and Kobus1993) examined multimodal influences on L1 vocabulary learning in isolated and intentional contexts. They found that processing speed was slightly slower, but memory recall more accurate, in visual + aural groups. Montali and Lewandowski (Reference Montali and Lewandowski1996) found that reading + listening groups outperformed either isolated modality in reading comprehension and memory among normal and poor L1 readers.
Whether the Redundant Signals Effect exists with reading and listening remains an open question, particularly how multiple simultaneous streams of information might facilitate learning. Researchers argue that L2 visual word processing may be dependent on the relationship between L1 and L2 (see Koda, Reference Koda1998 for an overview). Ota et al. (Reference Ota, Hartsuiker and Haywood2009) summarized a number of findings, suggesting that knowledge of L2 phonology plays a mediating role in visual word recognition in the L2, especially for learners whose L1 has a sound-symbol correspondence different than the L2. They argued that learners transfer L1 phonological contrasts in visual word recognition in the L2, resulting in confusion when presented with visual homophones (e.g., LOCK-ROCK for Japanese L1s learning English).
Hayes-Harb et al. (Reference Hayes-Harb, Nicol and Barker2010) tested L1 English participants for learning of new nonwords through auditory word/picture/visual word tasks, in which the visual word was either matched or mismatched to English spelling conventions. They found that participants were sensitive to inconsistencies in spelling conventions, and argued that orthographic and phonological information are both tapped in auditory word learning. Escudero et al. (Reference Escudero, Hayes-Harb and Mitterer2008) found that intentional L2 word learning was facilitated when learners heard and saw new words. Interestingly, their results also suggested that newly learned orthographic knowledge was stored, facilitating auditory word recognition in subsequent exposure.
Given empirical support for visual information facilitating aural processing and learning, it is curious that no work has explicitly asked whether the facilitating relationship between the two is bidirectional in L2 IVL. Mapping the sound-to-symbol connection would allow for mental resources previously allocated to phonological decoding from visual information to process form and meaning instead. At a theoretical level, a facilitative role for redundant simultaneous signals could exist. Listening while reading could facilitate deeper processing of the new words, but could also tap into individual memory differences by adding to the cognitive load on WM resources.
INDIVIDUAL DIFFERENCES AND NEW WORD LEARNING
One area of interest in reference to individual differences among learners has been the influence of working memory (WM), typically defined according to Baddeley’s (1998) model, on vocabulary learning abilities among L2 learners (Kormos & Sáfár, Reference Kormos and Sáfár2008; Martin & Ellis, Reference Martin and Ellis2012). If aural information facilitates deeper processing, including aural as well as visual information in new word learning would result in individual differences in WM to be involved, especially in IVL, where distracting information is the primary focus of the learning outcome.
Kormos and Sáfár (Reference Kormos and Sáfár2008) argued that the proficiency level of the students interacts with instructional setting/learning type, with low-proficiency learners relying more on WM to learn new words both intentionally and incidentally, while higher-proficiency learners rely on WM in IVL more. The present study tested whether increasing the load on memory in bimodal conditions would differentiate learners better on learning outcomes by individual differences, while accounting for differences in English proficiency in the model. Aural information presented by itself would potentially place higher demands on WM, so it was expected that including both visual and aural information would result in higher demands on WM than visual information alone.
To summarize across all three areas of IVL research, there has been considerable work done in exposure frequency, but very little consensus on definitions and learning outcomes. Few IVL studies have examined the effects of multiple simultaneous input modalities (Bisson et al., Reference Bisson, van Heuven, Conklin and Tunney2013, Reference Bisson, van Heuven, Conklin and Tunney2014 are exceptions), but others have reported effects in L1 intentional word learning (Lewandowski & Kobus, Reference Lewandowski and Kobus1993). To our knowledge, no IVL study has examined the effects of WM on learning outcomes. This study was designed to fill these gaps.
The Present Study
The primary goals of the present study were to test claims of frequency of exposure and AE effects separately in a controlled IVL environment, while accounting for group differences in English proficiency and examining the role of individual differences in WM. Five research questions were posed:
RQ 1: Controlling for duration of exposure, to what extent does initial vocabulary learning (form recognition) occur with minimal exposures (2×) to new words during IVL while reading?
RQ2: Controlling for duration of exposure, to what extent can initial form-meaning connections be made with only two exposures to target words during IVL while reading?
RQ3: Controlling for duration of exposure, are there differential effects of frequency of exposure with four exposures rather than two during IVL while reading?
RQ4: Controlling for duration of exposure, to what extent does AE facilitate IVL from two and four exposures?
RQ 5: To what extent (if any) does AE of reading passages influence effects of WM during IVL while reading?
PARTICIPANTS
Eighty-two intermediate-level ESL learners were recruited from two U.S. Intensive English Programs on university campuses. Two participants scored very poorly on reading and comprehension tasks, so their scores were removed from the final analysis, leaving 80 total participants. Participants were recruited by word of mouth, flyer, and e-mail advertising, and were either paid in cash or entered to win a gift card (depending on university requirements). Each participant took 90 to 120 minutes to complete the study. Upon agreement to participate, participants were randomly assigned to one of four treatment groups (two/four exposures, ± AE). For this study, no control group was necessary. All target words were quite rare and semantically opaque, so it was assumed that no treatment would result in zero learning for this population. Pilot testing helped to verify that the words were unfamiliar to this population, and a subset of study participants (n = 25) were asked explicitly after the conclusion of the study whether they had any prior knowledge of the target words. All clearly stated that they had no prior knowledge. Appendix AFootnote 1 summarizes L1, age, and education level of participants, and all reported normal or corrected-to-normal sight and hearing.
MATERIALS
Experimental passages were created, with a set of 32 target words embedded in four stories, eight in each story. The stories were initially written with four instances of each target word, distributed so that no target appeared twice on the same screen. Two-exposure conditions were created by substituting synonyms (e.g., “a bird” for “an ibis”) for two instances of each word, one near the beginning of the story, and one near the end. Thus, the resulting distribution of the words was approximately equated. Because all target words were concrete nouns, the reading experience in the two- and four-exposure groups was not substantially altered. The stories were presented in randomized order to avoid ordering effects. New to this study, the reading passages were presented on timed PowerPoint slides. This allowed for stricter control over exposures to the target words, which was useful in approximating true incidental learning.
The target words were of very low lemma frequency in English—beyond the 25,000-band range in the Corpus of Contemporary American English (COCA) (Davies, Reference Davies2008). As Reynolds and Wible (Reference Reynolds and Wible2014) noted, a problem in using frequency as a reference is that very few report the type of frequency used, making findings difficult to compare. Because COCA is self-updating, it allows for comparisons of frequency across time and language change. Each target word was a two-syllable concrete noun (see Appendix BFootnote 1 for the complete list). When targets were chosen, L1 speakers of Chinese, Spanish, and Arabic were consulted to see if any cognates or homophones existed. None were identified, so the target words were used. However, during the study, five participants from French-speaking African nations signed up to participate. Two of the target words had etymological origins in French (Doyen and Fracas), but all French-L1 participants claimed to have no prior knowledge of these words in French when asked.Footnote 2
Each target word was written in object position (direct object, indirect object, object of preposition) to minimize perceptual salience. The words were nonessential for answering reading comprehension questions, which was the primary stated task of the activity. This lessened the necessary semantic processing of targets during the reading, and limited exposures to timed readings. No target word appeared either at the beginning or end of a screen during the treatment. Screens consisted of five to nine lines, and were presented in 32-point Calibri font. All other words in the stories were within the 0 to 4,000 lemma frequency range according to COCA (Davies, Reference Davies2008). Waring and Takaki (Reference Waring and Takaki2003) argued that 96% to 99% of the words in a text should be familiar for reading comprehension to be high (p. 135). The proportion of target words to overall words in the treatment tasks is summarized in Appendix C,Footnote 1 with all four stories near the 96% threshold in the 0–4,000-band frequency range. Pilot testing demonstrated that this was appropriate for comprehension.
Simple reading comprehension questions were given for preview prior to reading, and at four points in each story, the slides paused for participants to answer questions (see Appendix DFootnote 1). This helped to ensure a focus on meaning during reading, which is essential to incidental learning (Dörnyei, Reference Dörnyei2009; Hulstijn, Reference Hulstijn, Doughty and Long2003). Pawlas et al. (Reference Pawlas, Ramig and Countryman1996) argued that L1 “rapid” speech rate is greater than 160 to 170 words per minute in English, and that a slower speech rate is below 160 (p. 85). For the present study, the stories were audio recorded at a rate of 120 to 140 words per minute. This allowed for detection of aural effects without being unnaturally slow, while also allowing slightly more time for participants in the reading-only conditions to read each line. Following each line of the aural recording, the line disappeared on the screen. This was true in all conditions, and participants were informed that the text would disappear. In this way, attention was drawn to meaning, while controlling for the additional variable of auditory input. Pilot testing revealed that this rate, even for the two visual-only conditions, was sufficient for participants to accurately answer comprehension questions. Scores indicated that all participants included in the analysis were focused on meaning, as all 80 participants in the final analysis scored at or above 75% accuracy on reading comprehension questions.
INSTRUMENTS
Vocabulary Learning Measures
Two outcome measures were used to assess learning of target words. First, initial stages of learning were measured with a simple form-recognition task (Chen & Truscott, Reference Chen and Truscott2010; Hulstijn et al., Reference Hulstijn, Hollander and Greidanus1996; Waring & Takaki, Reference Waring and Takaki2003; Webb, Reference Webb2007) (see Appendix EFootnote 1). Participants were given a randomized list of the 32 targets, along with 32 distractors, and told to circle words they recalled from the stories. Distractors were also two-syllable nouns in English in the same lemma frequency range as the targets. Participants were instructed to try not to guess, and each item was coded 1/0 (correct/incorrect) for both target words and distractors. Guessing was accounted for in two ways; by setting the baseline mean for learning at 50%, from which group scores could be compared to chance. A cross-tabulation of the results was also used to assess potential guessing. Correct and incorrect test items were examined for the total number of false positives, and these results are reported in Tables S4 through S8.Footnote 1 The form-recognition test was immediately followed by a multiple-choice form-meaning connection test, in which participants were presented with the target words and told to circle the correct semantic category for each word (e.g., a “dabbler” was “a type of person,” not “a type of plant” or a type of animal”) (see Appendix FFootnote 1).
Proficiency/WM Measures
Four additional measures were used. A cloze measure of written English ability originally created by Brown (Reference Brown1980) was given to assess proficiency, and is a measure originally validated and often used with intensive English program students (e.g., Chrabaszcz & Jiang, Reference Chrabaszcz and Jiang2014) to assess proficiency in English. Three measures of WM were also employed. A linguistic measure of phonological short-term memory was administered: the nonword span task (Gathercole, Pickering, Hall, & Peaker, Reference Gathercole, Pickering, Hall and Peaker2001; Linck et al., Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013). In this simple WM span task, lists of phonotactically plausible one- or two-syllable nonwords were presented for a short time (two seconds) in lists of seven. Participants were presented with a list of nonwords one by one, and asked to recall them from the most recent list. Ideally, this measure is conducted in the L1, as measurement in the L2 may not be equal to L1 (see Linck et al., Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013). However, given the diversity of L1s in the current study, it was logistically impossible to administer in the L1. Therefore, the test was given in L2 English.
A complex measure of WM processing was given through a modified version of the Operation Span (Ospan) task (Engle, Cantor, & Carullo, Reference Engle, Cantor and Carullo1992; see Foster et al., Reference Foster, Shipstead, Harrison, Hicks, Redick and Engle2015). Participants completed simple math equations by reading them aloud, along with a single uppercase English letter. Participants assessed whether the math equations were correct, while recalling the letters in blocks of two to five. Scoring was the percentage of the total number of accurately recalled letters from each block. This test was administered using DMDX software (Forster & Forster, Reference Forster and Forster2003). All participants scored above the 80% minimum threshold for accuracy on the math equations, indicating focus on the primary task. As discussed in the following text, five participant data points were adjusted using multiple imputation.
A nonlinguistic WM task—the Shapebuilder task (Atkins et al., Reference Atkins, Sprenger, Colflesh, Briner, Buchanan, Chavis and Dougherty2013)—was administered to measure complex WM processing. In this task, participants were presented with a series of colored shapes in varying patterns in a 4 × 4 grid, and asked to recall (a) spatial position, (b) order, and (c) color of the shapes in progressively more complex patterns. A score of five points for each component was given for each trial, with 15 available points for each trial. The reported score was the sum of the total points.
Reliability of Measures
Table 2 summarizes reliability types and coefficients for the measures. For both form-recognition and form-meaning outcomes, a Rasch analysis was conducted using WINSTEPS software (Bond & Fox, Reference Bond and Fox2012). This analysis included item-level assessment of the fit of the data within the probabilistic Rasch model. For both measures, all items fit the dataset (all infit mean square statistics were > .7 and < 1.3, respectively), so all were kept.
Reliability of measurements
PROCEDURE
Prior to the treatment, participants were administered the nonword span and proficiency measures. Following random assignment to a treatment group, each participant was guided in reading through the four stories, with explicit instructions for each participant not to return to an earlier section. Reading comprehension questions were given on paper, and participants followed on-screen instructions to answer them. Following the reading tasks, the two surprise vocabulary learning outcome measures were administered in paper-and-pencil format.
Success in ensuring that word learning was incidental was evidenced when many participants were visibly confused or laughed when given the vocabulary posttests, showing clear surprise. Following the two outcome measures, the Ospan and Shapebuilder WM measures were administered, followed by an informational survey. In keeping with incidental learning protocol, minimal information was given about the true nature of the study at the outset of the experimental sessions. On the consent form, participants were told they would be given an opportunity to practice their English skills, especially related to reading, and take some memory tests. This was to preserve the element of surprise for the vocabulary posttests, and a fuller account of the purpose of the study was given through a posttreatment debriefing form.
ANALYSIS
ASSUMPTIONS, GROUP-LEVEL DIFFERENCES, AND DATA CLEANING
Descriptive statistics and frequency histograms for all measures indicated that normality and homoscedasticity assumptions were met for the analyses. Six analyses were conducted:
(1) A one-way ANOVA comparing group scores on covariate measures;
(2) A 2 × 2 factorial multiple analysis of covariance (MANCOVA) model (RQs 1–2);
(3) Confirmatory ANOVA analyses using Estimated Marginal Means from the MANCOVA model (RQs 3–4);
(4) A cross-tabulation of correct/incorrect responses on the form-recognition measure;
(5) Correlational analyses examining relationships between WM and learning outcomes (RQ 5); and
(6) Regression analyses graphing the WM × group interaction effect (RQ 6).
A preliminary one-way ANOVA was conducted to determine if significant differences existed between groups on the covariate measures. Results yielded nonsignificant effects for three measures: (a) Ospan, F(3,76) = .99, p = .40; (b) nonword span, F(3,76) = .61, p = .61; and (c) Shapebuilder, F(3,76) = .1.74, p = .17. There was a significant omnibus effect for the cloze proficiency measure, F(3,76) = 2.91, p = .04. Post hoc Fisher's lease significant difference comparisons indicated that the FourV group outperformed the FourAV group (M difference = 4.20, p = .02), the TwoAV group marginally outperformed the FourAV group (M difference = 3.45, p = .06), the FourV group outperformed the TwoV group (M difference = 3.9, p = .03), and the TwoAV group marginally outperformed the TwoV group (M difference = 3.15, p = .08).Footnote 3 Given the differences between groups, it was necessary to include the proficiency measure as a covariate in the MANCOVA model.
During the study, explicit instructions were given on how to progress through the Ospan measure. However, five answer patterns clearly evidenced a lack of understanding about how to complete the letter recall task—that is, some answered by trying to recall all the letters across all blocks, while others only answered by entering one letter per block. These data were treated as missing, and multiple imputation was applied, with the imputed values used as the score for each participant. This allowed for a reasonable estimate of true Ospan scores for these five participants while keeping the dataset intact.
RESULTS
COMPARISONS OF GROUPS TO CHANCE
Research questions 1 and 2 were examined through a 2 (frequency of exposure type) × 2 (± AE) factorial MANCOVA. When estimated marginal means for each group had been generated by the MANCOVA model, these became the basis for chance (Tables 3-4) and group-level comparisons. The estimated means were compared with chance scores for each group on each measure through a series of independent-samples t-tests.
Independent-samples t-tests comparing Estimated Marginal Means from MANCOVA model with chance scores in form recognition
* p < .05
** p < .01
*** p < .001
Independent-samples t-tests comparing Estimated Marginal Means from MANCOVA with chance scores in form meaning connection
* p < .05
** p < .01
*** p < .001
The mean for chance scores on the form-recognition measure was 32 (k = 64), as the task was to accept or reject targets or distractors as having been in the stories. The mean for the form-meaning measure was 10.67 (k = 32), given that the task was a multiple-choice test with the target word and two distractors. Tables S1 and 3 summarize the results of the eight independent-samples t-tests, along with effect sizes (reported as Cohen’s d), and Figures 1 and 2 show the data graphically. Each group performed significantly above chance level, with large effects.
t-test results for estimated marginal means for form-recognition measure.
t-test results for estimated marginal means for form-meaning measure.
COMPARISONS BETWEEN GROUPS
To address RQs 3 and 4, the estimated marginal means and standard deviations were used to conduct confirmatory ANOVAs to test specific or “informative” hypotheses about group differences (Hoijtink, Reference Hoijtink2012; Kuiper, Klugkist, & Hoijtink, Reference Kuiper, Klugkist and Hoijtink2010). In contrast to post hoc group comparisons in the classic statistical test of the null and alternative hypotheses, confirmatory models allow for a priori predictions about differences between groups that are much more specific and theoretically motivated—essential components driving social science research. These hypotheses allow for stronger claims and evidence for effects. Confirmatory ANOVAs examined group-level differences on the outcomes from a probabilistic model-based approach, comparing the expected hypothesis with other plausible outcomes. Because there were differences between the four groups in proficiency, the estimated marginal means and standard deviations were used from the MANCOVA analysis to account for proficiency when comparing groups. Three a priori hypotheses for the form-recognition outcome were formulated:
• H 1: AE effects will exist between reading and reading + listening groups in form recognition, but without frequency effects. This hypothesis is represented by the formula M 1 > M 2 = M 3 > M 4. A plausible explanation of this hypothesis is that frequency effects would not be detectable until a higher number of exposures, given that the words were encountered only two or four times.
• H 2: Frequency effects will be found from two to four exposures, which AE will influence. Because learners exhibit increasing familiarity with new word forms by the four-exposure mark (Pellicer-Sánchez, Reference Pellicer-Sánchez2016), AE would not be expected to result in greater form recognition at four exposures. However, at initial stages of learning of form, the difference between two and four encounters would be sufficient to detect differences. Also, AE effects would exist at the initial recognition stage of two exposures, and the two-exposure group with AE would perform equally with both four-exposure groups, given the requisite processing load inherent in the multimodal condition. This hypothesis is represented by the formula M 1 = M 2 = M 3 > M 4. This hypothesis was the expected outcome.
• H 3: There will be frequency effects from two versus four exposures for both groups, and there will be AE effects between the groups with two exposures. Frequency effects would hold true with or without AE (i.e., comparing FourV to TwoAV). Also in this case, AE effects would exist only at the two-exposure level. This hypothesis is represented by the formula M 1 = M 2 > M 3 > M 4.
Each of these three order-restricted hypotheses were compared both with the null hypothesis (H 0), (i.e., that all four groups have equal outcomes) and the alternative hypothesis (H A), where there is no specific restriction or prediction of differences. A software program developed by Kuiper et al. (Reference Kuiper, Klugkist and Hoijtink2010) was employed for the confirmatory analyses, and Bayesian model selection (BMS) was conducted.
In BMS, the highest Bayes factor statistic is used to produce the proportion of posterior model probabilities, indicating which hypothesis is the best fit for the dataset. For the form-recognition measure, H 3 was identified as the actual best fit (as Table S2Footnote 1 indicates). The expected outcome was that AE would mitigate frequency effects from two versus four exposures, due to the increase in depth of processing in the multimodal condition. While AE effects were detected at two exposures, but not four, frequency effects held in the model from two to four exposures, regardless of AE.
A second confirmatory ANOVA was conducted on the form-meaning connection outcome estimated marginal means. The ordered informative hypotheses tested were as follows:
• H 1: Both frequency and AE effects will be found across all four groups. This hypothesis is represented by the formula M 1 > M 2 > M 3 > M 4. A plausible explanation is that AE results in deeper processing of new information, resulting in AE effects between groups at both two and four exposures, while frequency effects remain. In this hypothesis, deeper processing in the TwoAV condition would not be sufficient to eliminate frequency effects between FourV and TwoAV.
• H 2: Both frequency and AE effects will be found, with frequency effects between two and four exposures eliminated by AE. This hypothesis is represented by the formula M 1 > M 2 = M 3 > M 4. The deeper processing inherent in AE conditions results in greater semantic processing, and better form-meaning outcome scores.
• H 3: There will not be AE effects between the groups, represented by the formula M 1 = M 2, M 3 = M 4. A plausible reason for this hypothesis is that there is no deeper semantic processing in AE conditions.
The predicted hypothesis with this ANOVA was H 2 because previous research suggests that AE increases both level of processing and comprehension of new information. H 2 was clearly the best fit for the dataset (see Table S3Footnote 1).
ACCOUNTING FOR GUESSING IN FORM RECOGNITION
To account for guessing on the form-recognition outcome measure, a cross-tabulation of correct/incorrect responses was conducted. Tables S4 through S8Footnote 1 summarize these analyses, with whole dataset and group-level information. The relevant cells to guessing are the correct target word items (++) and the incorrect distractor items (−−), that is, words that were accurately identified by circling and words that were misidentified by circling. Guessing overall was quite low (4%), ranging from 3.1% to 5.7% by group. Figure 3 presents the accurately recalled target word (++) raw score percentages with the percentages of false positives (−−) subtracted by group, extended to TW items only, to give a very conservative adjusted estimate of form-recognition learning. Gains by group ranged from 10% to 30%, even with adjusted scores.
Raw scores on target word items (++) and adjusted scores ((++) – (−−)).
WM X GROUP RELATIONSHIPS
The MANCOVA model revealed a significant Group × WM interaction, F (3, 19) = 4.01, p < .01. Pearson product-moment correlations were calculated in to examine relationships between WM and outcome variables. Table S9 and Appendix GFootnote 1 show the strength of these relationships, including all three measures of WM and proficiency. As Table S9Footnote 1 indicates, vocabulary outcome scores significantly correlated with scores on the proficiency measure. Scores on the two outcome measures had a slightly smaller, but significant, correlation with each other, and Ospan and nonword span scores correlated significantly with the form-recognition measure.
At the group level, proficiency scores had a strong relationship with vocabulary outcomes in groups FourAV and TwoV (see Appendix GFootnote 1). Probably not coincidentally, these two groups also exhibited significantly lower proficiency levels. At lower proficiency levels, proficiency played a more important role than WM in initial stages of IVL. It also appears that stronger relationships between WM scores and form recognition existed in the two AE groups. Relationships between WM measures and the form-meaning outcome were less clear.
One phenomenon of particular interest in research into individual differences in instructed SLA is that of aptitude-by-treatment interaction (ATI) (see Roehr, Reference Roehr and Robinson2012 for an overview). To analyze ATI effects in the present study, the form-recognition outcome was regressed on a composite WM variable, with regression lines for each group compared to examine the interaction between the two variables by group.
Figures 4 and 5 illustrate ATI effects found in the present study. The composite WM variable consisted of combined nonword span and Ospan scores; the correlation between Ospan and Shapebuilder (r = .21) in the whole dataset was weaker than the correlation between Ospan and nonword span. Figures 4 through 5 clearly show evidence of an ATI effect, with the R 2 for AE groups indicating that WM accounted for much more of the variance in form-recognition outcomes than in non-AE groups. Figure 5 divides the groups by frequency. Differences are indicated in the pattern of crossed lines, as very little relationship was found between WM and form-recognition scores in non-AE groups, and stronger relationships in AE groups.
ATI for all groups.
ATI comparisons by exposure frequency.
DISCUSSION
The present study was designed to examine frequency-of-exposure effects during IVL, while controlling for aural listening while reading, and exploring the interactions between WM and treatment type. Incidental learning was operationalized through controlling duration of exposure to the target words, with WM as a potential predictor. Research questions 1 and 2 were answered affirmatively, in that participants performed significantly above chance in both form recognition and form-meaning connection at the level of two exposures. Measurable form-meaning learning in the two-exposure reading-only group was somewhat unexpected, given conflicting findings in IVL studies, but does follow other findings in controlled contexts (Rott, Reference Rott1999; Tekmen & Daloğlu, Reference Tekmen and Daloğlu2006; Zahar et al., Reference Zahar, Cobb and Spada2001).
This finding should be treated cautiously; however, effects for the other three groups were much larger, indicating significant form-meaning connections. These results align with other findings in both classroom and laboratory contexts, including multiple outcome measures at differing sensitivity levels (Chen & Truscott, Reference Chen and Truscott2010; Webb, Reference Webb2007; Zahar et al., Reference Zahar, Cobb and Spada2001). These findings provide support for initial representation of new word forms being established very early, while following the recent argument from Pellicer-Sánchez (Reference Pellicer-Sánchez2016) that multicomponent measures for word learning should be included in IVL studies.
Frequency effects were found between two- and four-exposure groups on both outcomes. It clearly matters how many times a learner encounters a word in IVL. These findings extend other research by supporting frequency effects at very early stages of learning. Effects were also found for AE, on both outcomes, but in different ways. In form recognition, four-exposure groups were not different, but as predicted, AE effects were found between two-exposure groups. The lack of AE effects at four exposures provides further evidence that new words are processed in similar ways to familiar words after three to four exposures (Pellicer-Sánchez, Reference Pellicer-Sánchez2016). If new words are processed more speedily after three to four exposures regardless of audio, form-level recall would not differ in AE and non-AE conditions at four exposures. It should be noted that, while AE effects existed at two exposures, AE did not eliminate frequency effects from two to four exposures, as was expected. The predicted informed hypothesis was incorrect, as frequency effects held.
AE effects in form-meaning connections were detected. Results indicate deeper processing of new word meanings when listening while reading, at both two- and four-exposure levels. These findings have two important implications. First, hearing the text while reading probably inflated reported learning gains from earlier classroom studies. Second, simultaneous aural information appears to enhance learning of semantic information of new words. This is the logical counterpart to the argument that visual information plays a role in new aural word learning (Escudero et al., Reference Escudero, Hayes-Harb and Mitterer2008; Hayes-Harb et al., Reference Hayes-Harb, Nicol and Barker2010). The relationship is likely facilitative in a bidirectional way, with aural information playing a role in IVL while reading. Other IVL studies have either ignored the aural component (Horst et al., Reference Horst, Cobb and Meara1998; Tekmen & Daloğlu, Reference Tekmen and Daloğlu2006; Webb & Chang, Reference Webb and Chang2015; Zahar et al., Reference Zahar, Cobb and Spada2001) or have been potentially confounded by intentional learning instructions to participants (Brown et al., Reference Brown, Waring and Donkaewbua2008). Further explorations into effects of simultaneous aural information on visual processing are warranted.
Other IVL input enhancement studies have examined contextual information (Hulstijn et al., Reference Hulstijn, Hollander and Greidanus1996), appositive cues (Godfroid et al., Reference Godfroid, Boers and Housen2013; Watanabe, Reference Watanabe1997), and marginal glosses (Watanabe, Reference Watanabe1997). The present study extends this research by providing evidence that concurrent audio support to reading can facilitate both form recognition and form-meaning connections at initial stages of lexical representation. The basic elements of this study can easily be extended to naturalistic settings, where reading versus reading-while-listening can be tested in longer texts.
The final research question related to a potentially differential role for WM in incidental word learning depending on treatment type. There has been much work on the relationships between individual differences in WM and new word learning, but only focused on intentional learning tasks (e.g., Kaushanskaya, Reference Kaushanskaya2012; Martin & Ellis, Reference Martin and Ellis2012; Papagno & Vallar, Reference Papagno and Vallar1995; Speciale, Ellis, & Bywater, Reference Speciale, Ellis and Bywater2004). To my knowledge, no previous studies have looked at potentially similar effects in an IVL context.
Additionally, relationships between WM scores and form-recognition outcomes indicate that reading while listening places higher WM demands on learners than reading alone. Including the aural component increased the memory load. In reading-only groups, relationships between WM scores and IVL outcomes were weak or nonexistent, while both AE groups demonstrated stronger relationships between form recognition and WM scores. An ATI effect was detected between groups, with WM as a predictor of form-recognition outcomes in AE conditions.
Interestingly, listening while reading helped to facilitate learning both of form and form-meaning connections, regardless of individual differences in WM. It was unexpected that the relationships between WM and the form-meaning outcome were not strong, and could be due to the nature of the WM tasks. Both tasks selected for the composite variable involved recalling basic linguistic information after a very short amount of lag time between memory presentation and testing, which parallels the form-recognition outcome measure closely.
Results concerning relationships between WM and IVL are similar to those of other studies of intentional word learning (e.g., Martin & Ellis, Reference Martin and Ellis2012), but only in AE conditions, and primarily in form recognition. No group demonstrated strong relationships between form-meaning connection and WM. This differs from other findings on new word learning, in which establishing knowledge of form and meaning is heavily dependent on available WM resources (see Gupta & Tisdale, Reference Gupta and Tisdale2009 for an overview). It should also be mentioned that greater demands on WM resources could have been due to aural information; because the present study was comparing word learning while reading or listening while reading, there was no experimental group that only listened to the texts. Including such a group would distinguish whether increased memory load was due to aural information alone, or aural + visual.
One important caveat concerning WM results concerns the population in the present study. Proficiency differences likely contributed to variability in relationships between WM and outcomes. The two groups with the lowest overall proficiency level (FourAV and TwoV) included participants exhibiting stronger relationships between proficiency and outcomes than the other two groups, and learners at higher proficiency levels were not as dependent on their general English proficiency to learn new words. Tapping into WM resources during IVL from reading appears to vary depending on learner proficiency.
Because the study was experimental, with strict control over duration of exposure, it sacrificed a degree of ecological validity. It is not a typical reading task for the text to disappear. Additionally, the selection of the target words, which were of low frequency and semantically opaque, possibly increased the perceptual salience of target words during the reading tasks. This potential threat to contextual validity was addressed through the timed nature of the tasks, but it is possible that the gap in frequency between familiar and novel words could have increased awareness of the new words.
The present study was also limited in that outcome measures were immediate posttests. As others have convincingly argued (Pellicer-Sánchez, Reference Pellicer-Sánchez2016; Waring & Takaki, Reference Waring and Takaki2003), durable learning of new words should be measured through delayed, as well as immediate, posttests. As the present study was concerned primarily with issues related to very early stages of lexical development, a delayed outcome measure was not administered. Future studies should test the longer-term durability of these results. It should also be noted that participants encountered the new word forms again during the form-recognition measure, and prior to the form-meaning measure. However, each instance of the target words in the reading task was in-context, while meaning inference was impossible from the form-recognition measure.
The population for the study was quite heterogeneous, with 12 distinct L1s represented. This limits L1-specific claims related to IVL, and dilutes strong claims for preexisting phonological knowledge supporting new orthographic knowledge. However, random assignment to groups, along with the distribution of L1s across the groups (see Appendix AFootnote 1), allowed for similar comparisons. Additionally, the learning task made the exposures to targets very intensive. The frequent repetition of the novel words in each story could have inflated initial learning results. Future studies should test these findings in less intensive texts, such as graded readers.
The study provides clear evidence that early IVL can be enhanced by audio support for reading. As audiobooks become increasingly accessible and affordable in all types of L2 learning contexts, classroom teachers can easily use and direct learners to these resources either in classroom reading exercises or in assignments outside of class. Modifying materials to flood the input with multiple exposures to target words is both time- and resource-consuming for a classroom instructor, but the inclusion of audio support to learners is simple and cost-effective. However, any encouragement to classroom teachers should be given with the caveat that participants in the present study were explicitly told to read carefully as they heard the listening passage. If learners do not attend to either the auditory or visual information, potential benefits from simultaneous AE would be lost.
An ATI effect was detected when comparing group relationships between WM and the form-recognition outcome measure; however, it did not exist in the form-meaning outcome. Learners tapped into WM resources more when learning new forms of words if they were listening while reading during IVL; however, implementing audio support for reading in the classroom need not exclude learners with lower WM ability. Audio support appears to interact with WM in establishing new word forms, but this interaction was not found when making semantic connections for new words. Further research should explore whether individual differences in WM differentially affect comparative outcomes in IVL among high- and low-WM participants separately, and whether this finding continues to hold for both form recognition and form-meaning mapping.
