INTRODUCTION
In recent years much of the material that is used as meaning-focused input for language learning has become available in both written and aural formats. Course books tend to include both aural and written versions of the same passages, and graded reading material can be purchased along with CDs that include the spoken versions of the texts. This allows for L2 input to be encountered in three modes: reading, listening, and reading while listening.Footnote 1 There are a large number of studies that have examined vocabulary learning in these different modes. The vast majority of these studies have investigated reading with results consistently demonstrating that L1 (e.g., Biemiller & Boote, Reference Biemiller and Boote2006; Gonzalez et al., Reference Gonzalez, Pollard-Durodola, Simmons, Taylor, Davis, Kim and Simmons2010; Jenkins et al., Reference Jenkins, Stein and Wysocki1984; Nagy et al., Reference Nagy, Herman and Anderson1985; Reynolds, Reference Reynolds2019; Shu et al., Reference Shu, Anderson and Zhang1995) and L2 words (e.g., Day et al., Reference Day, Omura and Hiramatsu1991; Dupuy & Krashen, Reference Dupuy and Krashen1993; Hulstijn, Reference Hulstijn1993; Pigada & Schmitt, Reference Pigada and Schmitt2006; Pitts et al., Reference Pitts, White and Krashen1989; Waring & Takaki, Reference Waring and Takaki2003) can be learned incidentally through reading. Although there are relatively few studies that have investigated incidental vocabulary learning through listening, this line of research has also shown that L1 (e.g., Elley, Reference Elley1989; Robbins & Ehri, Reference Robbins and Ehri1994) and L2 (Brown et al., Reference Brown, Waring and Donkaewbua2008; van Zeeland & Schmitt, Reference van Zeeland and Schmitt2013; Vidal, Reference Vidal2011) words can be incidentally learned through listening. Research suggests that reading while listening also contributes to incidental vocabulary learning (Brown et al., Reference Brown, Waring and Donkaewbua2008; Horst et al., Reference Horst, Cobb and Meara1998; Webb & Chang, Reference Webb and Chang2012a, Reference Webb and Chang2015a, Reference Webb and Chang2015b). Comparisons of the extent to which the three different modes contribute to incidental vocabulary learning are limited. However, initial findings indicate that reading while listening might be as effective (Brown et al., Reference Brown, Waring and Donkaewbua2008), or more effective than reading (Webb & Chang, Reference Webb and Chang2012a), and that listening may be the least effective mode (Brown et al., Reference Brown, Waring and Donkaewbua2008).
One question that remains to be explored is how these three modes of learning contribute to incidental learning of collocations. Because English tends to be made up of a large proportion of frequently occurring strings of words (e.g., Erman & Warren, Reference Erman and Warren2000) and learning these sequences tends to be a relatively slow process (e.g., Altenberg & Granger, Reference Altenberg and Granger2001; Laufer & Waldman, Reference Laufer and Waldman2011), it is useful to determine the extent to which collocations are learned through different modes of meaning-focused input. Although there is little research investigating the incidental learning of collocations, research suggests that sequences of words may be learned incidentally through reading (Pellicer-Sánchez, Reference Pellicer-Sánchez2017) and reading while listening (Webb et al., Reference Webb, Newton and Chang2013). However, there is reason to believe that listening may be more effective in learning sequences of words than for learning individual words because the prosodic forms of collocations might make them more salient in aural input than written input (Lin, Reference Lin2012). At present, however, there are no studies of incidental learning of collocations through listening. Nor are there any studies investigating the effects of different modes of meaning-focused input on the learning of collocations.
The first aim of the present study was to investigate incidental learning of collocations in three different modes of meaning-focused input: reading, listening, and reading while listening. Research examining incidental learning of sequences of words in different modes is needed for several reasons. First, it may highlight the way in which the majority of collocations are most likely to be learned. Individual words and collocations are learned in the classroom through intentional and meaning-focused learning. Moreover, vocabulary may also be learned outside of the classroom through intentional study using effective learning techniques such as flashcards (Nation, Reference Nation2013). However, because learners need to develop knowledge of thousands of words to understand different forms of discourse (e.g., Nation, Reference Nation2013; Webb & Nation, Reference Webb and Nation2017), a relatively small proportion of an advanced L2 learner’s vocabulary is likely to be acquired through deliberate study in the language learning classroom.Footnote 2 Moreover, when vocabulary is deliberately learned, it is individual words rather than sequences of words that tend to receive the most attention. Incidental learning of individual words and collocations may stimulate vocabulary growth when there is sufficient L2 input. Therefore, determining the most effective mode of learning word strings has great value. Second, because the prosodic form of word strings is most salient during listening (Lin, Reference Lin2012), the listening mode may have greater influence on learning collocations than it does for learning individual words. A third reason to examine incidental learning of collocations in different modes is that it may help to reveal the extent to which these items should be taught. Research suggests that L2 knowledge of collocations is far less than single-word items (e.g., Nguyen & Webb, Reference Nguyen and Webb2017) and that even advanced L2 learners often make mistakes when using word strings (e.g., Laufer & Waldman, Reference Laufer and Waldman2011). Further research investigating the extent to which collocations are learned in different modes of meaning-focused input may provide a more accurate assessment of the extent to which incidental learning may contribute to the learning of these items.
A secondary aim of the present study was to investigate the relationship between frequency of occurrence of collocations encountered in the different modes of input and learning. Research has consistently shown that frequency is a major indicator of whether single-word items are learned through reading a single text.Footnote 3 Studies examining the role that frequency plays in learning multiword combinations have been inconsistent. Research suggests that the frequency of a word string in English affects the speed with which it is processed by L2 learners (Ellis et al., Reference Ellis, Simpson‐Vlach and Maynard2008; Sonbul, Reference Sonbul2015; Wolter & Gyllstad, Reference Wolter and Gyllstad2013). However, Nguyen and Webb (Reference Nguyen and Webb2017) found that the frequency of collocations only explained 2.74% of the variance in L2 learners’ receptive knowledge of those items. Research has indicated that the number of encounters with items during reading while listening affected incidental learning (Webb et al., Reference Webb, Newton and Chang2013) but not during reading (Pellicer-Sánchez, Reference Pellicer-Sánchez2017). Further research in this area is clearly warranted.
LITERATURE REVIEW
We learn most L1 words incidentally. Incidental learning of L1 words occurs exclusively through listening for young children. Research suggests that children learn around 4,000–6,000 word families before they can read (Biemiller & Slonin, Reference Biemiller and Slonin2001; Goulden et al., Reference Goulden, Nation and Read1990). When children begin interacting with text, reading then becomes an important source of incidental vocabulary learning. Nagy et al. (Reference Nagy, Herman and Anderson1985) suggested that children are likely to encounter about 1,000,000 L1 words in written text each year, and that repeated encounters with unknown words during reading fuels their vocabulary growth. There have been many studies since then demonstrating that L1 and L2 words are learned incidentally through reading and that the more that words are encountered, the more likely they are to be learned (e.g., Jenkins et al., Reference Jenkins, Stein and Wysocki1984; Waring & Takaki, Reference Waring and Takaki2003; Webb, Reference Webb2007; Zahar et al., Reference Zahar, Cobb and Spada2001). If learners encounter a great deal of input, the lexical gains can be great. However, if learners encounter little input as is the case in many EFL contexts, the number of words learned incidentally will be very small. Thus, the extent to which L2 words are likely to be learned incidentally depends largely on the amount of L2 input that is encountered by learners. The contributions of listening to vocabulary growth have not been examined to the same extent as reading. However, both L1 (e.g., Elley, Reference Elley1989; Robbins & Ehri, Reference Robbins and Ehri1994) and L2 (Brown et al., Reference Brown, Waring and Donkaewbua2008; van Zeeland & Schmitt, Reference van Zeeland and Schmitt2013; Vidal, Reference Vidal2011) research suggests that words are learned incidentally through listening. Once learners are able to read, they might also learn words through reading while listening to a text.
Five studies have set out to compare the effects of learning L2 single-word items incidentally in different modes. Brown et al. (Reference Brown, Waring and Donkaewbua2008) investigated the differences in vocabulary learning gains made through reading, listening, and reading while listening to graded readers. They found that reading while listening tended to contribute to the greatest gains in vocabulary knowledge and that learning through listening led to the smallest gains. On immediate posttests participants demonstrated knowledge of 48%, 45%, and 29% of the target vocabulary on a multiple-choice test in the reading while listening, reading, and listening groups, respectively, and 16%, 15%, and 2% on a meaning recall test. The scores were significantly greater in the reading while listening and reading conditions than in the listening condition. Scores on 3-month delayed posttests were relatively consistent with the immediate posttest scores. The participants demonstrated knowledge of 43%, 41%, and 36% on the multiple-choice test in the reading while listening, reading, and listening modes. Their scores on the meaning recall test were 4%, 3%, and 1%, and were in the same rank order as the other tests. The participants were also surveyed about their preferred mode of learning; 72% reported that they favored reading while listening, 28% reported that they favored reading, and no participants responded that they preferred listening.
Teng (Reference Teng2018) compared the effects of reading and reading while listening to a graded reader on four aspects of L2 vocabulary knowledge: form recognition, grammar recognition, meaning recall, and collocation recognition. The participants were EFL students in their second year of studies at a university in China. The results revealed that the participants in the reading while listening condition produced significantly greater gains on all four tests of vocabulary knowledge than those who only read the graded reader. Teng’s findings are important because they indicate that mode of input may have an effect on gains in different aspects of word knowledge.
Webb and Chang (Reference Webb and Chang2012a) compared learning in two modes: the extent to which reading and reading while listening contributed to the incidental learning of single-word items for EFL students over 28 weeks. In both conditions, participants read or read and listened to the same text several times. Two tests were administered to determine the degree to which two sets of 50 words were learned. The results indicated that the students in the reading while listening condition learned significantly more words incidentally than those in the reading condition. A moderate effect size was found on one test and a strong effect size was found on the other.
Feng and Webb (Reference Feng and Webb2020) compared incidental learning of single-word items through viewing a L2 television program, reading a transcript of the television program, and listening to the audio of the program. The participants were Chinese learners studying EFL at a university in China. The results showed that there were significant gains in L2 vocabulary knowledge from pretest to immediate posttest and pretest to a 1-week delayed posttest for all learning conditions. However, there were no significant differences found between the three modes of input indicating that each mode contributed to statistically equivalent gains in vocabulary knowledge.
The fifth study that compared incidental vocabulary learning through different modes of L2 input was conducted by Vidal (Reference Vidal2011). She compared incidental learning of L2 single-word items through reading and listening. First-year university students studying English for specific purposes either read three short academic texts or listened to three short lectures. The readings and lectures were of a similar length and covered the same content. Participants in the reading and listening groups demonstrated knowledge of 22.7% and 15.5% of the target words on an immediate posttest, respectively. The superiority of the reading group in incidental vocabulary learning was also apparent on a 1-month delayed posttest. The reading group retained knowledge of 10.6% of the words while the listening group knew 7.8% of the words.
Taken together, these studies tend to suggest that the most effective mode for learning individual words may be reading while listening, while the least effective mode might be listening. However, with so few studies investigating the effects of mode of input on vocabulary learning, further research is needed.
COLLOCATIONS
In recent years there has been a great deal of research examining collocations. There is good reason to investigate the teaching and learning of collocations because they make up a large proportion of language. Hill (Reference Hill and Lewis2001) suggested that as much as 70% of language consists of multiword combinations, and Erman and Warren (Reference Erman and Warren2000) reported that almost 57% of speech and 52% of written text is made up of multiword units. Researchers tend to agree that ideally L2 lexical development should occur through both deliberate learning techniques and learning through meaning-focused input because each approach provides benefits to learners (e.g., Hunt & Beglar, Reference Hunt and Beglar2005; Laufer, Reference Laufer2003; Nation, Reference Nation2007, Reference Nation2013; Webb & Chang, Reference Webb and Chang2012b; Webb & Nation, Reference Webb and Nation2017). Studies have consistently demonstrated that collocations can be learned through deliberate learning techniques (e.g., Boers et al., Reference Boers, Demecheleer, Eyckmans, Bogaards and Laufer2004; Chan & Liou, Reference Chan and Liou2005; Laufer & Girsai, Reference Laufer and Girsai2008; Lindstromberg & Boers, Reference Lindstromberg and Boers2008; Sun & Wang, Reference Sun and Wang2003; Webb & Kagimoto, Reference Webb and Kagimoto2009, Reference Webb and Kagimoto2011). However, relatively little research has examined the extent to which collocations are learned through encounters in meaning-focused input.
The research findings focused on incidental learning of collocations through encounters in L2 input have been inconsistent. Szudarski (Reference Szudarski2012) found that there was no statistically significant difference in knowledge of collocation between participants who read texts containing target collocations over 3 weeks and participants in no treatment control conditions. In a follow-up study, Szudarski and Carter (Reference Szudarski and Carter2016) also found no statistically significant difference between participants who read six stories containing target collocations over 3 weeks and participants in a no treatment control condition. In contrast, two cross-sectional studies indicated that collocations can be incidentally learned through encounters in meaning-focused input (Pellicer-Sánchez, Reference Pellicer-Sánchez2017; Webb et al., Reference Webb, Newton and Chang2013). Pellicer-Sánchez (Reference Pellicer-Sánchez2017) found that L2 collocations were learned incidentally through reading, and Webb et al. (Reference Webb, Newton and Chang2013) found significant gains in knowledge of collocations that were encountered when reading while listening to a graded reader.
One question that remains to be investigated is the extent to which collocations are learned in different modes. There is great value in examining this issue because there is some evidence that listening might play a larger role for learning sequences of words than it does for single-word items. The form of formulaic sequences may be particularly salient when heard because there tends to be a lack of pauses or hesitations between component words (Bybee, Reference Bybee2002). In contrast, when formulaic language is encountered when reading, there is nothing to indicate the start or end of a sequence. Moreover, sequences may be particularly opaque during reading as they will often be broken up across lines on a page or from one page to the next (Webb et al., Reference Webb, Newton and Chang2013). The fact that preliterate children are able to produce sequences of L1 words (as well as individual words) before they have learned their meanings suggests that listening may contribute to learning and retaining the spoken forms of L1 words and multiword combinations (e.g., Brown & Bellugi, Reference Brown and Bellugi1964; Brown et al., Reference Brown, Cazden, Bellugi and Hill1969; A. M. Peters, Reference Peters1977). Peters (Reference Peters1977) found that while the word strings produced by young children were often not phonemically accurate, their rhythm and melody were more typical of adults. Lin (Reference Lin2012) thus concludes that the prosodic form of word strings may play a large role in learning and that listening may be particularly important for acquiring formulaic language.
FREQUENCY
All four of the studies that have investigated incidental learning of multiword combinations have also looked at the effects of frequency of occurrence of the target items as a factor that may affect the size of the learning gains. It is well established that the frequency with which unknown words are encountered affects whether they are learned. Research shows that higher frequency L2 words are more likely to be known than lower frequency words (e.g., Nguyen & Webb, Reference Nguyen and Webb2017; Schmitt et al., Reference Schmitt, Schmitt and Clapham2001; Webb & Chang, Reference Webb and Chang2012b), and that individual words that are encountered with greater frequency in a text are more likely to be learned than those that are encountered less frequently (e.g., Brown et al., Reference Brown, Waring and Donkaewbua2008; Waring & Takaki, Reference Waring and Takaki2003; Webb, Reference Webb2007; Webb & Chang, Reference Webb and Chang2015b).
The effects of frequency of collocations in a text and learning are still unclear. Webb et al. (Reference Webb, Newton and Chang2013) investigated how different numbers of encounters with collocations (1, 5, 10, 15) encountered in a graded reader affected learning. As the number of encounters with the items increased so did the size of gains on a receptive test of collocation. Twenty-seven percent of the unknown collocations were learned for items that were encountered once, 33% of the collocations were learned when items were encountered 5 times, 55% of the items were learned when encountered 10 times, and 76% were learned when the items were encountered 15 times. The results indicated that five encounters with collocations in the text contributed to significant learning and that 15 encounters led to significantly greater learning than all the other frequencies (1, 5, 10).
Pellicer-Sánchez (Reference Pellicer-Sánchez2017) investigated incidental learning of L2 collocations through reading. Participants encountered collocations in one of two versions of a text. The first included target collocations four times and the second included eight encounters with the target collocations. The results revealed that both conditions contributed to incidental learning of collocations. However, there was no statistically significant difference in the amount of learning found between the two conditions.
Research examining the extent of knowledge of multiword combinations at different word frequency levels is consistent with corresponding studies of single-word items. Research indicates that knowledge of single-word items tends to decrease as the frequency of words decreases (e.g., Schmitt et al., Reference Schmitt, Schmitt and Clapham2001; Webb & Chang, Reference Webb and Chang2012b). Learners tend to have greater knowledge of the most frequent 1,000 word families than they do for the most frequent 1,001–2,000 word families, and knowledge of this set of words tends to be greater than for words at the next frequency level, and so on. Similarly, Nguyen and Webb (Reference Nguyen and Webb2017) found that Vietnamese EFL learners had significantly less knowledge of collocations as the frequency of the items decreased. Knowledge of collocations was greatest for items made up of the most frequent 1,000 words, and smallest for items that included a node word from the 3,000 word level. Collocations from the 2,000 word level were better known than the items at the 3,000 word level but less well known than those at the 1,000 word level. The results indicated that rather than the frequency of the collocation as a whole, it was the frequency of the node word that best predicted whether it would be known.
THE PRESENT STUDY
To fill the previously mentioned gaps, the present study attempts to investigate to what degree L2 learners can incidentally learn collocations encountered in an intermediate-level graded reader. Three groups of participants studied a graded reader through one of the three modes of input: reading, listening, and simultaneous reading and listening. The present research addresses the following three research questions:
-
1. To what extent do L2 learners incidentally learn collocations in different input modes?
-
2. Does the mode of input affect the degree to which collocations are learned incidentally?
-
3. What is the relationship between frequency of occurrence and learning collocations?
METHODOLOGY
PARTICIPANTS
The participants of the present study were 138 second-year college students learning English as a foreign language in Taiwan. They had been learning English for an average of 9 years and their English proficiency was estimated to range from low- to high-intermediate. All the participants had experience reading and listening to graded readers before the intervention. The participants were randomly assigned to one of four treatments: reading, listening, reading while listening, and a no treatment control. The data for any participants who missed a treatment or testing session was excluded from the study. This left 112 students who completed the full intervention. Twenty-five participants completed the listening treatment, 28 completed the reading treatment, 26 completed the simultaneous reading and listening treatment, and 33 were in the no treatment control.
RESEARCH DESIGN
There were three experimental groups and one control group. The experimental groups encountered 17 target collocations in the same graded reader in one of three input modes: reading, listening, and reading while listening. The reading group read the text silently. The listening group listened to an audio version of the text. The reading while listening group listened to the audio version of the text and simultaneously read along. The control group did not read or listen to audio versions of the book, but completed the same dependent measures as the experimental groups. The inclusion of the control group ensured that any learning that occurred could be attributed to the treatments alone.
The treatment was conducted in six 50-minute classes over a 3-week period. All classes were taught by the same teacher. In the first five classes the participants completed two chapters per session. In the final class, the participants completed the final chapter followed by an immediate posttest. Knowledge of the target items was measured in a pretest 1 week before the treatment, in an immediate posttest at the conclusion of the treatment, and in a delayed posttest 4 weeks after the treatment. Sufficient time was provided for the participants to complete all learning conditions and tests. The design is summarized in Table 1.
Summary of the research design
READING MATERIALS
The graded reader, A Kiss before Dying (Cornish, Reference Cornish2005), was chosen as the reading and listening material for this study. A Kiss before Dying is an intermediate-level graded reader in the Macmillan Readers series. The book was written with a controlled vocabulary of 1,600 words and divided into three sections according to its storyline. The first two sections consist of four chapters each, and the third section is made up of three chapters. There were a total of 19,386 running words in the text. The audio version of the text that was created by the publisher was used as the listening material. The audio book was 156 minutes long with a speech rate delivered at 120 words per minute. All the participants had experience learning through listening to audio versions of graded readers and were comfortable with this rate of speech for extensive listening.
TARGET COLLOCATIONS
A total of 17 two-word collocations were selected as target items. Three criteria were used to select the target items. First, all target items had to occur in the graded reader—A Kiss before Dying. Second, data about the mutual information score were examined to ensure that there was a statistical strength of co-occurrence of the target items. Mutual information scores of 3.0 or higher indicate that two or more words occur together more frequently than would be expected by chance (Church & Hanks, Reference Church and Hanks1990; Hunston, Reference Hunston2002). All target items exceeded the 3.0 mutual information threshold. Third, the target items needed to be unknown to the majority of the participants. In a pilot study, students with a similar learning background who did not take part in the treatment did not know any of the items. Two additional items were found to be relatively well known in the pilot study. One of these items (personal files) was used as an example in the tests, and the other (hotel lobby) was included in the dependent measures to increase the participants’ motivation to do their best on the tests. If all items are unknown on a test, test takers may not make as much effort to answer all questions. Thus, scores, particularly on pretests where items are expected be unknown may not provide an accurate representation of vocabulary knowledge. Scoring for the item (hotel lobby) was excluded from the study. The frequency of occurrence of the items in the text ranged from 1 to 16. The target collocations, their frequencies of occurrence in the text, and their mutual information scores in Mark Davies (Reference Davies2008) Corpus of Contemporary American English are presented in Table 2.
Target items, their frequency of occurrence in A Kiss before Dying, and their mutual information scores from Davies (Reference Davies2008) Corpus of Contemporary American English
MEASURES AND PILOTING
Two tests were created to measure knowledge of the 17 target collocations. The first test (Test A) was made up of two components: a matching component and a meaning recall component. In the first component, one of the words in the two-word sequences was randomly put in one box (Box A) and the other words in the sequences were randomly listed in a second box (Box B). Randomization of the component words put in the boxes increased the difficulty of the test, as well as reduced the chance that participants might use strategies (rather than their knowledge of the target items) to complete the test. The participants’ first task was to match words from Box A with words from Box B to form the target collocations. It is important to note that the first word in the sequence did not necessarily appear in Box A. The participants’ second task was to translate the collocations that they created in the first task into Chinese. One example (personal files) was provided on the test sheet to help the participants understand what they were expected to do. Test A was piloted with native and nonnative speakers who did not take part in the study and found to work correctly; items in Box A were correctly matched with those in Box B and the sequences could be accurately translated into Chinese. Instructions were written in both English and the participants’ L1 to ensure that they were understood. The test is shown in Figure 1.
Test A. Note: Data for the multiword combination hotel lobby was not included in the analysis.
The second test (Test B) used a meaning recall format. It was considered a more sensitive measure than Test A because the collocations were provided. In Test B, the 18 collocations were presented to the participants and they had to translate the items into Chinese. For the two groups that encountered the target words in writing, the target items were presented in written form on a test sheet. For the participants in the listening only group, the target items were presented in spoken form because they had not been encountered in written form in the treatment. The target items were read aloud by a native-speaker of English and the participants wrote their meanings in Chinese on a corresponding test sheet. The example and three items are shown in the following:
The tests were administered to participants at three retention intervals: pretest, immediate posttest, and delayed posttest. However, only Test A was used as the pretest. Because Test B included information about the forms of the target items that may have contributed to learning, it was not included in the pretest and administered to participants only in the immediate posttests and delayed posttests.
SCORING AND DATA ANALYSIS
All test answers were marked as either correct (1) or incorrect (0). To prevent wild guessing on Test A, both form and meaning needed to be correct; otherwise no point was awarded. To answer the first and second research questions, paired samples t-tests were conducted between the pretest and posttests within each group and effect sizes were calculated. To answer the second research question, a one-way between-group multivariate analysis of variance was performed together with post hoc tests to make comparisons between groups. The Bonferroni adjusted α level was set at .01. To answer the third research question, Pearson product moment correlations were calculated to determine the relationship between frequency of occurrence and gains from the Pretest to Immediate posttest A for all the participants in the experimental conditions combined, as well as each of the three groups.
RESULTS
The descriptive statistics of means, standard deviations, reliabilities, and number of participants are presented in Table 3. The table shows that knowledge of the collocations was limited before the treatment. Each group produced fewer than one correct response on the pretest. On the posttests all the experimental groups achieved much higher scores than the control group. The higher standard deviations on the posttests indicate greater variation in the scores among participants than in the pretest. This is common within studies of incidental vocabulary learning (e.g., Feng & Webb, Reference Feng and Webb2020; Webb & Chang, Reference Webb and Chang2015a). Among the experimental groups, the reading while listening group scored the highest across the immediate and delayed posttests, and the listening group scored higher than the reading group. The reliabilities (Cronbach’s α) ranged from .88 to.92 for the immediate and delayed posttests.
Means and standard deviations of the learning conditions on the dependent measures (maximum score = 17)
Note: RL = reading while listening, R = reading, L = listening, C = control. Standard deviations in parentheses.
In answer to the first research question, there were statistically significant increases across modes from Pretest to Immediate posttest A and from Pretest to Delayed posttest A. Table 4 summarizes the analysis and reveals that the learning gains [(correct items/total unknown items)* 100] for the target items from Pretest to Immediate posttest were 28.46% (+4.64), 16.01% (+2.58), and 12.45% (+2.04), respectively, for the reading while listening, listening, and reading groups. The effect sizes were large across experimental groups. Scores from Pretest to Immediate posttest A for the control group regressed .45 points. The learning gains from Pretest to the Delayed posttest A were 56.91% (+9.28), 26.26% (+4.21), 23.01% (+3.82), and 4.46% (+.72) for the reading while listening, listening, reading, and control groups.
Learning rates and effect sizes across groups from pretest to immediate posttest A and delayed posttest A
Note: ***p < .001; *p < .05. The eta squared was obtained using the formula: t²/t² + (n − 1), .01 = small effect; .06 = moderate effect; .14 = large effect.
To answer the second research question, a one-way between-groups MANOVA was conducted to investigate mode differences on knowledge of the collocations (see Table 5). Five dependent variables were used: Pretest, Immediate posttests A and B, and Delayed posttests A and B. There was a statistically significant difference between input modes on the combined dependent variables, F(15, 287) = 6.83, p < .001, Wilks’s lambda = .43, and partial eta squared = .24. When the results for the dependent variables were considered separately, all the differences between groups except the pretest were statistically significant with large effect sizes.
Results of one-way between-groups MANOVA
Note: Alpha level set at .01. RL = Reading while listening, R = Reading, L = Listening, C = Control. Effect size measured using partial eta squared.
Table 6 presents the post hoc analysis using a Bonferroni adjusted level of .01. As shown, the reading while listening and the listening conditions contributed to statistically higher scores than the control condition on Immediate posttest A. There were no significant differences between any of the other groups. In Immediate posttest B, the reading while listening condition contributed to significantly higher scores than the other three conditions, and the listening and reading conditions had significantly higher scores than the control condition. The results for the delayed posttests were consistent with those from the Immediate posttest B. The reading while listening group remained the most effective mode in retaining what was learned; scores were significantly higher for this condition on both delayed posttests than the other three conditions. The listening and reading groups also outperformed the control group on both delayed posttest tests. Table 6 summarizes the post hoc analysis.
Post hoc Bonferroni tests between each subgroup (alpha set at .01)
Note: RL = Reading while listening, R = Reading, L = Listening, C = Control. *** p≤.001; * p≤.01
To answer the third research question, Pearson product moment correlations were calculated to determine the relationship between the frequency of occurrence of the target items and vocabulary learning gains. The analysis revealed that there was a large significant correlation between frequency of occurrence and vocabulary learning gains by the experimental groups as a whole from Pretest A to Immediate posttest A (r = .61, p < .01, n = 17). When each condition was examined separately, a large positive significant correlation was found between frequency of occurrence and vocabulary learning gains made from Pretest A to Immediate posttest A for the reading while listening group (r = .69, p < .01, n = 17). There were moderate correlations between frequency of occurrence and scores on Posttest A for the reading (r = .40, p = .12, n = 17) and listening groups (r = .24, p = .36, n = 17), but none were statistically significant. The difference in correlations between frequency and learning among the three conditions might suggest that the collocations were more salient in the multimodal reading while listening condition than in the other two unimodal conditions.
DISCUSSION
The present study expanded on earlier research in two ways. It is the first study to compare the effects of three modes of L2 input on learning collocations. Comparing the effects of L2 input modes has great value because researchers agree that learning vocabulary through input should play a role in L2 lexical development (Laufer, Reference Laufer2003; Nation, Reference Nation2013; Webb & Nation, Reference Webb and Nation2017), and so exploring how each input mode contributes to learning is necessary. Furthermore, it is also the first study to investigate the extent to which strings or words are learned through listening. The lack of research on learning formulaic language through listening is surprising given that vocabulary learning through spoken input is the sole contributor to preliterate children’s L1 lexical development. By investigating how listening contributes to learning collocations and comparing these gains to learning through other modes, the present study deepens our understanding of how each mode of input may potentially contribute to L2 vocabulary growth.
In answer to the first research question, the results indicate that the reading while listening condition led to the highest immediate posttest gains (28%) followed by the listening condition (16%) and the reading condition (12%). The amount of learning that occurred through the three modes was lower than in Webb et al.’s (Reference Webb, Newton and Chang2013) study that used a similar methodology (reading while listening to modified graded readers). This is surprising because the target items in the earlier study (e.g., run risk, pull strings, throw light) were rather opaque, while in the present study the target items were relatively transparent (e.g., birth certificate, municipal building, radio station). On an immediate posttest that was designed to measure recognition of collocations, Webb et al. found learning gains of 27%, 33%, 55%, and 76% when EFL learners encountered target items 1, 5, 10, and 15 times, respectively. The lower gains in the present study are likely due in part to a much more demanding test format; the participants needed to choose the correct collocate from five choices in the earlier study, while in this study they needed to match the correct collocations from 19 options and then correctly translate the item. However, it is also important to note that in Webb et al.’s (Reference Webb, Newton and Chang2013) earlier study, the graded reader was revised to include the target collocations, whereas in the present study the target collocations were originally written into the text. Thus, the learning gains in the present study may have greater ecological validity.
The relatively low immediate posttest gains for collocations are consistent with many studies investigating the incidental learning of single-word items through reading a single text. For example, Pitts et al. (Reference Pitts, White and Krashen1989) and Zahar et al. (Reference Zahar, Cobb and Spada2001) found lower gains (6.5%–8.6% and 7.2%, respectively), while both Brown et al. (Reference Brown, Waring and Donkaewbua2008) and Waring and Takaki (Reference Waring and Takaki2003) found higher gains for multiple-choice recognition tests (45% and 42%, respectively) and lower gains for meaning recall tests (15% and 18%, respectively). Earlier studies that have investigated learning in reading while listening conditions have also found gains that straddle those of the present study. Horst et al. (Reference Horst, Cobb and Meara1998) found learning gains of 22%, and Brown et al. (Reference Brown, Waring and Donkaewbua2008) found gains of 48% on a multiple-choice test and 16% on a meaning recall test. Brown et al.’s (Reference Brown, Waring and Donkaewbua2008) study is the only one to investigate the extent that EFL students learn individual words through listening to a graded reader. In that study listening contributed to gains of 29% on the multiple-choice test and 2% on the meaning recall test. The higher gains made for single-word items through listening to a graded reader are likely due to several factors such as a less demanding test, a difference in texts, and differences between the learners in the two studies.
The scores in the Delayed posttest A were higher than those for Immediate posttest A across groups. The improvements were 33% [(.37–.4) *100], 25%, 29%, and 3% for the RL, R, and L, groups, respectively; however, there was also a 3% increase for the control group. The higher scores on the delayed posttests were likely due to a learning effect from having taken Immediate posttest B, as well as the immediate posttests raising the participants’ attention on focusing on the target items in the test paper than reading tens of thousands of words in a text. Although the results of the delayed posttests cannot be attributed solely to the learning conditions, the research does reveal the contributions that tests can make to learning. Higher scores on delayed posttests in incidental vocabulary learning studies may be relatively common as several other recent studies have also revealed this to occur (e.g., E. Peters & Webb, Reference Peters and Webb2018; Webb et al., Reference Webb, Newton and Chang2013). One way to solve this methodological issue would be to double the number of groups and test half only on the immediate posttest and the other half only on the delayed posttest. This would require a large number of participants, however, which is not always possible. A second solution would be to simply exclude the immediate posttest and test all participants once at the longer retention interval. Although this appears to be a reasonable solution, it may misrepresent incidental learning gains. Because incidental learning occurs through repeated encounters with input, having a long delay between learning and testing suggests that in typical learning conditions recently encountered words are not encountered again. Although this may be the case for lower frequency items, this is not the case for higher frequency items. Thus, the immediate posttest gains may be an important indicator of the incidental learning of higher frequency words and collocations while delayed posttest may be the better indicator of the incidental learning of lower frequency items.
It is interesting to compare the results of Immediate posttest A with those of Immediate posttest B. Both tests involved writing meanings of the collocations. However, Immediate posttest A was the more demanding of the two tests because in addition to recall of meaning, the participants also needed to recognize the forms of the combinations and select the words that co-occur. As should be expected, the scores on Immediate posttest B were higher than those of Immediate posttest A. However, the large difference between the mean scores on the tests is somewhat surprising. Mean scores on Immediate posttest A for the reading while listening, reading, and listening groups were 5.44, 2.68, and 3.46, respectively. Mean scores on Immediate posttest B were almost twice as large for the participants learning through reading while listening (10.04), reading (4.84), and listening conditions (5.46). In studies of single-word items, recognition of form tends to be gained relatively easily in comparison to meaning recall (Pigada & Schmitt, Reference Pigada and Schmitt2006; Webb, Reference Webb2007, Reference Webb2008, Reference Webb2009a, Reference Webb2009b). The results of the present study suggest that recognizing the composition of collocations can be challenging. This supports the findings of Nguyen and Webb (Reference Nguyen and Webb2017) who found that after seven years of formal language instruction, Vietnamese EFL learners could recognize only 45% of collocations made up of the most frequent 1,000, 2,000, and 3,000 word families. The findings of the present study also support many studies investigating the use of L2 formulaic sequences that have revealed that developing the knowledge necessary to produce collocations can take considerable time (e.g., Altenberg & Granger, Reference Altenberg and Granger2001; Laufer & Waldman, Reference Laufer and Waldman2011).
Together, the results of these studies indicate a relatively small amount of incidental vocabulary learning for both single words and collocations in comparison to studies of deliberate vocabulary learning. However, lower gains made through encountering meaning-focused input should not discount the value of incidental vocabulary learning to lexical development. The gains that occurred in all these studies occurred through encountering a single text. However, Nagy et al.’s (Reference Nagy, Herman and Anderson1985) incidental vocabulary learning hypothesis suggests that words are learned by receiving large amounts of input (rather than a single text). By hearing and reading unknown words again and again over time, vocabulary knowledge is gradually gained until words are eventually learned. Research investigating L2 incidental vocabulary learning through reading multiple texts tends to support this; much larger gains tend to be made through reading multiple texts than through reading a single text (e.g., Cho & Krashen, Reference Cho and Krashen1994; Horst, Reference Horst2005; Pigada & Schmitt, Reference Pigada and Schmitt2006; Webb & Chang, Reference Webb and Chang2015b). Thus, it is likely that larger gains might be found to occur in a longitudinal study that investigated learning collocations in the three different modes. This would be a useful follow-up study.
A second reason not to discount relatively low incidental learning gains is that a great deal can be learned about target vocabulary (single or multiword combinations) through encounters in meaning-focused input. Through encountering words repeatedly in input, L2 learners are likely to gain awareness of how these words are used. Recognition of the grammatical functions, associations, collocations, and concept and referents are likely to be gained through repeated encounters in context. For example, although the target items in this study were collocations, the participants may still have gained knowledge of some of the words with which they co-occur. Verbs such as work were used regularly with advertising agency, write co-occurred several times with suicide note, and show was used regularly with birth certificate.
In answer to the second research question, the results indicated that the most effective mode for learning collocations was reading while listening. Participants who learned through reading while listening had significantly higher scores on Immediate posttest B and both Delayed posttests A and B than those who learned in the other conditions. This finding is supported by Webb and Chang’s (Reference Webb and Chang2012a) study that found that reading while listening contributed to greater knowledge of single-word items than reading. The findings are also partially supported by Brown et al.’s (Reference Brown, Waring and Donkaewbua2008) earlier study indicating that reading while listening contributes to greater incidental vocabulary learning of single-word items than listening, but similar gains in knowledge to reading.
One difference between the results of the present research and Brown et al.’s (Reference Brown, Waring and Donkaewbua2008) and Vidal’s (Reference Vidal2011) studies is that the listening condition was found to be as effective for learning as reading. In the present study, there was no difference in gains found between the listening and reading groups on all posttests. In contrast, Brown et al. (Reference Brown, Waring and Donkaewbua2008) found that both reading while listening and reading contributed to significantly higher scores than listening for single-word items, and Vidal (Reference Vidal2011) also found that reading contributed to significantly higher gains for words than listening. The difference in findings between the three studies suggests that mode of input may have differing effects on incidental learning of single and multiword items.
An important question is why might listening contribute to greater learning of sequences of words than individual words? There are three possible reasons for this. First, listening may help learners to process text as chunks and these chunks may be less transparent during reading. For example, collocations may often be heard without hesitations and pauses between words (Bybee, Reference Bybee2002). This may lead listeners to process sequences of words as individual processing units (Wray, Reference Wray2002). In contrast, the white spaces that separate words on a written page may lead readers to process each word separately and make sequences of words more opaque. Collocations may also be hidden to some degree when they are encountered over multiple lines or pages in a text making it more difficult for learners to notice them (Webb et al., Reference Webb, Newton and Chang2013). Second, distinct intonation contours that are encountered during listening may contribute to acquisition of the prosodic form of sequences, and this knowledge may be relatively durable as it enables children to reproduce them in their L1 (Lin, Reference Lin2012). Third, language learners may often use bottom-up processing during reading focusing on the meanings of individual words rather than on the meanings of combinations of words. In contrast, the lack of hesitations within formulaic sequences, as well as the prosodic features of these items when encountered during listening, may help to focus listeners on processing formulaic items.
In answer to the third research question, the results revealed that there was a large positive correlation between incidental learning of collocations and frequency of occurrence. This finding supports the results of Webb et al.’s (Reference Webb, Newton and Chang2013) study of incidental learning of collocations and contrasts the results of Pellicer-Sánchez (Reference Pellicer-Sánchez2017). The difference in findings may be due to the range in frequencies examined in the three studies. In the present study the frequency of occurrence of target items ranged from 1 to 16 encounters. Similarly, in Webb et al.’s (Reference Webb, Newton and Chang2013) study items ranged in frequency from 1 to 15 times. In contrast, the frequency of items in Pellicer-Sánchez (Reference Pellicer-Sánchez2017) was either four or eight encounters. The lack of a frequency effect in that study may thus be due to the narrow range in frequency. Indeed, studies examining frequency of single-word items have found that there may be no effect between items that were close in frequency (e.g., Rott, Reference Rott1999). Together, the results of these studies indicate the value of including a wide range in the frequencies of target items when investigating this variable.
The size of the correlation found between frequency of occurrence and incidental vocabulary learning in this study (.61) was higher than several studies of incidental vocabulary learning of single-word items through reading a single text. For example, Webb (Reference Webb2007) reported correlations of .23 and .43 between frequency of occurrence and learning on two tests, Saragi et al. (Reference Saragi, Nation and Meister1978) found a correlation of .34, while Horst et al. (Reference Horst, Cobb and Meara1998) found a correlation of .49. However, variation in the size of correlations between these studies may not necessarily indicate a stronger relationship for frequency of occurrence and learning collocations than single-word items. Instead the differences may be due to factors such as test formats (Webb, Reference Webb and Chappelle2014), as well as the length of time between encountering the target items and testing (Webb & Chang, Reference Webb and Chang2015b). We suggest that it would be more accurate to conclude that frequency of occurrence may have a similar impact to the learning of collocations as it does single-word items.
CONCLUSION
The findings provide support for the use of audio-assisted reading inside and outside the classroom. At present, there does not appear to be any research indicating that reading while listening has a negative impact on language learning. However, the present study and an earlier study by Webb and Chang (Reference Webb and Chang2012a) indicate that reading while listening may be superior to reading in developing vocabulary knowledge. Studies comparing audio-assisted and unassisted repeated reading also indirectly indicate that reading while listening may lead to greater improvement in reading rate and motivation to read (Blum et al., Reference Blum, Koskinen, Tennant, Parker, Straub and Curry1995; Chang & Millett, Reference Chang and Millett2015). Because a large proportion of course materials come with both audio and written versions, and the majority of graded readers have corresponding audio versions, there is the potential to do a large amount of reading with the support of audio. Moreover, online video and television often include captions (English subtitles of the audio soundtrack) that allow reading while listening to occur in multimodal L2 input. Because there are still relatively few studies that have compared learning in different modes of L2 input, further research is needed. In particular, it would be useful to further compare the language learning gains that are made through reading and reading while listening.
The results of the present study also indicate that listening may make a greater contribution to lexical development than has been indicated in earlier studies. Earlier research has indicated that reading and reading while listening are superior to listening in the learning of single-word items (Brown et al., Reference Brown, Waring and Donkaewbua2008; Vidal, Reference Vidal2011). However, the findings of this study suggest that sequences of words may be learned at a similar rate for both reading and listening. Because EFL learners in some contexts may receive the bulk of their L2 input outside of the classroom in the form of viewing subtitled television programs and movies (Lindgren & Muñoz, Reference Lindgren and Muñoz2013; E. Peters, Reference Peters2018), there may be the potential for spoken input in this mode to also contribute to the learning of collocations. This would be a useful avenue for future research.
Another useful avenue for future research is investigating the extent to which knowledge of the component words contributes to learning. In the present study, many of the component words that made up the collocations were high-frequency words (e.g., building, park, work). However, some items included less frequent component words that were more likely to be unknown (e.g., bulletin, lethal, pharmacy). It would be useful to investigate how learners’ knowledge of the component affects gains. It may be that a lack of familiarity with component words makes collocations more salient increasing their potential for learning, or it could be that familiarity with component words makes collocations more easily inferred. Moreover, it would also be beneficial to examine whether gains in knowledge of collocations are affected by prior vocabulary knowledge. Studies of single-word items have been inconsistent with several studies revealing positive effects of prior vocabulary knowledge on word learning (e.g., Tekmen & Daloǧlu, Reference Tekmen and Daloǧlu2006; Webb & Chang, Reference Webb and Chang2015a; Zahar et al., Reference Zahar, Cobb and Spada2001) and one study showing negligible effects (Reynolds, Reference Reynolds2019).
It is also important to note that the collocations selected for learning were likely to vary in difficulty within this study, as well as with the difficulty levels of items in other studies. The ease with which collocations are learned is affected by many factors such as semantic transparency (e.g., Boers & Webb, Reference Boers and Webb2018), the part of speech of the component words (e.g., Nesselhauf, Reference Nesselhauf2003), L1-L2 congruency (e.g., Nesselhauf, Reference Nesselhauf2003), and variation in the forms of component words across encounters (Reynolds & Wible, Reference Reynolds and Wible2014). Variation in the difficulty of target items is not unique to this study, target single-word and multiword items are likely to vary in how easily they are learned across all studies of vocabulary learning. Therefore, it is important to be cautious before generalizing the extent of learning gains from one context to another.
Finally, there would also be value in investigating the relative effects of frequency of occurrence of single-word and multiword target items. Research on single-word items tends to reveal an effect of frequency on learning when learning in a single session (e.g., Horst et al., Reference Horst, Cobb and Meara1998; Uchihara et al., Reference Uchihara, Webb and Yanagisawa2019). In the present study, a frequency effect was found for learning collocations in the reading while listening condition, but not in the listening and reading conditions. Contrasting results have also been found in earlier studies of multiword studies (Pellicer-Sánchez, Reference Pellicer-Sánchez2017; Webb et al., Reference Webb, Newton and Chang2013). There are several reasons why frequency may have differing effects on learning individual words and collocations. First, the frequency of a sequence of words can only be as frequent as the most frequent word in the word string. Thus, single-word items tend to be more frequent than multiword items. Second, single-word items typically convey meanings, but this may not be clear for many strings of words; learners might often focus on the meanings of the individual words that make up sequences rather than on the unit as a whole. Third, each time a single-word item is encountered, its occurrence may help to develop knowledge of that word. In contrast, each time multiword items are encountered, there may be an interval in which different component words of the string are encountered in other word strings (e.g., lose may occur in lose weight and then in lose sight of, and then in lose faith in before it is encountered again in the sequence lose weight). It is not clear how this might affect learning, but we might assume that it would have a different effect from the consistent repetitions of single-word items. Explicitly investigating the relative effects of frequency on individual and multiword items would help learners, teachers, and researchers to better understand how input may contribute to learning single-word and multiword items.
