2.1 Theoretical background

To date, the vocabulary research field has not given much attention to the assessment of spoken vocabulary knowledge. A recent review of vocabulary measurement research by Read (Reference Read2013) also reflects this: out of the 50 sources covered, only two deal with the assessment of spoken vocabulary knowledge.

This prevalence of written vocabulary testing is also visible in research on listening. Listening studies use written tests, for example, to predict learners’ comprehension of spoken input (Stæhr Reference Stæhr2009; van Zeeland & Schmitt Reference van Zeeland and Schmitt2013a), and to explore the role of vocabulary size in listening (Alderson Reference Alderson2005; Stæhr Reference Stæhr2008). However, the use of written tests for listening purposes is problematic. There is an increasing recognition in the field that learners’ written and spoken vocabulary knowledge might differ, and that they should be assessed separately (Read Reference Read2007; Milton Reference Milton2009). This has also led to the recent development of an aural version of the Vocabulary Levels Test (VLT), of which validation results look promising (McLean, Kramer & Beglar Reference McLean, Kramer and Beglar2015).

So far, very few studies have directly assessed learners’ spoken versus written vocabulary knowledge, and these have used the Yes/No test format to do so (Milton & Hopkins Reference Milton and Hopkins2006; Milton, Wade & Hopkins Reference Milton, Wade, Hopkins, Chacón-Beltrán, Abello-Contesse, Torreblanca-López and López-Jiménez2010). However, the use of a Yes/No test for spoken words has not been validated. This leaves us with very little understanding of the accuracy with which this test format predicts learners’ spoken vocabulary knowledge.

A second question that arises is to what extent knowledge of spoken isolated word forms is generally ‘transferred’ to the listening context. Research has found considerable word recognition problems in L2 listening (Field Reference Field2003), and researchers have cautioned that recognition of isolated word forms does not guarantee recognition in ongoing speech (Vandergrift Reference Vandergrift2007; Field Reference Field2008). Any assessment of isolated words (either written or spoken) may therefore overestimate learners’ knowledge in the actual listening context, i.e. their contextualised spoken word knowledge.

2.2 Study findings

The first study in the thesis (van Zeeland Reference van Zeeland2013) assessed how learners’ knowledge of isolated word forms (in a meaning recall interview) predicted their knowledge of these same words in reading and listening. This question was mainly relevant for the listening context, for it was expected that learners might struggle to successfully recognise ‘known’ words in continuous speech. The study showed that, where learners knew the meaning of a word in isolation, they managed to understand it more often in reading than in listening.

The second study (van Zeeland Reference van Zeeland2014a) explored the relationship between learners’ written and spoken vocabulary knowledge (in two meaning recall interviews), as well as the difference between their knowledge of spoken words in isolation and in short sentence contexts. In addition, the study measured how accurately learners could self-assess their own knowledge of written and spoken vocabulary on a Yes/No test. This would assess the validity of using the Yes/No format in both modes.

The results of this second study showed a significant but small advantage of learners’ written over their spoken word knowledge, and a stronger correlation between the two than reported in studies using the Yes/No test format (e.g. Milton & Hopkins Reference Milton and Hopkins2006). However, in the spoken mode, learners often (in 16.8% of the cases) failed to recognise a known spoken word form when it was presented in a sentence. This again shows the gap between knowledge and recognition in the spoken mode. This indicates that a vocabulary test that presents words in isolation is likely to overestimate learners’ knowledge while listening to continuous speech, even if this is a spoken vocabulary test.

The self-assessment tasks furthermore showed that learners overestimated their knowledge in both modes, but significantly more often in the spoken mode. Overestimation in the spoken mode was caused primarily by mishearing the presented form as another known word. This indicates that positive responses on a spoken Yes/No test cannot always be trusted to reflect knowledge of that particular spoken word form, questioning the use of this test format to assess spoken word knowledge.

2.3 Implications for research

As discussed, learners’ written vocabulary knowledge was slightly larger than their spoken vocabulary knowledge, but the two were highly similar. In light of this similarity, it can be questioned if it is really necessary to measure knowledge separately in the two modes: written tests appear to give quite a good approximation of both. It should be added, however, that the studies discussed here all involved high-proficiency learners of English, following an M.A. or Ph.D. course at a British university. If lower-proficiency learners also have comparable knowledge of written and spoken vocabulary, is a question for future research. Similarly, the relationship between spoken and written knowledge might differ between learners of various first languages (L1s) (Milton & Hopkins Reference Milton and Hopkins2006), and this also warrants further research.

If future research goes on to further compare L2 written and spoken vocabulary knowledge, it should use an instrument that works equally well in the two modes. The studies discussed here all used a meaning recall interview, in which learners could freely recall what they knew about the word meaning after having seen or heard its form. Needless to say, if a test is used that does not function in the same way in the two modes (e.g. the Yes/No test, as based on the results of Study 2), this automatically skews results. Perhaps the written and newly developed spoken version of the VLT could provide a valid comparison, although this would first require a comparative analysis of the tests results in both modes.

It should be kept in mind that the VLT and other traditionally used vocabulary tests reflect learners’ knowledge of individual, isolated words forms only, which likely overestimates their knowledge (recognition) in listening. This is particularly important to take into account where test scores are used as indicators of learners’ listening ability, such as in lexical coverage studies (Nation Reference Nation2006; Stæhr Reference Stæhr2008; van Zeeland & Schmitt Reference van Zeeland and Schmitt2013a). If a learner achieves a certain score on a test such as the VST (Vocabulary Size Test), VLT or Yes/No test (even if this is a spoken variant), this does not necessarily entail his/her ability to recognise the same degree of vocabulary in ongoing speech. The percentage gap found here was 16.8%, but this cannot be generalised. Lexical recognition likely depends on various learner factors such as proficiency and segmentation skills, as well as input characteristics such as speed, accent, and word frequency.

Efforts to contextualise spoken vocabulary knowledge assessment (e.g. Read Reference Read2007; Barclay Reference Barclay2013) reflect that researchers are aware of this difference between isolated and contextualised word knowledge. Read (Reference Read2007), for example, introduced the idea of a sentence-based spoken Yes/No test. This sounds promising, although the results of Study 2 question the suitability of the self-assessment format for the spoken mode. Moreover, Shillaw (Reference Shillaw, Fitzpatrick and Barfield2009) found that adding a sentence to the written Yes/No test led to inflated scores. Future research could look at the possible ways of assessing contextualised spoken vocabulary knowledge. However, before such efforts are made, it is perhaps more useful to further explore the need for such a test. This requires more research on learners’ ability to recognise known vocabulary in speech. Eye-tracking could be used for this purpose. This technique has been used to explore the recognition of contextual spoken vocabulary in relation to specific variables, such as sound contrasts problematic for L1-specific learners (e.g. Broersma & Cutler Reference Broersma and Cutler2011). It should therefore also serve well in investigating learners’ general contextual (vs decontextual) vocabulary recognition. Not only could this shed more light on the need for contextualised assessment; it could also explore what learner- and input factors might affect the gap between contextual and decontextual vocabulary recognition.

Besides exploring learners’ ability to recognise words in context, research could look at how this ability could be improved. Field (Reference Field1998) has presented exercises to help learners to develop their recognition skills, and has introduced ideas on how to facilitate recognition in spoken input. Future research could empirically assess the effect of such methods, possibly with learners of different L1s and proficiency levels. This would give teachers and learners concrete tools to work with.

3.1 Theoretical background

The second area in which listening has received less attention than reading is in the enhancement of vocabulary knowledge. This neglect is particularly evident in the area of lexical inferencing, but also in that of incidental vocabulary acquisition.

Lexical inferencing is an important vocabulary learning strategy (Nation Reference Nation2001), but virtually all lexical inferencing studies have involved reading. These studies have identified three main variables that affect lexical inferencing success: background knowledge (i.e. topic familiarity aids lexical inferencing; e.g. Pulido Reference Pulido2007), contextual clue types (i.e. so-called ‘local clues’, in the sentence context of the unknown word, lead to more successful inferencing than so-called ‘global clues’ elsewhere in the text; e.g. Qian Reference Qian2005), and L2 vocabulary size (i.e. the more words a learner knows, the more lexical inferencing success; e.g. Wesche & Paribakht Reference Wesche and Paribakht2010).

Incidental vocabulary acquisition has similarly been explored mainly in the realm of reading. Several studies (e.g. Webb Reference Webb2007; Pellicer-Sánchez & Schmitt Reference Pellicer-Sánchez and Schmitt2010) have explored how often a word needs to be read (i.e. the ‘frequency of occurrence’ required) for learners to acquire various dimensions of vocabulary knowledge (e.g. a word's form, grammatical function, and meaning). This so-called ‘dimensional’ framework is useful to reveal increments in learning. Learners generally develop the ability to recognise word forms before the ability to recall their meanings, for example, and multiple tests can tap into the acquisition of these different dimensions.

3.2 Study findings

Study 3 in the thesis (van Zeeland Reference van Zeeland2014b) assessed the extent to which L1 and L2 listeners’ lexical inferencing success is affected by the three variables found to play a role in reading research (discussed above). In addition, the study explored to what extent L2 listeners notice unknown vocabulary in listening, for learners can only go about inferring the meaning of a word once they have identified it as an unfamiliar form. A separate listening passage containing pseudowords was used for this purpose. Taking the results from Study 2, it was expected that learners might struggle to identify these pseudowords as unfamiliar while listening.

The study found lexical inferencing success rates of 59.6% and 35.6% by the native and non-native listeners. The significantly lower success rate of the non-natives was partly due to the relatively large number of cases in which they failed to spot the words to be inferred. Both the native and non-native participants showed the same as that found in reading research: inferences were more successful if listeners could use local (vs global) cues, had high (vs low) background knowledge, and had a large (vs small) vocabulary size (this third factor was only assessed in the L2 group). Results also showed that L2 listeners often failed to notice unknown vocabulary: they successfully identified less than half (44.2%) of the pseudowords in the passage. This suggests that learners’ lexical inferencing can be seriously hindered by limited lexical recognition skills.

Study 4 (van Zeeland & Schmitt Reference van Zeeland and Schmitt2013b) analysed L2 learners’ incidental learning from listening. It assessed the learning of three aspects of knowledge: form recognition, grammar recognition, and meaning recall. Results showed that listening primarily led to participants’ ability to recognise word forms. Word meaning was most difficult to acquire. Interestingly, results showed no clear relationship between learners’ acquisition of the target items and how often these items had appeared in the input (3 to 15 times), and any frequency effects on the immediate post-test had disappeared two weeks after listening. This suggests that, for listening to lead to durable learning, more than 15 occurrences of words are needed, regardless of which of these three knowledge dimensions is assessed.

3.3 Implications for research

The findings from Study 3 emphasise the importance of carefully selecting materials for both research and teaching. For research purposes, it is important to take into account background knowledge, clues, and participants’ vocabulary knowledge when setting up a new study as well as when interpreting results. For teaching and learning, these variables are similarly important to consider when selecting materials. Learners are more likely to infer (and likely also acquire) new word meanings if they can rely on background knowledge as a resource, and if the text contains sufficient local clues. Texts can be selected with these aspects in mind. In addition, the finding that vocabulary knowledge aids inferencing suggests that lexical inferencing might be a more appropriate learning strategy once learners have achieved a certain level of proficiency. However, the finding that even the natives inferred ‘only’ 60% of the words correctly emphasises that lexical inferencing is a complex process which depends on more than linguistic knowledge. While this study demonstrated three variables that play a role, there must be more, and this is an interesting question for future research.

Research that goes on to explore this issue may look at other variables than the ones chosen in this study. These three variables were chosen because they had been found to affect inferencing success in reading. While this was a logical starting point, it could also be argued that the study thereby took a ‘reading-biased’ approach. Especially in case of the clues: listeners are less likely to employ contextual clues than readers, for they do not have a static text to search for clues, nor the time to do elaborate processing of them. An interesting question for future research would be if listeners’ lexical inferencing success is affected by factors that suit the listening context better, such as certain spoken language characteristics (e.g. prosody and discourse markers).

The rather low pick-up rates found in Study 4 indicate that teachers and learners should not have high expectations from listening when it comes to incidental vocabulary learning. In particular, the form-meaning link was hardly picked up by participants. This suggests that listening may be more appropriate for strengthening partial knowledge rather than for establishing new knowledge. Perhaps it is best to teach or learn new word meanings explicitly, and use listening to gain deeper knowledge of words, such as nuances in meaning, collocations, and word frequency. The potential of listening for such learning targets is a further interesting point for future research.

The lack of a clear frequency effect on learning suggests that words need to be repeated many times for learners to acquire and retain knowledge of their form, grammatical function, and meaning. However, general listening materials are unlikely to offer sufficient repetitions. Webb & Rodgers (Reference Webb and Rodgers2009) carried out a frequency analysis of British and American television language, and found that the repetition rate of low-frequency vocabulary, which is the learning target for many learners, is not high. In 35.1 hours of viewing time, including 264,384 words, 14% of the low-frequency vocabulary occurred 3–4 times, 6% 5–7 times, and only 4% 10+ times. If listening is used as a learning source, it may therefore be best to use the ‘narrow listening’ approach in which learners listen to materials with a particular focus so that they encounter the same words more often than they otherwise would. This might also be an interesting focus for future research on incidental learning from listening.

It is most likely that the small frequency effect found here and in previous listening studies (Brown, Waring & Donkaewbua Reference Brown, Waring and Donkaewbua2008; Vidal Reference Vidal2011) is partly due to limited noticing of the target items, as suggested by the pseudowords spotting task in Study 3. Clearly, as noted by Nation (Reference Nation2001: 67−68), ‘a repetition can only be effective if the repetition is seen by the learner’. It appears that this is often not the case in listening. This questions the appropriateness of exploring the effect of frequency of occurrence on learning from spoken input, at least where it involves frequency thresholds that are quite close to one another. When it comes to acquisition from spoken language, it may be more appropriate to measure how a general increase in exposures aids learning rather than to use clear-cut frequency points.