2.1. How large is the estimated effect?

Glossing typically serves two functions: it directs the reader's attention to lexical items they may not yet be familiar with (the items are usually typographically enhanced in the text) and it clarifies their meaning (provided the reader finds the glosses clear enough). It stands to reason that glosses improve the chances of vocabulary acquisition relative to reading non-glossed texts, because in the latter case learners are less likely to attend to new words (see, e.g., Schmidt, Reference Schmidt and Robinson2001; Sharwood-Smith, Reference Sharwood-Smith1993, on the importance of attention) and when they do try to infer the meaning of unfamiliar items, they are often unsuccessful (Bensoussan & Laufer, Reference Bensoussan and Laufer1984; Laufer & Sim, Reference Laufer and Sim1985; Na & Nation, Reference Na and Nation1985; Nassaji, Reference Nassaji2003). That said, it is well established that vocabulary acquisition does also happen—albeit slowly—through reading texts without the support of glosses (e.g., Day et al., Reference Day, Omura and Hiramatsu1991; Elgort & Warren, Reference Elgort and Warren2014; Godfroid et al., Reference Godfroid, Ahn, Choi, Ballard, Cui, Johnston, Lee, Sarkar and Yoon2018; Horst et al., Reference Horst, Cobb and Meara1998; Pellicer-Sánchez, Reference Pellicer-Sánchez2016; Pitts et al., Reference Pitts, White and Krashen1989; Zahar et al., Reference Zahar, Cobb and Spada2001). The question is thus not whether adding glosses supports vocabulary acquisition, but rather how much more vocabulary acquisition can reasonably be expected thanks to glossing.

An early meta-analysis addressing this question was Abraham (Reference Abraham2008). This analysis focused on computer-mediated glosses, that is, hyperlinked annotations that learners can access during on-screen reading. Drawing on six studiesFootnote ¹ that compared scores on vocabulary tests administered shortly after the reading activity, Abraham found a large average effect in favour of glossing (Hedges’ g = 1.40). It is important to note that the effect sizes varied considerably across the individual studies, as captured by the 95% confidence intervals (0.73–1.76). In one study, Ben Salem (Reference Ben Salem2006) the effect of glossing was extraordinarily large (Hedges’ g = 2.90), reflecting a very wide gap between the scores obtained on vocabulary posttests after reading texts with versus without glosses. Such a gap does not necessarily entail a spectacular amount of vocabulary learning through glossed reading, though. It also emerges when the non-glossed reading condition brings about no vocabulary learning, while the glossed reading does (e.g., Abraham, Reference Abraham2007; Ben Salem, Reference Ben Salem2006).

A possible explanation for the negligible vocabulary learning from reading non-glossed texts in these experiments is that some of the texts used were not quite comprehensible without the glosses. When a text is not understood by learners, they will not be able to make use of context to infer the meaning of unfamiliar words (Laufer, Reference Laufer2020). Whether a text will be sufficiently well understood depends on numerous factors (Bernhardt, Reference Bernhardt2010; Koda, Reference Koda2005), and one such factor is what proportion of the words in the text the learner already knows. The more unfamiliar words there are in a text, the more challenging text comprehension will be. When more than 5% (that is, one in 20) of the running words are unfamiliar, text comprehension will be seriously compromised (Hu & Nation, Reference Hu and Nation2000; Laufer & Ravenhorst-Kalovski, Reference Laufer and Ravenhorst-Kalovski2010; Schmitt et al., Reference Schmitt, Grabe and Jiang2011). Going by glossing density (e.g., 8.4% of the running words were glossed in Abraham, Reference Abraham2007) as a rough indication of how many words the researchers considered unfamiliar to the participants in their studies, several of the texts used in the research on glossing to date were probably not comprehensible without the glosses (see section 2.2 for more examples). If so, the question needs to be asked whether it is realistic to require learners to read a text without any assistance when the text is in fact unsuitable for this.

To find out if a non-glossed text version was sufficiently comprehensible to a given group of participants, a measure of text comprehension is useful. It so happens that the primary studies in Abraham's (Reference Abraham2008) meta-analytic review that examined vocabulary learning also examined text comprehension, allowing Abraham to supply a list of effect sizes (p. 225) regarding how well the glossing benefited learners’ performance on text comprehension measures in comparison with the non-glossed reading. These effect sizes again vary widely across individual studies, but the largest effects were observed in the studies where the non-glossed reading brought about only negligible vocabulary learning (e.g., Abraham, Reference Abraham2007; Ben Salem, Reference Ben Salem2006).Footnote ² Calculating the correlation between the effect sizes for vocabulary learning for the six studies and their effect sizes for text comprehension yields r = 0.91 (r_s = 0.77). In short, the poorer the comprehension of the non-glossed texts in comparison with the glossed versions, the larger the effect of glossing on vocabulary learning tends to be.

Abraham (Reference Abraham2008) was an early and small-scale meta-analysis. A more recent and much more extensive meta-analytic review, by Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020), also lends strong support to the hypothesis that the effect of glossing on L2 vocabulary acquisition is substantial. Apart from the larger scale (42 studies), Yanagisawa et al.'s meta-analysis differs from Abraham's in the following respects: (a) it includes studies on glossing in both print and computer-mediated reading conditions, (b) it includes only data sets concerning glosses that provide a single annotation per target word (see further below for multi-gloss and multimodal gloss implementations), and (c) besides studies that directly compare glossed and non-glossed reading, Yanagisawa et al.'s meta-analysis includes studies that compare different kinds of glosses but without a non-glossed comparison condition. The advantage of including the latter studies is that more data sets can be added to the analysis. A disadvantage is that the comparison between glossed and non-glossed reading becomes less direct, since learning outcomes from experiments where learners only read glossed texts are now being compared with outcomes from other experiments where learners read other texts (with other target words) in both glossed and non-glossed versions. Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020) computed the average learning gains for all the data sets regarding glossed reading and then again for all the data sets regarding non-glossed reading. They found an average learning gain from reading glossed text of 28.4% according to recall tests (e.g., where participants are presented with the L2 words and asked to supply their meaning) administered shortly after the reading activities. Recognition tests (e.g., where participants are required to match the L2 words to their correct meaning, typically using a multiple-choice (MC) format) yielded an average of 54.6%. Average gains from reading non-glossed text were much smaller: 14.1% according to recall tests and 33.8% according to recognition tests, administered shortly after the reading activities.

It is important to note that the vocabulary gains show wide variation across data sets in Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020), as they did in Abraham (Reference Abraham2008). This variation is reflected by the 95% confidence intervals for the learning gains reported by Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020). For example, regarding the gains measured by recall tests, these CIs were 18.2%–38.7% after glossed reading and 6.4%–20.2% after non-glossed reading. Much of the variation in learning gains must be owing to differences in the design features of the studies. For example, while in most of the studies (e.g., Ko, Reference Ko2012) learners read a glossed text and were given a test on the glossed vocabulary immediately afterwards in the same session, others (e.g., Jung, Reference Jung2016) distributed reading tasks over separate sessions and what was labelled an immediate test by the researchers actually came with a delay regarding at least some of the reading texts. In the case of delayed tests, variation in outcomes may in part be owing to the differences in time lag between the reading activity and the test as well, ranging from one week (e.g., Watanabe, Reference Watanabe1997) to five weeks (e.g., Nagata, Reference Nagata1999).

The variation in outcomes must also be influenced by the design of the test instruments, especially in the case of multiple-choice tests (MCTs). For example, some MCTs used distractors that resembled the meaning of the target words (e.g., Miyasako, Reference Miyasako2002), making the test more challenging than using semantically distinct options (e.g., Yoshii, Reference Yoshii2006). Sometimes the distractors in the MCT were meanings that the participants had encountered during the reading activity (e.g., Jung, Reference Jung2016), while at other times participants could eliminate proposed meanings because they had not met them at all during the reading activity (e.g., Zhao & Ren, 2019). Furthermore, the fewer the number of response options in MCTs, the greater the likelihood of lucky guesses and thus the risk of inflated success rates. In the collection of primary studies examined by Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020), the number of options in the MCTs range from three (e.g., Ko, Reference Ko1995) to eight (e.g., Shahrokni, Reference Shahrokni2009).

All this variation makes estimating the overall effectiveness of glossing complicated, especially when data sets come from studies with and without direct comparisons of glossed and non-glossed reading. Another factor worth mentioning is that, in a few of the primary studies included in Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020), some or all of the target words occurred several times in the texts (e.g., Hulstijn et al., Reference Hulstijn, Hollander and Greidanus1996; Nagata, Reference Nagata1999; Rott et al., Reference Rott, Williams and Cameron2002). Repeated encounters with words help their acquisition (Uchihara et al., Reference Uchihara, Webb and Yanagisawa2019, for a meta-analysis), and this holds true also for glossed texts (e.g., Teng, Reference Teng2020). In a comparison of data sets from glossed or non-glossed reading conditions, an uneven distribution of datasets from experiments where the target words were encountered several times by the participants is thus also likely to influence the outcome.

2.2. Matters of text comprehensibility

Returning to an observation made previously, it is striking how little vocabulary learning happened through non-glossed reading in several experiments (e.g., Jung, Reference Jung2016; Samian et al., Reference Samian, Foo and Mohebbi2016; Watanabe, Reference Watanabe1997; Zhao & Ren, Reference Zhao and Ren2019). It needs to be reiterated that, in some studies, the non-glossed reading condition was not altogether realistic. If it is deemed necessary to add a gloss per every 20 or so running words in order to make the text comprehensible for a given learner group (e.g., Arpaci, Reference Arpaci2016; Azari et al., Reference Azari, Abdullah, Heng and Hoon2012; Jacobs et al., Reference Jacobs, Dufon and Hong1994; Sadeghi & Ahmadi, Reference Sadeghi and Ahmadi2012), then the flipside of the coin is that, without the glosses, the text is probably not going to be comprehensible. A well-informed teacher would probably not choose such material for their students’ unassisted reading in the first place. It is true that highly motivated learners may try to read texts that are far beyond their proficiency level in their spare time, for example because the texts concern a subject of personal interest, but such learners will then probably consult resources such as (online) dictionaries when they fail to comprehend a passage that they are really interested in. The participants in the non-glossed reading conditions in the experiments reviewed here had no access to any resources outside the actual texts.

To be fair, several of the research reports (e.g., Cheng & Good, Reference Cheng and Good2009; Ko, Reference Ko2012; Zhao & Ren, 2019) do include good justifications for the choice of texts (along with justifications for the choice of target words), with mention of piloting efforts and consultation of the students’ instructors. Often, though, the sole assurance given that the texts were likely to be comprehensible is a readability measure such as the Flesch-Kincaid formula (e.g., Farvardin & Biria, Reference Farvardin and Biria2012; Samian et al., Reference Samian, Foo and Mohebbi2016; Shahrokni, Reference Shahrokni2009; Watanabe, Reference Watanabe1997). It is worth noting that this formula concerns mostly syntactic complexity and was originally intended for estimating readability in a first language (L1). It may not suffice as assurance about text comprehensibility for L2 readers, for whom the lexical profile of a text is likely to matter more (Qian & Lin, Reference Qian, Lin and Webb2020, pp. 68–69). Administering a Vocabulary Levels Test (Schmitt et al., Reference Schmitt, Schmitt and Clapham2001; Webb et al., Reference Webb, Sasao and Balance2017) to estimate whether learners will be able to cope with a given text is probably a better assurance. There are also several research reports where no assurance is given regarding the suitability of the chosen texts for the given learner groups (e.g., Abraham, Reference Abraham2007; Lee et al., Reference Lee, Lee and Lee2016; Moradan & Vafaei, Reference Moradan and Vafaei2016; Nagata, Reference Nagata1999; Varol & Erçetin, 2016).

In addition, some researchers deliberately chose reading texts about a topic that was novel to their participants (e.g., Arpaci, Reference Arpaci2016; Jung, Reference Jung2016). This was done to control for topic familiarity, because topic familiarity is one of the factors known to aid text comprehension and, in turn, the ability to make effective use of context to infer word meanings (Pulido, Reference Pulido2004). Without background knowledge about the subject of a text, inferring the meaning of unknown words is bound to be especially challenging, and so vocabulary gains from non-glossed reading will likely be particularly small. Moreover, some researchers went to great lengths to make sure that the meaning of the target words was not inferable from context (Ben Salem, Reference Ben Salem2006), thus making vocabulary learning from the non-glossed version virtually impossible. Using texts that are selected, modified or created with a view to preventing successful inferencing can be helpful for the researcher to isolate the effect of glosses, because any learning of new word meanings from reading the glossed version of such a text will then be attributable to the glosses (rather than successful inferencing). At the same time, one may wonder whether teachers often ask their students to (a) read texts on totally unfamiliar topics without any preparation and (b) work with texts that have been specially modified to prevent successful inferencing.

There is no doubt that glossing assists vocabulary learning, but, as I argued above, the gap between the amount of vocabulary learning from glossed and non-glossed versions of texts could be narrower in actual pedagogic practice, where it is hoped that learners read texts that are within their comfort zone, and where consequently also non-glossed versions provide the learners with a chance of picking up at least a few new lexical items.

2.3. Diversity in reading tasks

What the participants were asked to do with the reading materials is bound to influence learning outcomes as well. In only a small number of the studies were students simply asked to read texts for comprehension. In most, the students were given tasks that engaged them intensively with the texts and their glosses. For example, students were asked to read a text with a view to reproducing its contents (e.g., Gan, Reference Gan2014; Ko, Reference Ko2012; Rott et al., Reference Rott, Williams and Cameron2002) or with a view to answering numerous questions that prompted the students to inspect the glosses (e.g., Samian et al., Reference Samian, Foo and Mohebbi2016). The effectiveness for vocabulary learning of text comprehension tasks that direct learners’ attention to target words has been demonstrated convincingly by Peters et al. (Reference Peters, Hulstijn, Sercu and Lutjeharms2009). Sometimes, the participants were explicitly told they should examine the glosses (e.g., Aizawa, Reference Aizawa1998; Lee et al., Reference Lee, Lee and Lee2016; Rassaei, Reference Rassaei2018; Salehi & Naserieh, Reference Salehi and Naserieh2013). Still, other studies examined the merits of MC glosses, which require the reader to choose which of two or more proposed word meanings matches the contextual use of the target word (see section 3.4). In some experiments (e.g., Rott, Reference Rott2005), the students were asked to first read a text without glosses and then with glosses, and to think aloud while interacting with the text and the glosses (also see section 4).

In most of the studies, learners worked at length with a short text passage (sometimes just one short paragraph), while in other cases they read several pages at a more natural pace. For example, in Arpaci (Reference Arpaci2016) students spent 25 minutes on a 250-word passage, whereas in Hulstijn et al. (Reference Hulstijn, Hollander and Greidanus1996) students spent the same amount of time on a text of over 1,300 words. It is likely that learners examine language features (including glossed lexical items) more closely if they are asked to work at length with a short text than when they read a long text at a more natural pace. It is unfortunate that few authors mention both text length and how long the reading activities took. Among the few studies included in Yanagisawa et al.'s meta-analysis that do mention this information, the time-on-text ratio varies widely, from just seven words per minute (Nagata, Reference Nagata1999) to 92 words per minute (Zhao & Ren, Reference Zhao and Ren2019).

To examine whether the time-on-text ratio is likely to influence vocabulary learning gains—as one would expect—I ran the following correlational analysis of a handful of data setsFootnote ³ included in Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020). For the sake of comparability, these data sets are exclusively from studies that used: (a) texts in which the words of interest occurred once, (b) meaning-recall posttests in the same session as the reading activity, and (c) marginal glosses offering a single meaning (no MC glosses—see section 3.4). I found a negative correlation between the average number of words dealt with per minute by participants and their vocabulary gains: r = −0.412 (n = 6). So, it is indeed when learners spend a long time working with short texts that substantial vocabulary gains tend to occur. This helps to put the average learning gains that Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020) found into perspective, because a large number of primary studies included in their meta-analysis engaged learners with short texts for considerable lengths of time, whereby these learners possibly considered these texts as materials for language-focused (and vocabulary-focused) study rather than materials for ‘mere’ reading practice.

Still, articles reporting research on glossing almost invariably characterize the observed learning as ‘incidental’, that is, as acquisition from activities that are focused on communicative content rather than the language code. It is true that participants in most of these studies were neither instructed explicitly to memorize the glossed items, nor were they informed a test about them would follow. Absence of test announcement is now considered by many scholars as the most straightforward way of operationalizing the distinction between incidental and intentional learning (e.g., Hulstijn, Reference Hulstijn, Doughty and Long2003; Webb, Reference Webb and Webb2020, for discussion). That said, if students are given a decontextualized text passage that is only about 50 words long and in which no fewer than five (10%) of the words are highlighted and clarified in a glossary (Azari et al., Reference Azari, Abdullah, Heng and Hoon2012), then is it not likely that the students will experience this input as material for deliberate vocabulary study?Footnote ⁴ This is an extreme example, but, as mentioned previously, there are numerous studies that used short texts where almost every sentence had a glossed word (Abraham, Reference Abraham2007; Cheng & Good, Reference Cheng and Good2009; Nagata, Reference Nagata1999; Yoshii, Reference Yoshii2013). Even though they are not explicitly forewarned about a vocabulary test, might the nature of such reading conditions not create an expectation that such a test will follow—especially if the participants were previously given a pretest about the words that they subsequently see clarified in a glossed text?

This discussion of whether vocabulary learning from reading glossed texts should always be considered incidental is arguably just of academic interest, while practitioners are concerned first and foremost with what is best for their students’ learning outcomes, regardless of the cognitive mechanisms presumed to generate those outcomes. That said, if reading a glossed text is perceived by learners as a vocabulary study activity, then a practical side effect of this may be that the text itself will no longer be of much interest to them. There are indeed anecdotes in publications that participants spent more time studying the glosses than reading the text (e.g., Abraham, Reference Abraham2007, p. 105). In some of the experiments, measures of text comprehension or recall of text content showed either no effect of glossing (e.g., Cheng & Good, Reference Cheng and Good2009; Jacobs et al., Reference Jacobs, Dufon and Hong1994; Jung, Reference Jung2016) or in fact better results for the non-glossed condition (e.g., Plass et al., Reference Plass, Chun, Mayer and Leutner2003). In such cases, the glosses clearly did not serve the purpose that they were originally intended for—scaffolding text comprehension.

It is probably when learners are given a relatively long text with moderate use of glossing (e.g., glosses for under 2.5% of the running words, as was done in Hulstijn et al., Reference Hulstijn, Hollander and Greidanus1996)Footnote ⁵, and told to read for text comprehension that vocabulary gains from glossed reading can safely be attributed to incidental acquisition—where incidental acquisition is characterized not only by the absence of test announcement but also by the content-focused nature of the activity. In Hulstijn et al. (Reference Hulstijn, Hollander and Greidanus1996), the average learning gain (for words that occurred once in the text) from reading the glossed version of the text was 16% according to a meaning-recall test. This was better than the gain from reading the non-glossed version (just 4%), but clearly less impressive than the average gain from glossed reading calculated in Yanagisawa et al.'s (Reference Yanagisawa, Webb and Uchihara2020) meta-analysis (28.4%). Apart from exceptions such as Hulstijn et al. (Reference Hulstijn, Hollander and Greidanus1996), reading activities in studies on the effects of glossing have overwhelmingly used short texts with high glossing densities. It would benefit future meta-analyses—especially ones interested in incidental acquisition—if more studies with longer texts and moderate use of glossing were conducted. That said, practitioners may justifiably ask the question why glossing should not also be tried as an explicit and intentional language-focused intervention, anyhow (e.g., Khezrlou et al., Reference Khezrlou, Ellis and Sadeghi2017). For example, how effective is reading glossed text in comparison with reading non-glossed text when the learners are told beforehand to expect questions about text content and vocabulary?

So far, I have presented evidence from meta-analytic reviews that glossing positively influences vocabulary learning, but have also argued that the general magnitude of this positive effect needs to be interpreted in a nuanced and cautious way, considering the wide diversity in study designs and instruments used in the collection of primary studies. The discussion so far concerned a broad comparison of glossed and non-glossed reading. Absent from the discussion so far has been the question whether some implementations or types of glossing are more effective than others. So, let us now turn to such comparisons. Topics to be discussed include the location of glosses (e.g., in the margin or elsewhere), the language of glosses (L1 or L2), the modality of glosses (e.g., a combination of text and picture), ways of stimulating cognitive engagement with glosses (e.g., presenting the glosses as MC quizzes), and the use of glosses when target words occur several times in the same text.

3.1. The location of glosses

One question that has attracted a fair amount of interest is where a gloss should be provided for learners to make the best use of it. Research findings suggest that listing the words and their meanings at the very end of a text, in the form of a glossary, is not optimal (e.g., Cheng & Good, Reference Cheng and Good2009; Jacobs et al., Reference Jacobs, Dufon and Hong1994), probably because learners are reluctant to interrupt the flow of reading to look for an unfamiliar word in the glossary (Ko, Reference Ko2012). Instead, placing the gloss closer to the location of the unfamiliar word in the text, so it can be inspected with less interruption, appears the better choice. This is often done by providing the gloss in the margin next to the line where the unfamiliar word occurs. Another possibility is to provide it within the text, directly adjacent to the unfamiliar word. However, according to Yanagisawa et al.'s (Reference Yanagisawa, Webb and Uchihara2020) meta-analysis, which examined the location of glosses as one of several moderating variables, such in-text glosses appear comparatively ineffective for vocabulary uptake. This was based on data from only four studies that used in-text glosses, however, and it may be skewed by one data set (Watanabe, Reference Watanabe1997), which implemented in-text glossing as the provision of so-called appositives, inserting a clarification after the new word without overt indication that it serves this clarification purpose (as I did just now regarding the term appositive). When some of the participants were asked to infer the meaning of the target words with the text at their disposal, they often failed to do so, which suggests they simply did not recognize the purpose of the appositives. In other studies, where in-text glosses were more clearly signalled as word clarifications, these did yield comparatively good results (Ouyang et al., Reference Ouyang, Huang and Jiang2020; Zhang & Webb, Reference Zhang and Webb2019, for recent examples). A re-assessment of the effectiveness of in-text glosses may become feasible when more studies comparing in-text and marginal glosses become available.

The number of glossed words may be a moderating variable in this specific evaluation as well (Nation, Reference Nation2013, p. 242), because, if a high number of words are glossed in the margin of the text, then this will clutter the margin and it will become difficult to position each gloss next to the text line where its corresponding target word occurs—which may possibly cause cross-item confusion. Even if researchers choose a text in which ‘only’ 5% (i.e., one in 20) of the running words are unfamiliar to a given group of readers, providing a marginal gloss with each occurrence of an unfamiliar word would amount to a gloss next to almost every line of a text such as the one you are reading, and it is likely that occasionally a single line will include two unfamiliar words. If so, it may be more judicious to use a glossary at the end of the text after all. Alternatively, one could select only some of the unfamiliar words for marginal glossing and hope that the meaning of the remaining unfamiliar words will become better guessable. Selecting 2% would amount to about three marginal glosses in the case of the present paragraph, for example, making it easier to put the glosses next to the relevant text lines and making the amount of information presented in the margin better manageable for the reader. That said, more research, including studies of learners’ perspectives, is needed to support this recommendation.

A fair number of studies about glossing (e.g., Lee et al., Reference Lee, Lee and Lee2016; Yeung, Reference Yeung1999) have used a ‘cognitive load’ perspective (Sweller, Reference Sweller1994) to evaluate how excessive glossing may interfere with learners’ ability to process a text and the accompanying information. In the case of computer-mediated reading, the above problem of cluttering text margins with glosses can be avoided, as the glosses can be made accessible one at a time through hyperlinks. This may also hold an advantage over glossaries added at the end of the text, because the reader just needs to mouse-click on the word or move the cursor over it to access the gloss, which is less cumbersome than scrolling down to the end of the text to find the relevant information (especially if the text is long). That said, where the hyperlinked glosses are presented probably matters as well. If the hyperlinked gloss is presented on a separate screen (e.g., Rassaei, Reference Rassaei2018), then readers are temporarily taken out of the reading text and will need to re-connect with it. Learners may therefore prefer a design where a pop-up window appears on the same screen as the text passage (e.g., Bowles, Reference Bowles2004; Lee et al., Reference Lee, Lee and Lee2016). A recent study by Varol and Erçetin (Reference Varol and Erçetin2019) indeed suggests that learners are more likely to access hyperlinked glosses if these appear in a pop-up window on the same screen as the text than if they appear on a separate screen. This preference is likely to be particularly strong if there are many hyperlinks and text comprehension is challenging (as was the case in Varol and Erçetin's experiment). On the other hand, if there are just a few hyperlinks and the learners find the text manageable, then they may not mind studying the glosses on a separate screen. Considering the evidence in support of retrieval-from-memory practice (e.g., Van den Broek et al., Reference Van den Broek, Takashima, Segers and Verhoeven2018), there may even be grounds for hypothesizing that briefly holding the information from the gloss in memory and then reconnecting it to the instance of the word in the actual text is more beneficial for retention than seeing the gloss together with the word in the text (see also section 3.5).

3.2. L1 versus L2 glosses

The meta-analysis by Yanagisawa et al. (Reference Yanagisawa, Webb and Uchihara2020) examined the language of glosses (L1 translations versus L2 definitions) as a moderator variable and found an advantage for L1 glosses. This was confirmed by yet another recent meta-analytic review (Kim et al., Reference Kim, Lee and Lee2020), which compared the benefits of L1 and L2 glosses by specifically examining a collection of primary studies in which language choice was the independent variable of interest. In this collection, 21 studies included measures of vocabulary learning. L1 glossing was found to be the most beneficial, although the overall effect sizes in its favour were on the small side (g = 0.44 and 0.28 in immediate and delayed posttests, respectively). Interestingly, the advantage of L1 glosses was most pronounced in the case of studies with learners at low proficiency levels, likely because it is low-proficiency learners who find L1 glosses much easier to understand than L2 glosses.Footnote ⁶ It stands to reason that comprehension can only be supported well by glosses if the glosses themselves are comprehensible to the learner, meaning L2 glosses should be carefully designed to match the learners’ proficiency, for example by avoiding words that the learners might not yet be familiar with.

It is surprising in this regard, that information about the L2 glosses that were used is absent in most of the research reports (e.g., Arpaci, Reference Arpaci2016; Ertürk, Reference Ertürk2016; Farvardin & Biria, Reference Farvardin and Biria2012; Jacobs et al., Reference Jacobs, Dufon and Hong1994; Rouhi & Mohebbi, Reference Rouhi and Mohebbi2012; Shiki, Reference Shiki2008; Taheri & Zade, Reference Taheri and Zade2014; Zarei & Hasani, 2011) while others provide just one or a couple of examples (e.g., Mitarai & Aizawa, Reference Mitarai and Aizawa1999; Miyasako, Reference Miyasako2002; Salehi & Naserieh, Reference Salehi and Naserieh2013). This makes it impossible to evaluate how clear to the study participants these L2 definitions were in comparison with the L1 glosses. Even under the assumption that they were crystal clear, clarifications in L2 may still take more time and effort to process simply because they are longer than L1 translation equivalents, especially when abstract words need to be defined. For example, in Salimi and Mirian (Reference Salimi and Mirian2019) the L2 definitions were up to 14 words in length and in Öztürk and Yorgancı (Reference Öztürk and Yorgancı2017) many of the definitions were over 20 words.Footnote ⁷ It is no wonder in such cases that learners are reluctant to read L2 glosses and by far prefer L1 glosses (e.g., Salimi & Mirian, Reference Salimi and Mirian2019). However, this preference may change when the L2 glosses are concise and clear enough (e.g., Ko, Reference Ko2012). Still, even if the L2 glosses are kept concise, readers tend to process L1 glosses faster (e.g., Kang et al., Reference Kang, Kweon and Choi2020).

If learners (especially at low proficiency levels) are less inclined to inspect L2 glosses than L1 glosses for the above reasons, then this may hold true in particular when the reading activity is done under time pressure and when an excessive number of glosses is available.Footnote ⁸ It is indeed striking that the studies that showed poor effectiveness of L2 glosses compared with L1 glosses tend to be ones that used reading materials with a high glossing density (e.g., Arpaci, Reference Arpaci2016; Yoshii, Reference Yoshii2006).

Eye-tracking research would be welcome to explore the potential interaction between gloss language and reading behaviour at different proficiency levels, at varying glossing densities, and in timed versus untimed reading conditions. If such eye-movement research were to confirm that L2 glosses are more often ignored than L1 glosses, then the question should be asked what could be done to increase the likelihood that learners do make good use of L2 glosses. This question is relevant because many textbooks and other instructional materials for language learning are produced for a global market or are meant for groups of learners with mixed L1 backgrounds. In those cases, L1 glosses are not a viable option (although this may change as computer-mediated reading becomes increasingly common and learners may be offered options to customize their reading experience).

3.3. The addition of pictures

Word meanings can also be elucidated by pictures. Several studies have compared the effectiveness of text-only, picture-only, and text-plus-picture glosses (e.g., Kost et al., Reference Kost, Foss and Lenzini1999; Yoshii & Flaitz, Reference Yoshii and Flaitz2002) or just text-only and text-plus-picture glosses (e.g., Boers, Warren et al., Reference Boers, Warren, He and Deconinck2017). Collectively, the results suggest that text-plus-picture glosses are the most beneficial. This, at least, is the conclusion reached by a meta-analytic review by Yun (Reference Yun2011), although the average weighted effect was on the small side (Hedges g = 0.37).Footnote ⁹ This analysis was based on only ten studies at the time, however, and there are additional studies that did not furnish evidence in favour of the addition of pictures (Acha, Reference Acha2009; Ben Salem, Reference Ben Salem2006; Boers, Warren et al., Reference Boers, Warren, He and Deconinck2017).

Half of the studies included in Yun (Reference Yun2011) were computer-mediated reading experiments—many of which were inspired by Mayer's (Reference Mayer2009) theory of multimedia learning—where learners were invited to click on highlighted words to call up one or more glosses (Akbulut, Reference Akbulut2007; Chun & Plass, Reference Chun and Plass1996; Jones & Plass, Reference Jones and Plass2002; Plass et al., Reference Plass, Chun, Mayer and Leutner1998, Reference Plass, Chun, Mayer and Leutner2003). In some reading conditions, only one gloss (a textual clarification or a picture) was available for the target word, while in another condition the learners could call up two separate glosses (a textual clarification and a picture). The two-gloss condition was found to be the most beneficial for word learning. Although authors of these studies tend to ascribe this effect to multimodality (i.e., exposure to both a textual and a pictorial clarification), an alternative explanation is that the availability of two separate glosses prompts learners to check information about the words twice instead of just once (Plass et al., Reference Plass, Chun, Mayer and Leutner1998). A meta-analysis by Ramezanali et al. (Reference Ramezanali, Uchihara and Faez2021) compared reading conditions with one and two hyperlinked glosses at the readers’ disposal and indeed found a general advantage of having two glosses to inspect.Footnote ¹⁰ This advantage is not necessarily owing to multimodality per se, however. After all, looking up a word meaning more than once improves the likelihood of word learning also if it concerns only a textual clarification (e.g., Montero Perez et al., Reference Montero Perez, Peters and Desmet2018; Peters et al., Reference Peters, Hulstijn, Sercu and Lutjeharms2009; Rott, Reference Rott2007). To separate the hypothesized effect of multimodality from that of multiple lookups, future experiments could offer two hyperlinked glosses per target word where in some reading conditions participants are informed that both glosses are textual or both are pictorial, while in another condition they are told one gloss is textual and the other pictorial. Computer log data could reveal under which condition the participants look up a word more than once and posttest data could then reveal if the combination of text and picture leads to better recall when the number of lookups is statistically controlled.

Turning back to Yun's (Reference Yun2011) meta-analysis, this also included five studies where the multimodal information (text and picture) was provided in a single gloss. Three of these found an advantage of text-plus-picture glosses. In Kost et al. (Reference Kost, Foss and Lenzini1999), these glosses led to better performance than only textual or only pictorial glosses in one of three posttests. Similar results were obtained by Yoshii and Flaitz (Reference Yoshii and Flaitz2002) and Yoshii (Reference Yoshii2006). A potential issue with the latter two experiments (which used the same materials) is that the text-only and the picture-only annotations appear to have underspecified the meaning of target words. The examples of gloss types given in Yoshii and Flaitz (Reference Yoshii and Flaitz2002, p. 37) concern the verb dash. This was defined as ‘to move very quickly’ in the textual gloss and elucidated by a drawing of a running figure in the pictorial gloss. In a posttest where the learners were required to supply the meaning of dash, two points were awarded if this included both the semantic features ‘fast’ and ‘run’ and one point was awarded if only ‘run’ was included (p. 53). Since the manner of motion (run) was not specified in the textual gloss, it is not surprising that learners in that reading condition failed to obtain good scores. It was the group of participants who had seen the combination of the textual and the pictorial information who stood the best chance on the test, because the two pieces of information together provided the meaning that was needed to obtain full points in the test. If the textual annotation had been more precise, such as ‘dash = to run somewhere quickly’, the results might have been quite different. Yeh and Wang (Reference Yeh and Wang2003) is another article claiming superiority of text-plus-picture glosses over L2 definitions only, but in this study the posttest scores were in fact not significantly different. The final study in Yun's (Reference Yun2011) meta-analysis, Al-Seghayer (Reference Al-Seghayer2001) used a within-participant design where seven words in a text were glossed by L2 definitions, seven others by a definition plus a picture, and another seven by a definition plus a video clip. The latter gloss type—which very likely invites longer engagement with the gloss—led to the best outcome, but no significant difference was found between definition-only and definition-plus-picture glosses.

In sum, when text-plus-picture glossing was found to be more effective in some of the experiments, it is not always clear why it was more effective, because other factors than multimodality may have played a part. Many authors invoke Dual Coding Theory (Paivio, Reference Paivio1986) to explain the proclaimed benefits of adding visuals to glosses. According to this theory, words that are associated with a mental image of their referents are easier to remember than words that only have a propositional representation in the mental lexicon. However, it is not entirely clear how this theory helps to explain the positive findings about pictorial glosses, because the words glossed with pictures usually have concrete meanings and are thus mentally imageable anyhow. If you learn that the French word cheval means ‘horse’, then surely you can conjure up a mental picture of the animal without being shown a picture. It is probably when concrete referents that you are not yet familiar with are introduced, such as objects or actions that are technical or that are alien to your own culture, that visuals are the most helpful (Mayer & Anderson, Reference Mayer and Anderson1992; Mayer & Gallini, Reference Mayer and Gallini1990). If Dual Coding Theory does not offer a convincing explanation for the observed benefits of multimodal glosses, then an alternative explanation may be that pictures attract attention where verbal definitions alone fail to do so. An eye-tracking study by Warren et al. (Reference Warren, Boers, Grimshaw and Siyanova-Chanturia2018) indeed revealed that marginal glosses providing only a definition (in L2) in their experiment were ignored by participants far more often than glosses containing a picture as well as a definition.

3.4. Stimulating cognitive engagement

It is possible that glosses fail to leave robust mental representations of word meanings when learners ‘passively’ take in the information and then carry on reading without giving further thought to the glossed item—which is a likely scenario when the reader is (naturally) interested first and foremost in the content of the text rather than how it is worded. Hulstijn (Reference Hulstijn, Arnaud and Bejoint1992) explored using MC glosses for the purpose of promoting more cognitive engagement with glossed words. MC glosses require readers to closely examine the word's use in the text to determine which of the proposed meanings in the gloss best matches it. Unless the correct choice is immediately obvious (which arguably defeats the purpose of triggering an interpretation effort), a downside of MC glosses is that learners may pick the wrong meaning. For example, Watanabe (Reference Watanabe1997) tried using glosses where two meanings were proposed, but these two options (e.g., cribs =  (1) babies’ beds; (2) small beds) were often both plausible matches for the reading context. Since there was no further evidence in the text for students to confirm or revise their interpretations, it is not surprising that the posttest scores of students who received MC glosses did not trump those of students who had been provided with standard, single-meaning glosses. An alternative is to give immediate feedback on the choices made in the MC glosses. In Nagata (Reference Nagata1999), students were explicitly asked to make a choice between two translations, after which they were informed if their choice was correct. This implementation of MC glossing led to better meaning recall than standard glossing. In Nagata's (Reference Nagata1999) experiment, the incorrect translations were not plausible, and so the risk of error was minimal. Yoshii (Reference Yoshii2013) conducted a similar study but with a greater likelihood of wrong choices. Yoshii's (Reference Yoshii2013) article is exceptional in that it includes helpful data per student regarding which items they got wrong in the MC glosses and in the MC posttests. Using these data, I tallied how often a wrong choice during reading was followed by a wrong choice in the test, despite the corrective feedback given at the reading stage. Of the incorrect responses to the MC glosses, 22.5% and 35% were followed by incorrect responses in the immediate and the one-week delayed posttest, respectively. This is appreciably higher than the average error rates in these posttests regardless of whether the MC glosses were answered correctly, which were only 14% and 25.5%, respectively. Evidently, the effectiveness of MC glosses can be compromised when learners make wrong choices at the initial learning stage, even if corrective feedback is given (e.g., Boers, Dang et al., Reference Boers, Dang and Strong2017; Strong & Boers, Reference Strong and Boers2019, for more evidence of this kind). The available collection of studies examining the merits of MC glosses again illustrates that different implementations of a given technique can yield markedly different learning outcomes,Footnote ¹¹ and so it also illustrates the need for caution and nuance if one wishes to recommend the use of MC glosses. It will be helpful if, like Yoshii (Reference Yoshii2013), future studies on MC glosses include data on participants’ response behaviour as an indication of how challenging the MC glosses were.

There may be additional ways of inviting cognitive engagement with glosses. One possibility regards polysemy. In Boers (Reference Boers2000), for example, students majoring in commerce and economics read a text containing lexical items such as hurdles (in the sense of difficulties to be overcome), wean off (as in weaning an industry off state support) and weed out (as in weeding out corruption). Although these items occurred in the text in these abstract meanings, they also have literal meanings, from which the abstract uses are derived through metaphorization: hurdles are the barriers that athletes jump over in track sports such as steeplechase, weaning off means gradually stop breastfeeding one's baby, and weeding refers to pulling out unwanted plants in one's garden. Half of the students in the experiment read the text with marginal glosses that only gave these literal meanings; the other half received glosses paraphrasing the abstract meanings in the text (e.g., hurdles = difficulties; weaning off = making independent; weeding out = removing). The students who had received only the literal meanings obtained the better scores on a productive recall test three days after the reading activity, even though the target items were again embedded in abstract contexts. Presumably these students had needed to invest some effort into working out the connection between the literal meaning given in the glosses and the contextual use of the items. This is perhaps where Dual Coding Theory does offer a plausible account—linking abstract word uses to their literal origins increases their imageability and can thus make them easier to remember. If glosses that prompt learners to think about the connection between related meanings of a lexical item can make the item more memorable, then this is promising because polysemy is very common. Boers (Reference Boers2000) was just one experiment, however, and one with serious shortcomings (such as lack of pretesting). Approximate and conceptual replications would therefore be welcome.

In essence, the purpose of MC glosses or other glosses that do not present readers immediately with the precise contextual meaning of lexical items is to create temporary ambiguity, where readers are made to wonder for a little while what exactly the contextual meaning of the item is. It does appear important, however, to reduce the risk of wrong interpretations lingering in memory. Encounters with the same lexical item in additional contexts should be helpful.

3.5. Increasing the number of encounters

Some of the studies on glossing have used texts with several instances of the same target words. For example, Rott (Reference Rott2007) inserted four instances of the same unfamiliar words in short texts (of about 350 words in length) and found that providing a gloss on each instance had a greater positive effect than presenting the gloss only with the first instance. It is likely that seeing the same words glossed several times on the same page left the students with the impression that these must be very important and thus worth studying. However, with target words occurring more than once in a text, it also becomes possible to create reading conditions where learners need to recall information from a previously seen gloss that they no longer have access to. Huang and Lin (Reference Huang and Lin2014) incorporated target words three times in a text, but each occurrence of the target word was displayed on a new screen, and the students could not return to previous screens.Footnote ¹² In one reading condition, the words were glossed on every occurrence, but in another reading condition only the first and third instance were glossed. The students in this condition were asked to supply the meaning of the words when they were met the second time. There was a third reading condition, where the students saw the words first without glosses and were asked to infer their meanings before seeing the glosses on the following pages. The best posttest results were obtained when students had seen the words explained first, then needed to retrieve this information from memory on the second encounter, and then saw the gloss again.

Huang and Lin's (2014) study adds to an already large body of evidence of the effectiveness of retrieval of information from memory (e.g., Barcroft, Reference Barcroft2007, Reference Barcroft2015). Posttest performance was not as good when students had first been required to infer word meanings before seeing the glosses, and this was found to be owing to the instances where students had started off with incorrect inferences. That incorrect guesses or inferences can interfere with later recall of correct form-meaning correspondences has been observed in a few other studies, albeit not ones directly concerned with glossing (e.g., Elgort, Reference Elgort2017; Elgort et al., Reference Elgort, Beliaeva and Boers2020; Mondria, Reference Mondria2003). The students who engaged in neither inferencing nor retrieval in Huang and Lin (Reference Huang and Lin2014) because the target words were always accompanied by their glosses obtained the poorest posttests scores in this experiment. Note that, unlike in the other two reading conditions, the students in this condition were not explicitly prompted at any time to think about the meaning of the target words. Approximate replications in which word-focused tasks are present or absent in all the reading conditions would be welcome (see my comment in section 2.3 about incidental vs. intentional learning).

A combination of MC glosses and repeated encounters may also be a fruitful way of first arousing curiosity about the meaning of an item and then offering further opportunities to fine tune one's interpretation. Rott (Reference Rott2005) used a text in which target words occurred four times each and were glossed only on their first instance. The participants were asked to think aloud as they read the text. The glosses were either MC glosses or standard, single-meaning ones. The first instance of each of the target words in the text did not come with much supporting context to pinpoint their meanings, but the subsequent instances were accompanied by clearer clues so as to guide learners to the correct form-meaning correspondence. The MC glossing condition was found to lead to better word retention, because—as evidenced by the think-aloud data—it aroused curiosity about the correct meaning of the target words and, when students initially made a wrong inference, the subsequent instances of the target word helped them to rectify this (an opportunity for rectification that was missing from other studies on MC glosses that I reviewed in section 3.4, such as Yoshii, Reference Yoshii2013). Such findings suggest that temporary ambiguity can trigger a quest for word meaning that can be quite beneficial—provided this quest is successful.

Temporary ambiguity may also have played a part in a study by Warren et al. (Reference Warren, Boers, Grimshaw and Siyanova-Chanturia2018). Target pseudo-words occurred three times in a story presented on a series of computer screens and were glossed in the margin of the text on their first instance only. The other two instances of each target word occurred on subsequent screens and the participants could not return to the earlier screens to revisit the glosses. Three kinds of glosses were used, depending on the reading condition that the participants were assigned to: a textual clarification in L2, a picture, or a combination of text plus picture. The learners’ eye-movements were tracked as they read the text. The text-only glosses were the least effective, but this was in part owing to the lack of attention they attracted—they were ignored about 20% of the time, while glosses containing a picture were seldom ignored. The expectation was that the richest representation of meaning (i.e., the text-plus-picture glosses) would have the greatest positive effect on retention, but instead it was the picture-only glosses that emerged as the most beneficial. The authors speculated that a picture alone leaves some uncertainty about the precise meaning of a word, which then influences readers’ engagement with later contextual instances of the same word. For example, one of the target words denoted a sleeveless jacket. A picture of a sleeveless jacket as such does not stipulate that absence of sleeves must be a defining feature of the word. It was by processing the subsequent text passages that readers could ascertain this, for example, when the protagonist of the story, who was wearing a sleeveless jacket, was said to have scratches on his arms after clambering through thorny bushes.

So, what predicts learning may not only be the extent to which learners engage with a gloss, but also how the gloss influences engagement with the remainder of the text and with the target words when these are encountered again.