The purpose of this study is to identify the optimal practice schedule for second language (L2) vocabulary learning in a paired-associate format. In a typical paired-associate learning (PAL) task, in the first encounter with a given item, learners are presented with a target L2 word together with its meaning (e.g., chakura = food) and are asked to make an association between them (e.g., Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007). In the second encounter, learners practice retrieval. In other words, they are presented with a L2 word and are asked to recall its meaning (e.g., chakura = ____?) or vice versa (e.g., food = ____?). Retrieval is usually followed by feedback, during which learners are presented with the L2 target word and its meaning (e.g., Kang, Lindsey, Mozer, & Pashler, Reference Kang, Lindsey, Mozer and Pashler2014; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007). The present study brings PAL into focus because it is not only common (e.g., Nakata, Reference Nakata2011; Schmitt, Reference Schmitt, Schmitt and McCarthy1997) but also effective and efficient for L2 lexical development (e.g., Elgort, Reference Elgort2011; Fitzpatrick, Al-Qarni, & Meara, Reference Fitzpatrick, Al-Qarni and Meara2008; Nation, Reference Nation2001; Webb, Reference Webb2009).
Previous studies suggest that the effects of PAL may be affected by factors such as the direction (i.e., receptive or productive) and frequency of retrieval, type of elaboration (i.e., semantic or structural), type of information provided (e.g., first language [L1] translations, context, or pictures), type and timing of feedback, and interference as well as item-related factors such as word length, parts of speech, pronounceability, concept imageability, and concept frequency (for reviews, see Laufer, Reference Laufer and Chapelle2012; Nation, Reference Nation2001). Research also shows that in PAL, spaced learning, which involves spacing between repetitions of a given item, yields superior retention than does massed learning, which does not involve any spacing; this phenomenon is known as the spacing effect (e.g., Cepeda, Pashler, Vul, Wixted, & Rohrer, Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006; Cull, Reference Cull2000; Cull, Shaughnessy, & Zechmeister, Reference Cull, Shaughnessy and Zechmeister1996; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Karpicke & Roediger, Reference Karpicke and Roediger2007; Logan & Balota, Reference Logan and Balota2008; Maddox, Balota, Coane, & Duchek, Reference Maddox, Balota, Coane and Duchek2011). Given that spacing increases learning, how should we space retrieval opportunities to optimize L2 vocabulary learning from PAL tasks? In other words, this question may be rephrased as follows: What kinds of absolute and relative spacing should be used to maximize vocabulary learning? Absolute spacing refers to the total amount of spacing that separates all repetitions of a given item (Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011). For instance, if a given item is encountered four times, and each encounter is separated by 2 min, the absolute spacing is 6 min (2 min × 3). Relative spacing refers to how study opportunities are distributed relative to one another (Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011). Examples of relative spacing schedules include equal and expanding spacing (e.g., Kang et al., Reference Kang, Lindsey, Mozer and Pashler2014; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007). In equal spacing, the intervals between encounters of a given item are held constant (e.g., 2, 2, and 2 min). In expanding spacing (also known as expanding or expanded rehearsal; Ellis, Reference Ellis1995), the intervals between encounters are gradually increased (e.g., 1, 2, and 3 min).
Research suggests that the effects of equal and expanding spacing are conditioned by several factors, such as feedback, task difficulty, absolute spacing, and the retention interval (RI; i.e., the interval between the treatment and posttest). These findings suggest that when examining the effects of expanding spacing, it may be useful to consider the effects of at least these four factors. None of the existing L2 vocabulary studies on expanding spacing, however, manipulated these four factors (Kang et al., Reference Kang, Lindsey, Mozer and Pashler2014; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007). With the limitations of the earlier studies in mind, the present study compared the effects of expanding and equal spacing on L2 vocabulary learning while controlling the effects of feedback, task difficulty, absolute spacing, and the RI. The findings of this study may allow us to determine whether expanding or equal spacing should be used to optimize L2 vocabulary learning from PAL tasks.
REVIEW OF THE LITERATURE
Previous studies have looked into the effects of two types of relative spacing: equal and expanding. Note that because only three studies compared the effects of equal and expanding spacing on L2 vocabulary learning (Kang et al., Reference Kang, Lindsey, Mozer and Pashler2014; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007), most previous studies reviewed in this section have investigated the learning of materials other than L2 vocabulary, such as L1 vocabulary. Examples of equal spacing include the 3-3-3 schedule. The numbers indicate an interstimulus interval (ISI; e.g., Logan & Balota, Reference Logan and Balota2008; Pyc & Rawson, Reference Pyc and Rawson2007), or an interval that separates repetitions of a given item. For instance, in the 3-3-3 schedule, encounters of a given item are always separated by three spacing units (trial or time). Examples of expanding spacing include the 1-3-5 schedule. Most existing studies (e.g., Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Karpicke & Roediger, Reference Karpicke and Roediger2007; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978; Logan & Balota, Reference Logan and Balota2008; Pyc & Rawson, Reference Pyc and Rawson2007) use the number of intervening trials as a unit of spacing.Footnote 1 In these studies, the 3-3-3 schedule refers to a schedule in which encounters of a given item are always separated by three trials for other items. Other studies use time as a spacing unit (e.g., Cull et al., Reference Cull, Shaughnessy and Zechmeister1996, Experiments 4 and 5; Dobson, Reference Dobson2011; Kang et al., Reference Kang, Lindsey, Mozer and Pashler2014; Storm, Bjork, & Storm, Reference Storm, Bjork and Storm2010). In the 3-3-3 schedule of these studies, encounters of a given item are separated by 3 min or days.
Note that when comparing the effects of different relative spacing schedules, the spacing schedules need to have equivalent absolute spacing (e.g., Carpenter & DeLosh, Reference Carpenter and DeLosh2005; Cull, Reference Cull2000; Cull et al., Reference Cull, Shaughnessy and Zechmeister1996; Karpicke & Roediger, Reference Karpicke and Roediger2007; Logan & Balota, Reference Logan and Balota2008). Comparing the 1-3-5 expanding and 5-5-5 equal schedules, for instance, may not be valid because they are not matched in absolute spacing: The 1-3-5 expanding schedule has an absolute spacing of nine units (1 + 3 + 5 = 9), whereas the 5-5-5 equal schedule has an absolute spacing of 15 units (5 × 3 = 15). As a result, any potential difference between the two schedules may be partly due to the difference in absolute spacing rather than in relative spacing per se. In contrast, comparing the 1-3-5 and 3-3-3 schedules, for instance, is considered valid because the two schedules have equivalent absolute spacing (nine units). Previous studies that failed to control for absolute spacing (e.g., Schuetze, Reference Schuetze2014) were excluded from the following literature review because relative and absolute spacing were confounded in these studies.
Theoretical Background: The Effects of Equal and Expanding Spacing
In this study, it is assumed that learning involves retrieval because research shows that retrieval increases L2 vocabulary learning (e.g., Barcroft, Reference Barcroft2007, Reference Barcroft2013). Expanding spacing is often regarded as the most effective relative spacing schedule (e.g., Baddeley, Reference Baddeley1997; Ellis, Reference Ellis1995; Hulstijn, Reference Hulstijn and Robinson2001; Nation, Reference Nation2001; Schmitt, Reference Schmitt, Cummins and Davison2007; see Roediger & Karpicke, Reference Roediger, Karpicke and Benjamin2010, for a review). The superiority of expanding spacing over other forms of relative spacing for retrieval practice is referred to as the expanding retrieval effect (e.g., Cull et al., Reference Cull, Shaughnessy and Zechmeister1996). The expanding retrieval effect can be accounted for by the retrieval practice effect and the retrieval effort hypothesis (Ellis, Reference Ellis1995; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Karpicke & Roediger, Reference Karpicke and Roediger2010; Logan & Balota, Reference Logan and Balota2008; Storm et al., Reference Storm, Bjork and Storm2010). The retrieval practice effect (Baddeley, Reference Baddeley1997; Ellis, Reference Ellis1995) refers to the phenomenon in which a successful retrieval from memory yields superior long-term retention than an unsuccessful retrieval. The retrieval effort hypothesis (Pyc & Rawson, Reference Pyc and Rawson2009) states that the degree to which a successful retrieval enhances memory increases with the difficulty of the retrieval practice.
Taken together, the retrieval practice effect and retrieval effort hypothesis imply that when a new lexical item is introduced, it should be practiced after a relatively short ISI. Otherwise, retrieval may be unsuccessful, and learners may not be able to benefit from the positive effects of retrieval success. The second retrieval may occur after a longer interval than the first one because success on the initial retrieval may facilitate subsequent retrieval, allowing the target word to be successfully retrieved after a longer ISI. The third retrieval may likely be successful after an even longer interval because success on the second retrieval can further strengthen memory. Similarly, gradually increasing the intervals between retrieval attempts may allow learners to increase retrieval effort while also facilitating successful retrieval. The preceding discussion suggests that expanding spacing is the most effective relative spacing schedule because it may allow learners to benefit from the positive effects of both retrieval success and retrieval effort (e.g., Ellis, Reference Ellis1995; Karpicke & Roediger, Reference Karpicke and Roediger2010; Logan & Balota, Reference Logan and Balota2008; Storm et al., Reference Storm, Bjork and Storm2010).
Note that empirical studies have yielded mixed results regarding the retrieval practice effect, one of the theoretical underpinnings of the expanding retrieval effect. Previous L2 vocabulary studies, for instance, have shown that although shorter absolute spacing (e.g., an ISI of 5 min) produces more successful retrievals than larger absolute spacing (e.g., an ISI of 10 min) during the learning phase, larger absolute spacing often leads to better retention over time (i.e., the lag effect; e.g., Crothers & Suppes, Reference Crothers and Suppes1967; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Nakata, Reference Nakata2013; Pashler, Zarow, & Triplett, Reference Pashler, Zarow and Triplett2003; Pyc & Rawson, Reference Pyc and Rawson2007). These findings are inconsistent with the retrieval practice effect. However, in all previous studies showing the expanding retrieval effect, expanding spacing produced more successful retrievals during learning than equal spacing (e.g., Karpicke & Roediger, Reference Karpicke and Roediger2007; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978; Logan & Balota, Reference Logan and Balota2008; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011; Storm et al., Reference Storm, Bjork and Storm2010), supporting the retrieval practice effect. At least in the studies on expanding spacing, therefore, retrieval success during learning is associated with superior retention.
Empirical Evidence Regarding the Effects of Equal and Expanding Spacing on L2 Vocabulary Learning
Studies comparing the effects of equal and expanding spacing on L2 vocabulary learning have failed to find any advantage of expanding spacing in their posttest scores (Kang et al., Reference Kang, Lindsey, Mozer and Pashler2014; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007). In Kang et al. (Reference Kang, Lindsey, Mozer and Pashler2014), for instance, 37 participants studied 60 Japanese-English word pairs under equal and expanding spacing conditions. The treatment was conducted over a period of 4 weeks, and the number of intervening days was used as a spacing unit. In the equal spacing condition, the target words were encountered on Days 1, 10, 19, and 28 (a 9-9-9 schedule). In the expanding condition, the target items were encountered on Days 1, 3, 9, and 28 (a 2-6-19 schedule). No statistically significant difference was found between the two schedules on the posttest administered on Day 83 (equal = 46%, expanding = 49%).
Karpicke and Bauernschmidt (Reference Karpicke and Bauernschmidt2011) and Pyc and Rawson (Reference Pyc and Rawson2007) used the number of intervening trials as a unit of spacing. In Karpicke and Bauernschmidt (Reference Karpicke and Bauernschmidt2011), 96 American undergraduate students studied 24 Swahili-English word pairs. Absolute (short, medium, and long) and relative (equal and expanding) spacing were manipulated, and the effects of the following six schedules were compared: 5-5-5 (short equal), 1-5-9 (short expanding), 10-10-10 (medium equal), 5-10-15 (medium expanding), 30-30-30 (long equal), and 15-30-45 (long expanding). Although the main effect of absolute spacing was significant on the posttest conducted 1 week after the treatment, no significant difference existed between equal and expanding spacing. The results suggest that absolute spacing has a larger effect on L2 vocabulary learning than does relative spacing. Pyc and Rawson (Reference Pyc and Rawson2007) also failed to find any significant difference between 1-5-9 expanding and 5-5-5 equal spacing conditions on the posttest conducted 40 min after the treatment. Note that in all existing L2 vocabulary studies, participants practiced receptive retrieval (i.e., retrieving the L1 translation corresponding to a L2 word form) rather than productive retrieval (i.e., retrieving the L2 word form corresponding to a L1 translation).
Non-L2 Vocabulary Studies on the Effects of Equal and Expanding Spacing
A number of studies have also compared the effects of equal and expanding spacing on the learning of materials other than L2 vocabulary, such as L1 vocabulary (e.g., sobriquet-nickname; Cull, Reference Cull2000; Karpicke & Roediger, Reference Karpicke and Roediger2007), weakly associated L1 word pairs (e.g., sheep-cloth; Balota, Duchek, Sergent-Marshall, & Roediger, Reference Balota, Duchek, Sergent-Marshall and Roediger2006; Logan & Balota, Reference Logan and Balota2008; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011), name pairs (e.g., Tony-Brown; Cull et al., Reference Cull, Shaughnessy and Zechmeister1996, Experiments 1 and 2; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978, Experiment 1), face-name pairs (Carpenter & DeLosh, Reference Carpenter and DeLosh2005, Experiments 2 and 3; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978, Experiment 2), general facts (Cull et al., Reference Cull, Shaughnessy and Zechmeister1996, Experiments 3, 4, and 5; Shaughnessy & Zechmeister, Reference Shaughnessy and Zechmeister1992), and text materials (Karpicke & Roediger, Reference Karpicke and Roediger2010; Storm et al., Reference Storm, Bjork and Storm2010). Because the learning of these materials may involve a different process compared with that of L2 vocabulary, the findings of these studies may not necessarily be applicable to L2 vocabulary learning. At the same time, because the spacing effect has been found to affect the learning of a wide range of verbal materials (see Cepeda et al., Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006, for a review), it is also possible that relative spacing affects L2 vocabulary learning in a similar way as the learning of other materials. It is useful, therefore, to survey the previous non-L2 vocabulary research on expanding spacing.
Non-L2 vocabulary studies have yielded mixed results regarding the expanding retrieval effect. Some studies failed to find any advantage of expanding over equal spacing in their posttest scores (e.g., Carpenter & DeLosh, Reference Carpenter and DeLosh2005; Cull, Reference Cull2000; Karpicke & Roediger, Reference Karpicke and Roediger2010; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011, Experiment 1; Storm et al., Reference Storm, Bjork and Storm2010, Experiment 1). Other studies found the superiority of expanding spacing under limited conditions: when (a) feedback is not provided, (b) the task difficulty is high, (c) the treatment involves long absolute spacing, and/or (d) the RI is shorter than 24 hr.
First, studies suggest that expanding spacing may be more effective than equal spacing when feedback (i.e., a correct response) is not provided after retrievals. Feedback was not given in all previous experiments that found the superiority of expanding over equal spacing in their posttest scores (Cull et al., Reference Cull, Shaughnessy and Zechmeister1996, Experiments 1 and 4; Dobson, Reference Dobson2011; Karpicke & Roediger, Reference Karpicke and Roediger2007, Experiment 1; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978; Logan & Balota, Reference Logan and Balota2008; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011, Experiment 2; Storm et al., Reference Storm, Bjork and Storm2010, Experiments 2 and 3), whereas experiments providing feedback failed to find any advantage of expanding spacing (e.g., Cull, Reference Cull2000, Experiments 1–4; Cull et al., Reference Cull, Shaughnessy and Zechmeister1996, Experiment 5; Karpicke & Roediger, Reference Karpicke and Roediger2007, Experiments 2 and 3; Karpicke & Roediger, Reference Karpicke and Roediger2010, Experiment 2). A possible interaction between feedback and relative spacing may be caused by the differential positions of the initial retrieval between equal and expanding spacing (e.g., Cepeda et al., Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006; Storm et al., Reference Storm, Bjork and Storm2010). Because the first retrieval opportunity usually occurs later in equal spacing than in expanding spacing (e.g., 3-3-3 rather than 1-3-5), equal spacing may produce a lower rate of initial retrieval success than expanding spacing. When the initial retrieval is unsuccessful and the treatment does not involve feedback, the subsequent retrievals are also likely to be unsuccessful because learners do not have opportunities to correct errors. Without feedback, therefore, equal spacing may lead to lower retrieval success throughout the treatment than expanding spacing, which in turn may result in poorer long-term retention (e.g., Cepeda et al., Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006; Storm et al., Reference Storm, Bjork and Storm2010). In contrast, as feedback allows learners to correct errors on subsequent retrievals, it may eliminate a possible difference in learning phase performances between equal and expanding spacing. As a result, an expanding retrieval effect may not be observed when the treatment involves feedback.
Second, non-L2 vocabulary research also suggests that expanding spacing is more effective than equal spacing when the task difficulty is high (Logan & Balota, Reference Logan and Balota2008; Storm et al., Reference Storm, Bjork and Storm2010). As previously noted, the expanding retrieval effect is based on the assumption that expanding spacing may produce higher success on the initial retrieval than other relative spacing schedules, which in turn may facilitate subsequent retrieval and may lead to better long-term retention. If the difficulty is high and memory decays rapidly, the initial retrieval needs to take place immediately after the initial encounter to facilitate successful retrieval. As a result, when the task difficulty is high, expanding spacing, which is associated with an immediate initial retrieval attempt (e.g., 1-3-5 rather than 3-3-3), may lead to higher levels of initial retrieval success than equal spacing, and a significant expanding retrieval effect may be observed. In contrast, if the difficulty is low and forgetting occurs slowly, the initial retrieval may be successful even in equal spacing, which is associated with a delayed initial retrieval attempt (e.g., 3-3-3 rather than 1-3-5). Because the assumption on which the expanding retrieval effect is based (i.e., expanding spacing produces higher success on the initial retrieval than does equal spacing) will not be met, an expanding retrieval effect may not be found when difficulty is low and forgetting occurs slowly.
Third, studies suggest that relative spacing may also interact with absolute spacing (Dobson, Reference Dobson2012; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011). Maddox et al. (Reference Maddox, Balota, Coane and Duchek2011) observed that experiments that found the advantage of expanding over equal spacing used longer absolute spacing (4.78 spacing units on average) than those that did not (3.66 spacing units). Based on their analysis, Maddox et al. argued that the expanding retrieval effect tends to be observed when absolute spacing is relatively long. Absolute spacing interacts with relative spacing presumably because long spacing increases task difficulty (Dobson, Reference Dobson2012). As previously discussed, expanding spacing may be particularly effective when the difficulty is high. Because long absolute spacing increases difficulty, a significant expanding retrieval effect may also be found when the treatment involves long absolute spacing.
Finally, non-L2 vocabulary research also suggests that relative spacing may interact with the RI and that the advantage of expanding spacing may diminish as the RI increases (e.g., Karpicke & Roediger, Reference Karpicke and Roediger2007, Reference Karpicke and Roediger2010; Logan & Balota, Reference Logan and Balota2008; Roediger & Karpicke, Reference Roediger, Karpicke and Benjamin2010; Storm et al., Reference Storm, Bjork and Storm2010). This interaction may also stem from the differential positions of the initial retrieval between equal and expanding spacing. Because the first retrieval opportunity usually occurs earlier in expanding spacing than in equal spacing (e.g., 1-3-5 rather than 3-3-3), learners may retrieve items from primary rather than long-term memory in expanding spacing, which may facilitate short-term but not necessarily long-term retention (e.g., Karpicke & Roediger, Reference Karpicke and Roediger2007; Roediger & Karpicke, Reference Roediger, Karpicke and Benjamin2010; Storm et al., Reference Storm, Bjork and Storm2010). In contrast, because equal spacing is associated with a delayed initial retrieval attempt, learners may be more likely to retrieve items from long-term rather than primary memory in equal spacing, which may not necessarily facilitate short-term memory but may lead to superior long-term retention. As a result, although expanding spacing may be more effective than equal spacing at a short RI, its advantage may disappear over time.
The preceding review of the literature suggests that when comparing the effects of equal and expanding spacing on L2 vocabulary learning, it is useful to consider the effects of at least the following four variables: feedback, task difficulty, absolute spacing, and the RI. Let us consider the implications of possible interactions between the expanding retrieval effect and the four variables just listed. First, studies indicate that expanding spacing may be particularly effective when the treatment does not involve feedback. Although this interaction is of theoretical importance, manipulating the presence or absence of feedback may have limited pedagogical value because not only does feedback enhance learning (e.g., Metcalfe & Kornell, Reference Metcalfe and Kornell2007), but also authentic PAL typically involves feedback (e.g., Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Karpicke & Roediger, Reference Karpicke and Roediger2007). To increase learning and ecological validity, therefore, it may be useful to always provide feedback after retrievals. Second, prior studies suggest that a significant expanding retrieval effect is observed when the task difficulty is high. This finding may be particularly pertinent to the direction of retrieval during learning because research (e.g., Griffin & Harley, Reference Griffin and Harley1996; Mondria & Wiersma, Reference Mondria, Wiersma and Laufer2004; Schneider, Healy, & Bourne, Reference Schneider, Healy and Bourne2002) suggests that productive retrieval (i.e., retrieving the L2 word form) is more demanding than receptive retrieval (i.e., retrieving the meaning). From a pedagogical perspective, productive retrieval may be more desirable because it may result in larger vocabulary gains than receptive retrieval (e.g., Mondria & Wiersma, Reference Mondria, Wiersma and Laufer2004; Steinel, Hulstijn, & Steinel, Reference Steinel, Hulstijn and Steinel2007; Webb, Reference Webb2009). Thus, although the use of productive retrieval can increase task difficulty and favor expanding spacing, it may be useful to use productive retrieval during learning. Third, a possible interaction between relative and absolute spacing suggests that it may be valuable to manipulate absolute spacing when examining the effects of expanding and equal spacing. Manipulating relative and absolute spacing simultaneously may also have pedagogical value because it may allow us to determine the relative importance of the two types of spacing on vocabulary learning (Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011). Fourth, a possible interaction between the expanding retrieval effect and the RI suggests that to obtain a comprehensive picture regarding the effects of equal and expanding spacing, we may need to give a posttest at more than one RI.
In summary, earlier non-L2 vocabulary studies suggest that when comparing the effects of equal and expanding spacing, it may be useful to meet the following four conditions: (a) provide feedback after retrievals, (b) use productive retrieval during learning, (c) manipulate absolute as well as relative spacing, and (d) give a posttest at more than one RI. None of the previous L2 vocabulary studies, however, meet all four of these conditions. Pyc and Rawson (Reference Pyc and Rawson2007) and Kang et al. (Reference Kang, Lindsey, Mozer and Pashler2014) satisfy only (a). Karpicke and Bauernschmidt’s (Reference Karpicke and Bauernschmidt2011) study meets only (c). With the limitations of the existing L2 vocabulary studies in mind, the present study compared the effects of equal and expanding spacing while meeting the four conditions previously outlined. By controlling variables that may influence the effects of equal and expanding spacing, the current study allows us to determine how relative spacing may influence vocabulary learning in a more rigorous manner than earlier studies. The research question of this study is as follows: Does absolute spacing influence the effects of equal and expanding spacing on L2 vocabulary learning from PAL tasks at different RIs when feedback is provided after retrievals and the treatment involves productive retrieval?
METHOD
Participants
The original pool of participants consisted of 167 Japanese learners of English. Forty-eight of the learners were first-year law majors from an English as a foreign language (EFL) class at a university in Osaka, Japan (hereafter, OU). Their average score on the first to the sixth 1,000-word frequency levels of the Vocabulary Size Test (Nation & Beglar, Reference Nation and Beglar2007) was 30.91 (SD = 7.03) out of 60. The other 119 participants were first-year engineering students from three EFL classes at a technical college in Gifu, Japan (hereafter, GTC). No data were available regarding their English proficiency. All of the participants had received at least 6 years of formal English instruction. Out of the 167 participants, 35 were dropped because they were absent on the day of the experiment, chose not to participate, or their data were lost due to technical problems, leaving 132 students. Furthermore, four students were excluded from analysis to (a) counterbalance the effects of target items across participants (Table 1) and (b) ensure that the four absolute spacing groups (massed, short, medium, and long) would consist of an equal number of participants from OU and GTC, leaving 128 students (see Appendix A for details).
Design of the present study
Note. Each item set consisted of 10 items.
Experimental Design
There were three independent variables. The first independent variable was absolute spacing: massed, short, medium, and long. The second independent variable was relative spacing: equal and expanding. The third independent variable was the RI: immediate and 1-week delayed posttests. Absolute spacing was a between-participant variable, and relative spacing and the RI were within-participant variables. The dependent variable was the number of correct responses on the posttest.
Table 1 summarizes the design of the current study. The participants were randomly assigned by a computer program to one of the four absolute spacing groups: massed, short, medium, and long. The treatment in the massed group did not involve any spacing, and all target items were encountered four times in a row. The massed group was included as the control group (e.g., Cull, Reference Cull2000; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978). Hereafter, the short, medium, and long spacing groups will be collectively referred to as the experimental groups, as opposed to the control (massed) group. The participants in the three experimental groups were randomly divided into two subgroups of 16 participants: Subgroups X and Y. Twenty target word pairs were also divided into two sets of 10 items: Sets A and B (see the Materials section for details). The two subgroups of participants in each experimental group studied both sets of word pairs under different relative spacing conditions (equal or expanding), thus counterbalancing the effects of target items (Table 1). Trial was used as an index of spacing (Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007), and the numbers in Table 1 indicate the number of intervening trials in each schedule. For instance, in the 5-5-5 (short equal) schedule, encounters of a given item were always separated by five trials for other target or filler items.
The six spacing conditions in Table 1 (5-5-5, 1-5-9, 10-10-10, 5-10-15, 30-30-30, and 15-30-45) were chosen because they allow us to test the hypothesis that the expanding retrieval effect may be observed when absolute spacing is relatively long (Dobson, Reference Dobson2012; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011). Specifically, pilot studies with eight Japanese learners of English suggested that when the six spacing schedules in Table 1 were used, the mean ISI would be 60.56, 147.22, and 359.26 s in the short, medium, and long spacing groups, respectively. Because all previous PAL experiments that have found an advantage of expanding over equal spacing used an average spacing of 60 s or shorter (5–60 s; Cull et al., Reference Cull, Shaughnessy and Zechmeister1996; Karpicke & Roediger, Reference Karpicke and Roediger2007; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978; Logan & Balota, Reference Logan and Balota2008; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011), it was judged that using the spacing schedules in Table 1 may allow us to investigate whether expanding spacing may be particularly effective when the treatment involves long absolute spacing.Footnote 2
As noted in the Review of the Literature section, when comparing the effects of different relative spacing schedules, the spacing schedules need to have equivalent absolute spacing (e.g., Carpenter & DeLosh, Reference Carpenter and DeLosh2005; Cull, Reference Cull2000; Cull et al., Reference Cull, Shaughnessy and Zechmeister1996; Karpicke & Roediger, Reference Karpicke and Roediger2007; Logan & Balota, Reference Logan and Balota2008). There are two common methods to control absolute spacing in different treatments. One way is to match the average number of intervening trials (e.g., Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007). In this method, if encounters of a given item are separated by 10 trials on average in two treatments, for instance, they are regarded as having equivalent absolute spacing. The other common method is to match the average amount of time between repetitions (e.g., Dobson, Reference Dobson2011; Kang et al., Reference Kang, Lindsey, Mozer and Pashler2014; Storm et al., Reference Storm, Bjork and Storm2010). According to this approach, if encounters of a given item are separated by 3 min on average in two treatments, for instance, they are considered to be controlled for absolute spacing. In the current experiment, absolute spacing was controlled using the first method, and the equal and expanding schedules in each experimental group were matched in the average number of intervening trials. For instance, in the short spacing group, encounters of a given item were separated by five trials on average in both the equal (5-5-5) and the expanding (1-5-9) schedules ([5 × 3] / 3 = 5; [1 + 5 + 9] / 3 = 5). Because the equal and expanding conditions were matched in the number of intervening trials, both conditions are regarded as having equivalent absolute spacing. At the same time, the average amount of time between repetitions was analyzed after the experiment to investigate whether absolute spacing in the equal and expanding conditions was equivalent when time, not the trial, was used as an index of spacing. The analysis indicated that the two relative spacing conditions could be regarded as having roughly equivalent absolute spacing whether trial or time is used as a spacing unit (see Nakata, Reference Nakata2013, for details).
Materials
Twenty English-Japanese word pairs were used as target items (see Appendix B for the list of target word pairs). They were divided into two sets of 10 items (Sets A and B) so that they would be of roughly equivalent item difficulty. Item difficulty was operationalized as the posttest scores in a similar previous experiment, in which another group of 95 OU students studied the same 20 English-Japanese word pairs (Nakata, Reference Nakata2013). Each item set consisted of six nouns and four verbs.
Thirteen items were used as filler items: abet, banter, exalt, husk, jibe, polemic, promontory, smudge, tyro, urn, usurp, valor, and vestige. They were studied and tested like target word pairs but were not included in the analysis. The filler items were included for two reasons. First, to manipulate spacing during the treatment, filler items were inserted in positions that were not occupied by any of the target items (e.g., Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007). Second, filler items were used as primacy and recency buffers during the treatment (e.g., Cull et al., Reference Cull, Shaughnessy and Zechmeister1996; Karpicke & Roediger, Reference Karpicke and Roediger2007; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978). There were three primacy and recency buffers at the beginning and end of the treatment in all four absolute spacing groups.
Procedure
The experiment was conducted during regular class hours. Participants studied and were tested individually with a computer program developed by the author. Participants first took productive and receptive pretests (see the Dependent Measures section). The pretest was followed by the treatment, in which participants studied 33 English-Japanese word pairs (including 13 filler pairs) using a computer-based flashcard program. The treatment in all four absolute spacing groups (massed, short, medium, and long) consisted of 130 trials. There were 80 trials for target items (20 items × 4 encounters) and 50 trials for filler items. In the three experimental groups, the trials for different target items were mixed together. In other words, before the first target item was encountered four times, the second target item was introduced. In the massed group, in contrast, all target items were studied four times in a row, and the second target item was not introduced until the first target item went through the full cycle of trials.
Target items were studied using one of the seven schedules in Table 1 (e.g., 0-0-0, 5-5-5, or 1-5-9). In the present study, the number of intervening trials was used as a unit of spacing (see the Experimental Design section). In the 5-5-5 schedule, therefore, encounters of a given item were always separated by five trials of other target or filler items. For instance, suppose a given target item assigned to the 5-5-5 schedule first appeared in the fifth trial. The second encounter with this item occurred five intervening trials after the first encounter (i.e., the 11th trial), the third encounter occurred five intervening trials after the second encounter (i.e., the 17th trial), and the fourth encounter occurred five intervening trials after the third encounter (i.e., the 23rd trial). Other target or filler items were studied in the intervening trials. Each target item was encountered four times throughout the treatment in all seven schedules. In the first encounter with each item, the English and Japanese words were presented together for 8 s per word pair (e.g., mane = たてがみ). In the remaining encounters, participants practiced productive retrieval, during which they were asked to type the target English word corresponding to the Japanese translation provided (e.g., たてがみ = ____?). Participants were allowed as much time as necessary to type a response. To ascertain whether retrieval times were roughly equivalent among different conditions, the average retrieval times were analyzed after the experiment (see the Study Time section for details). After each response, the target English word, Japanese translation, and learners’ response were shown for 5 s per response as feedback.
Following the treatment, the participants answered 10 two-digit addition questions as a filler task. They spent 72.94 s (SD = 20.18 s) on the filler task on average. This task was included to ensure that the posttests would measure learning rather than being a function of primary memory. The immediate posttest was administered after the filler task. On average, 90.71 s (SD = 17.76 s) elapsed between the treatment and immediate posttest. To measure retention, the delayed posttest was given 1 week after the treatment.
Dependent Measures
Pretest
Immediately before the treatment, participants took productive and receptive pretests, in that order. In the productive pretest, participants were asked to type the English word corresponding to the Japanese translation provided (e.g., たてがみ = ____?). To prevent learners from typing synonyms for a target word (e.g., hair for mane), one letter in the target word and the number of letters in the word were given as a hint (e.g., _ _ n _ for mane) in the productive pretest (see Nakata, Reference Nakata2013, for the protocol for determining the hints). In the receptive pretest, participants translated target English words into Japanese (e.g., mane = ____?).
Posttest
Productive and receptive tests, in that order, were given as the posttest. Unlike in the pretest, the hints (e.g., _ _ n _ for mane) were not provided in the productive posttest. Otherwise, the posttests were exactly the same as the pretests. Immediate and delayed posttests were administered in the present study. The immediate posttest was given immediately after the treatment and filler task. The delayed posttest was administered 1 week after the treatment without prior notice.
Scoring
Responses on the pretest and posttest were scored using strict and sensitive scoring procedures. For the productive test, using the strict scoring system, only responses without any misspellings were marked as correct. The sensitive scoring system was a modified version of Barcroft’s (Reference Barcroft2002) lexical production scoring protocol (LPSP-written), and responses that would be awarded a score of 0.75 using LPSP-written were also marked as correct. For the receptive test, using the strict scoring system, responses were marked as incorrect if (a) an intransitive verb was provided for a transitive verb (e.g., 怒る [intransitive] for pique) and vice versa or (b) responses were of a wrong part of speech (e.g., 後悔 [noun] for rue). With sensitive scoring, these two kinds of responses were both scored as correct.Footnote 3
RESULTS
Pretest
None of the participants exhibited prior knowledge of any of the target words on the productive pretest. The average receptive pretest score (SDs in parentheses) was 0.13 (0.34), 0.06 (0.25), 0.16 (0.37), 0.19 (0.64), 0.16 (0.37), and 0.22 (0.42) out of 10 with strict scoring and 0.16 (0.37), 0.13 (0.34), 0.19 (0.40), 0.22 (0.66), 0.19 (0.40), and 0.22 (0.42) out of 10 with sensitive scoring in the short equal, short expanding, medium equal, medium expanding, long equal, and long expanding schedules, respectively (see Table 1). When collapsed across the relative spacing conditions, the average receptive pretest score (SDs in parentheses) was 0.25 (0.57), 0.19 (0.47), 0.34 (0.79), and 0.38 (0.66) out of 20 with strict scoring and 0.31 (0.59), 0.28 (0.52), 0.41 (0.80), and 0.41 (0.67) out of 20 with sensitive scoring in the massed, short, medium, and long spacing groups, respectively.
Learning Phase Data
First, let us investigate whether any statistically significant difference existed between the equal and expanding schedules in the participants’ performance on the retrieval practice during the treatment. When collapsed across the three retrieval attempts, the average number of correct responses (SDs in parentheses) was 5.58 (2.48), 5.85 (2.33), 5.02 (2.29), 5.11 (2.04), 3.38 (2.31), and 3.18 (2.04) out of 10 with strict scoring and 7.33 (1.99), 7.67 (1.66), 6.82 (2.21), 6.75 (2.11), 4.68 (2.04), and 4.67 (1.77) out of 10 with sensitive scoring in the short equal, short expanding, medium equal, medium expanding, long equal, and long expanding schedules, respectively. According to a two-way, mixed-design 2 (relative spacing: equal or expanding) × 3 (absolute spacing: short, medium, or long) ANOVA, regardless of the scoring procedure, the main effect of relative spacing was not statistically significant, strict: F(1, 93) = 0.19, p = .662, ηpFootnote 2 = .002; sensitive: F(1, 93) = 0.48, p = .490, ηpFootnote 2 = .005. The interaction between relative and absolute spacing was also not significant, strict: F(2, 93) = 1.18, p = .313, ηpFootnote 2 = .02; sensitive: F(2, 93) = 1.10, p = .337, ηpFootnote 2 = .02. The 95% confidence intervals of difference were also narrow: with strict scoring, short [−0.27, 0.22], medium [−0.09, 0.22], and long [0.20, 0.22]; with sensitive scoring, short [−0.33, 0.21], medium [0.07, 0.21], and long [0.01, 0.21]. The results suggest that relative spacing may have little effect on learning phase performance.
Next, let us examine the effects of absolute spacing on learning phase performance. When collapsed across the relative spacing conditions and the three retrieval attempts, the average number of correct responses (SDs in parentheses) was 18.34 (1.49), 11.44 (4.66), 10.14 (4.19), and 6.55 (4.13) out of 20 with strict scoring and 19.56 (0.57), 15.00 (3.55), 13.57 (4.18), and 9.34 (3.52) out of 20 with sensitive scoring in the massed, short, medium, and long spacing groups, respectively. A one-way ANOVA found a statistically significant difference among the four groups with both scoring procedures, strict: F(3, 127) = 53.13, p < .001, η2 = .32; sensitive: F(3, 127) = 53.26, p < .001, η2 = .32. The Bonferroni method of multiple comparisons showed the following two things. First, no statistically significant difference existed between the short and medium groups regardless of the scoring procedure, showing small effect sizes (strict: p = 1.000, d = 0.29; sensitive: p = .500, d = 0.37). Second, the differences were statistically significant for all other comparisons, and medium-to-large effect sizes were found.Footnote 4 The findings suggest the following order in the degree of success on the retrieval attempts with both scoring methods: massed > short = medium > long.
Study Time
Because participants were allowed to take as much time as they needed to type a response during the treatment, the study time was different for each condition. On average, the participants spent 7.68 (1.47), 7.69 (1.14), 7.60 (1.12), 7.83 (1.26), 7.84 (0.97), and 8.06 (1.15) min (SDs in parentheses) studying the target items under the short equal, short expanding, medium equal, medium expanding, long equal, and long expanding schedules, respectively. The study time consists of the following: the initial presentation (8 s × 10 items = 80 s), feedback (5 s × 10 items × 3 trials = 150 s), and retrieval (short equal: 230.73 s; short expanding: 231.28 s; medium equal: 226.18 s; medium expanding: 239.71 s; long equal: 240.11 s; long expanding: 253.58 s).
To ascertain whether retrieval times were roughly equivalent between the equal and expanding spacing schedules, the average retrieval times were analyzed by a two-way, mixed-design 2 (relative spacing: equal or expanding) × 3 (absolute spacing: short, medium, or long) ANOVA. According to the ANOVA, neither the main effect of relative spacing, F(1, 93) = 3.01, p = .086, ηpFootnote 2 = .03, nor the interaction between relative and absolute spacing, F(2, 93) = 0.66, p = .517, ηpFootnote 2 = .01, was significant, and only small effect sizes were found. On the basis of the results, it may be possible to assume that the retrieval times were roughly equivalent between the two relative spacing conditions. The self-pacing of the retrieval practice, therefore, might not have significantly affected the results of this study regarding the effects of equal and expanding spacing.
When collapsed across the relative spacing conditions, the average study time (SDs in parentheses) was 12.72 (1.90), 15.37 (2.45), 15.43 (2.32), and 15.90 (1.87) min in the massed, short, medium, and long spacing groups, respectively. The study time consists of the following: the initial presentation (8 s × 20 items = 160 s), feedback (5 s × 20 items × 3 trials = 300 s), and retrieval (massed: 5.05 min; short: 7.70 min; medium: 7.76 min; long: 8.23 min). A one-way ANOVA found a statistically significant difference among the four groups in the average retrieval times, F(3, 127) = 14.36, p < .001, η2 = .07. The Bonferroni method of multiple comparisons showed that (a) all three experimental groups required significantly longer retrieval times than the control group (all p < .001), producing large effect sizes (massed vs. short: d = 1.21; massed vs. medium: d = 1.28; massed vs. long: d = 1.69), and (b) no statistically significant difference existed among the three experimental groups in retrieval times (all p = 1.000), and only small effect sizes were found (short vs. medium: d = 0.03; short vs. long: d = 0.24; medium vs. long: d = 0.22). Because there was a statistically significant difference in retrieval times among the four groups, the efficiency of the four groups was compared. Efficiency was defined as the number of words acquired per minute and was calculated by dividing the posttest score by the study time (e.g., Mondria, Reference Mondria2003; Pyc & Rawson, Reference Pyc and Rawson2007). The analysis indicated that although the massed group required the least study time, it was no more efficient than any of the experimental groups (see Nakata, Reference Nakata2013, for details).
Posttest Performance
Table 2 summarizes the immediate and delayed posttest results as a function of absolute and relative spacing. Cronbach’s alpha was .83 or higher (.83–.88) for all dependent measures, indicating good reliability. The productive and receptive test scores of the three experimental groups were analyzed by a three-way, mixed-design 2 (relative spacing: equal or expanding) × 3 (absolute spacing: short, medium, or long) × 2 (RI: immediate or 1-week delayed) ANOVA. As some items were answered correctly on the receptive pretest, gains from the pretest to the posttest were analyzed for the receptive test. Table 3 shows the results of the ANOVAs. The table indicates the following three things. First, the ANOVA showed a significant main effect of relative spacing with sensitive scoring on the receptive posttest. This means that when the results are collapsed across the three experimental groups and the RIs, they indicate that participants made significantly more gains on this measure in the expanding schedule compared with the equal schedule. However, only a very small effect size was observed (ηpFootnote 2 = .05, d = 0.12), and the 95% confidence intervals of difference were also narrow [0.03, 0.53]. The difference in the mean gains was also small (6.43 for expanding and 6.15 for equal spacing), which indicates that expanding spacing may increase learning by 4.6% relative to equal spacing. Taken together, the results show a limited, yet statistically significant, advantage of expanding spacing with sensitive scoring on the receptive posttest. Second, neither the interaction between relative and absolute spacing nor the interaction between the three variables was significant on any of the dependent variables. The results are inconsistent with the view that expanding spacing may be particularly effective when absolute spacing is relatively long (Dobson, Reference Dobson2012; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011). Third, the interaction between relative spacing and the RI was significant with strict scoring on the productive posttest and with strict and sensitive scoring on the receptive posttest.
Average number of correct responses on the posttest (standard deviations in parentheses)
Note. Eq = equal spacing; Exp = expanding spacing. In each absolute spacing group n = 32. The maximum score is 10 for equal spacing and expanding spacing and 20 for the total. There is no equal or expanding condition in the massed group because all target items were studied in the massed schedule (0-0-0; see Table 1).
Results of three-way ANOVAs for the posttest scores
Note. RI = retention interval. These are the results of a three-way mixed design 2 (relative spacing: equal or expanding) × 3 (absolute spacing: short, medium, or long) × 2 (RI: immediate or 1-week delayed) ANOVA.
As the interaction between relative spacing and the RI proved significant (see Table 3), the simple main effect of relative spacing was tested to investigate where the significant differences existed. The simple main effect tests showed the following three things. First, when the results are collapsed across all experimental groups, with strict scoring on the delayed productive posttest, they indicate that participants scored higher in the equal schedule (M = 2.73) than in the expanding schedule (M = 2.38). Yet the difference fell short of statistical significance, F(1, 95) = 3.58, p = .062, and only a very small effect size was observed (ηpFootnote 2 = .04, d = 0.16). Second, expanding spacing significantly outperformed equal spacing on the delayed receptive posttest with both scoring procedures, strict: F(1, 95) = 7.74, p = .007; sensitive: F(1, 95) = 9.76, p = .002. However, despite statistical significance, only very small effect sizes were found (strict: ηpFootnote 2 = .08, d = 0.17; sensitive: ηpFootnote 2 = .09, d = 0.19). The difference in the means between expanding (6.08 with strict and 6.23 with sensitive scoring) and equal (5.65 with strict and 5.74 with sensitive scoring) spacing was also small, which indicates that expanding spacing may increase learning by 7.6%–8.5% relative to equal spacing. These findings are also supported by the relatively narrow confidence intervals of difference: [−0.75, −0.13] with strict and [−0.80, −0.18] with sensitive scoring. The results indicate a limited, yet statistically significant, advantage of expanding spacing on the delayed receptive posttest. Third, the differences were not statistically significant on all other posttests regardless of the scoring procedure, producing very small effect sizes.Footnote 5 Overall, the present study demonstrated a limited advantage of expanding over equal spacing (a) with sensitive scoring on the receptive posttest when collapsed across the three experimental groups and the RIs and (b) on the delayed receptive posttest with both scoring procedures when collapsed across the three experimental groups.
Next, let us examine the effects of absolute spacing on posttest performance when collapsed across the relative spacing conditions (see the total columns in Table 2). The productive and receptive test scores of the four absolute spacing groups were analyzed by a two-way, mixed-design 4 (absolute spacing: massed, short, medium, or long) × 2 (RI: immediate or 1-week delayed) ANOVA. The ANOVAs showed a significant main effect of absolute spacing on both productive and receptive tests with both scoring procedures, productive (strict): F(3, 124) = 9.47, p < .001, ηpFootnote 2 = .19; productive (sensitive): F(3, 124) = 12.77, p < .001, ηpFootnote 2 = .24; receptive (strict): F(3, 124) = 7.93, p < .001, ηpFootnote 2 = .16; receptive (sensitive): F(3, 124) = 8.20, p < .001, ηpFootnote 2 = .17. The interaction between absolute spacing and the RI was significant on the productive posttest regardless of the scoring protocol, strict: F(3, 124) = 6.40, p < .001, ηpFootnote 2 = .13; sensitive: F(3, 124) = 3.89, p = .011, ηpFootnote 2 = .09. The interaction between absolute spacing and the RI, in contrast, was not significant on the receptive posttest, strict: F(3, 124) = 0.25, p = .865, ηpFootnote 2 = .01; sensitive: F(3, 124) = 0.27, p = .843, ηpFootnote 2 = .01.
To break down the significant interaction between absolute spacing and the RI on the productive posttest, the simple main effect of the RI was tested to examine whether the RI had differential effects on the four absolute spacing groups. The simple main effect tests showed that the delayed posttest scores were significantly lower than the immediate posttest scores in all four groups with both scoring procedures (all p < .001). With strict scoring, massed: F(3, 124) = 23.15; short: F(3, 124) = 98.32; medium: F(3, 124) = 92.59; long: F(3, 124) = 38.34. With sensitive scoring, massed: F(3, 124) = 31.42; short: F(3, 124) = 100.60; medium: F(3, 124) = 69.63; long: F(3, 124) = 42.12. Larger effect sizes were observed for the short group (strict: d = 1.78; sensitive: d = 1.79) compared with the other three (with strict scoring, massed: d = 0.96, medium: d = 1.58, long: d = 1.02; with sensitive scoring, massed: d = 1.00, medium: d = 1.34, long: d = 1.19). Together with the interaction graphs (see Appendix C), the results suggest that the productive posttest scores of the short spacing group may have decayed significantly more than those for the other three groups between the immediate and delayed posttests.
Due to the significant main effect of absolute spacing and the interaction between absolute spacing and the RI, the Bonferroni method of multiple comparisons was also used to investigate where the significant differences existed at each RI. The results of the multiple comparisons are summarized in Table 4. The table indicates the following three things. First, medium and long spacing significantly outperformed massed learning regardless of the posttest, scoring system, or RI, producing close to large (d = 0.79) or large (0.80 ≤ d ≤ 1.40) effect sizes. Second, short spacing significantly outperformed massed learning on the immediate productive posttest, and large effect sizes were observed (1.07 ≤ d ≤ 1.26). However, the difference was not statistically significant on the immediate receptive, delayed productive, and delayed receptive posttests, yielding small-to-medium effect sizes (0.39 ≤ d ≤ 0.67). Third, no significant difference existed among small, medium, and long spacing regardless of the posttest, scoring system, or RI, and only medium or smaller effect sizes were observed (0.04 ≤ d ≤ 0.50). Taken together, the results suggest that although spaced learning may be more effective than massed learning, the amount of spacing (short, medium, or long) may have little effect on L2 vocabulary learning.
Results of multiple comparisons for posttest scores
Note. The above table should be read as follows: With strict scoring on the immediate productive posttest, the difference between the massed and short groups was statistically significant (p < .001), and a large effect size was observed (d = 1.07). Effect sizes (d) of 0.20, 0.50, and 0.80 are indicative of small, medium, and large effects, respectively (Cohen, Reference Cohen1988).
DISCUSSION
The Effects of Relative Spacing
The present study found a statistically significant advantage of expanding over equal spacing (a) with sensitive scoring on the receptive posttest when collapsed across the three experimental groups and the RIs and (b) on the delayed receptive posttest with both scoring procedures when collapsed across the three experimental groups. The superiority of expanding spacing on the delayed posttest is especially useful because, from a pedagogical perspective, scores on the delayed posttest may be more important than those on the immediate posttest. The present study is also significant in that it is the first L2 vocabulary study to show a statistically significant expanding retrieval effect. The inconsistent results between the present and previous studies may be due in part to a methodological difference. Unlike previous L2 vocabulary studies (Kang et al., Reference Kang, Lindsey, Mozer and Pashler2014; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pyc & Rawson, Reference Pyc and Rawson2007), productive rather than receptive retrieval was used in this study. It may be that because productive retrieval is more demanding than receptive retrieval (e.g., Griffin & Harley, Reference Griffin and Harley1996; Mondria & Wiersma, Reference Mondria, Wiersma and Laufer2004; Schneider et al., Reference Schneider, Healy and Bourne2002) and because a significant expanding retrieval effect tends to be observed when the task difficulty is high (Logan & Balota, Reference Logan and Balota2008; Storm et al., Reference Storm, Bjork and Storm2010), the present study produced a larger expanding retrieval effect than previous L2 vocabulary studies.
Although statistically significant effects were found, it may be reasonable to assume that the present study demonstrated only a limited advantage of expanding spacing for three reasons. First, small effect sizes, small differences in the mean gains, and narrow confidence intervals of difference suggest that, although statistically significant, the difference between the two relative spacing conditions may not be large enough to be meaningful for learners, teachers, and materials developers (Nation & Webb, Reference Nation and Webb2011). Second, when collapsed across the three experimental groups and the RIs, although the difference between expanding and equal spacing reached statistical significance with sensitive scoring on the receptive test, no statistically significant difference was found with strict scoring. The results suggest that the effect of relative spacing was not very robust. Third, the present study found a statistically significant advantage of expanding spacing on the receptive but not on the productive posttest. The scores on the receptive tests, however, should be interpreted with caution because at each test administration, the productive test was given prior to the receptive test (see the Dependent Measures section), which may have affected performance on the receptive test. On the basis of the aforementioned reasons, it is reasonable to conclude that the present study found a statistically significant, yet limited, advantage of expanding spacing.
One may argue that the limited advantage of expanding spacing may be partially attributed to the rather narrow range of the mean ISIs used in the current study (58.97–355.91 s). Yet, as the average ISI used in this experiment was as long as or longer than those in all previous PAL experiments that have found an advantage of expanding spacing (5–60 s; e.g., Cull et al., Reference Cull, Shaughnessy and Zechmeister1996; Karpicke & Roediger, Reference Karpicke and Roediger2007; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978; Logan & Balota, Reference Logan and Balota2008), similar results might have been obtained even if longer absolute spacing had been used. Kang et al. (Reference Kang, Lindsey, Mozer and Pashler2014) also failed to find any benefit of expanding over equal spacing for L2 vocabulary learning, although they used a rather long mean ISI of 9 days (see the Review of the Literature section).
The limited advantage of expanding over equal spacing may be explained in part by learning phase performances. The expanding retrieval effect is based on the assumption that expanding spacing may produce more successful retrievals during learning than equal spacing, which in turn may result in better long-term retention (Ellis, Reference Ellis1995; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Karpicke & Roediger, Reference Karpicke and Roediger2010; Logan & Balota, Reference Logan and Balota2008; Storm et al., Reference Storm, Bjork and Storm2010). In the present study, however, no significant differences were found between the equal and expanding conditions in learning phase performances. It may be that, due to the lack of a significant advantage during learning, this study observed only limited superiority of expanding spacing on the posttest. Why was the advantage of expanding spacing not found during the learning phase? The results may be partially due to the provision of feedback. Previous studies suggest that feedback may eliminate a possible difference in learning phase performances because it may allow learners to correct errors on subsequent retrievals (e.g., Cepeda et al., Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006; Cull et al., Reference Cull, Shaughnessy and Zechmeister1996; Storm et al., Reference Storm, Bjork and Storm2010). As pointed out earlier, feedback was not given in all experiments that found the superiority of expanding over equal spacing in their posttest scores (e.g., Dobson, Reference Dobson2011; Landauer & Bjork, Reference Landauer, Bjork, Gruneberg, Morris and Sykes1978; Logan & Balota, Reference Logan and Balota2008; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011; Storm et al., Reference Storm, Bjork and Storm2010). Taken together, the present and previous studies indicate that, as long as the treatment involves feedback, the advantage of expanding spacing may be limited even though (a) the treatment involves long absolute spacing, (b) the task difficulty is high, and (c) the RI is shorter than 24 hr, the conditions in which expanding spacing has been found to be particularly effective. There remains a possibility that a larger expanding retrieval effect may be found when feedback is not provided. Although this possibility is of theoretical importance, it may have limited pedagogical value because not only does feedback enhance learning (e.g., Metcalfe & Kornell, Reference Metcalfe and Kornell2007) but also authentic PAL typically involves feedback (e.g., Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Karpicke & Roediger, Reference Karpicke and Roediger2007).
The Effects of Absolute Spacing
The present study also found a significant main effect of absolute spacing. Multiple comparisons showed that massed learning may be the least effective, regardless of the posttest, scoring system, or RI. The findings are congruent with the spacing effect (e.g., Cepeda et al., Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006; Cull, Reference Cull2000; Cull et al., Reference Cull, Shaughnessy and Zechmeister1996; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Karpicke & Roediger, Reference Karpicke and Roediger2007; Logan & Balota, Reference Logan and Balota2008). From a theoretical perspective, the results support the robustness of the spacing effect and suggest that L2 vocabulary learning may also be subject to this effect. At the same time, no statistically significant difference existed among the three experimental groups in their effectiveness. This result is inconsistent with the lag effect (e.g., Cepeda et al., Reference Cepeda, Coburn, Rohrer, Wixted, Mozer and Pashler2009; Crothers & Suppes, Reference Crothers and Suppes1967; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Nakata, Reference Nakata2013; Pashler et al., Reference Pashler, Zarow and Triplett2003; Pyc & Rawson, Reference Pyc and Rawson2007), according to which larger absolute spacing leads to better long-term retention than shorter spacing. Additionally, the lack of significant difference among the three experimental groups in their effectiveness suggests that, depending on the range of ISIs, larger absolute spacing does not necessarily lead to better long-term retention than shorter spacing. In other words, although an ISI of 1 day may be significantly more effective than an ISI of 5 min (Cepeda et al., Reference Cepeda, Coburn, Rohrer, Wixted, Mozer and Pashler2009), for instance, there may be only a small difference between the effects of 5-min and 10-min ISIs (see Crothers & Suppes, Reference Crothers and Suppes1967; Logan & Balota, Reference Logan and Balota2008; Pyc & Rawson, Reference Pyc and Rawson2007, for similar findings).
Compared with medium and long spacing, both of which were significantly more effective than massed learning on all posttests, short spacing produced a smaller spacing effect. Short spacing led to significantly better performance than massed learning on the immediate productive posttest but not on the other posttests. The results may be explained in part by the ISI-RI interaction, which refers to a phenomenon when short spacing tends to be effective at a short RI, whereas long spacing tends to be effective at a long RI (e.g., Cepeda et al., Reference Cepeda, Coburn, Rohrer, Wixted, Mozer and Pashler2009; Cepeda et al., Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006). Due to the ISI-RI interaction, the advantage of short spacing over massed learning disappeared on the delayed posttest. Whereas short spacing fared significantly better than massed learning on the immediate productive posttest, the difference between the two did not reach statistical significance on either the immediate or delayed receptive test. The results may be partially due to the test order. At each test administration, the productive posttest was given prior to the receptive test (see the Dependent Measures section). As correct responses in the receptive test were used as cues in the productive test, the productive test might have affected performance on the receptive test, possibly diminishing a potential difference between the two treatments.
Pedagogical and Methodological Implications
The results of the present study showed that expanding spacing increased learning by 4.6%–8.5% relative to equal spacing on the receptive posttest. Although the difference was statistically significant, it may be debatable whether expanding spacing should be recommended over equal spacing. On one hand, given the small effect sizes, small differences in the mean gains, and narrow confidence intervals of difference, one could argue that relative spacing has little effect on vocabulary learning. On the other hand, the superiority of expanding over equal spacing may have some pedagogical significance, considering that the treatment required only 7.60–8.06 min on average per relative spacing schedule (see the Study Time section). If learners study a larger number of items over a longer period of time, this may translate into a large difference, and the use of expanding spacing would be advisable. The findings of this study also have implications for how vocabulary learning software should be designed. The statistically significant expanding retrieval effect observed in this study suggests that it may be useful for vocabulary learning software to support expanding spacing. Nonetheless, because the present study demonstrated only a limited advantage of expanding spacing, further research is warranted.
Another implication of this study is that it may be useful to introduce spacing between encounters because massed learning turned out to be the least effective regardless of the posttest, scoring system, or RI. When correct spelling was required, medium and long spacing increased learning by 107.0%–112.5% relative to massed learning on the delayed productive posttest. The results have important pedagogical implications because, although spacing may have a large effect on learning, its benefits have not been exploited fully in traditional instructional settings (e.g., Cepeda et al., Reference Cepeda, Coburn, Rohrer, Wixted, Mozer and Pashler2009). Furthermore, learners are often unaware that spacing increases learning (see Son & Simon, Reference Son and Simon2012, for a review). On the basis of the results of the present study, it may be useful to raise awareness of the importance of spacing. Although no significant difference existed among the three experimental groups in their posttest scores, medium and long spacing may be slightly more desirable than short spacing, for two reasons. First, whereas medium and long spacing significantly outperformed massed learning on all posttests, the difference between short spacing and massed learning reached significance only on the immediate productive posttest. Second, the significant interaction between absolute spacing and the RI on the productive posttest suggests that the effects of short spacing may not be as durable as those of medium or long spacing. Although these results are inconclusive, on the basis of these two reasons, medium and long spacing may be recommended over short spacing.
It should be noted here that the experimental groups’ higher posttest scores may have resulted from the fact that they required significantly longer retrieval times than the control group, which is one limitation of the present study. At the same time, it is possible to argue that not controlling retrieval times may increase ecological validity because it reflects authentic learning. Requiring greater retrieval effort (as indicated by longer retrieval latency) is often cited as an advantage of spaced over massed learning (e.g., Karpicke & Roediger, Reference Karpicke and Roediger2007; Logan & Balota, Reference Logan and Balota2008; Pyc & Rawson, Reference Pyc and Rawson2009). In other words, retrieval times may need to be considered as inherent properties of treatments and not as a separate variable. Furthermore, the analysis of efficiency scores (i.e., the number of words acquired per minute; e.g., Mondria, Reference Mondria2003; Pyc & Rawson, Reference Pyc and Rawson2007) showed that although the massed group required the least study time, it was no more efficient than any of the experimental groups (see the Study Time section). These findings suggest that spaced learning may be more desirable than massed learning, despite potential differences in the retrieval times.
The results of this study also indicate that learning phase performance may not necessarily be a good index of long-term retention (e.g., Ellis, Reference Ellis1995; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Pashler et al., Reference Pashler, Zarow and Triplett2003). Although massed learning led to the best learning phase performance, it turned out to be the least effective on both the immediate and delayed posttests. The results are consistent with the desirable difficulty framework, according to which a treatment that increases the initial rate of acquisition does not always enhance long-term retention (e.g., Bjork, Reference Bjork, Metcalfe and Shimamura1994; Schmidt & Bjork, Reference Schmidt and Bjork1992). Pedagogically, these findings imply that it may be useful to raise awareness that making mistakes during learning is not necessarily a sign of ineffective learning. Massed learning may give learners confidence because it often produces a large number of correct responses during practice. However, learners should be aware that the advantage of massed learning tends to disappear over time. Spaced learning, in contrast, may appear ineffective in the short term because it may lead to unsuccessful learning phase performance. Yet learners should not be discouraged or demotivated because, despite resulting in a large number of errors during learning, spaced learning often yields superior long-term retention to massed learning (e.g., Cepeda et al., Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006; Cull et al., Reference Cull, Shaughnessy and Zechmeister1996; Karpicke & Bauernschmidt, Reference Karpicke and Bauernschmidt2011; Logan & Balota, Reference Logan and Balota2008; Maddox et al., Reference Maddox, Balota, Coane and Duchek2011).
For research methodology, the lack of a positive correlation between learning phase and posttest performance suggests that it may not be valid to use learning phase performance as an index of long-term retention (Bjork, Reference Bjork, Metcalfe and Shimamura1994; Ellis, Reference Ellis1995; Schmidt & Bjork, Reference Schmidt and Bjork1992). For instance, some empirical studies have used trials to criterion, or the number of trials required for learners to successfully recall a target item a given number of times during learning, as the dependent measure (for a review, see Nation, Reference Nation2000). However, considering that a treatment that maximizes learning phase performance does not necessarily lead to superior long-term retention, other types of dependent variables, such as posttest scores, may be a more direct and valid measure of learning outcomes.
CONCLUSION AND DIRECTIONS FOR FURTHER RESEARCH
The present study demonstrated the limited, yet statistically significant, advantage of expanding over equal spacing. The finding is valuable because this is the first L2 study to show a statistically significant expanding retrieval effect. The main effect of absolute spacing was also significant, producing large effect sizes. Taken together, the results suggest that gradually increasing spacing may facilitate vocabulary learning, although introducing spacing may have a larger effect.
At the same time, the present study suffers from some limitations. One limitation is that the participants took multiple posttests (i.e., immediate productive, immediate receptive, delayed productive, and delayed receptive posttests). As a result, earlier tests might have affected performance on later tests. Future research could administer fewer posttests per participant to reduce potential learning effects from posttests. Second, the sensitive scoring system for the productive test may have been too lenient because the response was scored as correct if 75% or more of the letters in the target word were found in the response, regardless of the position (e.g., Barcroft, Reference Barcroft2002). Third, retrieval practice was self-paced, and participants were allowed to take as much time as they needed to type a response during the treatment. Although this may have increased ecological validity, the experimental groups’ higher posttest scores may have resulted from the fact that they required significantly longer retrieval times than the control group. In future research, it may be valuable to control retrieval times. Future studies comparing equal and expanding spacing will be useful because they may further enable us to identify the optimal practice schedule for L2 vocabulary learning.
APPENDIX A
ADDITIONAL INFORMATION ABOUT PARTICIPANTS
To counterbalance the effects of target items across participants, each of the three experimental groups needed to consist of an equal number of participants from Subgroups X and Y (Table 1). The short spacing group, however, consisted of 16 Subgroup X and 17 Subgroup Y students. One Subgroup Y participant was excluded from analysis so that the short spacing group would have an equal number of participants (n = 16) from both subgroups. The participant to be dropped was determined in a way that minimized the effects on the mean posttest scores of Subgroup Y, the short spacing group.
When one Subgroup Y participant was excluded from the short spacing group, there were nine, eight, ten, and eight OU students in the massed, short, medium, and long groups, respectively, whereas each of the four groups consisted of an equal number of GTC participants (n = 24). Because OU and GTC students might differ in variables that may influence the ability to learn vocabulary from PAL tasks—such as English proficiency or memory capacity—the four absolute spacing groups needed to consist of an equal number of OU and GTC students. To redress the imbalance in the number of OU students, one and two OU students were dropped from the massed and medium groups, respectively. The participants to be dropped were determined in a way that minimized the effects of their scores on the mean posttest scores of the groups from which these participants were dropped. After three OU students were excluded, each of the massed, short, medium, and long groups consisted of 24 GTC and 8 OU students, leaving 128 students in total (32 × 4 = 128). The data from these 128 students were analyzed.
APPENDIX B
TARGET ENGLISH-JAPANESE WORD PAIRS
APPENDIX C
AVERAGE NUMBER OF CORRECT RESPONSES ON THE PRODUCTIVE POSTTEST
Brackets enclose ± 1 standard error.
