1. Introduction
Vocabulary knowledge is integral in learning a second language (L2). In addition, acquiring vocabulary is a key challenge for university L2 learners of English for academic purposes (Berman & Cheng, Reference Berman and Cheng2010; Evans & Morrison, Reference Evans and Morrison2011), particularly in classroom contexts with insufficient exposure due to time restrictions. As a result, teachers and students have been searching for effective alternative methods to traditional classroom courses.
In recent decades, mobile-assisted language learning (MALL) has garnered substantial interest in Chinese universities due to its pragmatic and functional applications. Mobile technologies (e.g. smartphones, PDAs, and tablet computers) are creating new learning protocols for students (Kim & Kwon, Reference Kim and Kwon2012), making it possible for individuals to learn language on an “ongoing basis, in a range of settings, according to a person’s ability and adapted to their needs” (Kukulska-Hulme, Lee & Norris, Reference Kukulska-Hulme, Lee, Norris, Chapelle and Sauro2017: 217), gradually blurring the boundaries of formal and informal learning. Furthermore, such increased use of mobile technologies provides learners with abundant opportunities to interact, using language to meet everyday communication needs and engage in different cultural experiences (Lu, Reference Lu2008). Mobile devices have also been found to be significantly beneficial for establishing, organising, and personalising appropriate goals for learning as well as customising protocol designs for teaching.
In the past several years, a growing amount of literature has explored the diverse modalities of mobile devices, email, short message service (SMS), multimedia message service (MMS), and apps for vocabulary learning. These mobile-based approaches may entail sending learners word lists, mini-lessons, or related text notifications via SMS, MMS, or email (Alemi & Lari, Reference Alemi and Lari2012; Kennedy & Levy, Reference Kennedy and Levy2008; Lu, Reference Lu2008; Saran & Seferoğlu, Reference Saran and Seferoğlu2010; Tabatabaei & Goojani, Reference Tabatabaei and Goojani2012; Thornton & Houser, Reference Thornton and Houser2005; Zhang, Song & Burston, Reference Zhang, Song and Burston2011). Alternatively, the strategy of using word cards (also called flashcards) has been adopted by a wide variety of mobile apps (Başoğlu & Akdemir, Reference Başoğlu and Akdemir2010; Nikoopour & Kazemi, Reference Nikoopour and Kazemi2014). Nevertheless, the majority of research into mobile-assisted vocabulary learning has exclusively focused on advances in vocabulary size (i.e. how many words are known), with only a few studies addressing other aspects of word knowledge. However, the basic receptive knowledge of a form–meaning link alone cannot ensure that words are used competently in context. The present study contributes to the growing body of research into mobile-assisted vocabulary learning by investigating and comparing the potential effects of mobile-based word cards and paper-based word cards on receptive and productive aspects of vocabulary knowledge gains. In particular, the study assesses both receptive knowledge of the form–meaning connection as well as productive knowledge required to recall collocations.
2. Literature review
2.1 Defining receptive and productive vocabulary
Knowing a word involves different aspects of vocabulary knowledge (Henriksen, Reference Henriksen1999; Nation, Reference Nation2001). One way of distinguishing various types of vocabulary knowledge is the receptive–productive dimension, which can help to estimate the level of mastery of comprehension and production of lexis (Henriksen, Reference Henriksen1999; Read, Reference Read, Bogaards and Laufer2004; Schmitt, Reference Schmitt2014). Receptive knowledge is often defined as the ability to comprehend words in listening and reading; productive knowledge, on the other hand, is the language ability required when producing words in speaking and writing (Schmitt, Reference Schmitt2010). The present study looks at receptive and productive mastery within Nation’s (Reference Nation2001) multi-component framework: form (spoken, written, word parts), meaning (form and meaning, concept and referents, association), and use (grammatical functions, collocation, constraints on use); each category can be broken down into both receptive and productive knowledge.
The majority of research on vocabulary teaching and learning tends to investigate knowledge of the form–meaning link, as this is inevitably the most central “sub-category” of knowledge of vocabulary (Laufer, Elder, Hill & Congdon, Reference Laufer, Elder, Hill and Congdon2004). Henriksen (Reference Henriksen1999) noted that knowing a word in greater depth encompasses not only mapping meaning onto form (extensional links between concept and referents) but also building a network between words (intensional links with other semantically related words; e.g. collocation). Consequently, collocation knowledge is also a key aspect of vocabulary learning. The term collocation often refers to word combinations that frequently co-occur within a given span (see, e.g., Ellis, Reference Ellis1996; Sinclair, Reference Sinclair1991). Lewis (Reference Lewis, Willis and Willis1996) stated that collocation knowledge is critical in the productive skills of speaking and writing. Laufer and Waldman (Reference Laufer and Waldman2011) also suggested that using collocations properly, rather than simply recognising them, is a significant challenge for L2 learners. Therefore, the present study focuses on both the receptive dimension of knowledge of the form–meaning link and the productive dimension of collocation knowledge in vocabulary learning.
2.2 Intentional and incidental vocabulary learning
The two main vocabulary learning processes include incidental learning and intentional learning, and these may also apply to vocabulary learning strategies and approaches (Gu, Reference Gu2003; Nation, Reference Nation2001). From the perspective of L2 pedagogy, incidental vocabulary learning is often believed to occur as the by-product of any non-vocabulary-learning-focused activity (Hulstijn, Reference Hulstijn and Robinson2001; Schmitt, Reference Schmitt2000). For example, learners may build vocabulary by using contextual information to infer the meaning of unfamiliar words during extensive reading. Existing research on MALL has attempted to utilise different modalities and media, such as e-books, podcasts or video blogs, to improve the incidental learning of vocabulary for English as a foreign language (EFL) learners (e.g. Arndt & Woore, Reference Arndt and Woore2018; Song & Fox, Reference Song and Fox2008). However, incidental learning does not necessarily lead to better knowledge retention if learners’ attention to lexical items is not sufficient during the message-focused process. Hulstijn (Reference Hulstijn and Robinson2001) suggested that processing new lexical information in a more elaborate way or in more depth (i.e. careful attention is focused on a word’s pronunciation, orthography, grammatical category, meaning, and semantic relations to other words) brings higher retention than engaging with the word in a less elaborate manner (i.e. limited attention is focused on only one or two of these dimensions). Intentional vocabulary learning, aiming at committing lexical and usage information to memory, explicitly focuses on vocabulary rather than on “other possible tasks or other possible stimuli in the learner’s … environment” (Barcroft, Reference Barcroft2015: 51).
Using word cards is one common intentional vocabulary learning strategy that has been widely adopted. Word cards are a set of cards with a foreign language word form on one side and information about that word, such as its meaning in the learner’s first language (L1), on the other side. Such cards allow learners to learn target items in a decontextualised paired-associate format (connecting word form and meaning) (Nakata, Reference Nakata and Webb2020). For example, in attempting to learn an English word, students could recall its L1 meaning when viewing the English word form on one side of the card or recall the L2 form when viewing the L1 translation on the other. After retrieval, learners obtain immediate feedback on their attempt by consulting the other side of the card. Although word cards are focused on knowledge of the form–meaning connection in the first encounter, word card learning does not exclude the possibility of acquiring other components of knowledge. Students typically learn individual vocabulary items by instruction when they also focus attention on the properties of collocation. For example, with a teacher’s guidance, students could be tasked with identifying collocations in a given text and then completing activities of memorisation, such as practice and repetition (Hill, Lewis & Lewis, Reference Hill, Lewis, Lewis and Lewis2000). Nevertheless, it is unlikely for learners to encounter a sufficient number of collocations in a limited amount of classroom time. Word cards, as a supplement, can be enhanced by placing one or two sample collocations related to the target word on the card, thereby providing L2 learners with opportunities to learn about the appropriate uses of words in context. The present study applied this thinking and approach by presenting vocabulary items accompanied by related collocations on both mobile-based word cards and physical word cards.
2.3 Vocabulary learning through mobile-based word cards
During the past decade, a number of studies have investigated the use of digital word cards in vocabulary learning (e.g. Başoğlu & Akdemir, Reference Başoğlu and Akdemir2010; Dizon & Tang, Reference Dizon and Tang2017; Ma & Yodkamlue, Reference Ma and Yodkamlue2019; Nikoopour & Kazemi, Reference Nikoopour and Kazemi2014). Ma and Yodkamlue (Reference Ma and Yodkamlue2019) investigated first-year university students’ vocabulary achievement and retention by using a mobile word cards app. The result showed that the mobile app enabled learners to learn and retain more words than learners who were using a paper-based word list. Nikoopour and Kazemi (Reference Nikoopour and Kazemi2014) compared the outcome of digital (mobile and online) as well as non-digital (paper) word cards delivery; advanced L2 learners’ gains in meaning knowledge were assessed using a multiple-choice format. It was found that the difference between the digitised and non-digitised group was not significant. However, the attitude of users of mobile word cards was more positive than the other two groups, and the quality of informal learning was enhanced by the use of mobile phones. This positive perception of mobile word cards apps is consistent with other studies (Başoğlu & Akdemir, Reference Başoğlu and Akdemir2010; Ma & Yodkamlue, Reference Ma and Yodkamlue2019), which reported that learners prefer the use of mobile devices for language learning due to their availability, portability, and accessibility. Dizon and Tang (Reference Dizon and Tang2017) compared the efficacy of mobile word cards and paper word cards in conjunction with three vocabulary learning strategies (dropping, association, and oral rehearsal) regarding receptive and productive L2 vocabulary gains. L2 university learners’ meaning recognition knowledge and form recall knowledge were elicited at the beginning and end of the treatment. The results showed that both the experimental and control group made improvements after the treatment, although no significant difference was found between the increase in the two groups of learners. They suggested that the reason for this result might be due to the impact of the explicit instruction of vocabulary learning strategies applied in the paper-based cards group, while no similar strategy instruction was applied in the mobile-assisted learning group. This finding implies that the incorporation of cognitive or metacognitive learning techniques may be a moderator variable for the word cards learning effect, thereby warranting further investigation. The current study presents a general learning situation without employing any specific vocabulary learning technique in both groups.
In sum, the aforementioned studies suggest that both mobile-based and paper-based word cards learning improve the gains in L2 vocabulary. However, more research is needed to compare the efficacy of the two delivery tools. The results of previous studies were inconsistent regarding which learning method is more effective, even when the same vocabulary assessment instrument was employed (Başoğlu & Akdemir, Reference Başoğlu and Akdemir2010; Nikoopour & Kazemi, Reference Nikoopour and Kazemi2014). In addition, these studies are also limited by the lack of empirical measurements of multiple word knowledge components. Although knowledge of the receptive form–meaning connection has been given much attention, other word knowledge components have seldom been investigated, with a few notable exceptions (e.g. Dizon & Tang, Reference Dizon and Tang2017). To explore the full picture of the efficacy of mobile-assisted learning and paper-based vocabulary learning, different receptive and productive dimensions of word knowledge need to be measured concurrently.
The present study sets out to investigate the vocabulary learning effects of mobile-based and paper-based delivery methods: a word cards app (Zhimi) and physical word cards on paper. Accordingly, we examine intermediate-level students’ receptive (form–meaning connection) and productive (collocation) mastery of words. The following research questions guide this study:
-
1. What are the differences, if any, in the EFL learners’ knowledge of vocabulary after using either the mobile word cards app or the paper word cards?
-
2. Between the two delivery methods, which is more effective in assisting EFL learners to acquire receptive vocabulary knowledge (form–meaning connection) and productive vocabulary knowledge (collocation)?
-
3. What are EFL learners’ attitudes towards the mobile app as a vocabulary learning tool?
3. Method
3.1 Participants
A total of 85 participants (56 female and 29 male) participated in the study. The participants comprised freshman students majoring in medicine at a university in China. They were aged between 18 and 22 years. Their language proficiency was at the B1 level of the Common European Framework of Reference for Languages, according to their scores on the National Matriculation English Test in China. The participants had been preparing to take the College English Test Band 4 (CET-4) several months after the study took place. CET-4 is a national exam in China, and desirable scores are valued in the job recruitment process. One challenge that Chinese university students generally face is building up vocabulary as they prepare for the test. Thus, participants expressed motivation in joining this study in the hope that they could find a vocabulary learning strategy that was both efficient and effective.
For the purposes of the study, students were allowed to select which group they preferred to join: the mobile app group or the paper-based word cards group. This gave each student the opportunity to select which educational tool would most likely suit his or her individual learning strategy. Those students who selected the mobile app formed the experimental group (n = 46), and those who opted to use the word cards formed the control group (n = 39). Prior to the intervention, a vocabulary pre-test was performed, and an independent samples t-test performed on the pre-test confirmed that there was no significant difference between the two groups regarding their vocabulary level before the treatment (see Table 1).
Vocabulary pre-test score results of the mobile app and word cards groups
3.2 Target words
A preliminary test involving 70 common English vocabulary words was designed prior to the study. The aim of the preliminary test, administered to a subset of 25 participants, was to identify suitable lexical items that learners would be expected to learn by identifying items with which participants had a low level of knowledge. This test was designed based on a revised Test of Academic Lexicon (Scarcella & Zimmerman, Reference Scarcella and Zimmerman1998), in which participants identify four levels of word knowledge: (a) I don’t know the word; (b) I have seen the word before but am not sure of the meaning; (c) I understand the word when I see or hear it in a sentence, but I don’t know how to use it in my own speaking or writing; (d) I can use the word in a sentence. The items provided in the test were extracted from the vocabulary list of the CET-4 exam and also cross-checked with the LTP Dictionary of Selected Collocations (Lewis & Hill, Reference Lewis and Hill1997) as well as the Academic Word List (Coxhead, 2000). Four points, three points, two points, and one point were respectively given for the answer to (d), (c), (b), and (a). The test demonstrated good reliability for testing depth of vocabulary knowledge; Cronbach’s alpha was .95. After analysis, the study retained 50 words with a mean score below 3 points, as participants were unlikely to have receptive and/or productive knowledge of these vocabulary items. These 50 items were then divided into five sets of 10 items. The study participants were tasked with studying one set of these items per day.
3.3 Materials
The study was conducted during English classes, and the vocabulary learning tools employed were a digital mobile app (Zhimi) and physical word cards. The mobile app, Zhimi, was chosen for its spaced repetition, audio pronunciation, L1 support, and in particular, collocation knowledge instruction, which is a feature not commonly employed in other apps. The app was programmed by the BlueBrain Educational Technology Co., Ltd, and was used by the experimental group. The app first presents an English vocabulary item with phonetic symbols and an audio pronunciation guide, then prompts the user to repeat the word out loud and guess the meaning of the word, as seen in Figure 1(a). If the user is unable to guess the English word’s meaning, the app will subsequently display its Chinese definition(s) and part of speech and provide additional collocation(s) to help put the word in a simple context, which assists the user in gaining a better understanding of the word’s practical usage, as seen in Figure 1(b). When the app introduces a word to the user for the first time, it is accompanied by two initial exercises designed to help review the word’s target meaning and related collocation knowledge. Figure 1(c) shows the recall of collocation meaning exercise. Figure 1(d) shows the recognition of collocation form exercise. The app features spacing algorithms that constantly monitor and record the user’s performance while completing the exercises in an attempt to gauge the user’s familiarity with the word and adjust the retrieval sequencing. When the user completes a set of items and the corresponding exercises, the app progressively offers new items to learn, and the sequence of words presented is decided by the app.
Screenshots of the Zhimi app used by the experimental group
The word cards used by the control group were designed according to the format provided by the Cambridge Flashcard Maker website (http://www.cambridgeenglishonline.com/Flashcard_maker/). One side of the word card presented an English word together with its phonetic symbols, and the other side of the card presented the word’s definition(s), part of speech, and related collocation(s). For the purposes of this study, the contents of the word cards had to match the contents of the mobile app. To ensure uniformity across both the word cards and mobile app, the same word items, Chinese definitions, phonetic symbols, part of speech, and sample collocations were duplicated from the app onto the word cards. The main differences that set the two learning tools apart were the following: the English vocabulary items in the app were automatically presented, and the users were subsequently tested via the collocational exercises; the performance and skill level of the user determined the app’s retrieval sequencing; the app provided standard audio pronunciations of the words; and finally, the app notified its users when they had completed a set of assigned learning tasks, while the word cards themselves could not reflect an individual’s learning progress in this way.
3.4 Instruments
3.4.1 Vocabulary pre-test and post-test
To assess the vocabulary knowledge of the students, a pre-test and post-test with a pencil-and-paper format were carried out during orientation and at the end of the study, respectively. The contents of both tests were identical. Three types of questions were chosen to gauge the development of vocabulary knowledge, and the questions were divided into two sections to measure receptive and productive aspects of knowledge. (See Appendix in the supplementary material for the complete test battery.)
The section that tested receptive knowledge was designed after the Academic Vocabulary Test (Schmitt, Schmitt & Clapham, Reference Schmitt, Schmitt and Clapham2001), which aims to measure receptive knowledge of the form–meaning connection. An example of one set of vocabulary items assessed is shown in Extract 1.
Extract 1 Sample of the section testing receptive knowledge
Please choose the right word to go with each meaning. Write the number of that word next to its meaning.
1 construction
2 feature __________________noticeable part of something
3 impact _______________joining something into a whole
4 obtain _______________collecting things over time
5 integration
6 accumulation
Productive knowledge was also tested in order to gauge how much the students gained regarding the appropriate use of words in collocations. Collocation knowledge involves grammatical collocation and lexical collocation (Bahns, Reference Bahns1993). Both types of collocations were tested in the productive knowledge section. In terms of the grammatical type, a cloze test was provided, based on the format used by Bahns (Reference Bahns1993) and Ha (Reference Ha1988). An example is provided in Extract 2.
Extract 2 Sample of section testing productive knowledge
Please fill in the blanks with prepositions to complete the sentences. The meaning of the verbal expression is in parentheses at the end of the line.
To my surprise, he refused to cooperate_________ us! (to work together)
Productive knowledge of lexical collocation was captured by a translation test, which required writing an English translation of a given Chinese sentence, where the sentence should contain the target word-related collocation. This may provide rich evidence of the learners’ ability to use the collocations appropriately in a context. The test helped students with certain words and phrases, which were not necessarily considered as targets, by giving further hints in parentheses. One example of a translation test item can be seen in Extract 3.
Extract 3 Sample of the section testing productive knowledge
Please translate the following sentences into English and pay attention to the expressions of the phrases that are underlined.
我通过阅读获得知识。_________________________________________
(Translation: I acquire knowledge through reading.)
Overall, the pre- and post-test of receptive and productive knowledge demonstrated a relatively high level of reliability (Cronbach’s alpha = .78, .89) for the two groups of learners.
3.4.2 Qualitative interviews
Following the completion of the post-test, interviews were scheduled within one week for five participants who had different levels of gains (small, medium, large) after the experiment. The interviews aimed to learn about the students’ attitudes, reactions, and reflections regarding the mobile app as a vocabulary learning tool. The study used a systematic sampling technique to select the five participants from the experimental group. These five students all gave their informed consent to participate in the interview. These interviews took place in the Chinese language over the instant messaging platform QQ, which is widely used by students. Their familiarity with QQ could have helped the students respond to the interview questions openly and honestly. The interview comprised three questions, which were as follows:
-
1. Do you think this mobile app is useful for acquiring vocabulary knowledge? (Please specify.) Do you enjoy using it?
-
2. What are the advantages and disadvantages of this mobile app? (Please compare it with other strategies you have used when learning vocabulary.)
-
3. How would you rate this application (from 1 to 5)? Would you keep using it even if you were not in class? (Please specify.)
3.5 Procedure
The experiment procedure for the mobile app group and word cards group is given in Table 2.
The experiment procedure of the two groups
3.6 Scoring and data analysis
The pre- and post-test answers of all participants were scored by the researcher. Two points were assigned to each correct answer, with a possible maximum total score of 100 points. To ensure the students had successfully acquired the target collocation, synonyms of the collocation were not counted as correct responses in the productive knowledge test, but minor mistakes were accepted, such as one- or two-letter spelling mistakes and grammatical mistakes. Descriptive statistics, independent and dependent t-tests were employed for quantitative data analysis. The interviews were transcribed, reflecting the attitudes of the participants toward the mobile app as a vocabulary learning tool. A qualitative analysis was then carried out according to the results of the interviews.
4. Results
4.1 Levels of vocabulary knowledge prior to and following the study
The first research objective of the study was to determine any possible difference in apparent vocabulary levels between the pre-test and post-test results of the mobile app and word cards groups.
As seen in Table 3, the dependent t-test found that the post-test score of the mobile app group (M = 69.57) was higher than its pre-test score (M = 28.43), t(45) = 21.25, p < .05. This result meant that the vocabulary knowledge of the students in the mobile app group significantly improved.
Comparison of the pre- and post-tests total scores of the mobile app group
* Indicates statistical significance at the 0.05 level.
As seen in Table 4, the dependent t-test found that the post-test score of the word cards group (M = 60.51) was statistically higher than its pre-test score (M = 28.62), t(38) = 13.74, p < .05. This result meant that the vocabulary knowledge level of the students in the control group also improved. In summary, the post-test results showed that both learning tools led to the improvement of vocabulary knowledge for all participants in the study.
Comparison of the pre- and post-tests total scores of the word cards group
* Indicates statistical significance at the 0.05 level.
An independent samples t-test was conducted to compare the mean gain scores of the mobile app group and word cards group before and after the treatment (see Table 5). The increase in mean score gained by the mobile group was 41.13 and 31.90 by the word cards group. The result indicated that there was a significant difference in the mean gain scores between the two groups (p < .05). This suggested that the mobile app may have been the more effective vocabulary learning tool.
Comparison of the pre- and post-tests mean gain scores of the mobile app and word cards groups
* Indicates statistical significance at the 0.05 level.
4.2 Comparison of receptive and productive vocabulary gains prior to and following the experiment
This study aims to answer a second research question regarding which learning tool, either the mobile app or the word cards, was able to improve receptive vocabulary knowledge (form–meaning connection) and productive vocabulary knowledge (collocational use) more effectively. Therefore, the two groups’ mean gain scores on the two tasks were compared using independent samples t-tests (see Table 5).
The results indicated that there was a statistically significant difference in the productive knowledge section of the assessment between the two groups (p < .05). The increase in mean score gained by the mobile app group and word cards group was 22.65 and 17.54, respectively. This meant that participants who used the mobile app produced on average more correct word-related collocations after the treatment. Cohen’s d indicated that there was a medium-sized effect (d = .56). The results also revealed a statistically significant difference in receptive knowledge between the two groups regarding overall gains (p < .05), although there was only a small- to medium-sized effect (d = .46).
4.3 Attitudes of participants towards the mobile app as a vocabulary learning tool
Interviews were conducted with the participants to ask about their experiences using the mobile app. There were three major themes that were observed from the interviews: accessibility, motivation, and interest. The comments of the interviewees were mostly positive toward the mobile app. Some of the statements given in the post-test interviews are as follows:
I always check whether my cellphone is in my pocket when I leave home; however, carrying word cards is not a habitual process in my life. It would be inconvenient to take paper word cards with me everywhere I go. (Interview, Participant 1)
I think the application is effective because learning each word is like advancing in a game, which boosts my motivation to overcome each vocabulary challenge. (Interview, Participant 3)
The application enhances my interests in memorising vocabulary to some extent. (Interview, Participant 5)
Previous studies have also found similar positive attitudes toward vocabulary learning via mobile apps. Freshmen students who participated in the study conducted by Wang, Teng and Chen (Reference Wang, Teng and Chen2015) reported that they felt more motivation and enjoyment than learning from traditional semantic maps. Moreover, Nikoopour and Kazemi (Reference Nikoopour and Kazemi2014) found that people valued the portable quality and convenience of mobile phones. The findings also reveal that positive learner attributes, such as learning motivation, learning strategy, and personal interests, may have contributed to better vocabulary learning effects for the mobile-assisted learning group.
In addition, learners in the mobile app group were asked whether they would like to continue learning vocabulary through the app. Some of the participants explained that it would be difficult to continue using it without supervision. One of the interviewees said,
I will try to keep using it, but honestly, I may not use it in my daily life as frequently as I used it for the experiment, because sometimes I am a bit lazy. (Interview, Participant 1)
Another participant pointed out that the factor of unique needs varies between individuals:
One or two months before an English exam, I may use it as carefully as I used it in this experiment. (Interview, Participant 2)
One week after the experiment, a particular student reported that he had continued using the app. He also compared the app’s learning experience with other traditional methods:
Compared with other paper-based materials, I tend to be more self-disciplined when I use this application, so I prefer to keep using it in my daily learning routine. (Interview, Participant 3)
These varied responses may suggest that not all learners would be keen on utilising the mobile app in the long run. Longitudinal research is needed to investigate the learning effects of digital word cards, also paying attention to whether any novelty effect can be observed. It is also worth noting that mobile platforms have the potential feature to facilitate learner-initiated and learner-directed vocabulary learning experience in and outside the classroom. Educators and designers could draw on the affordances that mobile technologies provide in order to create opportunities for L2 vocabulary learning within a more autonomous learning environment. Although there was a careful selection of the participants, only a relatively small number of participants were interviewed and, as a result of this limitation, the results must be interpreted cautiously.
5. Discussion
5.1 Levels of vocabulary knowledge prior to and following the study
The improvement of the two groups on total scores supports the findings of earlier studies that both paper word cards and mobile apps can provide an opportunity for effective vocabulary learning (e.g. Başoğlu & Akdemir, Reference Başoğlu and Akdemir2010; Dizon & Tang, Reference Dizon and Tang2017). Improvements observed are largely due to the effects of retrieval, in effect a type of learning activity that involves the process of recalling vocabulary-related information (e.g. form, meaning) previously encountered. The results are consistent with the finding that retrieval of material significantly increases L2 vocabulary learning (Barcroft, Reference Barcroft2007; Karpicke & Roediger, Reference Karpicke and Roediger2008). Both the mobile app and word cards presented the target items first, then required recall of the meanings and collocations afterwards. The process helped the students retrieve the items from their memory and gain access to new learning materials or correct their errors with feedback immediately and repeatedly, thus producing successful retention of vocabulary. One of the limitations of the study is that it did not administer a delayed test at an interval of longer than one week. Nakata (Reference Nakata2017) recently found that the benefits of repeated retrievals facilitate L2 vocabulary learning continuously four weeks after the treatment. However, future studies should aim to test more target words if possible and assess whether the positive effects of both paper- and mobile-based word cards learning could be retained over a longer time span.
The results further support previous research, which suggested that digital word cards provide a greater boost regarding learning outcomes when compared to physical word cards (e.g. Başoğlu & Akdemir, Reference Başoğlu and Akdemir2010). Although the expanding rehearsal strategy, also known as the spaced repetition technique (Baddeley, Reference Baddeley1997; Ellis, Reference Ellis1995), can optimise learning with both tools, most learners in the control group were observed in the classroom to be unable to utilise the word cards to their best advantage effectively. The control group participants were directed to sort the cards into separate decks during the experiment, with the most difficult vocabulary items placed at the beginning of the sequence, as this would have helped the participants encounter the most difficult vocabulary items with more frequency. However, sorting the word cards by difficulty would have required the students to accurately evaluate their vocabulary knowledge levels using significant metacognitive abilities (Nakata, Reference Nakata2008). Alternatively, the higher gains within the experimental group could have occurred because the mobile app with spacing algorithms afforded automatic assessment and analysis (e.g. user’s performance, retrieval interval), relieving the experimental group participants from the burden of complex self-evaluation.
5.2 Comparison of receptive and productive vocabulary knowledge gains prior to and following the experiment
Another question the current study raised is whether there were any differences in receptive and productive vocabulary knowledge gains that the mobile app and paper word cards promoted.
The mobile app group was found to have greater gains in productive vocabulary knowledge (collocational use) than the paper word cards group (d = .56). This was as expected as the finding is in line with the depth of processing theory (Craik & Lockhart, Reference Craik and Lockhart1972), which states that the deeper level of cognitive analysis of the stimuli leads to more elaborate, stronger, and longer memory retention. The mobile app helped the users to retrieve the target word in the form of word-related collocation, and it also provided collocational exercises for revision, whereas gains in productive knowledge for the paper word cards group had to depend on the users’ autonomy to direct their attention to the collocational information. It appears that productive knowledge was processed in more depth with the mobile app than the paper word cards. In addition, the results are consistent with the transfer-appropriate processing framework (Morris, Bransford & Franks, Reference Morris, Bransford and Franks1977), which states that most optimal learning occurs under the condition when learning matches testing. The collocational exercises of the app involve two types of retrieval: meaning recall (viewing the English translation of collocation and retrieving its Chinese meaning) and form recognition (selecting the correct form of the collocation from four options). The learning process may promote learners’ performance in producing appropriate collocation knowledge. In contrast, for the users of paper word cards, the testing practices during learning could be limited to the meaning/form recall of the target word. There is no opportunity for the users to review the collocations explicitly as the information is simply displayed on the other side of the cards.
The scores of the two groups indicated that there was a small- to medium-sized effect on receptive vocabulary knowledge (form–meaning connection) (d = .46). The participants in the mobile app group outperformed those in the paper word cards group; there was also a statistically significant difference between the digitised and non-digitised tools, which confirms and extends the findings in some previous studies (e.g. Başoğlu & Akdemir, Reference Başoğlu and Akdemir2010), but contradicts the findings in others (e.g. Nikoopour & Kazemi, Reference Nikoopour and Kazemi2014). Learning effects might be affected by multiple factors, such as the testing formats (multiple choice and matching) and sample sizes. Future research could use a larger sample size and different measurement instruments to explore how these factors affect vocabulary acquisition.
6. Implications for design
The findings suggest some starting points for the development of future smartphone-based word cards apps. First, auditory presentation plays a crucial role in verbal information retention (Baddeley, Gathercole & Papagno, Reference Baddeley, Gathercole and Papagno1998). The app in this study exposed the learners to native speaker pronunciation instructions when showing the form of the target word. Here, the multimodal, visual, and auditory presentation likely contributes to more successful recall of pronunciation. One participant reported, “I think one of the most significant benefits of this application is that it can teach me how to pronounce a word. If I were asked to pronounce these words according to their phonetic symbols, I would make many mistakes”. Future apps could follow this multimodal design, as it may benefit learners’ phonological knowledge. Furthermore, picture and video presentations could also be incorporated into the design of mobile vocabulary learning apps. The temporal contiguity principle (Mayer, Reference Mayer2002) suggests that displaying pictures simultaneously with the corresponding words may help students achieve better learning results. Visual information provides learners with opportunities to construct the meaning of the words in a contextualised learning environment, reinforcing the connection between form and meaning. Finally, intermediate to advanced EFL learners often exhibit a greater need to learn vocabulary in a more comprehensive manner, particularly for academic purposes (e.g. listening to lectures, writing academic essays). Mobile-assisted vocabulary learning and assessment tasks are, however, usually limited to the basic meaning of the word. Webb (Reference Webb2009) found that paired-associate learning improves learners’ multiple vocabulary knowledge constructs (i.e. orthography, association, collocation, grammatical functions, and form–meaning connection). The application used in this study focuses on the knowledge of single form–meaning mapping and collocational use, but other word components (e.g. phonological knowledge, polysemous meaning) should also be incorporated by future word cards apps.
By directly comparing the learning effects of receptive and productive word knowledge from a mobile app and paper word cards, this study adds to the ongoing research into mobile-assisted vocabulary learning and teaching. While most previous research on mobile-assisted vocabulary learning has focused only on the dimension of receptive knowledge, this study, in contrast, investigated both receptive knowledge of the form–meaning link and productive knowledge of collocations. The study demonstrated that both mobile- and paper-based word cards tools benefited vocabulary learning. The results also suggest that additional encounters, a deeper level of analysis, and the effective practice of vocabulary usage could promote greater gains in productive collocational knowledge with the app than with the physical word cards. At the same time, there was some limited evidence that the receptive knowledge of the form–meaning link was strengthened better with the app. It is a common belief that vocabulary knowledge gradually shifts from receptive knowledge to productive knowledge on a developmental continuum (e.g. Melka, Reference Melka, Schmitt and McCarthy1997). Therefore, attention and time on productive knowledge in the mobile-based vocabulary learning and recycling process make for a sensible pedagogical target.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0958344021000161
Acknowledgements
We would like to thank the anonymous reviewers for their valuable feedback. Special thanks also go to Associate Professor David Hirsh, who offered useful suggested changes.
Ethical statement
This research project was conducted in accordance with the requirements and approval of the Research Ethics Committee of City University of Hong Kong. Informed consent was obtained from all individual participants involved in the study. The authors declare that they have no conflicts of interest.
About the authors
Yan Li is a PhD candidate in the Sydney School of Education and Social Work, The University of Sydney. She has had teaching experience at the secondary and tertiary level in China. Her research interests include the acquisition and instruction of vocabulary in second languages and computer-assisted language learning.
Christoph A. Hafner is an associate professor in the Department of English, City University of Hong Kong. He has published widely on English for specific purposes, digital literacies, and language learning and technology. His latest book is Understanding Digital Literacies: A Practical Introduction (2nd edition, Routledge, 2021, co-authored with Rodney Jones).
Author ORCIDs
Yan Li, https://orcid.org/0000-0002-9480-483X
Christoph A. Hafner, https://orcid.org/0000-0002-9592-8092
