1 Introduction
Maximizing second language vocabulary acquisition (SLVA) rates for students attending an international school can be daunting, as a plethora of challenges must often be overcome. According to Carder (Reference Carder2008), such students can be at risk on several fronts; these include low second language L2 proficiency upon acceptance to the school (Robinson, Keough & Kusuma-Powell, Reference Robinson, Keough and Kusuma-Powell2004), culture and language shock (Haldimann & Hollington, Reference Haldimann and Hollington2004). Evidence also indicates that in general, English as a second language (ESL) learners are aware of their own lexical deficiencies (Folse, Reference Folse2010). Additionally, Hulstijin (Reference Hulstijin2001: 258) notes that many L2 learners are apprehensive toward the “burden of vocabulary learning” along with managing the difficult task of acquiring numerous lexical items.
When acquiring a new word, the learner must view it on five to sixteen occasions for it to be fully retained (Nation, Reference Nation1990). That being said, one begins to wonder how these learners can ever eventually attain even close to the average ESL post-graduate university student’s 8,000 to 9,000-word lexicon (Nation, Reference Nation2006), never mind an average native adult speaker's lexicon, which is estimated at 17,000 words (Goulden, Nation & Read, Reference Goulden, Nation and Read1990). Such a situation was encapsulated by Cobb, (Reference Cobb1999: 25), who noted that English for academic purposes (EAP) university students need to acquire “words measured in the thousands, not hundreds, but receive language instruction measured in months, not years”.
The appropriate path to improving these learners’ L2 lexical deficiencies, along with their anxieties, has been an area of fierce debate among researchers. Although it has been argued that L2 lexis can be acquired sufficiently from casual reading (see e.g. Krashen, Reference Krashen1989), yet conversely, there is an “extreme unlikelihood of developing an adequate L2 reading lexicon through reading alone” (Cobb, Reference Cobb2007: 38). Thus implicit learning should be buffered with explicit learning, via instructor-enhanced contextual learning, if L2 students are to acquire a sufficient and functional L2 lexicon (Zahar, Cobb & Spada, Reference Zahar, Cobb and Spada2001).
Thankfully, there are numerous wordlists which are easily accessible to L2 instructors and learners, who can utilize them for explicit learning practice. The General Service List (West, Reference West1953) is undoubtedly the most prominent and extensively utilized (Brezina & Gablasova, Reference Brezina and Gablasova2015). Another influential list, designed for university students, is the Academic Word List (Coxhead, Reference Coxhead2000). However, even while using such wordlists, it is inevitable that L2 learners will encounter unfamiliar words while reading. Knight (Reference Knight1994: 286) notes that, “in addition to using contextual cues to guess the meaning of an unfamiliar word, the L2 reader often chooses another strategy: looking the word up in a dictionary”. This latter strategy has been and remains a fundamental aspect of the traditional L2 classroom, and in international schools it is no exception.
In our modern age, in classrooms that are pretty high tech (and oddly enough, even in those that are not), practices such as learner created vocabulary books and word lists for rote memorization or translation into the L1, may seem antiquated and ready for the pedagogical dustbin. Yet, far from being vestigial, such practices can be revitalized with computer assisted language learning (CALL) in order to maximize their SLVA potential. In this particular situation for international school students, the CALL application is data-driven learning (DDL), which is something of a CALL enigma, due to its viability in high-tech classrooms, as well as in those devoid of Information and Communication Technology (ICT).
1.1 A brief history of DDL
CALL has been around since the early 1960s (Davies, Reference Davies2004). According to Johns (Reference Johns1991), it was only the advent of storage capacity and the affordability of personal computers, starting in the mid-1980s, which set the stage technologically for DDL. In 1990 Johns coined the term DDL, which can be described as the implementation of concordancing in the classroom (Johns, Reference Johns1991). More specifically, DDL means that learners are shown copious amounts of concordance output, which they will begin exploring and delving into, and through this process, they will acquire an enhanced ownership for their language learning (Boulton, Reference Boulton2009). Moreover:
DDL can be argued to be firmly grounded in cognitive linguistic theory as learners analyze masses of input in a quest to become more familiar with structural regularities via inductive means.
(Geluso & Yamaguchi, Reference Geluso and Yamaguchi2014: 227)
By not producing any form of answer to a learner’s question, concordancers create a situation where the learners must bring about for themselves the appropriate answer (Bloch, Reference Bloch2009; Gavioli, Reference Gavioli2001). Also, Yoon and Hirvela (Reference Yoon and Hirvela2004) note that learners can analyse concordance results to determine how a word functions with other words in context. Additionally, Kettemann (Reference Kettemann1995: 1) postulates that DDL is “motivating and highly experiential” for the language learner. Moreover, Johns (Reference Johns1991: 30) notes that DDL allows the L2 learner to essentially “cut out the middleman”.
Such a situation has led Johns (Reference Johns1997: 101) to describe students using concordancers as being akin to “language detectives”. Furthermore, Johns (Reference Johns1991: 30) notes that EFL learners who are using the DDL approach are in a unique situation where “the materials… reflect authentic usage and go a long way toward dispelling the myths and distortions that have arisen from reliance on intuition-based ‘armchair’ linguistics”.
Since its advent, there has been a steadily increasing number of studies being conducted in this field (Balunda, Reference Balunda2009; Braun, Reference Braun2007). Additionally, there is a diverse spectrum of linguistic phenomena being studied, such as vocabulary (e.g. Balunda, Reference Balunda2009; Cobb, Reference Cobb1997, Reference Cobb1999; Stevens, Reference Stevens1991) reading (e.g. Cobb Reference Cobb2007; Johns, Lee & Wang, Reference Johns, Lee and Wang2008), grammar (e.g. Biber, Reference Biber2001; Kettemann, Reference Kettemann1995; Sun, Reference Sun2003), writing (e.g. Gaskell & Cobb, Reference Gaskell and Cobb2004; Huang, Reference Huang2014; Yoon & Hirvela, Reference Yoon and Hirvela2004), and collocations (e.g. Shei & Pain, Reference Shei and Pain2000; Sun & Wang, Reference Sun and Wang2003).
1.2 Establishing empirically the beneficial effects of DDL
Since the early 1990s, there has been something of a DDL revolution in select specialized L2 fields of research. This has been in part catalyzed by assertions like Oghigian and Chujo’s (Reference Oghigian and Chujo2010), who note that there exists a general consensus that DDL in the second language classroom produces positive results. There is growing empirical weight behind such statements, as there have been significant findings in numerous publications that confirm DDL’s effectiveness (see e.g. Cobb, Reference Cobb1997, Reference Cobb1999; Johns et al., Reference Johns, Lee and Wang2008; Stevens, Reference Stevens1991).
Not only is DDL proving itself an effective language learning method, but it is also changing the very nature of the L2 classroom. Along these lines, Johns (Reference Johns1991) described how DDL had led to innovative language learning and teaching methods. Furthermore, the pedagogic outlook for DDL appears quite promising when:
Many researchers and teachers in the TaLC (Teaching and Language Corpora) tradition are convinced that DDL can empower learners to find out things for themselves, and that corpora have a great pedagogic potential.
(Römer, Reference Römer2009: 119–120)
Such sentiments coupled with empirical findings have paved the way for creating a pedagogic environment which is highly compatible with DDL.
1.3 DDL’s applicability across a range of educational levels
It is relatively easy to envision a rapid and widespread implementation of DDL rendering it utilizable by most L2 learners. Particularly in the 21st century, there is generally easy access in L2 teaching environments to ICT facilities. Yet, quite paradoxically, DDL has remained predominantly in the peripheral teaching realms, seemingly ever destined for specialized or university language learners as evidenced by Boulton (Reference Boulton2011b), who notes that most DDL studies focus on learners in higher education. Furthermore:
It is… assumed that DDL will not work with learners at lower levels – even to the extent that the objection seems to apply to level of sophistication and language proficiency simultaneously, as if the two were inextricably connected.
(Boulton, Reference Boulton2009: 39)
Hence it unfortunately appears that most secondary L2 students remain unaffected by DDL. Braun (Reference Braun2007: 308) notes poignantly that “the overall absence of corpus work in secondary school might seem overwhelming in view of the large amount of research into corpus-based language learning”. This lends further credence, as well as a fairly palpable sense of urgency, to Boulton’s (Reference Boulton2010b) notion that it is meaningful to pursue DDL research which focuses on L2 learners at the secondary school age levels. It is hoped that once substantial empirical DDL research in secondary education has been conducted, a threshold will be surpassed, propelling DDL into mainstream secondary L2 classrooms.
1.4 Commonly cited problems and solutions during DDL implementation
A host of apparent hurdles to the implementation of DDL have been outlined in the literature, including the learners, the classroom’s ICT facilities, and instructors. As for the latter, Talai and Fotovatnia (Reference Talai and Fotovatnia2012) noted that it is critical that instructors receive sufficient DDL training regarding concordancing usage in the classroom. They also note that instructors should be taught how to train language learners in both DDL and the basics of using concordance software or online concordancers.
As for classroom facilities, Oghigian and Chujo (Reference Oghigian and Chujo2010) note obstacles to DDL in the classroom, for instance not having a computer lab, or adequate ICT arrangements. They gave a variety of practical solutions. In the above situations, they suggest that DDL exercises could be projected to train learners how to use the corpus applications, and individualized concordancing could be done at home or in a school library. Although ICT access is vital for DDL, in some L2 classrooms, the teacher is the only one with access to a computer, and even then, output can be printed off or even put onto a blackboard for students to utilize.
Regarding DDL learners, there exists a spectrum of different learners and respective learning outcomes, ranging from the untrained novice to those having extensive knowledge and training in DDL. It is interesting to note that in both situations, the DDL user is still gaining significant results (for the former, see Boulton, Reference Boulton2010a; for the latter, see Horst & Cobb, Reference Horst and Cobb2001), but this may be relative to their degree of training and L2 proficiency. The latter factor was observed by Owen (Reference Owen1996), who noted that the learner may not have sufficient language proficiency or familiarity with the topic to make sense of the concordance output. Thus, if a learner is to move away from the novice end of this spectrum, then they may need more extensive training (Talai & Fotovatnia, Reference Talai and Fotovatnia2012). In the same vein, Boulton (Reference Boulton2011a) states that online concordancers require substantial preparation for learners in order to be used to their full extent, following a previous observation: “the more experience and training learners have, the more responsibility and autonomy they can assume, and the more sophisticated the use of corpora can be” (2009: 40).
1.5 DDL activities: The spectrum of learner autonomy
Concerning DDL exercises, Mukherjee (Reference Mukherjee2004: 12) noted that they “can be plotted on a cline of learner autonomy, ranging from teacher-led and relatively closed concordance-based exercises to entirely learner-centered corpus-browsing projects”. At one end we have Bernardini’s (Reference Bernardini2004: 22) idea of “serendipitous corpus browsing”, which allows completely unguided and autonomous concordancing. At the other lie completely closed activities which are instructor led, often utilizing paper-based printouts. Balunda (Reference Balunda2009) and Boulton (Reference Boulton2010a) have both advocated the usefulness of printouts.
In the initial stages, according to Johns (Reference Johns1997), DDL should be teacher directed. This approach was tested by Boulton (Reference Boulton2010a), who found that learners with lower language levels and no prior exposure to DDL gained significantly from paper-based DDL activities. As students progress along the DDL cline described above, they begin to exhibit the hallmarks of student-centered learning, that of self-reliance and self-empowerment toward their L2 learning.
1.6 DDL and its implications for SLVA
There has been a tremendous boom of activity regarding SLA vocabulary learning research over the past decade and a half (Nation, Reference Nation2011). In light of this, it should come as no surprise that, in a review of CALL research by Stockwell (Reference Stockwell2007), there is a diverse spectrum of CALL studies regarding vocabulary learning. Moreover, DDL vocabulary research within CALL has seen a flurry of interest over the last decade, with a wide range of implications from dictionary usage (Loucky, Reference Loucky2005) and design (Granger, Reference Granger2002), to L2 course design (Horst & Cobb, Reference Horst and Cobb2001; McCarten, Reference McCarten2007). Thus, the effects of corpus research have the potential to have a large influence upon traditional SLVA methods. The effectiveness of this method since its advent has been corroborated by numerous researchers (see e.g. Cobb, Reference Cobb1997, Reference Cobb1999, Reference Cobb2007; Horst, Cobb & Nicolae, Reference Horst, Cobb and Nicolae2005; Stevens, Reference Stevens1991; Thurstun & Candlin, Reference Thurstun and Candlin1998). Furthermore, according to Shaw, (Reference Shaw2011:12):
Using DDL in vocabulary teaching fits logically. Their close relationship provides great opportunities for learning more about vocabulary and how to apply DDL in the language classroom.
It is evident that vocabulary exercises must provide the L2 learner with more to learn than just lexical items and their definitions (Horst et al., Reference Horst, Cobb and Nicolae2005). For many SLVA exercises, once the dictionary shuts, there is a clear void, and it is at this critical juncture that the implementation of DDL is most pertinent, to link metalinguistic awareness with a lexical item’s definitional meaning. This juncture is firmly located in the CALL field, where finding the right balance between explicit and implicit SLVA methods is crucial for the learner.
There are numerous studies regarding DDL and various aspects of vocabulary acquisition; many of these provide qualitative results such as L2 student or language instructor perceptions of DDL, or do not involve empirical quantitative research. Cobb and associates are a most notable exception to the above. In one study (Cobb, Reference Cobb1999), two methods of vocabulary acquisition were compared: one had learners viewing the word in numerous lines of concordance output, the other provided the word used in a sentence along with its definition. It was found that learners utilizing the concordancing method, in the short and long term, were significantly better at applying their new linguistic understanding of the word to novel contexts as well as producing a good understanding of word meanings. Furthermore, it was found that ‘traditional’ dictionary learning methods (simply going to the dictionary and looking up the word, and copying down the definition) led to learning, but this was not retained for the long term, nor was this group able to apply these newly acquired words to novel situations. In another study (Horst & Cobb, Reference Horst and Cobb2001), university ESL students enrolled in a course designed for academic SLVA. These L2 learners were offered a variety of CALL methods, including online dictionaries and concordancing, along with traditional monolingual dictionaries. All of these were utilized in the hopes of increasing their academic lexicon growth. One of the more intriguing findings of this study was that students who utilized concordancing were likely to experience vocabulary improvements.
1.7 The impetus for implementing DDL in international schools
International schools can be classified into a variety of differing types, and according to Matthews (Reference Matthews1989) such schools can be classified along a cline, between “market” and “ideologically” driven schools. For the sake of simplicity, in this study, an international school will be defined as a school that utilizes a curriculum, often taught in the medium of English, which is not offered in a country’s national schools and caters to internationally mobile children as well as to affluent host nationals.
It is also pertinent to note that international schools are booming worldwide. According to ISC Research (2014), there are over 7,000 schools open contemporarily, with 3.5 million students attending these schools. Furthermore, Asia contains the majority of these schools, and Vietnam was distinguished as a country with a growing market for such schools (ISC Research, 2014).
Demographics in international schools (particularly those located in Southeast Asia) have been steadily changing. One driving factor behind this is the ever-increasing level of enrollment of host national students from high socio-economic status families, as they often equate enrollment with the provision of a subsequent pathway to a foreign university (Hayden & Thompson, Reference Hayden and Thompson2008). The majority of these students will eventually enroll in universities abroad, which has wide implications for these future L2 university students and their L2 proficiency.
While DDL has found some uptake among L2 learners in higher education, those in secondary education in general, and in this case those attending an international school, have yet to feel the direct effects of concordance work. The exceptions to this are those students who, often by happenchance, utilize corpus-driven grammar books, websites, or dictionaries.
Therefore, this study may be one of the first to directly implement DDL in international education, concerning international school students, and one of a handful implementing DDL for secondary students at lower levels of English language proficiency. The impetus for this is that with the growing trend of international schools catering to L2 learners, DDL can be one of many techniques (for more on teaching L2 students in international schools, see Carder, Reference Carder2007) that can be employed to facilitate these students’ L2 learning in order to eventually prepare them for the L2 requirements of gaining admission to universities abroad. Being able to successfully implement DDL in the curriculum and validating the linguistic benefits at the secondary level will contribute to cementing DDL’s place in the secondary student’s L2 classroom.
2 Methodology
2.1 Population
This population of students are predominantly (96%) EFL speakers attending an international school in Ho Chi Minh City, Vietnam. They are between eleven and fourteen years of age with a mean age of thirteen. There are 51 females and 49 males. Some 74% of these students are host national educational CCKs (Vietnamese). The remaining 26% are TCKs from Korea (11%), Singapore (4%), India (2%), Malaysia (2%), Taiwan (2%), Australia (2%), New Zealand (1%), China (1%), and Sri Lanka (1%).
Students who attend international schools can be classified into several categories, and the students involved in this study are no exception; they have been classified either as ‘third culture kids’ (TCKs) or ‘educational cross-cultural kids’ (CCKs). A TCK was originally described by Pollock in Reference Pollock1988 (Pollock & Van Reken, Reference Pollock and Van Reken2009: 13) as being:
An individual, who, having spent a significant part of the developmental years in a culture other than the parents’ passport culture, develops a sense of relationship to all cultures while not having full ownership in any. Elements from each culture are incorporated into the life experience, but the sense of belonging is in relationship to others of similar experience.
The term CCK was developed by Van Reken and is defined as:
A person who is living or has lived in – or meaningfully interacted with – two or more cultural environments for a significant period of time during childhood (up to age 18).
(Pollock & Van Reken, Reference Pollock and Van Reken2009: 31)
An “educational” CCK is a student attending a school, in their own country, which has a school culture and student make-up different than those traditionally found in their country’s national schools (Pollock & Van Reken, Reference Pollock and Van Reken2009).
2.2 Methods
There were three EFL instructors involved, one being the researcher of this study. The other two instructors were trained in DDL before the implementation of this study and they were provided with support throughout. DDL training was conducted through tutorials, one-on-one training sessions, as well as professional development meetings held in the ICT lab accompanied with a suggested DDL reading list.
All classes, ranging from grade 6 to grade 8 (aged eleven to fourteen), were separated into two groups, a control and an experimental group. The average number of students in the control group was fourteen, and 20 in the experimental group per grade level. Each grade level’s population by treatment group is reported in Table 1. This was done by using nationality to determine grouping. Classes comprised solely of CCKs formed the control group, whereas the classes with a mix of TCKs and CCKs acted as the experimental group. In total, there were 33 CCKs and 26 TCKs in the experimental group, and 41 CCKs in the control group.
Distribution of students in the control and experimental groups by grade level
The intermediate-level EFL curriculum for these grades was expected to produce a minimum exit result of IELTS 3.5 to 4.5 (elementary to lower-intermediate) according to grade level. Both groups, for grades 6, 7, and 8, were given an average of eighteen new vocabulary items weekly, presented in a vocabulary list. These words were selected from their EFL course book at the beginning of each week by utilizing the Compleat Lexical Tutor’s (http://www.lextutor.ca) keywords function. This function compares a text (reading comprehension texts from students’ EFL textbooks) to two general English corpora, the British National Corpus (BNC) and Corpus of Contemporary American English (COCA) (http://corpus.byu.edu/coca) and generates a ‘keyword list’. This list contained every word from the L2 text extract that had a frequency which was at least 25 times greater than in the reference corpus. The list is indicative of the key role that these high-frequency words play in the original text. Thus, for each grade level, the experimental and control groups for that grade level received eight consecutive weekly keyword lists, which were identical for both groups for each grade level. The average student was exposed to 144 new words; however, the number of words in the weekly word lists varied, primarily due to how many high-frequency words were generated by the keyword function.
Each word covered in this study was analyzed using the word frequency function found in COCA. This corpus is the largest free corpus of American English and currently comprises over 450 million words (Davies, Reference Davies2008). The word frequency function allows the user to search for a word from a list of the 100,000 most common words found within COCA, ordered by frequency. The overall average word frequency for all three grades was 5635. Weekly word frequency averages varied slightly between the three grades (Table 2).
Average word frequency for each grade level
Both groups recorded each week’s new words into their e-vocabulary book (Figure 1). For this, learners used online-dictionary learning methods. Ellis (Reference Ellis1995: 12) discusses several of the advantages of such methods, the first being that “on-line dictionaries allow ready access to the lemma”. Another is being able to use a split screen, with content on one side, and a definition on the other, allowing the student to switch from one to the other with ease. For this study, in addition to the above method (students were shown several methods to split the screen), online dictionaries (Cambridge and Cobuild were recommended) were not just utilized to search for word definitions, but to also acquire information about type of speech, synonyms, and L1 translations. Participants were instructed to submit such information, along with a sentence of their own construction, into their weekly e-vocabulary book.
Sample page from a student e-vocabulary book from the control group.
Prior to the experiment, all students in the experimental group received training in the ICT lab (7.5 hours minimum) for the Compleat Lexical Tutor’s English language concordancer interface. This training was conducted in a group, as well as in one-on-one sessions, over a period of two weeks. Students were taught a host of different concordancing skills: corpus identification and selecting the most suitable corpus for their L2 purpose. For example, students were shown how to select corpora of graded readers with a vocabulary of just one or two thousand words specifically designed for use by beginner and intermediate L2 learners. They were also taught how to skim key word in context (KWIC; cf. Figure 3) data for grammar patterns, identifying words to the left and right of the search item, how to view the original text, how to view collocations by frequency, and how to use a search application on the KWIC output data. The training followed the advice set out by Johns (Reference Johns1991, Reference Johns1997), to initially have the instructor lead DDL exercises while modeling the approach, and to follow this up later with semi-autonomous pre-designed DDL activities.
The experimental group was asked to concordance each vocabulary item, and using the concordance output, fill out a DDL vocabulary handout each week (see Figure 2). The control group did not receive DDL training or use a concordancer for their weekly vocabulary learning.
Sample page from an e-vocabulary book from the experimental group.
Sample example question from a weekly vocabulary test.
Weekly vocabulary tests were conducted to record the results of the two methods of vocabulary learning over the course of the study. These tests were generated using output from the Lextutor Multi-Concordance function. Ten words were selected randomly from the previous week’s vocabulary list, and eight KWIC examples were given for each word (see Figure 3), with the keywords omitted. These series of sentences provided learners with ample contextual clues to enable them to discern the meaning of the gapped item. Additionally, students were given a word bank containing the keywords which they were able to select answers from.
This was conducted in a controlled setting and took no more than fifteen minutes. For the experimental group, a blank version of the quiz was projected on an overhead device afterwards and answers were elicited from the class, along with relevant grammar structures, collocations, or particular phrases which aided them in ascertaining the correct answers which were highlighted for the class.
2.3 Missing data
As there were instances of missing data, the most cutting edge approach was used, which is multiple imputation. This is a Monte Carlo technique where missing values are replaced by simulated ones. After data cleaning, all missing data accounted for >5% of the total responses (1.4% total). Missing data was attributed to student absences from school on that particular week’s testing day.
3 Analysis
The software package used to conduct all statistical analyses was SPSS 20. A mixed between/within subjects analysis of variance was initially conducted. This test is utilized when there is one independent between-subjects variable and another within-subjects variable, and is employed to find differences between more than one group while utilizing repeated measures. It was thus a suitable test to evaluate the effect of the two different vocabulary acquisition methods on the L2 students’ weekly vocabulary test scores over time.
No significant interaction was discovered between vocabulary learning method and time: Wilks Lambda=.94, F (7, 88)=.76, p=.63, partial eta squared=.06. Furthermore, no significant within-subject effects were found for the covariates regarding the subjects’ age, gender, grade level, or nationality. These results were respectively Wilks Lambda=.89, F (7, 88)=1.52, p=.17, partial eta squared=.11; Wilks Lambda=.96, F (7, 88)=.5, p=.84, partial eta squared=.04; Wilks Lambda=.9, F (7, 88)=1.34, p=.24, partial eta squared=.1; Wilks Lambda=.89, F (7, 88)=1.6, p=.16, partial eta squared=.11. The result for nationality is of interest as it confirms that the treatment group’s selection process did not impact the main effects.
A significant main effect was found for time: Wilks Lambda=.8, F (7, 88)=3.06, p=.006, partial eta squared=.2. Both groups showed an average increase on test scores across the eight time periods (Table 3). The experimental group scored 10% higher on average than the control group, with the former scoring on average 8.26 correct answers out of 10 and the latter 7.28 correct answers out of 10.
Mean distributions across eight weeks for the control and experimental groups
A linear mixed effects analysis measures fixed and random effects and can be used for repeated measures, and provides a more interpretable output than classical repeated measures. For fixed effects, grade level was entered; for random effects, word frequency and number of words were used. This analysis was utilized to better determine these covariates’ relationship on vocabulary acquisition scores over time. Main effects were utilized with type III fixed effects tests, which included an intercept. The covariates of word frequency and number of words given were not significant, with F (1, 537)=.3, p=.3; and F (1, 343)=.56, p=.46 respectively.
From the mixed between/within results, regarding between-subject effects, two factors were found to be significant. The main effect comparing the two types of learning methods was significant: F (1, 94)=4.8, p=.03, partial eta squared=.049 (Figure 4).
Longitudinal distribution of means for the control and experimental group.
An additional main effect comparing the subjects’ grade level was significant: F (1, 94)=1.7, p=.001, partial eta squared=.02. The remaining covariates of gender, nationality, and age were found to have no main between-subjects effect with the following respective results: F (1, 94)=1.45, p=.23, partial eta squared=.02; F (1, 94)=2.1, p=.15, partial eta squared=.02; and F (1, 94)=3.3, p=.07, partial eta squared=.03.
To begin teasing apart the differences between the experimental and control groups, a univariate repeated measure analysis of variance was conducted for both groups to calculate further the effect of time on the two different methods of vocabulary acquisition. For the experimental group, Wilks Lambda=.70, F (7, 52)=3.25, p<.006, partial eta squared=.30; and for the control group, Wilks Lambda=.63, F (7, 34)=2.91, p<.017, partial eta squared=.38.
Additionally, univariate post hoc tests utilizing the Bonferroni correction found no significant results for the control group. However, for the experimental group, the second week of DDL results, with a mean result of 7.78/10 correct answers, was significantly different (see Table 4) from weeks 5 (p=.012), 6 (p=.049) and 8 (p=.040), which had mean vocabulary scores of 8.71/10, 8.58/10 and 8.73/10, respectively.
Post hoc Bonferroni for the experimental group
Based on estimated marginal means
*The mean difference is significant at the .05 level.
a Adjustment for multiple comparisons: Bonferroni.
Thus it can be concluded that significant gains were found for time, in conjunction with significant results after the fifth week of DDL and generally beyond (with the exception of week 7). This indicates that DDL coupled with online dictionary learning methods produces significantly better acquisition results initially, as well as significant long-term results.
Regarding the between-subject differences between the experimental and control groups on specific grade levels, the largest difference in overall acquisition was found for grade 6(Figure 5). As mentioned above, the impact of the students’ grade level on vocabulary acquisition scores over time was analysed using the linear mixed model, and a significant effect was found using type III fixed effects, with F (2, 651)=33.1 p=.003. A further Bonferroni comparison using the mixed model found that grade 6 was significantly different from grades 7 and 8 (p=.0001 and p=.009), while grades 7 and 8 were not significantly different (p=1.0 for both).
Longitudinal distribution of test means across grades 6, 7, and 8.
Thus a repeated measure analysis of variance was conducted on the grade six data. For this grade, there was only one significant interaction found (Figure 6), which was between vocabulary learning method and time: Wilks Lambda=.36, F (7, 14)=3.54, p=.02, partial eta squared=.84.
To further understand the difference for grade 6, a univariate repeated measures test was conducted upon the control and experimental group. A significant result was found for both groups: Wilks Lambda=.20, F (7, 5)=34.18, p<.001, partial eta squared=.98 for the control group, with sphericity assumed for the within subject test; F (1, 12)=3.03, p<.007, partial eta squared=.20 for the experimental group. For grade 6, univariate post hoc tests utilizing the Bonferroni comparison found significant results for both groups. For the control, it was discovered that weeks 1 and 2 were significantly different from week 4 (p=.005 and p=.018 respectively), and week 4 was significantly different from week 5 (p=.0001). This indicates that the control group had a major drop in their vocabulary scores from the initial weeks, to a low in week four (see Figure 6). A significant gain was made from weeks 4 to 5; however; overall longitudinal gains were not made. For the experimental group, a significant rise in results was found, with week 1 significantly different from week 3 (p=.002) and week 2 significantly different from week 3 (p=.019). Furthermore there was an overall improvement of vocabulary scores longitudinally (see Figure 6).
Distribution of weekly means for experimental and control groups in grade 6.
4 Discussion
The finding that both groups are significantly affected by time, and not the alternatives of just one, or neither, is quite important. If these latter alternatives had occurred, then either the usage of dictionary methods or DDL, depending upon which group was not significant, would have shown that this method was not making longitudinal vocabulary gains. This was certainly not the case. As anticipated, both methods produced significant results.
The overall longitudinal success of the experimental group could be attributed to their having been exposed to the KWIC format and trained in this, along with the possibility that they could have viewed the KWIC examples prior to testing. However, the latter situation, albeit plausible, would depend upon the subjects wholly disregarding their DDL training, and reading each KWIC line verbatim. In order to complete their weekly DDL tasks, possibly hundreds of lines of KWIC data were skimmed through for each lemma, with attention paid mainly to words to the left and right of the target item. It is therefore unlikely that students in the experimental group would remember whole KWIC lines, when they were exposed to so many lexical items, along with the very mode of viewing discouraging reading the entire KWIC sentence verbatim (as most sentences will only be visible in an incomplete form). Furthermore, the KWIC examples in the weekly tests were chosen at random from the data output, and were not exclusively taken from the corpora that the students were utilizing (K1 and K2), but from additional corpora such as the Brown, BNC, and COCA. This limited the chance exposure of subjects to KWIC data found in the weekly tests.
The gains were longitudinal, and occurring significantly after several weeks. Thus, if familiarization with the KWIC format was the sole explanation for better test scores, then it would be reasonable to expect dramatically different results between the control and experimental groups (for example, seeing experimental results from week 5 occurring initially in week 1, and sustained throughout the duration of the experiment). This was not the case, indicating that, indeed, it is quite unlikely that the significant gains found in the experimental group were the result of prior exposure to testing items and familiarity with the KWIC format, and that these gains are certainly due to the process of DDL in conjunction with online dictionary work.
Another interesting finding from this study is that one would expect the control to be disadvantaged on the KWIC-based weekly vocabulary tests when compared to the experimental group due to their lack of exposure to the KWIC format. However, there is empirical evidence which reinforces the fact that students do make improvements when using the KWIC format, without having had any formal training (Boulton, Reference Boulton2008, Reference Boulton2009). This helps dispel the notion that students’ lack of familiarity with the KWIC format would unfairly advantage the experimental group in this study. Moreover, as both grades 7 and 8 in the control group made significant longitudinal gains, further reinforcing Boulton’s above findings that students can accurately utilize the KWIC format, and make successful longitudinal gains. However, this was not the case for the lowest grade level in the control group.
The contrasting results for grade 6 indicated an intriguing situation. The experimental group who were using DDL showed significant gains just as did all other experimental groups. Yet the control showed a significant decrease, and then a slight increase, in vocabulary acquisition. Moreover, the control group’s results for grade 6, on average, were roughly 30% below the average results for the control scores for grades 7 and 8. What caused this grade level to perform significantly lower than the other grade levels? It is possible that they were simply experiencing some of the more commonly cited difficulties associated with viewing KWIC data; however, this seems unlikely, with their peers in control groups in grades 7 and 8 making significant gains on tests utilizing the same KWIC format. The most dramatic alternative explanation would be that students in the lower-secondary age levels (eleven years on average) are simply incapable of successfully utilizing KWIC data effectively without prior instruction, and would need to be followed up empirically. Other explanations are much more plausible and mundane. For example, the control group could have received poor explanation of the vocabulary exercises, might have used their e-vocabulary books irregularly, or simply not have been prepared for the vocabulary tests. The latter scenario is the most plausible, as this group did in fact experience a significant turnaround in their results in week 5. This gain could be indicative of a reversal of the class’s motivation to learn their new vocabulary, or a willingness to attempt using the new format for vocabulary testing, as this testing had replaced prior methods of weekly vocabulary testing.
Overall, the control group’s results (excluding the control group in grade 6) reinforce Ellis’s (Reference Ellis1995) position on the value of online-dictionary learning. These results also clearly show that online dictionary learning methods can be associated with other techniques, like DDL, and be just as successful over time – or, as in this case, much more successful. This appears to empirically corroborate Kettemann’s (Reference Kettemann1995) notion that DDL is suitable for teaching the meanings of unfamiliar vocabulary through sentence contexts. As this study has shown, and as noted by Cobb (Reference Cobb2003), a coupling between DDL and online dictionary methods is seen as an idyllic scenario. This union between methods clearly adds credence to Balunda’s (Reference Balunda2009: 15) claim that:
Instead of promoting DDL over the use of more traditional vocabulary teaching methods and activities, DDL should be seen as but one part of a holistic plan to teach vocabulary and used in conjunction with traditional methods and activities.
Furthermore, the significant improvements by the experimental group found in the later weeks of the experiment show that the real dividends from DDL vocabulary exercises in conjunction with online-dictionary learning methods not only appear in the short term, but really begin to show strong results longitudinally after the second fortnight, with an estimated twelve and a half hours of DDL exposure time accrued (including time spent during training sessions). Significantly better results following continuous exposure to DDL gives one much food for thought. How many other DDL experiments would have shown significant results, or even greater results, had they been conducted longitudinally?
Overall, the results of this study confirm Cobb’s (Reference Cobb1999) findings that DDL facilitates vocabulary acquisition from contextual clues over the long term as well as Horst and Cobb’s (Reference Horst and Cobb2001) findings that DDL use promotes vocabulary improvements. Also, these results are a clear answer to Frankenberg-Garcia’s (Reference Frankenberg-Garcia2014: 142) query regarding “the long-term effects of presenting learners with multiple corpus examples”. Furthermore, this study also indicates a direct confirmation of Cobb’s (Reference Cobb2003: 190) prediction that “online, corpus-based, capacity unlimited dictionaries do not presently replace the value of concordances for language learners, and they are unlikely to do so in the future”.
Another facet of this study’s findings is the successful results of the intensive DDL group training and individualized feedback that students received. The rationale behind this observation is that over the course of the study, the mean time per week utilizing DDL was an hour. Thus the amount of training combined with the mean exposure time equals fifteen and a half hours, over a ten-week time frame. This is interesting when compared to the findings by Boulton (Reference Boulton2010b), who in a review of 27 DDL studies, found that the average exposure time for the L2 learner utilizing DDL itself barely exceeded ten hours. Thus in the present study, the amount of time was 50% greater than the average. It therefore seems that the ample training time allowed learners to get through the initial “teething” stages of DDL and well on their way to becoming DDL proficient, as evidenced by the experimental group’s results in the first week in comparison to the control. At the secondary school level and with their limited L2 proficiency, autonomous DDL browsing may not be feasible, nor is it appropriate to expect the serendipitous browsing described by Bernardini (Reference Bernardini2004). However, it appears that the approach that Johns (Reference Johns1991, Reference Johns1997) advocated has been quite successful in this study. Furthermore, these results appear to confirm Geluso and Yamaguchi’s (Reference Geluso and Yamaguchi2014: 10) results, whereby once the DDL learners were “past the initial learning curve, less than half of the students felt that the actual concordancing was ‘difficult’”.
The experimental method’s results for this study clearly dispel the widely touted CALL myth that DDL is unsuitable for lower-level L2 secondary-school students. Inadvertently supporting this myth has been the fact that DDL has been researched almost exclusively among undergraduate and graduate level EAP and English for special purposes (ESP) students. This study shows the very real viability of this method for lower L2 secondary students, when taking into careful consideration the time constraints and initial ‘teething’ problems this method involves for students to become confident and proficient in DDL. The weeks prior to the study were surely invaluable, as on every day of the training, the EFL office was inundated by students vying to have their individual questions regarding DDL methods answered or receive feedback on DDL handouts.
This experimental method also shows that, in the field of international education, particularly in Asia where a predominant number of students are EFL speakers, DDL can, hypothetically at least, be implemented in most L2 curriculums relatively painlessly by a fairly small number of dedicated L2 educators who have received relevant DDL vocabulary training and instruction. An additional advantage is that once DDL has been incorporated into a secondary school’s L2 curriculum, then it is readily available for the language teacher to supplement other linguistic domains, such as in students’ regular grammar, writing, and reading lessons.
There is one aspect of this study that requires mulling over, which is that the EFL population is fairly atypical in a myriad of ways from other L2 learners found in other non-international secondary-school learning contexts. These privileged CCKs and TCKs are often raised in ‘cosmopolitan’ and affluent environments, attend international schools, and expect to eventually go abroad to attend university. In a study by Cottrell and Useem (Reference Cottrell and Useem1993a), the longitudinal educational and L2 outcomes for ‘adult’ TCKs after graduating, is highly promising, with 81% achieving an undergraduate degree and 80% of them regularly using their L2 (Cottrell & Useem, Reference Cottrell and Useem1993b). Therefore, for TCKs (and quite possibly CCKs as well), there certainly appears to be a tripartite relationship between the TCK experience, attending an international school, and their linguistic and academic prowess.
According to Tokuhama-Espinosa (Reference Tokuhama-Espinosa2003), one of the most beneficial aspects of being a TCK is being able to acquire a second language. Furthermore, more often than not, they develop into what is known as “cultural chameleons”, which facilitates their quick acquisition of cultures, mannerisms, and languages (Pollock & Van Reken, Reference Pollock and Van Reken2009). This is further espoused by McCaig (Reference McCaig1996: 100), who claims that:
Intercultural and linguistic skills are the markings of the cultural chameleon – the young participant observer who takes note of verbal and non-verbal cues and readjusts accordingly, taking on enough of the coloration of the social surroundings to gain acceptance while maintaining some vestige of identity.
The cultural chameleons found in this study are predominantly Asian TCKs and Vietnamese educational CCKs, who are learning in their L2. This brings up the intriguing question of how this determines their response to DDL vocabulary exercises – and, for that matter, to foreign language acquisition as a whole. These are important considerations when thinking about how the results of this study tie into the broader CALL field and SLA in general. This specific L2 population has been well documented in the sociological and educational fields for traditional (Western) TCKs (see e.g. Pollock & Van Reken, Reference Pollock and Van Reken2009). Yet it has been shown that non-Western TCKs, particularly those hailing from Asia, need to be more widely studied (Kwon, Reference Kwon2006; Tanu, Reference Tanu2008). Karras (Reference Karras2011) coined the term ANNES-TCKs to describe Asian non-native English speaking TCKs to help describe and delineate them from the traditional ‘Western’ TCK category, while advocating more SLA research to better understand this steadily growing group of L2 speakers.
5 Conclusion
This experiment provides empirical evidence that DDL, utilized in conjunction with online-dictionary learning methods, can produce longitudinally better results for vocabulary acquisition when compared to the online-dictionary learning method alone. What is more, it has shown that DDL is a viable CALL method for lower L2 level secondary students, and that the common complaints that students have while learning DDL and which can stymie results, can be assuaged with sufficient student training time. Furthermore, it appears that ample longitudinal DDL exposure results in improved vocabulary acquisition rates, which shows that the results of DDL are not limited to the short term, but become accentuated over the long term.
However, with all of this in mind, there is still plenty of further research needed. At this point, extrapolating the findings from this study should be done with great care, until further experimentation and replication can empirically confirm that these results are consistent when conducted in other learning environments besides international schools and with learners other than the secondary ANNES-TCKs in this study. Other questions still linger and a much longer longitudinal study would complement these findings to determine several unknowns. For example, at what point-in-time do the two treatment groups peak in vocabulary proficiency, and are these peaks sustainable? Such questions indicate that although DDL research has been around for over 20 years now, it appears that in the field of CALL, we are only just beginning to take the lid off of what we understand about the measurable effects of DDL.
