The overall aim of this study is to report on the design of, and to further develop, a profiler for lexical sophistication (i.e., use of advanced vocabulary) specifically created for Swedish second language (L2) learners’ French and Italian. The study is part of a more comprehensive project concerned with the possibility of L2 speakers attaining nativelike proficiency in the domain of vocabulary. A lexical profiler is software that automatically classifies the words of a text according to preset criteria—most commonly the frequency by which words occur in the target language (TL). For the present study, advanced vocabulary refers to low-frequency vocabulary, with the exception of such words that, although they are not frequent, can be assumed to be easy for L2 learners either due to cognateness or because they belong to so-called thematic vocabulary. Cognates can be assumed to be relatively easy, or basic, because they resemble words already known to the learner, as are thematic words because they are normally introduced early in a course or a textbook. A secondary aim of this study, consequently, is to discuss the role of cognates and thematic vocabulary in relation to frequency. To determine whether certain cognates and thematic vocabulary might be assumed to be easy or belong to a basic vocabulary, the judgments of experienced teachers were used (see Tidball & Treffers-Daller, Reference Tidball and Treffers-Daller2008).Footnote 1
The profiler that is the focus of this investigation was inspired by the lexical frequency profile (LFP; Laufer & Nation, Reference Laufer and Nation1995) and has been elaborated on in two earlier studies. In a study by Lindqvist, Bardel, and Gudmundson (Reference Lindqvist, Bardel and Gudmundson2011), frequency bands on the basis of existing corpora of oral production data from native speakers (NSs) were created and used as a benchmark when measuring learners’ lexical sophistication in French and Italian L2.Footnote 2 The learners were classified in different groups at different proficiency levels on the basis of morphosyntactic traits of interlanguage development, among others. Morphosyntactic stages were considered to serve as a potential point of departure in a given phase of elaboration and testing of a lexical measure. Furthermore, they made it possible to compare groups of learners and NSs.
The purely frequency-based profiling method was then tested, which allowed for the pinpointing of statistically significant differences between these groups of learners in reference to the proportion of low-frequency words in their oral production. Some of the learners displayed nativelike lexical profiles—when compared to native controls—in the sense that they produced similar proportions of low-frequency words, even though they did not reach a nativelike level according to the morphosyntactic criteria suggested by Bartning and Schlyter (Reference Bartning and Schlyter2004). This indicated that L2 learners can reach a nativelike level of lexical sophistication—as defined by the use of low-frequency vocabulary—and thereby that vocabulary knowledge may be a domain of L2 knowledge that is less limited than others, such as grammar and phonology, which is an interesting result indeed. Moreover, some individuals within the groups displayed unexpectedly advanced profiles such that they had larger proportions of low-frequency words. These individuals not only were advanced when compared with other learners at the same proficiency level (as defined by morphosyntactic development) but also had even larger proportions than any of the NSs.
In research by Bardel and Lindqvist (Reference Bardel, Lindqvist, Roberts, Pallotti and Bettoni2011), a qualitative analysis of low-frequency words was conducted that led to the conclusion that further developments of the method were needed to obtain profiles that better reflected the individual’s lexical repertoire. Bardel and Lindqvist suggested that although low-frequency words, in general, are good indicators of lexical richness, advanced vocabulary is not necessarily indicated by low-frequency cognates and words belonging to thematic vocabulary related to teaching materials (e.g., textbooks). In the present study, certain low-frequency words (cognates, on the one hand, and thematic words, on the other) were recategorized, and it was found that these words should not be considered advanced. To make this kind of modification of the method, teachers’ judgments (TJs) were used as a tool (see Daller, Van Hout, & Treffers-Daller, Reference Daller, Van Hout and Treffers-Daller2003; Tidball & Treffers-Daller, Reference Tidball and Treffers-Daller2008). Because the modifications imply that aspects other than frequency are taken into account, the new version of the method will simply be referred to as a lexical profiler.Footnote 3
A majority of key studies carried out on vocabulary acquisition have analyzed cross-sectional data, comparing groups of learners at different levels of proficiency. Much less attention has been paid to individual learners’ lexical proficiency. A reliable tool needs to function not only at the group level but also at the individual level. The new lexical profiler was therefore tested on data from 54 individuals (14 learners of French and 10 NSs of French; 20 learners of Italian and 10 NSs of Italian), used in Lindqvist et al. (Reference Lindqvist, Bardel and Gudmundson2011).
FREQUENCY AND OTHER FACTORS THAT AFFECT VOCABULARY ACQUISITION
As a natural consequence of the inherent complexity of a word, knowledge of words is multifaceted such that the degree of difficulty of a word can be discussed in terms of word-internal or word-external factors. According to Laufer (Reference Laufer, McCarthy and Schmitt1997), there are certain intralexical traits that are fundamental for complete word knowledge: pronunciation, spelling, morphology, syntactic pattern, meaning, lexical relations (e.g., synonymy, antonymy, hyponymy), and collocations. As Laufer points out, all these aspects can be more or less well known. The more advanced a learner, the more aspects of a word are likely to be known, and the more developed the different aspects will be (e.g., the more the meanings of a polysemic word are known, the more synonyms, collocations, and idiomatic expressions are mastered). Given this line of reasoning, it can be concluded that all these aspects contribute to the difficulty of a word to different degrees.
Milton (Reference Milton2009) also discussed word-internal factors that may have an impact on the learning burden—namely, the form of the word, cognateness, abstractness, word length, and part of speech. The role of cognateness as a facilitating factor has also been emphasized by Horst and Collins (Reference Horst and Collins2006), Rule, Myles, Mitchell, David, and dos Santos (Reference Rule, Myles, Mitchell, David and dos Santos2010), and Willis and Ohashi (Reference Willis and Ohashi2010). Additionally, Milton also mentions the role of thematic vocabulary in teaching materials as a factor that is related to the input that learners receive: “Materials for learning language are usually thematically arranged and words for teaching cannot be selected purely by frequency. Inevitably, learners will learn the thematically important words in addition to the highly frequent function words” (2009, p. 29).
Many researchers take an approach to vocabulary learning that focuses mainly on the role of frequency in the input, and they thereby give less weight to other factors and claim that frequency is a particularly powerful factor in vocabulary learning (see Cobb & Horst, Reference Cobb, Horst, Bogaards and Laufer2004; Laufer & Nation, Reference Laufer and Nation1995; Milton, Reference Milton2009, for adult L2 learning; Vermeer, Reference Vermeer2001, Reference Vermeer, Bogaards and Laufer2004, for child first language [L1] and L2 learning). According to this perspective, learners basically learn words in an order that reflects the frequency in the TL input: “Whether in a classroom or a naturalistic setting, learners tend to acquire L2 vocabulary in rough order of frequency” (Cobb & Horst, Reference Cobb, Horst, Bogaards and Laufer2004, p. 17). This might appear as a somewhat extreme assumption, and other factors have indeed been acknowledged (see discussion in Vermeer, Reference Vermeer, Bogaards and Laufer2004). However, it is often assumed that high-frequency words will be learned early, and that low-frequency words will be learned later during the acquisition process and can be considered as advanced: “The less proficient learner is less likely to use rare vocabulary than the advanced one” (Laufer & Nation, Reference Laufer and Nation1995, p. 311). Milton states that other factors “do not seem to reverse this trend” (2007, p. 51)—that is, the trend that frequency is the most powerful factor for vocabulary learning, at least when considering large groups of learners. He also suggests that it is likely that factors other than frequency are decisive when looking more closely at the vocabulary of individuals.
In this study, it is argued that other factors—namely, cognateness and thematic words—can have an impact on the frequency model. This suggestion implies that this model would benefit from being modified to account for aspects other than frequency in L2 vocabulary acquisition.
MEASURING LEXICAL SOPHISTICATION
Read (Reference Read2000) distinguishes four dimensions of lexical richness in L2 acquisition: lexical density, lexical diversity, lexical sophistication, and proportion of errors. In this study, lexical sophistication will be investigated. To measure lexical sophistication, researchers have considered the percentage of sophisticated or advanced words in a text (Laufer & Nation, Reference Laufer and Nation1995). As pointed out by Laufer and Nation (Reference Laufer and Nation1995), the words labeled as advanced will depend on each researcher’s definition. In the majority of previous studies, word frequency has been used to determine what advanced words are in the sense that low-frequency words represent advanced or difficult words (Hyltenstam, Reference Hyltenstam1988; Laufer & Nation, Reference Laufer and Nation1995; Read, Reference Read2000; Vermeer, Reference Vermeer, Bogaards and Laufer2004).
To measure lexical sophistication, Laufer and Nation (Reference Laufer and Nation1995) have developed the LFP, which measures the proportion of low-frequency versus high-frequency words in a learner’s production by means of Vocabprofile, a computer program developed by Tom Cobb (www.lextutor.ca/vp). The original version of the program divided the words of a text according to frequency bands: the 1,000 most frequent word families, the second 1,000 most frequent word families, and the Academic Wordlist, supplying an output that illustrates the lexical profile of a text. For instance, a lexical profile might consist of 80% of its words belonging to the 1,000 most frequent word families, 10% of its words belonging to the second 1,000 most frequent word families, and 5% of its words belonging to the Academic Wordlist, with the remaining 5% of its words not belonging to any of these categories but ending up in a not-in-the-lists category. A high proportion of low-frequency words in a text is supposed to be indicative of a high level of lexical proficiency in the TL. Even though the LFP was originally developed for written texts, it has been suggested that it is a valid measure of lexical richness both in written production (Laufer & Nation, Reference Laufer and Nation1995) and in oral production (Ovtcharov, Cobb, & Halter, Reference Ovtcharov, Cobb and Halter2006). These studies have shown that the LFP method distinguishes between proficiency levels.
TEACHERS’ JUDGMENTS
Daller et al. (Reference Daller, Van Hout and Treffers-Daller2003) proposed two new measures of lexical richness: the Advanced Guiraud and the Advanced TTR (both of which are based on calculating the ratio of advanced, or low-frequency, types to all tokens). In doing this they elaborated the notion of advanced vocabulary, combining rarity and variation, or in other words, combining frequency measures and ratio measures. Examining Turkish-German bilinguals, they used an existing basic vocabulary list for German. As there was no reliable corresponding list for the Turkish language, the authors used TJs as a tool to define a basic vocabulary, as opposed to an advanced vocabulary, in Turkish. Seven teachers of Turkish as a foreign language were asked to indicate the words belonging to a more basic vocabulary and those belonging to a more advanced vocabulary; 693 types occurring in picture retells were judged. The teachers were asked to classify these words into four categories: words suitable for (a) beginners, (b) intermediate students, or (c) advanced students, and (d) words regarded as not suitable for any of these categories, such as dialectal or slang items. The lexical items that were judged as basic by a majority of the teachers were then included in the Turkish basic vocabulary.Footnote 4
Another study that aimed to distinguish between basic and advanced vocabulary from the sophistication perspective was Tidball and Treffers-Daller (Reference Tidball and Treffers-Daller2008). They suggested that TJs might be a more reliable tool than methods based on different frequency lists for defining basic versus advanced vocabulary, on the basis of the assumption that teachers might be sensitive to the role of factors—other than frequency—such as cognateness (Tidball & Treffers-Daller, Reference Tidball and Treffers-Daller2008). Their methodology was slightly different from Daller et al. (Reference Daller, Van Hout and Treffers-Daller2003) in that three experienced teachers of French were asked to judge 932 types produced by learners in an oral picture description task. They were asked to rank the words on a 7-point scale, in which 1 was the most basic and 7 the most advanced. To define a basic vocabulary, the sum of the scores given by the raters was calculated for each item. The words that obtained total scores in the lower quartile were selected for the basic vocabulary.
To investigate lexical richness in two groups of learners of French L2 and one group of NSs, Tidball and Treffers-Daller then compared different operationalizations of the Advanced Guiraud and concluded that “using teacher judgement is a better way to obtain a basic vocabulary list than using frequency data” (2008, p. 308). Operationalizations that were based on TJs were found to better discriminate between groups at different proficiency levels and were therefore more powerful than frequency-based measures of Corpaix (Véronis, 2000) and Le français fondamental 1er degré (Gougenheim, Reference Gougenheim1959). It is this research on lexical richness that provides the fundamental basis for the development of more sensitive profilers of L2 French and Italian explored in the present study.
TOWARD A LEXICAL PROFILER FOR SPOKEN L2 FRENCH AND ITALIAN
In previous research, a lexical profiling method based on the distinction between high-frequency versus low-frequency words was developed. The aim of the present study is to integrate cognates and thematic words into the lexical profiler as criteria aside from frequency. This implies a shift from a purely frequency-based approach to a distinction between basic versus advanced words. Teachers’ judgments will be used as a tool for choosing which low-frequency cognates and thematic words should be recategorized as basic.Footnote 5
The lexical profiler in the present study has been elaborated in a series of previous studies. Lindqvist (Reference Lindqvist2010) analyzed lexical richness in advanced learners’ oral production of French L2 and compared two groups of advanced learners of French and native controls. The study was conducted within the French LFP framework, using the program Vocabprofil (available at www.lextutor.ca/vp). In a similar study, Ovtcharov et al. (Reference Ovtcharov, Cobb and Halter2006) showed that the French LFP was a valid tool for measuring lexical richness in advanced learners of French L2, even though the learners’ spoken data were run against frequency bands that were based on written data. However, Lindqvist (Reference Lindqvist2010) could not confirm the validity of the LFP because the results did not show significant differences between the learner groups; rather, Lindqvist’s study raised some issues regarding the use of written corpora when analyzing oral data and suggested using oral data as a benchmark when studying oral production.
Lindqvist et al. (Reference Lindqvist, Bardel and Gudmundson2011) set out to elaborate frequency bands on the basis of spoken data for both French and Italian. This work resulted in the Lexical Oral Production Profiler (LOPP), a package of three frequency bands in each language, on the basis of corpora of spoken L1 French and L1 Italian, and a program used to run learner data against the frequency bands. To create frequency bands that were based on spoken TL data, the Corpaix corpus (Véronis, 2000)Footnote 6 for French and the C-Oral-Rom and LIP corpora for Italian were used.Footnote 7 A program was also developed to run data against the frequency bands. The lexical profiles of learners of French and Italian at two different proficiency levels according to morphosyntactic criteria were then compared to that of NSs. The lexical profiling analysis reflected this classification, as significant differences in lexical richness were found between the learner groups. However, there were intragroup individual differences that needed further investigation; for instance, as already mentioned, some learners had an unexpectedly high proportion of low-frequency words, and one of them had results even higher than the native controls.
Bardel and Lindqvist (Reference Bardel, Lindqvist, Roberts, Pallotti and Bettoni2011) studied two learners of French and two learners of Italian of different proficiency levels from a qualitative viewpoint. The results showed that among the low-frequency words, there were many cognates and thematic words. This was especially the case for the less advanced learners, for whom a relatively high number of such words were found in proportion to their total production. This result suggests that these words, although infrequent in the TL corpora, should not necessarily be considered advanced and that analyses of learners’ lexical sophistication would benefit from the integration of aspects other than frequency. The aim of the present study is therefore to expand on the current method by recategorizing some of the low-frequency words.
THE STUDY
Research Questions
The study was guided by two principal research questions:
1. Will teachers judge low-frequency cognates and low-frequency thematic words as easier than other low-frequency words?
2. Will a method that includes TJs of cognates and thematic words produce profiles that are more homogeneous within groups at different proficiency levels?
Method
As previously shown, TJs have been successfully used by other researchers as a tool for characterizing words from the sophistication perspective. In fact, Tidball and Treffers-Daller (Reference Tidball and Treffers-Daller2008) concluded that using TJs was the optimal way to define a basic vocabulary of French. In the present study, like in Tidball and Treffers-Daller, experienced teachers categorized words as basic (easy) or advanced (difficult). This procedure was used in order (a) to develop a profiler for lexical sophistication (i.e., use of advanced vocabulary) specifically created for Swedish learners’ French L2 and Italian L2 and (b) to discuss the role of cognates and thematic vocabulary in relation to frequency. First, all low-frequency words (band 3 and off-list) produced by the informants—both learners and NSs—in the previous study (Lindqvist et al., Reference Lindqvist, Bardel and Gudmundson2011) were extracted. This yielded 981 French words and 1,056 Italian words. Starting with the French words, there were 626 words that had cognates either in Swedish L1 or English (the first L2 of all the learners in the study), 47 thematic words, and 308 other low-frequency words that were neither cognates nor thematic words.
Cognates can be defined in several ways. In this study, a broad definition was applied to the term (see Horst & Collins, Reference Horst and Collins2006) that takes into account Swedish (the L1 of the learners), English (known by all the learners), and either French or Italian (depending on the TL of the learners).Footnote 8 Words were considered cognates if they shared meaning and had similar forms across languages—that is, word forms that have a common origin but are not necessarily identical, such as the following French-English examples from Horst and Collins (Reference Horst and Collins2006): estomac (stomach), surpris(e) (surprised), compliqué(e) (complicated), and so forth. If the meaning differed considerably, words with the same or similar form (so-called deceptive cognates) were not considered.
It is important to note that the notion of similar forms is itself vague. As observed by Milton (Reference Milton2009), cognates can be more or less similar. He proposed a way of measuring cognateness that takes into account the degree of similarity, that is, in general, the more letters two words have in common, the higher the degree of cognateness. According to Milton, the higher the degree of cognateness, the easier the word will be from a learner’s perspective. However, it is also important to keep in mind that pronunciation of cognates may differ even though spelling is rather similar. Hence, it may also be of interest to take into account differences in pronunciation when discussing cognateness as a facilitating factor for vocabulary learning, especially in studies on L2 production. If the pronunciation of cognates differs highly between two languages, it cannot be taken for granted that the learner detects and makes efficient use of the cognateness. An advanced level of the L2 may be required for the use of some cognates. Considering this, it is acknowledged that not all cognates are easy.Footnote 9
To define thematic vocabulary, a bottom-up method was used in which all the low-frequency words produced by the learners in the present study were matched against the classification of themes of the Common European Framework of Reference for Languages (CEFR; Council of Europe, 2001).Footnote 10 In this way a list that contained all low-frequency words in the subcorpus, defined as cognates or thematic words, was compiled, and it was these words that were to be classified as either basic or advanced by TJs. Because all of these were low-frequency words, there was a risk of misclassification, as the judge might regard some words as relatively easy compared to other words in the list. Such a relative judgment could give a skewed result. Therefore, 129 randomly selected words were added from the high-frequency categories. In contrast, 128 words from the low-frequency categories that were neither cognates nor thematic words (i.e., difficult words according to the frequency perspective and with no facilitating effect of cognateness or saliency in teaching materials) were also added. This was done to achieve approximately the same number of test words as in the study by Tidball and Treffers-Daller (Reference Tidball and Treffers-Daller2008). In total, the final list of French words contained 929 randomly sorted words. Assuming that cognates can have greater or lesser difficulty—depending on their meaning and their degree of similarity—they were further divided into two subcategories, basic and advanced (see note 9 and Bardel & Lindqvist, Reference Bardel, Lindqvist, Roberts, Pallotti and Bettoni2011).Footnote 11
Among the Italian words, there were 553 words that had cognates in either Swedish L1 or English L2, 67 thematic words, and 432 other low-frequency words. As in the case of the French word list, randomly selected high-frequency words (n = 155) and low-frequency words (n = 155) were added, to reach a total of 930 test words. Examples of words in the five categories are shown in Table 1. The teachers were presented with the lists and asked to indicate, for each word, whether it was a basic or advanced word on a one to six scale, in which one was most basic or easy and six was most advanced or difficult.
Examples of words in the different categories
Note.
Swedish cognates have been provided in parentheses where available.
Before administering the test to the teachers, a pilot judgment was carried out by one experienced teacher of Italian. First, she was given the complete list of Italian test words and asked to judge each word as previously described. This task took her approximately 50 minutes. Four days later, she was given a list of 10% of the test words from the original list and was asked to judge them once again. The within-reliability (intercorrelation) of this particular teacher was calculated and found to be high and significant (r = .89, p < .0001).
Two groups of teachers—one for French (n = 9) and one for Italian (n = 11)—agreed to participate in this study. These were all experienced teachers, NSs either of French or Italian or of Swedish, all with a very high level of proficiency in their respective language. As in the pilot study, some days later, all teachers were given a retest containing 10% of the words from the original list. The results showed high within-reliability (intercorrelation) for all teachers: the r values ranged from 0.71 to 0.95 for the French teachers and from 0.57 to 0.98 for the Italian teachers. For each individual, the correlations between test and retest were significant (p < .0001) for all French and Italian teachers. An ANOVA showed a good between-reliability (intracorrelation) between teachers. The between-reliability was measured by the intracorrelation coefficient (ICC), proposed by R. A. Fisher. The ICC values for the French teachers were 0.98 on both occasions and for the Italian teachers 0.94 on both occasions. The number of words judged, as pertaining to the different categories, and the mean score of each category for the two languages are shown in Table 2.
Results of teachers’ judgments: mean values of the test words in the different categories
The results of the TJs are very similar in French and Italian and follow the same pattern, as can be seen in Figure 1. The low-frequency words were judged as the most difficult ones, followed by the advanced cognates. The categories that obtained the lowest mean values were the high-frequency words and the basic cognates. An ANOVA showed significant differences between the categories for both French, F (4, 924) = 156, p < 0.001, and Italian, F (4, 925) = 154, p < 0.001. Post hoc comparisons (Dunnett T3) showed significant differences (p < .05) between all categories in the Italian data. In the French data the post hoc comparisons showed significant differences (p < .05) between all categories except for between the basic cognates and the high-frequency words.
Mean values of the teachers’ judgments of the different categories.
Each word was then given a mean value, calculated from the total of all the teachers’ individual scores. To recategorize the low-frequency words of interest (i.e., words from categories 1, 2, and 3), the words that had a mean value lower than 4 were selected. This resulted in a recategorization of 610 words in the French data and 494 words in the Italian data from the low-frequency (advanced) categories to the high-frequency (basic) categories.
RESULTS AND DISCUSSION
In this section, a comparison will be made between two methods. First, the results from the previous study, in which the purely frequency-based LOPP method was used (Lindqvist et al., Reference Lindqvist, Bardel and Gudmundson2011), are summarized. The data from that study is then reanalyzed using the new version of the LOPP method, which relies on a distinction between advanced and basic vocabulary according to the TJs. To distinguish the two methods from each other, they will be termed LOPPf (f for frequency) and LOPPa (a for advanced), respectively.
Results Using the Purely Frequency-Based Method: LOPPf
In Lindqvist et al. (Reference Lindqvist, Bardel and Gudmundson2011), the lexical sophistication of 14 Swedish learners of French and 20 Swedish learners of Italian was analyzed. The data were taken from the InterFra and the InterIta corpora, which were compiled at Stockholm University.Footnote 12 Both corpora consist of oral data, retrieved from interviews conducted by a NS with learners of French or Italian. The topics of the interviews are roughly the same in the two corpora: family, studies or work, spare time activities, future plans, experiences in Italy or France, and so forth. The learners of both languages were divided into two subgroups on the basis of their proficiency levels, as measured by morphosyntactic traits.
Table 3 shows the results from the previous study. The table shows the proportions of words belonging to different frequency bands elaborated from the LIP, the C-Oral-Rom, and the Corpaix corpora, produced by the learners and by NS controls: band 1 (the 1,000 most frequent words in the reference corpus), band 2 (the next 1,000 most frequent words in the reference corpus), band 3 (words ranging from 2,000 to 3,000), and the off-list (words produced that are not found in any of the frequency bands).
French and Italian lexical profiles (%) using LOPPf
Note:
Adv.lo = advanced low level, adv.hi = advanced high level, int. = intermediate level, adv. = advanced level, NS = native speakers, B = band, and OL = off-list.
There were significant differences in lexical sophistication (i.e., the proportion of words belonging to band 3 and the off-list) between the learner groups in both languages, and between the advanced L2 learners of Italian and the NSs of Italian. There was no significant difference between the most advanced group of French L2 learners and the French NSs. However, there were intragroup individual differences that required further investigation. Several learners had an unexpectedly high proportion of low-frequency words, and one learner of French reached higher than all the NS controls. This can be seen in Figures 2 and 3, which show the proportion of low-frequency words produced by each individual.
The proportion of low-frequency words (band 3 + off-list) in the French data using LOPPf. All names are pseudonyms.
The proportion of low-frequency words (band 3 + off-list) in the Italian data using LOPPf. All names are pseudonyms.
Results Using the New Method: LOPPa
In this section, the new profiles—on the basis of the distinction between basic versus advanced vocabulary—are presented, along with the results from the recategorization of words. Because the new profiles are no longer based exclusively on frequency, the learners’ lexical production was divided into the two categories basic versus advanced vocabulary. The basic vocabulary category contains the 2,000 most frequent words plus the low-frequency words that were evaluated as easy or basic by the teachers. The advanced vocabulary category contains low-frequency words beyond the 2,000 most frequent words.Footnote 13
Table 4 shows the results at group level, using the new method on the data from Lindqvist et al. (Reference Lindqvist, Bardel and Gudmundson2011). As expected, the proportion of low-frequency words was generally lower when compared to the profiles in the previous study. To investigate whether the differences were still significant between the learner groups, ANOVA tests were run. For French there were significant differences between the three groups, F (2, 21) = 15.58, p < .001. The post hoc test (LSD) shows that there were significant differences between all groups: between the advanced low and the advanced high L2 learners (p < .05); between the advanced low L2 learners and the NSs (p < .001); and between the advanced high L2 learners and the NSs (p < .05). Thus, the new method LOPPa allows for the discrimination between the advanced high group and the NSs. This is probably due to the fact that the groups were more homogeneous using the new method.
French and Italian lexical profiles using LOPPa
Key.
Basic voc = basic vocabulary, adv. voc = advanced vocabulary, adv.lo = advanced low level, adv.hi = advanced high level, int. = intermediate level, adv. = advanced level, and NS = native speakers.
As for Italian, the ANOVA test showed significant differences between the groups, F (2, 27) = 57.12, p < .001. Equal variances were not assumed; therefore Dunnett T3 was used for the post hoc test. The post hoc test showed significant differences between all groups: between intermediate and advanced L2 learners (p < .001); between intermediate L2 learners and NSs (p < .001); and between advanced L2 learners and NSs (p < .001).
The individual profiles of the French data are shown in Figure 4. This figure shows that all informants had a lower proportion of advanced words when the new method was used. For instance, Knut, who had an extremely high proportion of low-frequency words in the previous study (see Figure 2), is now approaching the other learners in his group, although he still has the highest proportion of advanced words. Some of the words that have been moved from the low-frequency categories to the basic category—opéra (opera), nordique (Nordic), reportage (report), médaille (medal), and douche (shower)—all have cognates in Swedish and were considered to be easy or basic by the teachers. Examples of words in Knut’s production that remain advanced are chaleureux (warm, friendly), revers (reverse), relier (to link), and dépasser (to pass).
The proportion of advanced words in the French data using LOPPa. All names are pseudonyms.
As seen in the data from the Italian learners, all the individual profiles are lower than before the recategorization of words when Figures 3 and 5 are compared. In the previous study, Anita, Alva, Clelia, and Kristina all had higher proportions of low-frequency words than one of the NSs, something that was unexpected, especially in the case of Anita, who had been classified at an intermediate level on the basis of her morphosyntax. Anita’s profile presented a proportion of almost 5% low-frequency words in the previous study. In her case, the method that removes basic cognates and thematic words considered relatively easy by teachers lowered her proportion of advanced words to 0.60%. Examples of words produced by Anita that were removed from the advanced to the basic category are etnologia (ethnology), romantico (romantic), stress (stress), timido (timid), turista (tourist), and grammatica (grammar), all of which have cognates in Swedish. The few words that remain are cassapanca (chest container) and scrivania (desk).
The proportion of advanced words in the Italian data using LOPPa. All names are pseudonyms.
As for Kristina, although her score has lowered from a level of 4.50% (see Figure 3), she remains ahead of the other learners of Italian, with a proportion of advanced words amounting to 2.32%. The number of low-frequency tokens is still high (n = 42), and examples of her sophisticated vocabulary (e.g., affatto [at all], anziché [rather than], badare [look after], apposto [all right], incinta [pregnant], and noioso [boring]) show words that clearly do not fall within the categories of cognates or thematic words. Remaining words that could be considered cognates—mainly in relation to English words—in the proportion of advanced words in Kristina’s production included concedere (to grant, to give, to concede), distinguere (to distinguish), socializzare (to socialize), femmina (female), and restutire (to return, to repay, to restore). These were considered rather difficult by the teachers and should be evaluated as examples of advanced cognates.
To evaluate the effect of the new tool, effect sizes were calculated by Eta2, which is analogous to r 2 in correlation and regression analysis. As Table 5 shows, the LOPPa method obtains higher Eta2 than the LOPPf method, for both the French data and the Italian data. This means that the inclusion of some low-frequency words (i.e., cognates and thematic words)—words that turned out to be considered easy by the teachers—in a basic vocabulary represents an improvement compared to a purely frequency-based method.
Effect size for the two methods LOPPf and LOPPa
CONCLUSION
In this study, two methods for lexical profiling were compared: one in which advanced vocabulary is defined in terms of low frequency, and one that distinguishes between basic and advanced vocabulary and now includes certain low-frequency cognates and thematic vocabulary in the category of basic vocabulary. A new method, LOPPa, was tested that reconsiders some low-frequency cognates and thematic vocabulary judged by teachers as easy or basic words, and by doing so, advantages were found in refining the purely frequency-based method used in previous studies. A limitation of the study is that it is concerned with two particular language combinations. However, it could be replicated with any combination of languages, following the methodology presented in this article.
Advanced vocabulary is not only seen as low-frequency vocabulary but also as a set of difficult words in which difficulty depends on two powerful factors: the quality of the input of the teaching situation and the knowledge of other languages. It remains to be investigated whether other factors that come into play are strong enough to have an effect on the frequency model for vocabulary acquisition. It would be interesting to investigate some of the traits related to form and meaning mentioned by Laufer (Reference Laufer, McCarthy and Schmitt1997) and Milton (Reference Milton2009) by means of factor analysis.
The new method resulted in more homogeneous groups at different proficiency levels and allowed for the discrimination between all learner groups and the NSs. With the previous method, some learners had a high proportion of low-frequency words, which apparently created an inaccurate impression of their lexical sophistication. The new method allowed for a more clearly distinguished lexical profile of advanced L2 learners and NSs. A further advantage of this method, if compared to measures of lexical richness relying on mathematical formulas such as variants of the type-token ratio measure (e.g., Advanced Guiraud; Tidball & Treffers-Daller, Reference Tidball and Treffers-Daller2008), is its qualitative dimension, in the sense that the output provides the researcher (and other plausible users, such as teachers and learners) with lists of words belonging to the two categories of basic versus advanced vocabulary. Of course, a widespread use of the tool would require the realization of user-friendly software. Such an application was not implemented in this project but could be a natural extension of this work.
It has been shown by the comparison of the two methods that TJ is a good tool to determine basic and advanced vocabulary in L2. It was clear from the judgments that teachers characterize many low-frequency cognates and thematic words as relatively easy, compared to other low-frequency words. The TJs gave an indication of what had been suspected—namely, that not all low-frequency words are necessarily difficult or advanced. By taking cognateness and the notion of thematic words into consideration, it was found that the LOPP method improved, and that the learners’ profiles became more homogeneous within the different learner groups (intermediate and advanced for Italian, advanced low and advanced high for French) after moving certain items from the advanced to the basic vocabulary. Because the difference between these groups was statistically significant, it can thus be concluded that even though the words that remained in the advanced category were relatively few after the modifications were made, the new method still captured the differences in vocabulary sophistication between proficiency levels. The fact that the new method, LOPPa, was a better operationalization was also shown by comparing the effect sizes of the methods.
In summary, the study indicates that the LOPPa method is appropriate both when comparing lexical sophistication at the group level and when looking more closely at individual profiles. It also has pedagogical implications, for both syllabi and teaching materials design as well as diagnostic tests and assessment of proficiency levels.
