4.1 Measuring L2 vocabulary knowledge and use

A major attraction of working with lexis is that it seems to lend itself quite comfortably to measurable, quantitative study. The first two theses reviewed in this section (Kamimoto Reference Kamimoto2008 and Fitzpatrick Reference Fitzpatrick2003) are primarily quantitative in nature, focusing on tools which can be used to approximate the size, or relative size, of a learner's L2 lexicon. Ronald (Reference Ronald2006) also measures vocabulary knowledge in a quantitative way, but his study is concerned with incremental gains in both depth and breadth of knowledge, and the ways in which these are affected by reading and dictionary use. Pemberton's work (Reference Pemberton2003) highlights the gap for L2 learners between word knowledge, when measured in a discrete, low-context test, and word use in terms of recognition of those same items in connected speech. Where Pemberton's learners struggle to recognise the spoken form of a word, Emery's (Reference Emery2005) have difficulty producing the written form. Despite the fact that they are concerned with different types of word knowledge, and very different groups of learner, Pemberton and Emery reach similar conclusions about the causes of their learners’ problems.

Kamimoto's thesis, ‘Nation's vocabulary levels test and its successors: a re-appraisal’ (Reference Kamimoto2008), provides an example of the sort of in-depth, detailed analysis which can result from a series of very precise research questions. Like Zhao (above), Kamimoto challenges the performance of an existing test, but whereas Zhao's redesign is informed by users’ feedback, his is based on rigorous statistical analysis of the interaction between test format and test-taker performance. Kamimoto's thesis is based on the premise that the Vocabulary Levels Test (VLT) (Nation Reference Nation1983; Schmitt, Schmitt & Clapham Reference Schmitt, Schmitt and Clapham2001) has been pressed into use as a relatively high-stakes measurement tool before being subjected to a sufficiently rigorous validation process (notwithstanding Read Reference Read1988, Beglar & Hunt Reference Beglar and Hunt1999 and Schmitt et al. Reference Schmitt, Schmitt and Clapham2001). Kamimoto's introduction explains that the VLT is a diagnostic test, consisting of target words from each of five frequency bands. Word knowledge is tested by presenting the target words in clusters of three, plus three distractors, and requiring test-takers to match three of the six words (the targets) to corresponding definitions. It is this format which Kamimoto identifies as problematic, since items are not fully independent, and results may therefore be influenced by the guessing behaviour of test-takers. In his literature review he discusses studies which have attempted to revise or adapt the VLT, and observes that these studies have variously criticised the ‘wordlists, word levels, the number of target words per level, and definition language’ (Reference Kamimoto2008: 67) of the VLT, but have failed to address the confounding influence of the 6–3 matching format and the resulting complexity of learners’ guessing behaviour.

Kamimoto presents six experimental studies, with the first three focusing on item independence and the role of the distractor. A correlation analysis of test versions with different cluster compositions finds that item difficulty is dependent on the composition of clusters. Nation originally claimed that the inclusion of item, rather than definition, distractors in the clusters strengthens the test, by demanding knowledge of twice as many words (Nation Reference Nation1983: 14). Kamimoto tests this claim, firstly by devising a translation test of targets and distractors and comparing performance on this with VLT scores, and secondly, by designing a test in the VLT format, using distractors from Schmitt et al. (Reference Schmitt, Schmitt and Clapham2001) as targets. He finds that, contrary to Nation's claim, distractors and target items are not necessarily equal in difficulty, and he discusses the uncritical use of frequency corpora as a possible factor underlying this problem.

The remainder of Kamimoto's experimental chapters are concerned with the guessing behaviour of test-takers. He investigates this using think-aloud protocols with learners from two distinct proficiency levels, and concludes that Schmitt et al.'s (Reference Schmitt, Schmitt and Clapham2001) assertion that higher proficiency learners are more successful at guessing is not necessarily true. The most popular guessing strategy of elimination, as well as strategies based on partial word knowledge, loanwords, and ‘pure blind guessing’, are common across proficiency levels, though higher proficiency learners seem to use more semantically-oriented strategies. In Kamimoto's final experiment he proposes an adaptation to the VLT which uses a confidence rating scale to correct for the effect of guessing, and shows that this produces more reliable results than the original dichotomous scoring system. In his final chapter Kamimoto considers the implications of his findings for future applications of the VLT. It is indicative of this quest for improvement of the test, rather than criticism of it, that he includes here a Q&A dialogue with Paul Nation, author of the original VLT test. Kamimoto's thesis represents an exemplary contribution to the ‘never-ending process’ (Reference Kamimoto2008: 271) of validation and development of the widely-used and often misinterpreted VLT.

Test development and validation are central, too, to Fitzpatrick's ‘Eliciting and measuring productive vocabulary using word association techniques and frequency bands’ (Reference Fitzpatrick2003). She identifies an apparent gap in the portfolio of existing vocabulary tests, claiming that there is currently no test of vocabulary which elicits productively a relatively large sample of items from the learner's lexicon, and measures them in a principled way. Her thesis presents a series of experimental studies in which she develops and hones a test to fill this gap. The test, Lex30, elicits vocabulary by means of a word association test; test-takers are asked to write four responses to each of 30 high-frequency cue words. The test is scored by awarding one mark for every low-frequency (i.e. outside the first 1,000 band) word produced, the premise being that more proficient learners will produce a higher proportion of low-frequency words. The cue words used in Lex30 are selected in a principled way so as to minimise the effect of predictable and frequent (i.e. non-scoring) responses. Fitzpatrick's initial experimental findings are promising. She finds high, significant correlations both between scores in a test–retest, and with scores on a frequency-based test of receptive vocabulary knowledge. Following Bachman (Reference Bachman1990: 248), Fitzpatrick then attempts to determine the ‘concurrent criterion relatedness’ of the test by comparing learner scores with those of native speakers, but her findings here are not straightforward. Although the mean native speaker score is significantly higher than the mean learner score, there is considerable overlap between groups, with several learners scoring higher than some of the native speakers. Her final study compares Lex30 scores with those from two other tests of productive vocabulary, Laufer & Nation's Productive Levels Test (Reference Laufer and Nation1999), and an L1 > L2 translation test. The resulting correlations are significant but moderate. Fitzpatrick attempts to explain her rather complex findings in a number of ways, arguing, for example, that the test is perhaps not sensitive enough to distinguish very proficient learners from native speakers. Her discussion of the moderate correlations between Lex30 and the other two tests of productive vocabulary draws attention to a ‘catch 22’ of validity testing: the rationale for designing her test was that no satisfactory productive test of discrete vocabulary items existed, so any test used for a concurrent validity study will not, by definition, measure the same abilities in the same way. Fitzpatrick compares the different ‘activation models’ of each test, arguing that where Lex30 elicits vocabulary solely by means of a semantic stimulus, the Productive Levels Test uses semantic, collocational and orthographic stimuli. Finally, she proposes that productive vocabulary knowledge is a complex construct, and that different tests tap into different aspects of it.

The work reported in Kamimoto's and Fitzpatrick's theses represent different stages of the validation process. Fitzpatrick concludes that her test has some potential, but recognises that the studies she reports represent the very beginnings of a validity argument (an argument which has in fact been continued in subsequent papers and is summarised in Fitzpatrick & Clenton Reference Fitzpatrick and Clenton2010). Kamimoto, on the other hand, offers a rigorous analysis of the design of a test which has already been subjected to a number of validation investigations. Both theses present a thorough and logical progression of argument through experimental studies, but both give cause for concern that their findings may not be widely replicable. Kamimoto acknowledges that his findings might only be relevant to learners with Japanese as their L1. Some of Fitzpatrick's studies use a worryingly small participant group (16 for the test–retest), which somewhat undermines the validity of her findings.

Ronald's thesis, ‘Second language vocabulary acquisition through dictionary use’ (Reference Ronald2006), is also concerned with measuring knowledge of discrete lexical items. His focus, though, is not on assessment of general L2 proficiency or vocabulary size, but on the effects of a specific type of learning intervention, monolingual dictionary use, on the acquisition and retention of vocabulary. In his introductory chapters he identifies a number of methodological challenges inherent in this kind of investigation, including how to record a learner's dictionary use, and how to measure word knowledge which ‘is very often not a clear-cut matter of know/not know’ (Reference Ronald2006: 104).

Ronald's first set of studies aims to measure comprehension and retention of previously unknown words, encountered by participants in various intervention tasks. Scores for learners who have used monolingual dictionary entries for information about unknown words are compared with those for learners who have been given corpus-based sample sentences containing the target words. Findings indicate a tendency for the dictionary users to perform better on the comprehension task, but Ronald concludes that a more sensitive experimental procedure might reveal more precise information about the incremental vocabulary growth taking place. His next chapters therefore describe case studies in which learners undertake multiple readings of long texts, and both their dictionary use and changes in their knowledge state of target vocabulary items are recorded. In these studies Ronald uses a more sensitive measure of vocabulary uptake, V_States (Meara Reference Meara2001), which asks learners to rate, on a four-point scale, the degree to which they know a word. The study design allows for a large number of items (300) to be tested, and the test events occur after each of eight consecutive readings of the same novel. The first three readings were completed without dictionary access, and for subsequent readings the learner used a monolingual learner dictionary, fixing a Post-It tab in the dictionary for each word looked up so that the researcher could keep track of them. This sophisticated longitudinal study design allows Ronald to track incremental gains in word knowledge and to link these to the learner's look-up behaviour. The use of a repeated, multi-state vocabulary test such as V_States affords Ronald an additional analysis opportunity. By using a transitional probability matrix (Meara & Rodriguez-Sanchez Reference Meara, Sánchez, Bax and Zwart2001) he is able to make projections of future vocabulary development under the without-dictionary and the with-dictionary reading conditions.

Ronald finds that use of the monolingual dictionary during reading accelerates vocabulary growth, and that previously unknown target words are better retained if they have been looked up. Vocabulary gains are greatest after the first without-dictionary reading, and after the first with-dictionary reading. This is an impressively thorough – and lengthy – piece of work. Ronald effectively addresses problematic issues which have arisen in earlier studies, such as how to test enough items, how to measure small gains in knowledge, and how to quantify the benefits of dictionary use. The use of probability matrices to model future change, based on real data points, adds an innovative dimension to his work. Reflecting on his research, though, Ronald notes that while the V_States instrument does expose areas of partial knowledge, its self-report nature makes it susceptible to individual learner differences with regard to confidence. A number of the theses reviewed here rely to some extent on self-report data, but few engage to this degree with the need to normalise these kinds of data, or the problems inherent in doing so.

Both Kamimoto and Ronald warn against the assumption that lexical knowledge is an all-or-nothing phenomenon, and take pains to consider and measure partial word knowledge. Pemberton's thesis, ‘Spoken word recognition and L2 listening performance’ (Reference Pemberton2003), highlights the difference between ‘knowing’ a word when it is presented in a decontextualised, or written, mode, and recognising it when it is used in connected speech. He argues that research into L2 listening processes has focused disproportionately on comprehension skills and neglected the prerequisite skills of word recognition and identification. In each of the experimental studies presented in Pemberton's thesis, groups of Hong Kong learners of English listen to BBC news broadcasts and perform tasks which allow the researcher to identify their initial, and subsequent, perceptions of speech segments. His first experiments find that the intermediate learners recognise only three in every four of the most frequent words of English when the news items were played at normal speed. By asking participants to identify part of the broadcast which they could not understand, and by using codes to distinguish between types of difficulty (e.g. failed to recognise known word, unknown word, mis-segmented section, section too fast), Pemberton was able to determine that around 40% of errors happened when the learner failed to recognise a word which, when later encountered in the written transcript, they knew. Unsure of the accuracy of this self-report data, Pemberton next asked learners to transcribe a section of a news broadcast. He allowed them multiple listening events (they took away a copy of the broadcast to replay at will), yet the recognition rate was still only 75%. Pemberton hypothesises that the advantage of replaying the tape was offset by the more accurate scoring in this experiment (the researcher scored the transcriptions), resulting in the same overall recognition rate as for the previous experiment. In order to test his assumption that recognition rates were much lower in the learners’ first attempts at transcription, Pemberton devised a sophisticated recording mechanism for his next experiment. This time, the typed transcriptions were kept in dribble files, which enabled the researcher to access all stages of transcription, from the first listening onwards. He found that on the first attempt, about one in three words were recognised, and that this increased to only two in three by the final attempt. To maximise the learners’ opportunities for word recognition, Pemberton then recorded a slower version of the news story (150 words per minute instead of the original 200). This time, after multiple listenings, the recognition rate for frequent words increased, but only to 80%. In his discussion, Pemberton explores potential causes and implications of the failure of these learners to recognise in speech words which, in a written test, they would claim to ‘know’. He cites two possible causes of this gap in his learners’ performance. Firstly, he suggests that they have insufficient exposure to spoken English which, in any case, has an ‘inherent messiness’ (Reference Pemberton2003: 106). Secondly, and insightfully, he points out that the L1 reading instruction of these Hong Kong students focuses on logographic script, which does not convey phonemic information. Pemberton suggests that this L1 reading background impairs learners’ ability to match sounds and symbols in later L2 performance, and he calls for a pedagogical approach which more explicitly focuses on phonological awareness training. In a later publication (Reference Pemberton, Fitzpatrick and Barfield2009), Pemberton pursues this hypothesis, comparing the L2 listening performance of Hong Kong-educated Cantonese speakers with those from mainland China, whose L1 reading instruction had used the phonemic Pinyin system.

Emery (Reference Emery2005) is also concerned with difficulties related to word form as presented by a specific group of learners: university-level learners of English studying in the United Arab Emirates. Her thesis, ‘An investigation into the nature and causes of reading and spelling errors made by Arab ESL learners’, follows up the findings of Al Sulimani (Reference Al Sulimani1990) and Ryan (Reference Ryan1993) that spelling and reading errors in L1 Arabic learners of English are similar to those presenting in Developmental Surface Dyslexics, where the phonological processing route is intact but the orthographic route is impaired. Emery's experimental studies investigate these processing routes using Frith's reading and spelling model (Reference Frith, Patterson, Marshall and Coltheart1985) to determine the stage they are at: alphabetic spelling, alphabetic reading or orthographic reading. She compares them with EFL learners of a similar level but with a different L1 (Spanish), and concludes that the Arabic L1 learners’ word recognition and reading development did not arrest at a particular stage, as implied by Al Sulaimani (Reference Al Sulimani1990) and Ryan (Reference Ryan1993), but rather suffered a significant developmental lag. However, Emery's review of L2 reading strategy research leads her to conclude that their L1, Arabic, should not impair these learners. Returning to Frith's model, which emphasises the role of appropriate instruction in development of reading skills, she reviews the UAE state education system through a series of interviews, observations and textbook analyses. Emery concludes that ‘a major cause of the reading and spelling errors made by Arab ESL learners is inappropriate instruction in the schools and insufficient reading practice’ (Reference Emery2005: 223). Interestingly, the final comments of her thesis broadly mirror those of Pemberton: she recommends that teachers are trained to teach phonics, to emphasise the importance of accurate spelling, and to provide extensive reading practice. Encouragingly, she notes that, in Oman, moves towards such developments are already beginning.

Ronald's comment that word knowledge cannot be measured in a binary way (‘know’ or ‘not know’) is relevant to all the theses reviewed in this section. Nation's VLT, investigated by Kamimoto, and Fitzpatrick's Lex30 do not test complete word knowledge, but minimal knowledge of form (Lex30 scores do not demand accurate spelling) and meaning (VLT definitions are of the central meaning of the target word). Word knowledge that is incomplete, rather than lacking, is what impairs Pemberton's and Emery's learners, with the former struggling to recognise spoken forms and the latter struggling to produce written forms accurately. A theme common to these theses, then, is that the initial attraction of lexis as a measurable, quantifiable aspect of language acquisition is deceptive. As well as counting the words that learners ‘know’, we need ways of identifying, describing and assessing partial knowledge. It also emerges that word frequency, used as a grading device by Fitzpatrick and in the VLT, is not a straightforward proxy for word difficulty; Pemberton's learners could not identify 25% of the most frequent words, when heard in continuous speech. One of the most challenging aspects of vocabulary knowledge, and perhaps the one most difficult to address discretely (it includes syntactic and phonological elements) is collocation, the theme common to the theses discussed in the next section.

4.2 Collocation and association

Taking different perspectives, the theses in this section focus on collocation, and, by extension, on associative word links (Schur Reference Schur2003; Wolter Reference Wolter2005). Barfield (Reference Barfield2006) observes that most collocation research looks, rather negatively, at collocation errors or at collocation types. His thesis investigates ways of measuring the development of L2 collocation knowledge, and he proposes a developmental model on the basis of his findings. McGee's (Reference McGee2006) work focuses on awareness of collocation in native speakers. The relevance of his work to second language acquisition is that his informants are teachers of English as a foreign language, and his study is inspired by their apparent inability to retrieve, for their learners, accurate information about L1 collocations. Wolter and Schur both compare native and non-native speakers’ behaviour when asked to identify related words. Wolter's study is designed to contribute to assessment of depth of word knowledge; Schur's approach is more theoretical, and investigates the appropriacy of the metaphors we often use in relation to the mental lexicon.

Barfield's ‘An exploration of second language collocation knowledge and development’ (Reference Barfield2006) begins by addressing the problem of defining collocation, and discusses phraseological, typological and textual interpretations. His review of previous research into L2 collocation knowledge concludes that there has been a disproportionate focus on adjacent two-word collocations, on error, and on advanced level learners, and that, typically, relatively few items are tested. He suggests a more positive approach which measures existing and developing collocation knowledge, especially in less proficient L2 learners, and identifies factors which help or hinder that development. His experimental studies benefit from a systematic use of frequency ranks and word class to investigate collocation behaviour. Recognising that knowledge of collocation can be partial, he uses a knowledge-state ranking system similar to the one used by Ronald, to measure learners’ recognition of collocations. Barfield finds that, while discrete noun and verb recognitions correlate significantly with general English proficiency, recognition of collocations does not. However, he also finds that some learners are more ‘collocationally proficient’ than others, and that these learners tend to prefer different kinds of word combinations from their peers. He pursues this finding in a qualitative, longitudinal study with three learners, in which he explores learners’ individual constructs of vocabulary and collocation learning and use. Using semi-structured interview protocols to gather data over the course of an academic year, Barfield asks the learners about their vocabulary learning processes related to a specific long-term study task, and he classifies their comments into four broad categories, one of which is collocation. From his findings he concludes that collocation learning requires certain conditions to be met in the learning process, namely that learners: 1) break free of exam-based grammar-translation learning; 2) distinguish between new vocabulary and important vocabulary; 3) balance ‘knowing’ with ‘using’; 4) use paraphrase as a means to reach the edge of the Zone of Proximal Development (Vygotsky Reference Vygotsky, Cole, John-Steiner, Scribner and Souberman1935), where collocation use is most likely to occur (Barfield Reference Barfield2006: 308). An interesting aspect of Barfield's thesis is that he seems to shift from a largely corpus-based interpretation of collocation (with attention to frequency counts, etc.) towards a more psycholinguistic-oriented interpretation. The model of collocation development which he ultimately proposes in fact includes both perspectives, incorporating both L1 and L2 influence on collocation acquisition and use, and demonstrating ways in which the acquisition and integration into the lexicon of new items feeds into collocational knowledge as the lexicon restructures itself.

This tension between the representations of collocation in corpora and in the mental lexicon is central to McGee's work. McGee's thesis ‘Lexical intuitions and collocation patterns in corpora’ (Reference McGee2006) stems from his observation that language teachers are often asked by students to provide examples of vocabulary usage. He notes that the answers and examples given by teachers, based on native speaker intuition, often do not match evidence from corpus linguistics, particularly when it comes to identifying common collocates of a word. To give an example from his own data, when native speakers were asked for a word which collocates with front, the most popular response was row, whereas the British National Corpus (BNC) gives door as the most common collocate (1,669 instances, with only 25 instances of row). McGee reports a series of experiments which use data from native speaker (English) language teachers, and from advanced level teachers of English whose L1 is Arabic. He found that while some common collocates are readily produced in the elicited data, others seem less available. Further analysis reveals that words which are found together in idioms or collocational frameworks are usually (but not always) unavailable as ‘choices’ in the respondents’ search for collocates. On the other hand, words which are part of a dyad, or a ‘frozen collocation’ (Greenbaum Reference Greenbaum and Cowie1988), where one of the pair has a highly restricted set of collocates (e.g. blonde > hair), are more readily and reliably elicited. Searching for patterns in his data, McGee finds that his respondents behave in unexpected ways. For example, they seem unaware of the typical prosody of some words, or attach to them a prosody which is not reflected in the corpus data (that the word moment is positive, for example), or attach a primary meaning to a word which is not reflected in the most frequent collocations.

In order to determine whether this phenomenon is a product of producing, rather than recognising, collocates, McGee gave his native and non-native speaker English teachers a list of target words, and a choice of three potential collocates for each. The collocates were ranked 1, 10 and 20 in the BNC lists. In nearly all cases, the participants in both groups correctly identified the most frequent collocate; the success rate was more pronounced when nouns rather than adjectives were used as stimuli. The anomaly between elicited data and corpus data, then, seems to be one of recall rather than recognition. McGee, citing Aizawa, Mochizuki & Meara, notes that ‘native speaker's intuitions about frequency are not always straightforward’ (Aizawa et al. Reference Aizawa, Mochizuki and Meara2001: 80): indeed, his findings suggest that native speakers do not fare much better than advanced non-native speakers in recalling common collocations. However, his findings indicate that there is a ‘principled bias affecting collocate and collocation frequency judgements’ (Reference McGee2006: 312), which demands that future research categorise collocations in a way that goes beyond considerations of frequency and word class.

For Wolter (Reference Wolter2005), collocation and association are aspects of depth of vocabulary knowledge, and his thesis, ‘V_Links: A new approach to assessing depth of word knowledge’, describes the development of a test to measure this knowledge. In his introductory chapters Wolter identifies the challenges in designing a test of depth of knowledge, and in particular the difficulty of defining ‘depth’. He discusses the limitations of widely used tests such as the Vocabulary Knowledge Scale (Paribakht & Wesche Reference Paribakht and Wesche1993) and Read's Word Associates Test (Reference Read1993), and aims to improve on these with his Productive Vocabulary Scales Test, which asks learners to self-rate their ability to use target words in terms of meaning, grammar, collocation and register. However, as in several other studies reviewed here, he finds that self-assessment produces unreliable results, and he raises concerns about the validity of what he calls ‘word-based’ models of assessment: tests which target discrete aspects of knowledge of specific, pre-selected words.

Wolter next describes a series of studies using what he calls a ‘lexicon-based’ approach, resulting in the development of V_Links, a test of lexical organisation rather than of depth. The format of V_Links is a computer program which presents test-takers with groups of ten highly-frequent words, randomly selected for presentation. Test-takers are instructed to make links between the five pairs of ‘most closely related’ words, and their score is the number of acceptable links found. An acceptable link is one previously chosen by at least eight out of twenty-five native speakers. In order to be sure that his subjects were matching words based on their linguistic, rather than conceptual knowledge (they could transfer the latter from their L1), Wolter conducted an item analysis, and, working on the basis that conceptually related pairs would be recognised by both native and non-native speakers, selected for his test only those pairs which native speakers had identified frequently, but non-native speakers had not. Wolter does not seem completely comfortable with this selection process, though, wondering whether the recognition of a connection through conceptual knowledge might in fact represent an important first step in acquiring collocation knowledge. In his discussion he returns to consider the interface between conceptual and lexical knowledge, an issue which he finds troublesome in the context of V_Links, and he concludes that further research is needed on ‘the potential role played by L1 lexical/conceptual knowledge’ (Reference Wolter2005: 233). As a test tool, V_Links is found to distinguish between native and non-native speakers, and broadly between different levels of learner. Native speakers seem to rely more on collocational and phrasal knowledge, non-native speakers on conceptual links.

Wolter's thesis is interesting in that its direction evolves through each experimental study. He sets out to design a test of vocabulary depth, but does not shy away from changing direction and pursuing the most salient questions which arise from his findings. This leads him to reflect on and question the theoretical underpinnings of his own and similar research and, ultimately, to look at implications for network models of the lexicon. He concludes that V_Links is perhaps more valuable as a research tool allowing us to investigate the development of lexical links, rather than as a measure of proficiency.

Schur (Reference Schur2003) also takes a network model approach to her investigation of the mental lexicon in ‘An exploration of the structural properties of L2 vocabulary networks: a graph theoretical approach’. Through her exploration she confronts the use of metaphors such as network and cobweb, which are commonly used to describe the complex nature of the lexicon, arguing that its complexity is ‘not shrouded in mystery as these metaphorical accounts imply, but (is) amenable to empirically based and principled explanation’ (Reference Schur2003: 271). The empirical approach she takes uses a word association task to elicit data representing structural aspects of the lexicon. She interprets her findings as suggesting that the semantic networks of native and non-native speaker participants can be viewed as small world networks. These networks are densely interconnected, so that despite the fact that they contain very many nodes, the pathways between nodes are relatively short (typically six or seven ‘steps’), and distinct clusters and hubs emerge from their apparently random creation (Watts Reference Watts1999). The main contribution of Schur's thesis is its attempt to unpack and challenge an ‘intuitively plausible but fuzzy metaphor’ (Reference Schur2003: 250) and, as such, the thrust of her thesis is not central to our review, but is a good complement to Wolter's attempts to investigate lexical links.

The theses reviewed in this section focus not on the acquisition of words as discrete entities, but on the way in which they are integrated into an existing lexicon. The fact that, as Schur points out, we tend to use metaphor to talk about these lexical connections is testimony to the complexity of the concept. Indeed, McGee finds that some collocations are so firmly integrated into his native speakers’ lexicons, they find it difficult to access and extract constituent elements of them. The final selection of theses approaches the complexity of the relationships between L1 lexis, L2 lexis and conceptual knowledge from a psycholinguistic perspective.

4.3 Organisation of the bilingual mental lexicon

The theses reviewed below are concerned with the role of L1 in L2 processing, and with the ways in which a bilingual's two languages, and their conceptual or semantic knowledge, are organised. Giuffrida (Reference Giuffrida2004) and Wu (Reference Wu2008) are psycholinguists, and the laboratory-based methods they use are informed by psychology research. Namba's (Reference Namba2008) data is natural language, and his approach includes more qualitative analyses. While the research methods selected by these authors are very different, though, the questions underpinning their research have much in common.

Giuffrida (Reference Giuffrida2004) investigates ‘Semantic and language organisation in bilinguals’, using as her framework the predictions of Kroll and Stewart's Revised Hierarchical Model (RHM) (Reference Kroll and Stewart1994). Her studies compare the performance of L1- dominant and L2-dominant bilinguals (L1 Italian, L2 English). First, she measures reaction times on a word translation task, and finds that while L1 > L2 translation is significantly faster than L2 > L1 translation in L2 dominant bilinguals, there is no significant difference in the two translation direction conditions in the L1-dominant group. She includes cognate items in the translation task and her findings support the prediction from the RHM model that translation of cognates bypasses conceptual representation and is therefore faster.

The next set of studies uses a lexical decision task to investigate cross-language priming effects. Words are presented in pairs (L1 + L1, L1 + L2, L2 + L2 and L2 + L1) with half the pairs semantically related. A priming effect is found in all conditions except, for the L1-dominant group, in L1 + L2. These findings contradict previous studies, and Giuffrida explores possible explanations for this. She concludes that when L2 becomes the dominant language, links between L2 words and their semantic representation are strengthened; L1 mediation may only apply for less fluent bilinguals.

Her next two priming experiments investigate links between semantic and lexical representations, using pictures and words. Contrary to her prediction, she finds that neither L1 nor L2 words prime semantically related pictures: in fact she detects a negative priming effect, and suggests that this delay is due to the participant fitting their mental image, elicited by the word, to the actual image in the picture. However, when the order of presentation is reversed (picture then word), again no priming effect is found. In her final picture-based experiment, Giuffrida finds that the L2 dominant group is slower at matching words with pictures in both languages equally. Again, she interprets this as evidence that semantic representation can be accessed from both languages in these more fluent bilinguals. Giuffrida summarises her findings from these studies as indicating that ‘the bilingual memory system is evolving in different stages in accordance with the level of dominance and the general fluency in the second language’ (Reference Giuffrida2004: 71).

In her final experimental chapter, Giuffrida returns to the question of whether cognate words are translated through lexical mediation only. She presented participants with 18 uncommon words, half of which are English–Italian cognates, and asked them to choose, first, the correct translation, then the correct meaning, each time from four possible options. Giuffrida's results support her prediction that cognate words will be translated more accurately than non-cognates, because this process will only use lexical links: the correct translation option was chosen for cognate words even when the meaning was not known. For the non-cognate words, though, the opposite is true: incorrect translations were selected even when the meaning was known. Giuffrida concludes that non-identical cognate words are stored separately in the bilingual memory, but that translation between these items is performed mainly through lexical links. Non-cognate words, in contrast, can only be translated through conceptual mediation.

Giuffrida concludes that her studies support the theoretical framework of the RHM in that L1 mediates access to the concepts, but that this is only the case up to a certain level of fluency. Her results, she claims, show that language dominance affects cognitive procedures adopted by bilinguals when they access semantic features. She offers adapted versions of the RHM, with links between the three basic components (L2 lexicon, L1 lexicon and semantic features) changing in terms of strength and direction depending on language dominance. She concludes with a plea to researchers to pay attention to language dominance, and to regard bilingualism as dynamic and developmental.

Giuffrida's experimental work is thorough and methodologically robust, and her challenge to the predictions of the RHM is principled. While the distinction between ‘language dominance’ and ‘language proficiency’ is a fine one, it is justified by her focus on activation and performance, rather than latent knowledge. The status of cognates in the RHM and in similar bilingual activation models is certainly complex, and her study goes some way towards unpicking this. However, an approach rooted in applied linguistics rather than psychology would investigate the role of form-based guessing strategies among the participants in her cognate-focussed experiments.

Wu's thesis, ‘The nature of cross-language activation in late Chinese–English bilinguals: a behavioural and event-related potential investigation’ (Reference Wu2008), also investigates the operating principles underlying bilingual word processing, focussing on the level and nature of L1 activation during L2 processing. From his review of existing models of the bilingual lexicon, Wu compiles a list of questions still to be answered, including: a) what is the nature of cross-language lexical interaction in languages which have radically different orthography (English and Chinese)? and b) what is the nature of the activation in the non-target language (phonological or orthographic)? Wu's methodology is informed by experimental cognitive psychology, and is based on ERPs (event-related potentials), an observational technique increasingly used in neuroscience. Wu gives a balanced description of how ERPs work and how they have been applied in bilingualism research, explaining that the ERP detects electrical activities in the brain by measuring voltage changes on the participant's scalp, and observing that the main advantage of ERP use is that they ‘reflect the complexity of cognitive processes from the onset of stimulus presentation up to the response and beyond’ (Reference Wu2008: 45).

Wu then presents three studies which compare English language processing in monolingual English speakers with that of English–Chinese bilinguals (L1 Chinese). The studies are designed to necessitate the processing of different categories of L2 word pairs. For example, study one includes (i) words which are semantically related (e.g. post – mail) and whose Chinese translations have a repeated character (You Zheng – You Jian); (ii) words which are semantically unrelated (train – ham) but whose Chinese translations have a repeated character (Huo Che – Huo Tui); (iii) words which are semantically related (love – rose) but whose Chinese translations have no repeated character (Ai Qing – Mei Gui); and (iv) words which are not related semantically and whose Chinese translations have no repeated character (apple – table, Ping Guo – Zhuo Zi). The words were presented in pairs, and participants indicated with a keystroke whether the two words were related or not. ERP data and reaction time data were collected throughout these activities.

In the first study, repeated word-form elements were both phonological and orthographic. The second study distinguished between repeated phonology and repeated orthography in order to tease apart the respective roles of these representations in L1 mediation during L2 word processing. In the third study, participants were presented with picture cues, and had to decide, in the English condition, whether the names of the pictures rhymed, and in the Chinese condition, whether a character was repeated. Wu claims four main findings for his research. Firstly, he finds that ‘translation equivalents in the L1 are accessed when bilinguals read, listen, and retrieve the phonological form of words in L2’; secondly, ‘cross-language activation is automatic, unconscious, simultaneous to semantic access, and mediated by the phonological route during reading and listening’; thirdly, he finds that cross-language activation is ‘delayed, conscious, and unidirectional (L2 to L1) during picture naming tasks’; and finally, he finds that ‘bilingual individuals can access conceptual representations from pictures without accessing corresponding lexical labels in either language, an ability that is unseen in the monolingual individuals’ (Reference Wu2008: 151–152). Wu's study is exemplary in that it uses a relatively new methodology in a balanced and principled way, addressing precise research questions and using each set of findings to progress logically to the next experiment. He offers clear answers to his questions about cross-language activation, but chooses not to discuss potential implications of his findings for language learning and teaching processes.

Namba (Reference Namba2008), like Wu, is interested in the interaction between languages in bilingualism, but his study differs greatly from Wu's in both the method of investigation and the bilingual profiles of participants. Namba's thesis, ‘English–Japanese bilingual children's code-switching’, centres on conversational data from two young (aged 3–9) English–Japanese bilingual siblings. The children have Japanese input from their father and English from their mother, and, during the period of the case study, lived alternately in Japan and the UK. Namba's specific focus of analysis is the bilingual clause: the occurrence of code-switching within one clause. He first analyses his data using the framework of insertional code-switching, but finds instances where this cannot be applied. He concludes that code-switching is a complex phenomenon, and can be influenced by psycholinguistic, pragmatic and morphosyntactic considerations, as well as language convergence. He uses Myers-Scotton's Matrix Language Model (Reference Myers-Scotton1993) as a basis for analysis; the model proposes that in potential code-switching contexts, there is a ‘Matrix Language’ which provides the underlying morphosyntactic clause structure, and an ‘Embedded Language’ into which some items are switched. The first unit analysed in the study is single-item insertion, and although Namba notes a number of phrases where the Matrix Language cannot be identified, on the whole his findings comply with the principles of the Matrix Language framework. He moves on to look at the more complex phenomenon of multi-item insertions, or ‘Embedded Language Islands’ (Myers-Scotton Reference Myers-Scotton1993). In these ‘Island’ insertions, the syntactic framework of the Matrix Language does not apply, and Namba discusses two possible explanations for this: that the Embedded Language insertion carries its own morphosyntactic framework, or that these multi-word insertions should be regarded as fixed phrases. He interprets his data as tending to support this second explanation and follows this up by exploring in more detail formulaicity in code-switching. In order to do this, he first devises a set of criteria by which a language string may be identified as ‘formulaic’. He then analyses formulaic strings in the data, according to propositions by Backus – that code-switching involves holistically-produced units (Reference Backus2003) – and Azuma – that it involves complete syntactic constituents (Reference Azuma1996). Backus’ claim is more strongly supported by Namba's data, and he finds that code-switching occurs either between formulaic sequences, or within the variable ‘slot’ in a phrase frame, but not within the set part of the sequence.

Whereas Giuffrida and Wu are interested in activation of the two languages during receptive language processing, Namba focuses on evidence of a similar sort of dual activation during natural language production. However, unlike the controlled, laboratory-based studies which tend to produce quantitative results, fuzzy boundaries and subjective judgements run through Namba's study. One such fuzzy boundary is that between borrowing and insertion, which Namba claims occupy a continuum rather than being two discrete phenomena. In his conclusion, Namba reflects that it is not possible to generalise from his findings, as they are based on one pair of siblings. He regards as limitations the facts that there is evidence of developmental changes in the children's language use, and that he has only analysed bilingual clauses in his study. He also recognises that applying the criteria to identify formulaic strings requires some subjective judgement by the researcher. He does not comment on the advantages or disadvantages of studying one's own children in this kind of research (he is the father of the two study participants), but the content of the thesis implies that this is more advantageous than otherwise, with the author able to give precise information about the language input received by the children, and their various interactional behaviours.

The theses reviewed above present an interesting juxtaposition: Wu and Giuffrida examine the effect of L1 on language processing; Namba's approach examines evidence from language performance. The approaches, methods and conclusions of the first two authors are clearly influenced by the psychology traditions in which they have been trained. Namba's study is an example of an applied linguist's approach to the same sort of question.

5.1 Failing the learner

A lot of the research reviewed here carries the rather sobering message that education systems, and educators, are failing learners in a number of ways. All the theses mentioned in section 2 highlight problems in teaching methods, teaching materials or provision of resources which disadvantage certain groups of learners. Most of these authors, one guesses, were aware of these problem areas at the outset of their Ph.D. study, and used their research to enable them to identify, explore, and in some cases address, the issues more precisely. Emery and Pemberton, though, come to criticise teaching methods only after long investigations into the causes of persistent language errors which they have observed, and only after eliminating a number of other potential causes. For Zhao and Li, it is the assessment of learners which most needs revision; they show that current methods of assessing the spoken English of Chinese learners are neither useful nor accurate. Some of these theses are dedicated to the identification of a particular teaching problem, where others propose specific solutions. Zhao, in her thesis, makes an impressive attempt to do both, and to evaluate the solution she proposes.

It is worth mentioning here two other theses which recognise problems with the education experience of learners, but which, in addressing these, put the emphasis back on the learner. Sappapan (Reference Sappapan2007), like Emery, is concerned with the low English reading proficiency of students on a university foundation course in Thailand. Her solution to this problem, though, focuses on reading strategies rather than fundamental phonics skills. Her study, based on pre- and post-intervention tests, questionnaires, verbal reports and logs, compares two learner groups, one of which receives reading strategy training, and concludes that such training results in increased reading proficiency. Alexiou (Reference Alexiou2005) takes a concept which is most commonly applied to adult learners – language aptitude – and explores the way in which it might be applied, and measured, in young learners. She devises cognitive tasks suitable for these children, which target memory, classification, spatial ability, etc., and finds that scores on these tasks predict, very broadly, foreign language scores. Alexiou's study is somewhat weakened in that she does not seem to tackle the problematic question in aptitude testing of whether scores across these tasks are truly comparable. Her main claim, though, is that aptitude is not fixed but rather, in young learners at least, is a malleable quality. Alexiou does not venture to detail the educational implications of these findings, but the inference here is that young learners can (and presumably should) be trained in some generic way to be better language learners.

5.2 Complexity

One theme running through most of these theses is the bewildering complexity of language acquisition processes. Paradoxically, it is in the theses which focus on the more minute and specific aspects of language acquisition (e.g. Barfield, McGee, Giuffrida, Namba), that this complexity most clearly emerges. Miller's ‘Vocabulary and reading medical texts’ (Reference Miller2003) is also extremely specialised in its focus (and therefore did not fit comfortably into the sections above), setting out to answer the very clear and specific question: how important is vocabulary knowledge for Spanish speakers reading medical texts in English? Like other theses reviewed here, Miller describes a series of experiments designed to investigate the roles of different, carefully manipulated, variables. For Miller, though, each set of findings reveals another layer of complexity, so his ultimate conclusion is that ‘text comprehension (is) a complex process involving the interaction of multiple components’ (Reference Miller2003: ii). Miller reflects usefully on the general implications of his own research experience, and emphasises the importance of an incremental approach to increased understanding, suggesting that ‘research should be concerned with the steady elaboration of increasingly complete theoretical models’ (Reference Miller2003: 228), and that ‘sporadic, one-off studies’ cannot usefully address the inevitably complex nature of any aspect of language acquisition.

The majority of theses relevant to this review are concerned with lexical issues. As mentioned above, lexis is perhaps an attractive area of study because it initially seems less complex than grammar or phonology, and word knowledge seems relatively easy to measure (words are countable, and we have convenient lists to tell us how frequent they are, at what age they are acquired, what they collocate with, etc.). The studies reviewed in 4.1 and 4.2 above, though, soon dispel these myths. We find that word knowledge is partial, and incremental, and messy, and that knowing a word does not necessarily equate with using it correctly. When it comes to measuring word knowledge, Kamimoto finds guessing behaviour to be a confounding influence, and Fitzpatrick finds that productive vocabulary knowledge, which had seemed a straightforward concept at the beginning of her study, is in fact itself a complex construct. Collocation is perhaps one of the most perplexing aspects of lexical study. McGee, Barfield and, to an extent, Wolter, present thorough investigations into aspects of collocation, but all conclude with lists of (well-informed) questions for further study. For Giuffrida, cognates are the lexical phenomena she struggles to fit into her models.

The heartening thread running through these studies, though, is that the authors seem to embrace the complexities they are exposing. There is an underlying impression here that, perhaps unlike the first generations of applied linguists (it is, after all, still a very young discipline), these researchers recognise that their contributions to the field are small and incremental, and that the complexities they reveal will take many years of research to unpack.

5.3 Interdisciplinarity and bilingualism

The objective of this paper is to review theses from Welsh universities, within the given time frame, that are relevant to language learning and teaching. Through my selection process I found theses which had been produced in departments of Linguistics, Applied Linguistics, Communication, Psychology and Education. This is encouraging: it means that questions relevant to language teaching are being addressed by researchers trained in a variety of fields and methods. While Ph.D. study is essentially an individual pursuit, one might hope that current incentives for interdisciplinary post-doctoral projects mean that these researchers will have the opportunity to inform each others’ expert knowledge in future work.

Wales is a bilingual country, which is perhaps why here, especially, bilingualism pervades so many different fields of research. Although (disappointingly in my opinion) no theses within this period addressed Welsh language teaching per se, the bilingual context is an evident influence on much of the work which takes place here. We can perhaps expect to see this influence become more apparent as, for example, the ESRC Centre for Research on Bilingualism in Theory and Practice (Bangor University) – a truly interdisciplinary centre – begins to produce Ph.D. theses, and as the Welsh Assembly ‘Welsh for Adults’ initiative gains momentum. In the meantime, though, we should probably note that of the 25 theses mentioned here, all but three have English as their ‘learner’ language.