3.1 Main claims about the relationships between WA, word knowledge, and general proficiency

Despite a frustrating lack of reliably consistent findings in WA research, a number of claims about the way WA relates to proficiency have been generated. The strength of these claims varies, and some findings are contradictory (marked ! below). Later in this paper we examine in more detail the methodological approaches and analyses underlying these claims. First, though, we note some indicative claims, with examples of studies supporting them:

• • Number of responses produced

  • ○ As learners become more proficient, there is an increase in the number of responses they produce to cues in a multiple-response WA task (Lambert, Reference Lambert1956; Randall, Reference Randall1980; Zareva, Schwanenflugel, & Nikolova, Reference Zareva, Schwanenflugel and Nikolova2005).

  • ○ As learners become more proficient, they will increasingly resemble L1 speakers in the number of responses they provide to specific cues (on the basis that some cues are more ‘provocative’ – generate more responses – than others) (Lambert, Reference Lambert1956).

  • ! Vocabulary measures correlate more strongly with the number of responses learners produce than with stereotypy scores (Zareva, Reference Zareva2005).

  • ! Proficiency measures (cloze test and TOEIC scores) correlate less strongly with the number of responses learners produce than with stereotypy scores (Munby, Reference Munby2018).

• • Similarity to L1 norms (also referred to as stereotypy, native-like commonality)

  • ○ As learners become more proficient, most produce slightly more responses that are the same as those of L1 speakers (Randall, Reference Randall1980; Schmitt, Reference Schmitt1998b; Fitzpatrick, Reference Fitzpatrick2012).

  • ! Similarity to L1 norms correlates moderately with cloze test scores (Kruse, Pankhurst, & Sharwood Smith, Reference Kruse, Pankhurst and Sharwood Smith1987; Wolter, Reference Wolter2002), TOEIC scores (Munby, Reference Munby2018) and vocabulary knowledge scores (Zareva, Reference Zareva2005).

  • ! There is no significant correlation between similarity to L1 norms and grammar monitoring test scores (Kruse et al., Reference Kruse, Pankhurst and Sharwood Smith1987).

  • ! Similarity to L1 norms could distinguish between L1 speakers and learners, but could not detect differences in proficiency level (Zareva, Schwanenflugel, & Izura, Reference Zareva, Schwanenflugel and Nikolova2005; Zareva & Wolter, Reference Zareva and Wolter2012).

• • Weighted and unweighted scoring systems

  • ○ Scoring systems that account for the position of a response on L1 norms lists (weighted scoring) are not significantly more sensitive to learner proficiency than those that don't (unweighted scoring) (Kruse et al., Reference Kruse, Pankhurst and Sharwood Smith1987; Wolter, Reference Wolter2002).

• • Reliability of WA measures

  • ○ There is a moderately strong test-retest correlation for number of responses produced and more modest correlations for stereotypy (Kruse et al., Reference Kruse, Pankhurst and Sharwood Smith1987).

• • Receptive knowledge of WA (as determined by the word associates format (WAF) approach, whereby participants identify items that are associates of the cue/target)

  • ○ Rasch analysis indicates that the WAF test is reliable (high person separation reliability) (Read, Reference Read and Kunnan1998).

  • ○ There is a strong correlation between WAF scores and reading and vocabulary test scores (Read, Reference Read1993, Reference Read and Kunnan1998; Qian, Reference Qian2002; Qian & Schedl, Reference Qian and Schedl2004).

  • ! High and low WAF scores correspond with performance in a vocabulary-focused interview (Schmitt, Ng, & Garras, Reference Schmitt, Ng and Garras2011).

  • ! WAF performance can be contradictory, with learners selecting a combination of correct and incorrect responses to some items (Schmitt et al., Reference Schmitt, Ng and Garras2011).

3.2 WA stereotypy research in L2 – methodological approaches

A significant proportion of L2 WA research focuses on comparing learners’ responses with those of expert users of the language – in most cases these being L1 speakers. In this section we examine the methods used in studies that attempt to assess learner proficiency by examining the stereotypy (likeness to L1 speaker norms) of the associations they produce.

Measures of WA stereotypy are dependent on norms lists – lists of responses given to cue words by particular participant groups, ranked by response frequency. These can be used to investigate properties of particular words (e.g. black is considered to have a dominant primary response, because it so often elicits white), or of specific populations (based on, for example, age, gender, occupation, education, cultural background). Norms lists commonly used in applied linguistics WA research include:

• • the Postman and Keppel collection (Reference Postman and Keppel1970)

• • the Edinburgh Associative Thesaurus (Kiss, Armstrong, Milroy, & Piper, Reference Kiss, Armstrong, Milroy, Piper, Aitkin, Bailey and Hamilton-Smith1973)

• • the University of Florida Free Association Norms (Nelson, McEvoy, & Schreiber, Reference Nelson, McEvoy and Schreiber2004)

• • the Small World of Words Norms lists in English and (so far) 11 other languages (databases for most of the latter are rather small at the time of writing, but the collection project is still live; De Deyne et al., Reference De Deyne, Navarro, Perfors, Brysbaert and Storms2018).

The predominance of English lists here prompts us to note two things: (1) valid, peer-reviewed WA research in languages other than English is relatively hard to find, and (2) if Meara is correct in his suggestion that English has ‘particularly high levels of stereotypy compared to other languages’ (Reference Meara1980, p. 234), then norms lists in English are likely to be more reliably indicative of a population's response patterns than those in other languages. A norms list, much like a language corpus, is only as useful as its provenance and its fit with the target research question. Fitzpatrick, Playfoot, Wray, and Wright (Reference Fitzpatrick, Playfoot, Wray and Wright2015) demonstrate that researchers’ choice of norms list can influence stereotypy-related WA scores. Their study focused on (L1) WA responses to one hundred cues by participants in two distinct age cohorts: 16-year-olds and over 65s. Participants within each cohort were split into two experimental groups (16A, 16B, 65+A, 65+B) in a manner that matched exactly for age (since the participants were twin pairs), and a ‘norms’ list was compiled for each of the four groups. Each participant's response set was then scored against each of the four norms lists, with a point awarded for every response that appeared at the top of the norms list. Thus, four separate norms list scores were generated for each participant. The mean score from the ‘same age group’ norms was significantly higher (26 out of 100) than those calculated from the two ‘other age group’ norms (19 and 19.2 out of 100), indicating that norms list data is influenced by age (or generation). Even before that study, many researchers had chosen to create their own bespoke lists to match the characteristics of their target population, but the Fitzpatrick et al. (Reference Fitzpatrick, Playfoot, Wray and Wright2015) findings offer explicit evidence that norms list selection can critically affect stereotypy scores. This contributes to uncertainty about the validity of the ‘L1 speaker norm’ construct, as discussed below (see also Zhang & Koda, Reference Zhang and Koda2017).

Insofar as the performance of an L1 speaker can be considered a benchmark against which to measure L2 proficiency, the expectation underpinning this stereotypy research – that the more proficient the learner, the closer their WA responses will be to the norms of L1 speakers of their target language – seems, on the face of it, a reasonable one. Two developmental considerations seem to support this: first, as proficiency progresses, responses are likely to reflect knowledge of multiple meanings and connotations of a word; second, in cases where response norms are translation equivalents across languages, the production of an L1 speaker-like response indicates that the learner has acquired the appropriate item in their L2. Nevertheless, despite promising early forays into this line of enquiry (Meara, Reference Meara1978; Randall, Reference Randall1980), researchers have been confounded in attempts to find predictive relationships between L1 speaker-like responses and L2 proficiency. Wondering whether this problem is related to a WA protocol issue, many have tried adjusting various features of the task design, including:

• • WA cue words – attention has been given to word class, frequency, propensity to elicit strong dominant responses (or not), and the number of cues used.

• • Instruction – the number of responses requested for each cue has been varied.

• • Scoring system – options include awarding ‘stereotypy’ points (a) for any response in the top e.g. three in the norms list; (b) for the percentage of the norming population giving the response; (c) according to the ranking of the response on the norms list; (d) for any response that appears anywhere in the norms list; (e) for a response that is the dominant response on the norms list (list adapted from Fitzpatrick et al., Reference Fitzpatrick, Playfoot, Wray and Wright2015, p. 33). In much of the literature, (b) and (c) are referred to as weighted stereotypy scores, and the others as unweighted.

In a proof of concept paper intended to establish a research tool for measuring learner proficiency, Schmitt (Reference Schmitt1998a) uses L1 speaker data to address problems he sees as inherent in WA protocols to date. He considers that (1) the conventional requirement of one WA response per cue is inadequate for demonstrating word knowledge, and (2) production of a common (i.e. frequently produced) L1 speaker response might indicate a different (higher?) degree of word knowledge than production of an uncommon/idiosyncratic L1 speaker response.

Schmitt's proposed solution was to ask for three responses per cue, and from these to compile a norms list for 17 words from one hundred L1 English speakers. He was concerned about variation in response dominance of cue words (some cues attracted the same response more often than others), and devised a complex graded score system to account for this, whereby a ‘maximum’ notional score was calculated for each cue. Participants’ scores were calculated by adding up the number of norms contributors who had given each of the participant's responses, and expressing this as a proportion of the maximum possible score. Schmitt calculates a mean proportion score for L1 speakers, but finds it challenging to identify a ‘threshold’ score at which learner responses become native-like; he resolves this by identifying four calculable levels of ‘nativelikeness’ in response behaviour. In a follow-up study (Schmitt, Reference Schmitt1998b), he tracks development of learner vocabulary using this scoring method, but results suggest backsliding and lack of progression (towards L1 norms), indicating that the method is still not fit for purpose. However, Schmitt (Reference Schmitt1998a) provided a useful platform for subsequent WA research, articulating clearly a number of WA characteristics that have subsequently been taken up for examination: variability of cue words’ propensity to attract dominant responses, variability in L1 speaker responses, and the elusiveness of a conclusive definition of ‘native-like response’.

Taking on board the problem of variable cue word behaviour, Wolter (Reference Wolter2002) selected cues whose responses were heavily weighted towards the first three responses in the norms list data (i.e. a high proportion of respondents gave at least one of those three responses). He too asked for three responses to each cue. For his learner and L1 speaker participants he calculated weighted scores (accounting for the number of norms list contributors who gave that response) and unweighted scores (a point for any response also on the norms list). Although group means saw L1 speakers score higher than learners, some L1 participants scored below the learner mean. Correlations with a proficiency measure were moderate (less than 0.5 for both scoring methods). Like Schmitt, Wolter considers WA still to hold promise as a proficiency measure, and calls for three revisions to WA protocols in order for their potential to be fulfilled:

• • Avoid cue words that are delexicalized (i.e. have little inherent meaning, and are usually used with a noun phrase; e.g. make, get) or that elicit personal experience (e.g. travel).

• • Exclude from analysis the lowest scoring response to each cue word, and/or prompt words that tend to produce low scores; this would reduce the confounding effects of individualized responses.

• • Use a more fitting proficiency comparison measure.

Together with Zareva, Wolter returns to the challenge of WA analyses (Zareva & Wolter, Reference Zareva and Wolter2012) with an evaluation of three measures: associative commonality, lexico-syntactic patterns, and collocative analysis. The first of these is relevant to this section of our paper, addressing the question ‘At the higher levels of proficiency … to what extent do learners’ primary responses to familiar vocabulary correspond with native speakers’ associations?’ (Reference Zareva and Wolter2012, p. 47). Using a principled methodology, the authors categorized responses of L1 speakers, advanced and intermediate learners against L1 speaker norms, as ‘nativelike common, nativelike idiosyncratic, and non-nativelike idiosyncratic’ (Reference Zareva and Wolter2012, p. 51). They found significant differences between scores of L1 speakers and learner groups, but no difference between the intermediate and advanced learners; both produced a higher proportion of idiosyncratic than common, native-like responses. The authors conclude that this method of scoring was not particularly promising. An earlier study by Zareva and colleagues (Zareva, Reference Zareva2005; Zareva et al., Reference Zareva, Schwanenflugel and Nikolova2005) had also found that comparison with L1-derived norms could distinguish between response data from learner and L1 participants. However, other ways of measuring the response data (number of associations generated, within-group consistency of associations) proved more sensitive to differences in proficiency than did L1 stereotypy. They suggest that extralinguistic factors, including cultural drivers, affect associative links, and that the native-likeness score cannot therefore be seen as a clean reflection of proficiency.

While the above studies focused on scoring protocols as the key to extracting proficiency information from WA responses, other studies focused on cue word selection and norms list compilation. Munby's work is a key contribution here (Reference Munby2011; Fitzpatrick & Munby, Reference Fitzpatrick, Munby, Milton and Fitzpatrick2014; Munby, Reference Munby2018). His starting point was a scrutiny of Kruse et al. (Reference Kruse, Pankhurst and Sharwood Smith1987), who had investigated the viability of WA as a measure of proficiency, comparing responses of 15 Dutch learners of English with those of seven L1 speakers, using a multiple-response protocol: participants provided up to 12 responses to each of ten cue words. Kruse et al. used three measures: number of responses given, weighted stereotypy, and non-weighted stereotypy (using Postman & Keppel Reference Postman and Keppel1970). No strong correlation was found between any of these scores and proficiency measures (the highest was r = 0.576), nor did they find any clear difference between scores of learners and the L1 speaker control group. The authors conclude that ‘contrary to the expectations raised by earlier studies, we find that word association tests do not show much promise for the specific role created for them in L2 research’ (Reference Kruse, Pankhurst and Sharwood Smith1987, p. 153). This relatively small study seemed to temporarily stop proficiency-related WA research in its tracks (Meara, Reference Meara2009, p. xi); little research was conducted on this topic for a decade following its publication.

Scrutinizing the methodological detail in Kruse et al. (Reference Kruse, Pankhurst and Sharwood Smith1987), Fitzpatrick and Munby (Reference Fitzpatrick, Munby, Milton and Fitzpatrick2014) note as potentially problematic (1) the selection of cue words, and (2) the scoring of multiple responses against norms lists that had been created from single responses. They set out to create a new version of the Kruse et al. test, using ten cue words painstakingly selected for maximum sensitivity to proficiency, and creating a bespoke multiple-response norms list from 114 L1 speakers of English. Seventy-one Japanese learners of English gave up to 12 responses to the new and the original Kruse et al. cues, and for each set of cues, an unweighted stereotypy score (using the Postman Keppel norms lists for the original Kruse et al. cues, and the bespoke norms list for the new cues) was compared with performance on three proficiency tests. Correlations between proficiency tests and scores from the original cues were similar to those reported in Kruse et al., but response stereotypy for the authors’ new cue set yielded a correlation of 0.7 (P < 0.01) with TOEIC scores.

Munby's next step was to question the use of L1 speaker norms lists in these studies. He and colleagues (Racine, Higginbotham, & Munby, Reference Racine, Higginbotham and Munby2014) make a strong argument against the use of L1 speaker norms as a benchmark for learners’ WA performance, on the basis that (1) L1 speaker responses are not homogeneous; (2) socio-cultural and demographic differences between L1 speakers and learners hinder linguistic comparability (this echoes Zareva et al.'s (Reference Zareva, Schwanenflugel and Nikolova2005) concerns about extralinguistic factors); and (3) such measures rest on the assumption that an L1-like variety of English is the learner's goal. Munby (Reference Munby2018) addresses this by compiling two separate norms lists for 50 cues: one from L1 English speakers (n = 114) and one from L1 Japanese speakers who were highly proficient users of English (n = 114). Eighty-two L1 Japanese learners of English then completed the WA task, giving multiple responses to each of 50 cues. Their responses were awarded two scores, according to whether they were on (1) the L1 and (2) the L2 norms lists. Learners scored significantly higher on the L2 than the L1 norms lists; Munby suggests this is perhaps because of shared cultural background. However, the marginally stronger – though still moderate – correlations with proficiency came from the L1 norms scores. Munby suggests this could be due to a native variety bent in proficiency tests, or to the fact that both proficiency and L1-like response behaviour are positively influenced by increased exposure to L1 English input.

It is our assessment that despite the sophisticated and dogged attempts by these researchers to find a method of stereotypy scoring that can reliably evaluate learner proficiency or word knowledge, this is likely to be an unattainable goal (though further calibrations of cue words and scoring protocols may chip away at statistical findings). We suggest two fundamental reasons for this. The first is a paradox connected with language development: the more proficient a learner becomes, the more lexical items are available as potential responses in a WA task. However, the more proficient s/he becomes, the more honed and precise word selection can be, meaning fewer items present themselves as candidates for a WA response. It is possible that these things affect learners at different stages of development (vocabulary grows, then becomes more honed); production of native-like responses is therefore unlikely to operate on a steady cline.

The second reason for our assertion that stereotypy-based evaluations of L2 proficiency are unlikely to succeed is derived from usage-based theories of language development (e.g. Bybee & Beckner, Reference Bybee, Beckner, Heine and Narrog2010; Bøyum, Reference Bøyum2016; Ellis, Römer, & O'Donnell, Reference Ellis, Römer and O'Donnell2016; Thwaites, Reference Thwaites2019, and see Section 5.3). These theories assert that an individual's linguistic system is the result of their unique experience with language. If this is the case, then it follows that the WA responses of learners, whose L2 experiences are often largely based on tightly controlled snippets of language found in classroom contexts, are unlikely to reflect the rich and diverse linguistic experience of an L1 speaker of the same language.

The next area of WA research we consider avoids managing unwieldy and unpredictable learner data sets, by examining not the associations a participant produces, but the associations s/he recognizes from a restricted set of options.

3.3 Word associates format (WAF) – testing depth of word knowledge

Whereas the stereotypy research strand examines WA responses generated by participants, the WAF presents participants with a set of predetermined items, and requires them to identify which are associated with the stimulus word. In this sense the WAF is rather different from WA models we focus on elsewhere in this paper: it does not ask for words to be produced, but rather for connections between words to be identified. This means that item cues, associates, and distractors can be manipulated according to research or testing aims. In terms of word knowledge, the focus is firmly on the stimulus word – it is the depth of knowledge of that word that is being investigated. Target associates are typically selected on the basis that they have collocational or definitional relationships with the stimulus, and so this kind of task can explicitly target aspects of word knowledge relating to meaning and syntactic relationships, and to polysemous uses of a target item. Because the WAF has verifiably ‘correct’ answers, and because teachers/testers can select target items according to curriculum needs, the WAF is useful to practitioners as well as to researchers, and is often proposed as a depth of (word) knowledge test. Zhang and Koda (Reference Zhang and Koda2017) offer a comprehensive review of WAF, and in particular its validity as a research tool.

The WAF was pioneered by Read (Reference Read1993, Reference Read1995, Reference Read and Kunnan1998, Reference Read2000, pp. 178–187), and he has worked systematically to develop it into an effective tool for assessing learners’ knowledge of the meaning(s) and collocations of vocabulary items. In the last 20 years the WAF has been taken up by other researchers, who have further refined it, and applied it to data from a variety of languages and age groups.

In the original version of Read's test (Reference Read1993), each stimulus word was presented alongside eight associates and distractors. The test-taker's task was to identify the associates, which could be ‘paradigmatic’ (defined by Read as wholly or partially synonymous), ‘syntagmatic’ (collocates), or analytic, which Read defines as representing one aspect or component of the target word, and likely to form part of its dictionary definition (Reference Read1993, p. 181). For the example stimulus word edit, the associates/distractors are arithmetic, film, pole, publishing, revise, risk, surface, text. The intended correct associates are revise (paradigmatic), film, text (syntagmatic), and publishing (analytic).

Trials found a strong correlation with a word definition test, and Rasch analysis of test items indicated a good general level of reliability. However, post-task interviews revealed several issues which led Read to make a number of revisions to the format. One change of particular significance was the presentation of the eight associates and distractors in two distinct sets – paradigmatic and syntagmatic. Participants selected the four target associates (these were not necessarily distributed evenly across the two categories). Read's walk-through account of this process meticulously explains the complex set of decisions involved in test development (Reference Read2000, pp. 180–187).

While refinements continue to be made to the WAF, it is important to note that studies have generally shown that scores on WAF tests can predict performance on linguistic tasks such as reading comprehension; the Depth of Vocabulary Knowledge (DVK) studies are important contributions in this regard (Qian, Reference Qian2002; Qian & Schedl, Reference Qian and Schedl2004). In addition, Schmitt et al. (Reference Schmitt, Ng and Garras2011) found that WAF-style tests correlate closely with measures of depth of word knowledge derived from post-task interviews. These correlations, and those between the WAF and proficiency measures reported by Read, Qian, and Qian and Schedl are stronger, on the whole, than those obtained in the stereotypy studies reported in the previous section, and might indicate a more straightforward relationship between the two, or at least one with fewer confounding variables at play. We note, however, Horiba's (Reference Horiba2012) finding that for Korean (but not for Chinese) L1 learners of Japanese, scores on the WAF test did not explain any additional variance on reading comprehension tasks than that which was explained by vocabulary breadth tests. For further discussion of this and the possible interaction with orthography it implies, see Zhang and Koda (Reference Zhang and Koda2017).

Around the same time as Qian developed the DVK test, Greidanus and Nienhuis (Reference Greidanus and Nienhuis2001) created a WAF test for Dutch L1 learners of French, and reinstated the analytic set of associates/distractors originally proposed by Read. Greidanus and Nienhuis focus on the nature of the distractors, comparing performance on items with semantically related and semantically unrelated distractors. An example of the former is the item Fr. rive ‘bank’, with associates/distractors artificiel ‘artificial’, bord ‘edge’, côté ‘side’, fleuve ‘river’, gauche ‘left’, vague ‘wave’. Intended correct associates are bord (paradigmatic), gauche (syntagmatic), and fleuve (analytic). Their participants found the items with semantically related distractors challenging, and the authors conclude that these are therefore the appropriate kind to use; they also make guessing harder. Schmitt et al. (Reference Schmitt, Ng and Garras2011) make a similar comparison – between use of meaning-related and non-related distractors – and find an interaction with the number of options participants had to select from. Zhang and Koda conclude that the impact of distractor type should be considered alongside other influencing variables (Reference Zhang and Koda2017, p. 14). These findings and their interpretation give us pause, because in any language encounter, including a WAF task, we are driven to make meaning, and in order to suppress the sense of a connection between, for example, rive ‘bank’ and vague ‘wave’ (we might visualize a wave lapping against a riverbank), it is perhaps necessary to employ analytic, metalinguistic skills. This is not in itself problematic, but it makes it difficult to be sure about whether knowledge of a language or knowledge about language is being tested.

Dronjic and Helms-Park also express concern about the type of knowledge being targeted in WAF tests; they consider the DVK to be a ‘highly metalinguistic test’ (Reference Dronjic and Helms-Park2014, p. 211). Their paper resonates with the introduction to this State-of-the-Art piece, by calling for lexical acquisition studies to be better informed by psycholinguistic research. Specifically, they argue that in order to understand how WA tasks work, more information is needed about (1) lexical links in stable (L1) and developing (L2) networks; and (2) the nature of lexical activation in a WA context. On the basis that any test used with L2 learners should be able to produce consistent responses from L1 users, they analyse WAF data produced by L1 English undergraduates. Even in this relatively homogeneous group, they find high levels of variation in the identification of associations, particularly syntagmatic ones. They conclude that this lack of consistency in L1 data, together with concerns about differences between lexical activation in WA tasks and in natural speech production, means that WA tasks are unfit for the evaluation of learner word knowledge, and should be used for exploratory research purposes only.

Mulder and Hulstijn (Reference Mulder and Hulstijn2011) conduct such an exploratory investigation in their critical examination of the notion of native speaker proficiency. They use an abridged WAF test along with a series of other tasks, to examine lexical knowledge and availability in adult L1 speakers of Dutch in three age ranges (18–35, 36–50, 51–76). Participants were presented with just one associate and one distractor for each target item, and were required to choose one. The researchers recorded an additional measure, response time, and found that older participants responded more slowly but with equivalent accuracy to the younger participants. They conclude with cautionary remarks about the use of L1 speaker benchmarks in L2 studies, noting the variability in L1 speaker performance on the tests they administered.

Response-time data is also used in Cremer and Schoonen's (Reference Cremer and Schoonen2013) study of semantic knowledge in bilingual and monolingual children. They measure both response time and accuracy in a WAF task, and compare scores with reading comprehension performance. They identify a difference between lexical availability (word knowledge) and lexical accessibility (fluency) and find that accessibility accounts for some variance in reading comprehension that is not accounted for by word knowledge. Zhang and Koda conclude from these findings that omission of accessibility measures in WAF tests might underplay the predictive power of WAF tests (Reference Zhang and Koda2017, p. 25).

As exemplified in Cremer and Schoonen (Reference Cremer and Schoonen2013), the operational and ecological advantages of WAF-based tests – taking the test is not onerous and does not require language production but merely the circling of words – makes the format particularly suitable for use with young participants. Schoonen and Verhallen (Reference Schoonen and Verhallen2008) capitalized on this, using a WAF test for 9- and 11-year-old Dutch L1 or L2 speakers, in order to assess their vocabulary knowledge. They asked participants to select from six items the three associates that ‘fit with the stimulus best’ or ‘always go together’ with the stimulus (Reference Schoonen and Verhallen2008, p. 218). They give the example of their stimulus item banana and associate/distractors fruit, peel, yellow, slip, monkey, nice (translated from Dutch); the first three are the most fitting associates. The authors find that performance on this test correlates strongly with that on a definition task. The reliability and validity analyses conducted in this study are sophisticated and the authors conclude that their results can guide further investigation into depth of word knowledge in children.

The final study to mention in this section, and one we will return to later, is Henriksen's (Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008) data-rich study of the production and identification of associations in two languages (Danish L1 and English L2) by participants representing three age groups: 13–14, 16–17, and early 20s. Her version of the WAF (she calls it the ‘word connection’ task) is tied more closely to conventional WA work than those described above, in that her associates/distractors are all drawn from existing WA norms lists. For each of her 24 stimulus items (high frequency adjectives from the Kent Rosanoff lists), ten potential associates are provided: five are the most common responses to the stimulus item according to norms list data, and five are only given by one respondent in the norms lists – but are still semantically related to the stimulus. So, for the example stimulus word cold, the following possible associates are given: war, water, frost, hand, hot, warm, snow, pain, winter, ice. The instruction is to indicate the ‘five words they consider to be most strongly related to the stimulus word’. The author suggests that this gives participants ‘greater opportunity to draw on both their conceptual  … . and their meta-semantic knowledge’ than the equivalent productive (conventional) WA task (Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008, p. 43); that productive task, using the same cues as the word connection task, is discussed in Section 4. Henriksen's multi-layered study offers comparison potential across several dimensions: participant-wise, there is an L1 (English) control group, and three groups of Danish L1 students at different ages (therefore educational levels) and stages of L2 (English) development; language-wise, equivalent data is collected from the students in Danish and English; task-wise, participants not only undertake a conventional WA task and the ‘word connection’ task but also essay writing and a lexical inferencing task (reported by Albrechtsen and Haastrup respectively, in the same Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008 volume). This makes for rich pickings for WA researchers, and while the research reported in the 2008 volume likely does not exhaust the data's potential, some insightful observations and careful justifications emerge, not least the following from Henriksen, which relates to questions about L1 speaker benchmarks that arise elsewhere in the current paper: ‘One could ask why the ability to identify or supply a native-like response is seen as a hallmark of a well-developed mental lexicon. The underlying assumption is that native-like associational patterns will reflect the types of conventional access routes or lexical pathways in the mental network available to a fully competent language user in communicative situations’ (Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008, p. 44).

Juxtaposing the various operationalizations of the WAF discussed above enables us to evaluate its contribution to the WA research field. Superficially, the precise instruction and restricted response options lead us to suppose that it is more straightforward to identify the construct being tested here than in the more unwieldy conventional WA tasks, where the potential response pool is enormous. For each target word, manipulation of associates can reveal test-takers’ knowledge of specific collocations or multiple meanings of polysemous words, and so on. However, some studies suggest that different knowledge constructs are targeted in the identification of syntagmatic and paradigmatic (and perhaps analytic) associations (e.g. Greidanus & Nienhuis, Reference Greidanus and Nienhuis2001; Batty, Reference Batty2012; Dronjic & Helms-Park, Reference Dronjic and Helms-Park2014; see also Zhang & Koda, Reference Zhang and Koda2017). This links with considerations of response types addressed in the following section, and rather than being seen as intractably problematic, we suggest that the designed distinction between association types in WAF tests might offer a potential route to unpicking some of the conflicts and white noise associated with categorization of responses in the research discussed in the following section.

We also detect in this WAF research a tension between constructs that goes beyond types of word knowledge. This tension is between (1) intuitive judgements, especially L1 speaker intuition, which has been shown to be unreliable, for example by McGee (Reference McGee2009); (2) knowledge of individual vocabulary items (the stimuli); and (3) metalinguistic knowledge, which Dronjic and Helms-Park claim is key to successful completion of the WAF. Again, the interplay of these different dimensions in WAF task completion might be exploited, perhaps in addressing a question that has emerged quite strongly from this section: that of the extent to which L1 speaker data provides a useful or even valid benchmark for learner WA tasks.

Finally, we note that this research strand is less dominated by English than those described in the previous section; it is likely that the ‘high levels of stereotypy’ offered by English over other languages (Meara, Reference Meara1980) are not so relevant to the format of the restricted choice WAF, which can be manipulated to elicit different layers of knowledge.

4.1 Main claims in category-based WA research

There has been a steady momentum to this strand of research in the last two decades, and despite persistent confounds, some broad patterns of WA behaviour can be detected, and are summarized below. The weight of evidence behind these varies, and occasionally findings are contradictory (marked ! below). It will be noted that in addition to the paradigmatic, syntagmatic, and clang categories mentioned above, the terms meaning-based, position-based, and form-based are used as category labels. These are sometimes considered more user-friendly categories, and are often subdivided further. Specific categorization systems are discussed further in Section 4.2.

Indicative findings from category-based WA research, with examples of studies reporting them, include:

• • In the majority of studies, paradigmatic or meaning-based responses are most prevalent, and clang or form-based responses least prevalent.

• • The likelihood that a participant will produce a paradigmatic response or a meaning-based response is increased:

  • ○ if they know the cue word well enough to use it in a sentence (Wolter, Reference Wolter2001)

  • ○ if the (L2) cue word is relatively familiar (Söderman, Reference Söderman and Hammarberg1993; Wolter, Reference Wolter2001)

  • ○ if they are an expert user (L1 or advanced L2) of the language of the WA task (Zareva, Reference Zareva2007; Jiang & Zhang, Reference Jiang and Zhang2019)

  • ○ the more proficient in L2 they are (Söderman, Reference Söderman and Hammarberg1993; Orita, Reference Orita2002; Zareva, Reference Zareva2007; Zareva & Wolter, Reference Zareva and Wolter2012; Khazaeenezhad & Alibabaee, Reference Khazaeenezhad and Alibabaee2013)

  • ○ if the cue words are nouns (Nissen & Henriksen, Reference Nissen and Henriksen2006)

  • ○ the older they are (Namei, Reference Namei2004)

  • ○ if they are adult (Cremer, Dingshoff, de Beer, & Schoonen, Reference Cremer, Dingshoff, de Beer and Schoonen2011)

  • ○ if they are a heritage L2 speaker (Kim, Reference Kim2013).

• • The likelihood that a participant will produce a clang or orthographic/phonological response is increased:

  • ○ the younger they are (Namei, Reference Namei2004)

  • ○ if they are using their L2 (Wolter, Reference Wolter2001; Namei, Reference Namei2004; Fitzpatrick, Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006; Norrby & Håkansson, Reference Norrby and Håkansson2007; Hui, Reference Hui2011; Jiang & Zhang, Reference Jiang and Zhang2019)

  • ○ if they are learning L2 outside the target language environment (Håkansson & Norrby, Reference Håkansson and Norrby2010)

  • ○ the less proficient in L2 they are (Söderman, Reference Söderman and Hammarberg1993; Orita, Reference Orita2002; Khazaeenezhad & Alibabaee, Reference Khazaeenezhad and Alibabaee2013)

  • ○ if the cue word is relatively unfamiliar or newly acquired (Söderman, Reference Söderman and Hammarberg1993; Wolter, Reference Wolter2001).

• • The likelihood that a participant will produce a syntagmatic/collocational/position-based response is increased:

  • ! if they are using their L2 (Norrby & Håkansson, Reference Norrby and Håkansson2007; Zareva, Reference Zareva2007; Håkansson & Norrby, Reference Håkansson and Norrby2010)

  • ! if they are using their L1 (Fitzpatrick, Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006; Fitzpatrick & Izura, Reference Fitzpatrick and Izura2011)

  • ○ if they are learning their L2 as a foreign rather than second language (Norrby & Håkansson, Reference Norrby and Håkansson2007)

  • ○ the less proficient they are (Söderman, Reference Söderman and Hammarberg1993; Orita, Reference Orita2002; Zareva, Reference Zareva2007; Zareva & Wolter, Reference Zareva and Wolter2012; Khazaeenezhad & Alibabaee, Reference Khazaeenezhad and Alibabaee2013)

  • ○ if the cue words are adjectives (Nissen & Henriksen, Reference Nissen and Henriksen2006).

• • The likelihood that a participant will produce a synonym response is increased:

  • ○ if they are using their L1 (Fitzpatrick, Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006; Fitzpatrick & Izura, Reference Fitzpatrick and Izura2011)

  • ○ the more proficient they are (Khazaeenezhad & Alibabaee, Reference Khazaeenezhad and Alibabaee2013).

• • The likelihood that a participant will produce a loose conceptual association response is increased:

  • ! if they are using their L2 (Fitzpatrick, Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006; but not supported by Kim, Reference Kim2013).

• • The likelihood that a participant will produce a quality-based association response is increased:

  • ○ if they are using their L1 (Kim, Reference Kim2013).

These findings must be considered in context, of course; results that feed into the broad statements above are nuanced according to specific proficiency levels, languages investigated, cue words used and so on, and scrutiny of these and other variables is necessary to avoid further research being driven down cul-de-sacs by false assumptions. Additionally, some of the studies conflate variables that others examine separately; age and proficiency in Orita (Reference Orita2002) and Namei (Reference Namei2004), for example. Zareva's (Reference Zareva2007) identification of items unknown to participants in order that the confounding effect of these might be excluded from analysis is a good but rare example of an attempt to manage data so as to eliminate confounds. Henriksen (Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008) implicitly conveys a warning against assumptions of linear or binary models of WA behaviour, in the finding that learners simultaneously produce more ‘canonical’ (prototypical) and more low-frequency responses as development progresses. Crucially, it should be noted that there are few, if any, precise replications of the studies considered here, and in our view these are crucial in order to weed out true findings from context-dependent ones.

In the next section we unpack some of the methodological differences, challenges and approaches found in this research strand.

4.2 WA response categorization – methodological approaches

Scrutiny of the studies in this research strand reveal differences on multiple dimensions which, as noted above, make it difficult to compare findings across studies. Differences include:

• • the precise definition of the paradigmatic/syntagmatic/clang categories

• • the number of cue words used and the number of responses elicited for each cue

• • whether/how the data is prepared for analysis (e.g. Zareva & Wolter, Reference Zareva and Wolter2012 operate detailed lemmatization criteria)

• • the theoretical models (or absence of) motivating the study

• • whether comparisons are between-subject or within-subject, and if the latter, whether they are longitudinal, comparing participant data as proficiency progresses (e.g. Fitzpatrick, Reference Fitzpatrick2012); or cross-sectional, comparing participants’ L1 and L2 behaviour (e.g. Politzer, Reference Politzer, Honsa and Hardman-de-Bautista1978; Nissen & Henriksen, Reference Nissen and Henriksen2006)

• • whether between-subject comparisons are made on the basis of L2 proficiency level (e.g. Orita, Reference Orita2002), identity as learner or L1 speaker (e.g. Wolter, Reference Wolter2001), or some other criterion

• • whether the focus is on the holistic organization of the lexicon, or on individual lexical items (e.g. Söderman, Reference Söderman and Hammarberg1993; Wolter, Reference Wolter2001)

• • which languages are investigated, and the extent to which features of a specific language affect response behaviour (see Kim, Reference Kim2013 on Korean).

To further confound matters, the reach of this research strand has been (and continues to be) reshaped in terms of both methodology and the target variables. The range of target variables has in some cases been extended (for example to include learning context: foreign language, second language, or heritage language), and in others has been reduced to a focus on particular developmental or organizational features (such as the influence of word form). Most notably, researchers have developed more fine grained, or transparent, or fit-for-(specific)-purpose categorization systems, reflecting the theoretical drivers of their research. In Table 2 we present an overview of some of the most salient and/or cited studies from the last 20 years, organized according to the approach taken to classification of responses.

Table 2.

WA studies using category approaches: a methodological overview (post-2000)

Initial forays into the application of the paradigmatic/syntagmatic classification in L2 research were promising in that findings supported the hypothesis that the paradigmatic > syntagmatic shift can be detected in L2 as well as L1 development. Politzer's (Reference Politzer, Honsa and Hardman-de-Bautista1978) comparison of L1 (English) and L2 (French) WAs, though vulnerable to criticism for the ambitious interpretation of findings, is notable for its relative statistical power (n = 203) and the alignment of findings with the L1 literature: he found proportionally more paradigmatic responses, and fewer clang responses, in the L1 than in the L2 task, and found that scores on a French grammar test correlated positively with the number of paradigmatic responses produced. Following Politzer, the paradigmatic/syntagmatic/clang categorization was applied in a series of L2 studies, with the label ‘phonological’ sometimes used in place of ‘clang’. However, as Osgood et al. (Reference Osgood, Sebeok, Gardner, Carroll, Newmark, Ervin and Lounsbury1954) had predicted, categorization was not straightforward; some studies give detailed explanations of how the classification was operationalized (e.g. Namei, Reference Namei2004; Zareva, Reference Zareva2007), some add an ‘other’ category (e.g. Orita, Reference Orita2002; Nissen & Henriksen, Reference Nissen and Henriksen2006; Yu, Xu, & Sun, Reference Yu, Xu and Sun2011), and several researchers explicitly set out the drawbacks of the paradigmatic/syntagmatic/clang classification. Weaknesses observed include:

• • There is a lack of ‘clear and objective criteria’ for assigning responses to categories (Henriksen, Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008, p. 46).

• • The breadth of the categories may mask information within them (Fitzpatrick, Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006, p. 126; Nissen & Henriksen, Reference Nissen and Henriksen2006, p. 46).

• • The paradigmatic/syntagmatic distinction ‘is very difficult to work in practice, especially when you cannot refer back to the testee for elucidation’ (Meara, Reference Meara1983, p. 30); categorization often entails making assumptions about context or mental mapping (Roux, Reference Roux2013, p. 87).

• • It can be difficult or impossible to assign an association to (only) one of paradigmatic or syntagmatic (see e.g. Maréchal, 1995; cited in Singleton, Reference Singleton1999, p. 234; Fitzpatrick, Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006; Nissen & Henriksen, Reference Nissen and Henriksen2006).

• • Hyponymous semantic relationships are typically paradigmatic, but are sometimes represented by words of different classes (some categorization systems explicitly address this – see Wolter, Reference Wolter2001, p. 52 and Namei, Reference Namei2004, p. 372).

• • Depending on the way in which the category distinction is operationalized, some associations do not fit into any of them (for example, idiosyncratic associations, which are often emotional or experiential; Henriksen, Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008, p. 46).

• • Category of association depends in part on word class; ‘nouns seem to be predominantly organised in paradigmatic relations, whereas verbs and adjectives are characterised by syntagmatic relations’ (Nissen & Henriksen, Reference Nissen and Henriksen2006, p. 46).

These observations have led to the augmentation of the paradigmatic/syntagmatic/clang system in some studies, and the application of alternative classification systems in others. Augmentations have included the addition of a translations category (Norrby & Håkansson, Reference Norrby and Håkansson2007; Håkansson & Norrby, Reference Håkansson and Norrby2010) and extensive use of sub-categories (e.g. Khazaeenezhad & Alibabaee, Reference Khazaeenezhad and Alibabaee2013). Of the alternative categorization systems to have been proposed, some are theoretically motivated and others driven by data characteristics.

The most widely used alternative classification systems somewhat echo the traditional paradigmatic/syntagmatic/clang approach, despite deriving from different theoretical frames. Fitzpatrick acknowledges that her broad Meaning/Position/Form distinction, for example, bears similarity to the traditional approach, despite being partly inspired by pedagogic word knowledge frameworks (Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006, pp. 130–131). The division of those main categories into sub-categories, though, deviates from previous approaches and goes some way towards addressing problems noted above: category labels are reasonably transparent, and it is fine-grained enough to observe group differences that would be masked by the conventional three categories – for example, within the Meaning category, L1 speakers are found to produce significantly more synonymous responses (e.g. church > cathedral), and fewer responses with loose conceptual links (e.g. culture > cathedral) than learners (Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006, pp. 132–133). Use of multiple categories results in the creation of a ‘profile’ for each dataset, consisting of the number/proportion of responses in each category. The statistical challenge of comparing profiles, rather than single scores, is addressed by applying proximity analyses (see Fitzpatrick, Reference Fitzpatrick2007, Reference Fitzpatrick, Fitzpatrick and Barfield2009). The advantage of the profile approach is that it enables holistic comparisons that are inclusive of all data produced in a WA task. Findings have indicated that individuals’ profiles are relatively stable across test times (Fitzpatrick, Reference Fitzpatrick2007) and across languages, with a suggestion that an individual's L2 profile will move closer to their L1 profile as proficiency increases (Fitzpatrick, Reference Fitzpatrick, Fitzpatrick and Barfield2009). There are difficulties inherent in Fitzpatrick's system too, though, including:

• • Having up to 17 sub-categories necessitates a relatively large data set from each participant, in order for responses to be distributed densely enough for robust statistical analysis.

• • As with the traditional approach, it can be difficult to make a call between categories for many responses, challenging rater reliability (e.g. Roux, Reference Roux2013, p. 87).

• • Related to this, the categories are such that some responses fall squarely into more than one (sub-)category – this is the case, for example, for some of Henriksen's (Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008) ‘canonical’ responses, such as fork > knife (fitting both collocation and lexical set categories).

• • As with the traditional approach, it is not always possible to identify the nature of the association without referring back to the respondent, and even time-consuming post-task interviews depend on participants’ capacity to account for a spontaneous response (see Fitzpatrick, Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006).

Fitzpatrick's system has been tweaked in subsequent attempts to address some of these weaknesses, and/or to match the system better to specific research questions. For example, Hui (Reference Hui2011) adds the sub-category ‘personal feelings’, Kim (Reference Kim2013) distinguishes between subject-verb and object-verb collocations to better fit the language of enquiry (Korean), and Fitzpatrick (Reference Fitzpatrick2007) minimizes the dilution of data power across categories by reducing the number of sub-categories from 17 to 9. Fitzpatrick and Izura (Reference Fitzpatrick and Izura2011) include dual-link categories, such as ‘meaning and collocation’ (e.g. rubbish > bin) and ‘form and meaning’ (hairdresser > hairdryer). While the adaptations may have made the system more rater-friendly, the fact that the system has not been held steady across studies makes replication and comparison very difficult, and problems of rater confidence and theoretical heft remain. On the other hand, the apparent consistency of individual profiles across test times and between languages is a positive indication of the system's validity (Fitzpatrick, Reference Fitzpatrick2007, Reference Fitzpatrick, Fitzpatrick and Barfield2009).

Other recent studies have used rather different approaches, for example focusing on the particular kinds of association that are pertinent to their research question. Examples of this are Jiang and Zhang's focus on form (Reference Jiang and Zhang2019), Yokokawa et al.'s focus on semantics (Reference Yokokawa, Yabuuchi, Kadota, Nakanishi and Noro2002), Cremer et al.'s interest in conceptual versus semantic knowledge (Reference Cremer, Dingshoff, de Beer and Schoonen2011), and Zareva and Wolter's concentration on collocation (Reference Zareva and Wolter2012); these are discussed further in Section 4.3. Finally, several studies layer categorization of responses with other measures: for example, Fitzpatrick and Izura (Reference Fitzpatrick and Izura2011) include response-time measures, and Henriksen (Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008) incorporates frequency of the response item within her sophisticated measurement system.

4.3 Theoretical influences on categorization methodology in WA

Meara (Reference Meara1983) lamented the lack of theoretical models applied to WA behaviour, and it is still the case that while most WA researchers are linguistically aware enough to associate the paradigmatic/syntagmatic distinction with the work of Saussure, surprisingly few of them explicitly expose or explore the provenance and implications of these terms. Stronger adherence to the Saussurian classifications might in fact serve WA researchers well, given the clarity of theoretical distinction between the ‘in praesentia’ (evident) co-presence of ‘combinations based on sequentiality’ – i.e. syntagmatic relationships, and the ‘in absentia’ ‘connexion in the brain’ of words ‘linked by meaning only’ – what Saussure calls associative relationships, and which are later labelled paradigmatic (Saussure, in Bally & Sechehaye, Reference Bally and Sechehaye1966). Saussure's considerations of these notions include a number of features that have not systematically been transferred to WA research despite being, in our view, highly relevant. For example, within the syntagmatic class he includes combinations of parts of words and compound words, as well as entire words, and associations can be either double (‘based on form and meaning’) or single, based ‘on form or meaning alone’ (Reference Bally and Sechehaye1966, p. 124). Operationalizations of the paradigmatic/syntagmatic distinction in WA literature vary in their closeness to the Saussure conceptualization, but most WA category systems, as we have seen, distinguish in some way between three relationship axes that are identifiable in Saussure's work:

• • relationships based on meaning, similarity of signifier, substitution, mnemonic groupings, paradigm, mental association

• • relationships based on combination, expressions, position (within utterance), interdependence of units within an utterance

• • relationships based on word form and/or formation, phonology and/or orthography.

The extent to which theoretical models of any kind are referred to in connection with categorization systems varies considerably. Many (e.g. Roux, Reference Roux2013) take previous empirical studies as a starting point, with the implicit assumption that the systems used in these have theoretical validity. Others (e.g. Namei, Reference Namei2004) examine theoretical underpinnings in previous WA research, but do not connect these with the specific categorization systems they or others use. For Wolter (Reference Wolter2001), theoretical modelling drives the research questions; he challenges the prevalent belief that a syntagmatic > paradigmatic shift indicates development of the lexicon, and proposes an alternative model that considers the trajectories of individual words, and ascribes changes in WA behaviour to a shift from ‘semantically meaningless … . to semantically meaningful responses’ (Reference Wolter2001, p. 63). What is known about a particular word also contributes to Henriksen's classifications (Reference Henriksen, Albrechtsen, Haastrup and Henriksen2008), and the various iterations of Fitzpatrick's (Reference Fitzpatrick, Foster-Cohen, Medved Krajnovic and Mihaljevic Djigunovic2006) system. Zareva and Wolter's (Reference Zareva and Wolter2012) consideration that there are multiple aspects of the learner's lexical organization, and that WA methods might tap into these differently, might also be considered a learner- or data-driven account.

In contrast, theoretical models of lexical organization or development explicitly drive the methodology of some WA studies. Jiang and Zhang (Reference Jiang and Zhang2019) explore the hypothesis that the L2 lexicon is more form-driven than the L1 lexicon, and consequently two of their four WA categories are ‘morphological’ and ‘form-related’. Yokokawa, Yabuuchi, Kadota, Nakanishi, and Noro (Reference Yokokawa, Yabuuchi, Kadota, Nakanishi and Noro2002) use fine-grained semantic categories to investigate semantic organization of the lexicon. Zareva and Wolter's classification of +/− collocational, in the third analysis of their 2012 paper, explores claims that acquisition is collocation-driven (citing Wray, Reference Wray2002 and Hoey, Reference Hoey2005).

Finally, the prevalence of English language focused WA studies should not obscure the role of linguistic typology in classification systems; there is an element of language specificity in the way participants respond to WA cues. The differences Fitzpatrick (Reference Fitzpatrick, Fitzpatrick and Barfield2009) finds in the number of forward and reverse collocations produced in English and Welsh is almost certainly caused by word order differences (Welsh word order configuration is Verb-Subject-Object (VSO) with noun typically preceding adjective), and Kim (Reference Kim2013) re-designs her classification system specifically to accommodate the highly-inflected, Subject-Object-Verb (SOV) Korean. Typological differences are a particular challenge in studies comparing bilinguals’ responses across two languages, and the degree to which the structures of the relevant languages are accounted for differs; in our view, too little attention is paid to this.

Examining the theoretical underpinnings and challenges of WA research can expose opportunities for future research as well as weaknesses in previous work. In the next section we identify and explore three themes that we believe are key to future development of WA research.

5.1 Mapping WA findings onto network models of the lexicon

In a recent overview chapter, De Deyne and Storms (Reference De Deyne, Storms and Taylor2015) identify three approaches to researching WA: microscopic, mesoscopic, and macroscopic. The research discussed in Sections 3 and 4 above largely takes a microscopic approach – it is concerned with relations between individual cues and responses. This approach can, as Wilks and Meara have pointed out, ‘be criticised for attempting to generalize the features and properties of a large-scale phenomenon – the mental lexicon – on the basis of small-scale snapshots of that phenomenon’ (Reference Wilks and Meara2002, p. 306).

Meso- and macroscopic approaches take a wider view. Researchers in these traditions assert that a key feature of WA research is its ability to yield structured patterns of connections between words. This strand of research was founded by Deese (Reference Deese1962a, Reference Deese1966), who studied patterns of association related to cue words from the Minnesota Norms list (Russell & Jenkins, Reference Russell and Jenkins1954). In his 1966 study, responses to those cues were then used as cues in subsequent rounds of WA. Responses to all of these words can then be tabulated to form a matrix revealing how frequently each word in the dataset yields each other word when presented as a cue. Using this approach, Deese revealed that WA responses cluster semantically. For example, butterfly yielded responses such as moth, insect, blue and color [sic]. After responses to these new words were collected, factor analysis revealed several clusters of mutual association. One, for example, related to winged creatures (butterfly, moth, bird, wing), while another centred around colours (color [sic], blue, yellow).

Deese's findings led straightforwardly to the mesoscopic WA research tradition, which focuses on the hypothesis that the overlap in associative distribution of two words can be taken as a measure of the semantic similarity of those words. For example, since there is greater overlap in associative response distributions between moth and butterfly than between colour and butterfly, it can be asserted that the former pair are more similar than the latter. Results from studies investigating the capacity for WA data to generate semantic similarity ratings in this manner (e.g. Steyvers, Shiffrin, & Nelson, Reference Steyvers, Shiffrin, Nelson and Healy2004; Andrews, Vinson, & Vigliocco, Reference Andrews, Vinson and Vigliocco2008; Van Rensbergen, De Deyne, & Storms, Reference Van Rensbergen, De Deyne and Storms2016) have been impressive: such ratings have been shown to correlate with human judgements of semantic similarity, and to explain variance on psycholinguistic tasks such as lexical decision (Steyvers et al., Reference Steyvers, Shiffrin, Nelson and Healy2004; De Deyne, Navarro, & Storms, Reference De Deyne, Navarro and Storms2012). These results suggest that networks built from WA data are not only suitable for use as simple estimates of semantic similarity for psycholinguistic experiments, but also that they capture important aspects of lexical processing.

Deese's work was important also in the development of network approaches to lexical research. This view proposes that word knowledge is organized in a massively interconnected, web-like structure (Wilks & Meara, Reference Wilks and Meara2002; Aitchison, Reference Aitchison2012). Research in the macroscopic tradition has used WA data to build approximations of such networks, and to explore their implications for lexical development, attrition, and global structure. Although research into network structure oversimplifies actual lexical knowledge, it nevertheless offers a crucial link between the lexicon and lexical fluency, since, as Strogatz puts it, ‘structure always affects function’ (Reference Strogatz2001, p. 268).

An early example of macroscopic WA research is found in Kiss's (Reference Kiss1968) application of graph theory to WA-derived networks. Graph theory is a way of describing networks, such that nodes (corresponding, in WA networks, to a single word) are connected by links (also called arcs or edges). In some models, these links are weighted. In WA-derived networks, weights are derived from the association strength between two words. Links can also be directed, meaning that each link travels in only one direction. This allows WA-derived networks to correspond to the directed nature of some WA pairs. For example, in a weighted, directed network, two links would join writing to pen, since both words yield the other in free association. However, since writing yields pen much more commonly than the reverse (according to the South Florida norms; Nelson et al., Reference Nelson, McEvoy and Schreiber2004), the link from writing to pen would be stronger than that of pen to writing.

A number of important measures can be derived from these networks, some focusing on individual nodes, and others measuring the properties of the network as a whole. An important measure of individual nodes is their degree. This refers to the number of links to other nodes in the network that an individual node possesses. In directed networks, which differentiate between incoming and outgoing links, the terms in-degree and out-degree are used. In WA networks, in-degree refers to the number of cues to which word A is given as a response, while out-degree measures the number of different responses given to word A when it is presented as a cue. Continuing the example above, in the South Florida norms (Nelson et al., Reference Nelson, McEvoy and Schreiber2004), writing was given as a response to a total of 14 cues, yielding an in-degree of 14. When writing was presented as a cue, 21 response typesFootnote ³ were given as responses, resulting in an out-degree of 21.Footnote ⁴ Words which possess many links to other words can be described as being central to the network. Some research suggests that associative network centrality may influence online lexical processing, since measures of associative centrality can explain unique variance in lexical processing tasks such as lexical decision, categorization, and visual word recognition (Griffiths, Steyvers, & Firl, Reference Griffiths, Steyvers and Firl2007; Duñabeitia, Avilés, & Carreiras, Reference Duñabeitia, Avilés and Carreiras2008).

Researchers have suggested that central words play a formative role in the growth of the L1 lexicon. This hypothetically occurs through a process known as preferential attachment (e.g. Steyvers & Tenenbaum, Reference Steyvers and Tenenbaum2005), whereby newly acquired nodes preferentially attach to nodes which are already central to the network. This leads to the prediction that frequent and early-acquired words should become increasingly central, because they have the highest probability of attracting links to new nodes. Support for this theory has been provided by De Deyne and Storms (Reference De Deyne and Storms2008), who found that both word frequency and age of acquisition were correlated with centrality in a WA-derived network; and from simulations run by Meara (Reference Meara2007). One caveat to these findings is that, because few studies have utilized the most sophisticated of the available measures, it remains unclear whether WA-derived centrality captures aspects of processing not explained by other measures (see e.g. De Deyne & Storms, Reference De Deyne and Storms2008; Yap, Tan, Pexman, & Hargreaves, Reference Yap, Tan, Pexman and Hargreaves2011). Nevertheless, the study of network centrality in L1 lexical development raises the possibility that this approach may shed light on some of the determinants of L2 lexical acquisition.

Lexical researchers have been particularly interested in Small World networks (Milgram, Reference Milgram1967; Watts & Strogatz, Reference Watts and Strogatz1998). This type of network displays several properties not found in random networks: sparseness (e.g. a network of 122,005 nodes, created from the WordNet database by Steyvers and Tenenbaum (Reference Steyvers and Tenenbaum2005, p. 51), had an average of only 1.6 links per node); short average path lengths (e.g. an average of just 10.56 links connected any two nodes in the same network); and high clustering, indicating the presence of highly interconnected neighbourhoods, as exemplified by the semantic clusters discovered by Deese (Reference Deese1966), and possessing relatively few links to other clusters. Networks such as these depend on ‘hubs’Footnote ⁵ – a small number of words which have a very in-high degree, including connections between several neighbourhoods – to facilitate fast search and retrieval of items analogous to the fast semantic access of which humans are capable (De Deyne & Storms, Reference De Deyne, Storms and Taylor2015, p. 477).

The studies reported above seem to confirm that WA-derived networks display small world properties. However, they all take a between-subject, cross-sectional approach, resulting in networks based on responses from a wide range of participants, whose individual response preferences are not known. As such, they may be best understood as reflecting broad possibilities regarding network structure, rather than as revealing the properties of any one person's actual associative network. A study by Morais, Olsson, and Schooler (Reference Morais, Olsson and Schooler2013) took a different approach, exploring the structure of six associative networks each derived from WA responses given by one person. This study revealed marked variation in the size, connectedness, path length, and clustering of each network, suggesting clear individual variation in the properties of the systems or processes underlying WA. Importantly, however, all of these networks demonstrated the same general small world structural properties, in spite of their surface-level differences. Related to this approach is work by Beckage, Smith, and Hills (Reference Beckage, Smith and Hills2010) who, in a study of the L1 networks of young children, linked delayed emergence of these small world properties with later language processing difficulties. These studies suggest that small world network properties may be reflective of fully developed associative knowledge, and may also be related to online language processing. This is a potential area of interest for L2 researchers: if the development of efficient networks of lexical knowledge can be demonstrated to be a precondition of verbal fluency, this would appear to prioritize research on how the emergence of such networks in L2 learners can be facilitated in the classroom.

The application of these approaches to L2 research has been hampered by methodological challenges. A central problem, which applies to several of the studies discussed below, is that L2 responses tend to be less consistent than those produced by L1 respondents (Meara, Reference Meara2009). This inconsistency has led some researchers to abandon the use of productive WA tasks for the construction of L2 associative networks, and instead develop receptive tasks in which participants identify associated words from lists of provided options (similar to the WAF tasks discussed in Section 3.3, but without the notion of correct/incorrect answers). Using this approach, Wilks and Meara, in a series of studies (Wilks & Meara, Reference Wilks and Meara2002, Reference Meara2007; Wilks, Meara, & Wolter, Reference Wilks, Meara and Wolter2005; Wilks, Reference Wilks, Fitzpatrick and Barfield2009), suggest that L1 networks are significantly denser than those of L2 respondents, and they observe that increases in L2 network density are linked to L2 proficiency.

Meara (Reference Meara, Brown, Malmkjaer and Williams1996b, Reference Meara2007) progresses the application of network theory to L2 lexical research in his discussion of the concept of depth of word knowledge. Rather than viewing this concept as a property of individual words (see Section 3), he argues for a network-based measurement of global depth of vocabulary knowledge which conceptualizes depth as the total number of links within a learner's lexicon. However, he acknowledges that a sufficiently large number of test items and an appropriate test methodology are required in order to generate a meaningful indication of depth of vocabulary knowledge, and methodological problems have beset trials using receptive WA tests (e.g. Meara & Wolter, Reference Meara and Wolter2004; Wilks & Meara, Reference Wilks, Meara, Daller, Milton and Treffers-Daller2007).

Another attempt to measure L2 network properties through receptive WA tasks was made by Schur (Reference Schur, Daller, Milton and Treffers-Daller2007), who investigated the small world properties of L2 networks through a task in which monolingual and bilingual (Hebrew/English and Chinese/English) participants identified associates from a list of 50 frequent English verbs. Although this methodology proved too limited to fully achieve its aims, it did reveal interesting differences in participant groups: the Chinese participants’ responses yielded networks with low levels of overall connectivity, including many isolated clusters of just two items. The Hebrew learners, on the other hand, produced much longer chains of interconnected responses, with fewer isolated clusters. Schur interprets these in terms of language learning background. Citing ‘informal feedback’ from the Chinese participants and their teachers, she suggests that the Chinese learners’ shallow networks may be the result of a rote style of learning which placed little emphasis on communication, whereas the Hebrew learners had learnt English in a much more communicative context. Schur suggests that this may have made them more aware of the multiple meanings of words and different potential links between them (see Section 5.3 for further discussion of language learning background on associative knowledge).

Finally, it is worth reiterating that when researchers create WA-based networks, they are creating networks of WA response data, not accurate models of the human mind: the network model remains a metaphor. This is evident in the way that different methodological choices, such as the use of single-versus multiple-response tasks, lead to different network properties. Expanding on this, Wilks and Meara (Reference Wilks, Meara, Daller, Milton and Treffers-Daller2007) have questioned whether WA tests provide the ‘direct access’ to the lexicon that they are often assumed to. Much WA research, both L1 and L2, appears to define two words as being ‘associated’ simply because one yields the other in a WA task. Wilks and Meara suggest that this is not necessarily the case, since WA responses tend to vary depending on respondent strategies (Reference Wilks, Meara, Daller, Milton and Treffers-Daller2007; Riegel et al., Reference Riegel, Ramsey and Riegel1967) and task demands (see for example Suzuki-Parker & Higginbotham, Reference Suzuki-Parker and Higginbotham2019, on differences between oral and written WA responses). Similarly, De Deyne, Verheyen, and Storms (Reference De Deyne, Verheyen, Storms, Mehler, Lücking, Banisch, Blanchard and Job2016) have demonstrated that lexical networks built from corpus data differ in fundamental ways from those built from WA data. Research such as this provides a reminder that no single method of lexical network construction is likely to perfectly replicate the contents of the human mental lexicon.

5.2 The influence of lexical variables on WA

This section explores the numerous ways in which the semantic and distributional properties of words influence the WA task. We here take the view that the word association task is a form of psycholinguistic elicitation, and as such reflects properties of cue words in ways which may not be true of other psycholinguistic processes (Mollin, Reference Mollin2009; Nordquist, Reference Nordquist2009). The network studies described in the previous section have been important to the endeavour of exploring these properties, since their holistic view has facilitated a shift from seeing lexical variables as influencing only local connections between words, to an approach which sees lexical properties as a key determinant of the structure of the associative network. The vast majority of research conducted in this area has been carried out using L1 respondents, and to date very few studies have explored the extent to which L1 findings are applicable to L2. These are discussed at the end of this section; the following review of lexical variable effects is largely informed by L1 literature.

Numerous distributional and semantic properties of words have been shown to influence WA. The most widely researched of these characteristics are described in Table 3.

Table 3.

Lexical variables

An entry point into research on these variables is a study by De Deyne and Storms (Reference De Deyne and Storms2008). Their experiment involved the creation of a large network of Dutch L1 WA responses collected using 1,424 cue words. The network was analysed in order to determine the extent to which the lexical properties of its constituent words influenced its structure. Two measurements are particularly useful for picking apart these lexical effects: out-degree and in-degree (see Section 5.1 for definitions). De Deyne and Storms found that lexical variables interact with these measures in four ways:

1. 1. Distributional variables correlate only weakly, or not at all, with out-degree (Frequency r = 0.14, P < 0.01; AoA r = 0.02, P > 0.05).

2. 2. Distributional variables correlate much more strongly with in-degree (Frequency r = 0.7; AoA r = −0.64, both P < 0.01).

3. 3. Only one semantic variable, imageability, was used in the study; it did not correlate significantly with out-degree.

4. 4. Imageability did, however, correlate with in-degree (r = 0.30, P < 0.01).

De Deyne and Storms’ findings offer a holistic perspective against which we can evaluate findings from other studies.

The first finding might imply that the distributional properties of cue words hold only a very modest influence over the generation of responses. Other studies have generally supported this implication. Studies on cue frequency, for example, have revealed only weak effects: a seminal study by de Groot (Reference de Groot1989) found slightly slowed responses and greater response heterogeneity for higher frequency cues, but these effects were modest and emerged only when cue frequencies were very widely spaced. Compounding this weak effect are contradictory findings presented in earlier reviews by Brown (Reference Brown1971) and Cramer (Reference Cramer1968), which suggested that higher frequency cues sometimes yield slightly faster responses. Both Brown and de Groot conclude that cue frequency has only a marginal effect on response patterns. Indeed, Stolz and Tiffany (Reference Stolz and Tiffany1972) suggest that WA frequency effects are better viewed as word familiarity effects than as pure frequency effects – a suggestion which has been important to L2 WA researchers. Research into the influence of AoA is slightly more clear: both Brysbaert, Van Wijnendaele, and De Deyne (Reference Brysbaert, Van Wijnendaele and De Deyne2000) and van Loon-Vervoorn (Reference van Loon-Vervoorn1989) find that WA responses are produced more quickly when the cue is an early-acquired word. Finally, the view that distributional data holds little influence over response patterns is supported by Van Rensbergen, Storms, and De Deyne (Reference Van Rensbergen, Storms and De Deyne2015), who found that a cue's distributional properties were very poor predictors of the same properties of their responses (AoA R² = 0.04; Contextual diversity R² = 0.01; Frequency R² = 0.01).

The second finding emerging from De Deyne and Storms’ research suggests that frequent and early-acquired words are much more likely than others to assume the role of hubs in the associative network, since they appear to possess a higher number of incoming links than less frequent or later acquired words do. Although few studies have attempted to replicate these findings, they are nevertheless given conceptual validity by the theory of preferential attachment, which suggests that early-acquired words develop high centrality because they provide an anchor for new knowledge (Steyvers & Tenenbaum, Reference Steyvers and Tenenbaum2005, and see Section 5.1). This is an area with important implications for L2 research. If words learnt early in life help to anchor lexical knowledge in an L1, then it is useful to ask whether these same words play a similar role in L2 acquisition; more generally, what are the properties of words central to L2 lexical networks? The contrasting influence of distributional variables when measured as cue vs response properties also point to the importance of large-scale network research: such findings are simply invisible to microscopic research into local connections between isolated words.

Third, De Deyne and Storms’ research suggests that a cue word's imageability (and perhaps, by extension, other semantic variables) holds no significant influence over response generation. This implication is not, however, supported by WA studies using a wider range of methods. An initial indication of the importance of cue semantics to WA response patterns is given by Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015). They showed that while a cue's distributional properties were poor predictors of the same properties of responses, the cue's semantic features mirrored response properties to a much greater extent (Concreteness R² = 0.20; Valence R² = 0.31; Arousal R² = 0.17; Dominance R² = 0.15; all P < 0.001). Further examples of the importance of specific semantic properties to response generation include:

• • Concrete cues have been shown to be responded to more quickly (Brown, Reference Brown1971; de Groot, Reference de Groot1989; van Hell & de Groot, Reference van Hell and de Groot1998), receive fewer blank responses (de Groot, Reference de Groot1989; Bøyum, Reference Bøyum2016), yield more homogeneous distributions (Brown, Reference Brown1971; de Groot, Reference de Groot1989), and have higher response availability (de Groot, Reference de Groot1989) than less concrete ones.

• • Van Rensbergen et al. (Reference Van Rensbergen, De Deyne and Storms2016), taking a mesoscopic approach, have shown that WA networks can be used to generate norms for valence, dominance, and arousal which correlate to a high degree with human judgements of affective strength; reviews by Cramer (Reference Cramer1968) and Brown (Reference Brown1971) offer further examples of the influence of affective variables.

The last of the findings in the list above implies that words with strong semantic profiles (e.g. high imageability or strongly affective properties) may be more central to the lexicon than more semantically opaque words (De Deyne & Storms, Reference De Deyne and Storms2008). Van Rensbergen et al.'s (Reference Van Rensbergen, De Deyne and Storms2016) work on generating affective norms from WA data appears to support this implication. However, as with the distributional data described, the lack of research on outgoing associative connections to date means that it is too early to draw any conclusions.

The L1 findings presented here suggest that WA may offer a promising method for exploring the properties of L2 lexical structure, for example by identifying the properties of words which are central to the L2 lexicon. It is critically important, however, not to assume that L1 findings will transfer straightforwardly to L2. Second languages are typically less completely acquired than first languages, and this is reflected in findings pertaining to frequency effects in L2 WA. Several studies have found that low-frequency cues lead to less consistent responses (Meara, Reference Meara2009), more blank responses (Higginbotham, Reference Higginbotham2010), and lower response availability (Zareva, Reference Zareva2011) than high frequency words, suggesting that WA in an L2 is more sensitive to frequency effects than L1 association. This has led some researchers to adopt Stolz and Tiffany's suggestion that these findings may be better understood as word familiarity effects. For example, Wolter (Reference Wolter2001) demonstrated that both L1 and L2 respondents produce increasing numbers of form-based responses as word familiarity declines, while Riegel and Zivian (Reference Riegel and Zivian1972) found a similar pattern among trilingual participants (see also Wilks, Reference Wilks, Fitzpatrick and Barfield2009). Results such as these led Zareva (Reference Zareva2011) to call for dedicated studies disambiguating the role of cue frequency and familiarity in L2 WA.

Another justification for rejecting assumptions of L1 and L2 associative similarity can be derived from the theory of preferential attachment (see Section 5.1). If early-acquired vocabulary knowledge provides an anchor for later learning in an L1, it follows that newly learnt L2 items may also preferentially attach to L1 words or concepts. At least two studies have hinted that this might be the case, and that lexical variables such as concreteness might in fact be one determinant of the extent of such linguistic and conceptual entanglement (see Section 5.3). First, Kolers (Reference Kolers1963) found that bilinguals producing responses in both of their languages generated a higher number of translation equivalents when the cue in question was a concrete word than when it was more abstract. Second, research by van Hell and de Groot (Reference van Hell and de Groot1998) found that concrete cues were more likely than abstract ones to result in participants producing the same response in both L1 and L2 tasks. Responses were also produced more quickly to concrete cues in both L1 and L2. Van Hell and de Groot additionally imply that the existence of (L1) conceptual knowledge might be a complicating factor in the interpretation of L2 WA research.

The comparison of L1 and L2 WA studies in this section has exposed a critical methodological requirement for L2 WA work: approaches should be developed which allow the unique role of each lexical variable to become clear. Several studies investigating the role of grammatical class (GC) in L2 WA stand in evidence of this. In L1 studies, numerous effects of this variable have been identified in both micro- and macroscopic studies. In general, these findings suggest that nouns are more central to associative networks than words of other classes, since they tend to be the most common type of response to noun, verb, and adjective cues, and dominate lists of hubs in associative networks (Deese Reference Deese1962b; Entwisle, Reference Entwisle1966; Cramer, Reference Cramer1968; De Deyne & Storms, Reference De Deyne and Storms2008, Reference De Deyne, Storms and Taylor2015). Methodological issues have obscured such findings in L2 research, however. For example, Nissen and Henriksen (Reference Nissen and Henriksen2006) found that the pattern of influence from cue GC in L2 WA mirrors that found in L1. However, they also acknowledged that their experimental design did not control for the effects of important variables such as concreteness and age of acquisition, leaving some question marks over their findings. Similarly, Zareva (Reference Zareva2010, Reference Zareva2011) reported an interaction between participant proficiency, cue GC, and cue frequency, which resulted in significant differences in syntagmatic and paradigmatic responses, but was unable to identify the specific locus of this interaction, partly because of the lack of statistical power resulting from her use of only 12 cues from each GC. These points underline the need for methodological clarity in future L2 WA studies.

Perhaps the most important methodological concern facing research into the effect of lexical variables on L2 associative structure is the lack of large-scale L2 network-based WA research. While some researchers have attempted to apply a network perspective to their research, they have tended to use simulation methods rather than empirical data (e.g. Meara, Reference Meara2006, Reference Meara2007; though see Schur, Reference Schur, Daller, Milton and Treffers-Daller2007; and Zareva, Reference Zareva2010, for small-scale network-oriented behavioural studies). What this means in practice is that while L1 research has provided good evidence that various properties of words strongly influence the structure of the lexicon, for example by providing anchors for the integration of new knowledge, none of the structural properties of L1 associative networks described in the first half of this section can yet be asserted in an L2.

5.3 Mapping WA findings onto psychological constructs

Scholars have long been interested in the psycholinguistic systems and processes that underlie the formation and production of word associations, as documented in Section 2. The classical assumption regarding the formation of associations was that words become associated in the mind through textual contiguity – that is, when they co-occur in texts (see Deese, Reference Deese1966; and Warren, Reference Warren1916, for historical overviews). This belief has led some researchers to argue that the generation of word associations also involves recall of these co-occurring words. For example, Wettler, Rapp, and Sedlmeier (Reference Wettler, Rapp and Sedlmeier2005, p. 116) have shown that a corpus-derived model of WA can predict a number of primary human WA responses, and generally behaves in a manner similar to human WA respondents. The authors conclude that ‘the behaviour of participants in the free association task can be explained by associative learning of the contiguities between words’.

This is not the view of all scholars, however. Others point to the weakness of correlations between corpus co-occurrence and WA response strengths (e.g. Mollin, Reference Mollin2009; Kang, Reference Kang2018), and warn against assuming that the processes underlying the learning of association are necessarily also the ones responsible for the retention and generation of words in WA tasks (Hutchison, Reference Hutchison2003; McRae, Khalkhali, & Hare, Reference McRae, Khalkhali, Hare, Reyna, Chapman, Dougherty and Confrey2012). These researchers argue that while textual contiguity may be, in the words of Kang (Reference Kang2018, p. 110), a ‘starting point’ for the development of associative knowledge, the lexical information drawn on during WA response generation is, in fact, semantic in nature (Guida & Lenci, Reference Guida and Lenci2007; Mollin, Reference Mollin2009; McRae et al., Reference McRae, Khalkhali, Hare, Reyna, Chapman, Dougherty and Confrey2012; De Deyne & Storms, Reference De Deyne, Storms and Taylor2015; Thwaites, Reference Thwaites2019). According to this view, contiguities between words provide input which is then acted upon by the mind. WA responses therefore reflect not the contiguities themselves, but the mental system which results from this semantically driven cognitive activity (cf. Deese, Reference Deese1966). Correspondences between corpus and WA data such as those presented by Wettler et al. (Reference Wettler, Rapp and Sedlmeier2005) are explained as reflecting the co-occurrence of semantically related words in text. Evidence in support of this view includes the following:

• • Priming effects have been reported between words which are semantically related but do not co-occur: no reliable priming effects have been demonstrated for words which co-occur but are not semantically related (Hutchison, Reference Hutchison2003).

• • While several studies have demonstrated that non-semantic associations can be learnt under experimental conditions, the process is time-consuming, and the resulting associations are not generalized to new tasks (Dagenbach, Horst, & Carr, Reference Dagenbach, Horst and Carr1990; Schrijnemakers & Raaijmakers, Reference Schrijnemakers and Raaijmakers1997).

• • Associations presented in sentential contexts are learnt more easily than decontextualized ones (Prior & Bentin, Reference Prior and Bentin2003); the process of integrative semantic processing of sentences can account for the retention of associations (Prior & Bentin, Reference Prior and Bentin2008).

• • Most studies comparing WA responses to corpus data reveal significant differences between the two types of data (Mollin, Reference Mollin2009; Kang, Reference Kang2018); word associations tend to reflect prototypical interpretations of words (e.g. erupt-volcano), while figurative uses (e.g. erupt-violence), often found in corpora, are uncommon in WA (Thwaites, Reference Thwaites2019).

• • Responses to concrete noun cues in WA correspond to a high degree (72.5%) with defining semantic features of their cue (Vivas, Manoiloff, Garcia, Lizarralde, & Vivas, Reference Vivas, Manoiloff, Garcia, Lizarralde and Vivas2018).

• • Semantic aspects of cue words (e.g. verbs pertaining to weather, bodily processes, or motion) influence proportions of responses of different types (e.g. synonyms, thematic roles; Guida & Lenci, Reference Guida and Lenci2007).

This evidence suggests that the process of generating WA responses, at least in a first language, is largely a semantic one.

The neural basis of this semantic knowledge has attracted much research, both in WA and the wider psycholinguistic research community. Two broad viewpoints have been put forward. The first is the amodal position. Researchers such as Fodor (Reference Fodor1975, Reference Fodor1989) have suggested that words exist in the mind within a single dedicated neural system which organizes them in relation to one another. The system works by converting linguistic experience, such as the books we read or the conversations we have (as well as, in some amodal models, aspects of our perceptual experience such as sensory input and motor activity), into a network of semantic symbols unrelated to perceptual brain states. Expressed somewhat crudely, amodal systems posit that we know what words mean because we are aware of their similarities and differences to other words.

The opposing, modality-specific, view, which encompasses theories of embodied cognition, is that words are not stored in a single brain system, but are distributed across the brain together with the motor, perceptual, and introspective experiences to which they correspond. Processing words such as grasp, according to these models, will therefore activate those neural systems which underlie the physical act of grasping (e.g. Wilson, Reference Wilson2002; Bergen & Chang, Reference Bergen, Chang, Östman and Fried2004; Barsalou, Reference Barsalou2008; see also the Distributed Lexicon model in Wray, Reference Wray2002). Proponents of this view have suggested that these models provide a priori explanations of numerous linguistic phenomena which are less satisfactorily explained by amodal models (see Martin, Reference Martin2007 for a review; also Barsalou, Reference Barsalou2008).

Evidence in support of both of these viewpoints has been provided in WA studies. For example, the viability of the amodal position as an explanation of L1 associative network properties is suggested by Gruenenfelder, Recchia, Rubin, and Jones (Reference Gruenenfelder, Recchia, Rubin and Jones2016), who found that these properties can be replicated using only corpus data (i.e. without the need for additional perceptual information). On the other hand, several studies have lent support to one particular modality-specific model: Barsalou's Language and Situated Simulation model (LASS; Barsalou, Santos, Simmons, & Wilson, Reference Barsalou, Santos, Simmons, Wilson, De Vega and Glenberg2008). LASS posits that discrete neural systems handle linguistic and conceptual knowledge; linguistic events (including WA) activate both of these systems. Importantly, however, the former is accessed more quickly than the latter. This leads to a number of predictions. First, it should be possible to identify two distinct patterns of brain activity during WA, corresponding to linguistic and conceptual processing respectively, since separate neural systems are posited to handle these. Second, there should be a different time course for the responses originating from the two different systems, with conceptual responses produced more slowly. Both of these predictions have been supported experimentally (De Deyne & Storms, Reference De Deyne and Storms2008; Simmons, Hamann, Harenski, Hu, & Barsalou, Reference Simmons, Hamann, Harenski, Hu and Barsalou2008; Santos, Chaigneau, Simmons, & Barsalou, Reference Santos, Chaigneau, Simmons and Barsalou2011), though in some cases methodological issues such as the use of novel categorization schemes mean that these findings are in need of independent replication.

A somewhat intermediary position between modal and amodal views is offered by ‘hub-and-spoke’ models (Jackson, Hoffman, Pobric, & Lambon Ralph, Reference Jackson, Hoffman, Pobric and Lambon Ralph2015), which propose an integrative, dynamic semantic system (the ‘hub’) which collects information from linguistic, introspective, and perceptual sources within a single system (and is therefore somewhat similar to the system proposed in amodal models), while retaining links (the ‘spokes’) to modality-specific systems which contain more specific conceptual information. Such hybrid systems, which do not insist upon hard divisions between linguistic and perceptual processing, offer a potentially exciting alternative to the models described above, but are yet to be explored in a WA context (Andrews, Vigliocco, & Vinson, Reference Andrews, Vigliocco and Vinson2009; Andrews, Frank, & Vigliocco, Reference Andrews, Frank and Vigliocco2014; Fernandino et al., Reference Fernandino, Binder, Desai, Pendl, Humphries, Gross and Seidenberg2016).

Several models of semantic knowledge have the potential to contribute to debates regarding the automaticity of lexical processing in WA. LASS, for example, may imply that linguistic WA responses are somewhat more automatic than conceptual ones, since the latter might be slowed by conscious generation of images and memories.

Playfoot et al. (Reference Playfoot, Balint, Pandya, Parkes, Peters and Richards2016) looked more closely at the concept of automaticity, investigating whether WA responses are in fact the first word which comes to mind, as is generally assumed. Across two experiments, the authors found working memory capacity effects on both a standard free association task and a novel creative association task, in which participants were asked to generate associations which they felt would be unique. Individuals with higher working memory capacity produced more stereotypical responses on the free task, and more unique responses on the creative task. Since this interaction of task demands and working memory potentially suggests strategic responding (i.e. selection of a response which was not the first which came to mind), the authors designed a further experiment in which working memory use was inhibited. They found that the proportion of stereotypical responses produced per participant increased in this time-constrained condition. This appears to suggest that the stereotypical responses generated in the free response task are in fact the first word that comes to participants’ minds. Playfoot et al.'s study therefore suggests some level of automaticity in WA response generation.

Studies on psycholinguistic aspects of second language WA have tended to focus on the influence of L1 lexical knowledge on L2 association. Some studies have attempted to link various aspects of associative response patterns to L2 teaching methods. An early example of this type of research is a study by K. F. Riegel, Ramsey, and R. M. Riegel (Reference Riegel, Ramsey and Riegel1967), who compared responses from bilingual (L1 English, L2 Spanish; L1 Spanish, L2 English) participants on two tasks: a traditional WA task, and a constrained task in which respondents were asked to produce specific types of association, such as words which shared semantic features with the cue. The authors note the different linguistic background of the two sets of participants: the Spanish L2 group had learnt in a classroom environment, while the L2 English participants (who reported slightly lower proficiency) were living in the US and using English communicatively. They found that the L2 English participants gave fewer blank responses and showed greater response homogeneity than the Spanish L2 group. However, their responses also showed less overlap with English L1 responses on the constrained WA task than did the L2 Spanish group. According to the authors, this implied that while the L2 English group had a more fluent grasp of word meanings, the Spanish L2 group possessed more detailed conceptual representations of word meanings. Riegel et al. suggest that this might be due to the different L2 acquisition contexts, with an immersion context leading to greater lexical fluency and a classroom one to greater lexical precision.

While this line of enquiry has not yet been extensively pursued, it is worth recalling, first, the findings of Schur (Reference Schur, Daller, Milton and Treffers-Daller2007; see Section 5.1), who suggested in the light of L2 associative network research that learning English in a communicative classroom may have afforded her Israeli participants more interconnected lexical networks than the Chinese participants who had studied in classrooms dominated by rote learning of vocabulary; and second those of Håkansson and Norrby (Reference Håkansson and Norrby2010), who found that language learning context influenced the number of clang responses produced by L2 WA participants (Section 4). Both of these findings additionally resonate with the predictions of usage-based theories of language (see Section 3.2) that the nature of linguistic experience shapes the structure of linguistic knowledge.

Riegel et al.'s (Reference Riegel, Ramsey and Riegel1967) study makes an important assumption – that L2 WA reflects the degree to which L2 conceptual representations have been acquired. This view implies that language learning necessarily involves the development of new conceptual representations for words, corresponding to those of speakers of the target language. A contrasting view is that language learners do not develop new representations, but instead simply apply their pre-existing L1 representations to new L2 words. Several studies have argued for the latter, suggesting that three cue properties – concreteness, cognate status, and grammatical class – might mediate the extent to which new L2 representations are developed (Kolers, Reference Kolers1963; Taylor, Reference Taylor1976).

In an important examination of this issue, van Hell and de Groot (Reference van Hell and de Groot1998) found that bilingual (Dutch L1, English L2) participants were most likely to give the same response in both languages to concrete cognate nouns; abstract cognates, concrete non-cognates, and abstract non-cognates respectively yielded declining levels of response repetition, as did non-noun cues. The authors interpreted these findings as supporting a distributed, network model of conceptual representation, according to which the ‘conceptual units’ which make up word meanings are shared between languages only where they overlap. Since there is greater between-language overlap for concrete than for abstract concepts, for cognate than for non-cognate words, and for nouns than for other grammatical classes, it is considered that conceptual representations for these words (concrete, cognate, nouns) are more likely to be shared between languages. From a language teaching perspective, this may also suggest that time spent helping learners to develop an awareness of conceptual differences between cognates may be beneficial.

A different approach to studying the relationship between first and second language lexical access was taken by Fitzpatrick and Izura (Reference Fitzpatrick and Izura2011). Their study had two aims, first to test whether responses of different categorizations (meaning-, position-, or form-based, or a combination of two of these) differed in response speed (in L1 and/or L2); and second, to test (via a lexical decision task) whether L1 translation equivalents had been accessed during L2 response generation. The study also tested whether L2 vocabulary knowledge (as a proxy for proficiency) influenced these measurements. The participants were 24 bilinguals (L1 Spanish, L2 English) living in the UK.

A number of findings emerged from the study. First, L1 response times were faster than L2 response times for the corresponding categories. Second, the fastest responses (irrespective of response language) were words which were both meaning- and position-based (i.e. dual category responses such as pen > paper); responses of non-equivalent meaning to the cue (e.g. party > celebrate; accountant > numbers) were slower than other response types. Third, L2 participants with higher vocabulary test scores responded more quickly than those with lower tests scores, except in the case of the aforementioned meaning- and position-based dual category responses. Finally, an L1 priming effect was discovered: lexical decision response times were faster for L1 translation equivalents of L2 cues than for L1 filler words. The priming effect appeared to be influenced by proficiency level, since it was only statistically significant for participants who scored below average on the test of vocabulary knowledge. Fitzpatrick and Izura's results are therefore supportive of accounts of bilingual lexical processing which view L2 representations as parasitic, at least in the early stages of learning, upon L1 representations. Examples of this type of theory include the Revised Hierarchical Model (Kroll & Stewart, Reference Kroll and Stewart1994) and MacWhinney's (Reference MacWhinney, de Groot and Kroll2005) Unified Model of Language Acquisition. The study also suggests the value both of using response-time measures in WA, and of testing WA findings using other psycholinguistic measures, such as lexical decision tasks.

While the L2 studies above are too few in number to offer a decisive view of the psycholinguistics of L2 WA, they do suggest significant potential for future research. The influence of learning environments (e.g. classroom methodologies and periods of immersion), and the role of L1 knowledge in L2 development may be particularly worthy of further investigation.