Introduction
English lexicography is undergoing a transformation so profound that both dictionary makers and users need new strategies to cope with the challenges of today's technologies and to take full advantage of their potential. Rundell has rightly stated that dictionaries have finally found their ideal platform in the electronic medium (Reference Rundell, Granger and Paquot2012: 15), which provides quicker and more sophisticated access to large data collections that are no longer subject to space restrictions. But the innovations go far beyond storage space and ease of access - customization, hybridization and user-input are amongst the most promising trends in electronic lexicography. Customization means that dictionaries can be adaptable, i.e. manually customized by the user, or even adaptive, i.e. automatically adapted to users’ needs on the basis of their behaviour (Granger, Reference Granger, Granger and Paquot2012: 4). Paquot lists genre, domain as well as L1 as examples of fruitful areas for customization (Reference Rundell, Granger and Paquot2012: 185). In the electronic medium, the barriers between different language resources such as dictionaries, encyclopaedias, databases, writing aids and translation tools are disappearing, a development referred to as hybridization (Granger, Reference Granger, Granger and Paquot2012: 4). And the concept of user-input is exemplified by the well-known platforms Wiktionary and Urban Dictionary, both of which are online reference works based on contributions by users.
This article focuses, however, on another innovation within English lexicography, one that originated several decades ago but has recently provided fresh impetus for the discipline. Corpora, i.e. electronic collections of authentic language data, have been used within dictionary making at least since the 1980s. In keeping with Granger, they play an important role ‘both upstream, as raw material which lexicographers mine and refine to produce rich lexical entries, and downstream, as an integral part of the electronic dictionary to which users have direct access and which they can mine for themselves’ (Reference Rundell, Granger and Paquot2012: 3). Both aspects are investigated below: After a brief discussion of the history of corpus use in English lexicography, this article provides an overview of selected corpus tools currently used for dictionary making. The findings show that the process of compiling a dictionary has changed substantially due to these tools, and that future developments may even render lexicographers obsolete. As stated, some electronic dictionaries allow their users to query corpora directly within the dictionary interface, providing them with a large amount of language data for self-study purposes (e.g. collocational and grammatical patterns). The potential and usefulness of this feature is also touched upon. Altogether, this paper aims to portray the ever-growing role of corpus tools for dictionary makers and users, illustrating the paradigm shift that has recently taken place.
A brief history of corpus use in English lexicography
In pre-corpus lexicography, dictionary makers largely had to rely on their own intuition and (hand)written citations, not seldom focusing on words in isolation and thus failing to capture their full phraseological behaviour. Samuel Johnson used slips of paper to collect the quotations for his Dictionary, and the OED had a ‘corpus’ of about five million index cards, each providing a citation. The earliest electronic corpora such as LOB and Brown, both of which were created in the 1960s, had only a minorFootnote 1 impact on English lexicography, simply because they were not big enough to yield reliable data. A corpus of their size - roughly one million words - did not allow statistically significant co-occurrences of words to be distinguished from chance co-occurrences (cf. Hanks, Reference Hanks2012b: 403). The first English dictionary that relied heavily on a corpus was COBUILD1 Footnote 2 (Reference Sinclair1987). It was produced using the Birmingham Collection of English Texts (BCET), a corpus of originally 7.3 million words, which ‘was nevertheless too small to support reliable generalizations about anything but the main meanings and uses of the core vocabulary of English’ (Rundell, Reference Rundell, Granger and Paquot2012: 18). During the 1990s, many of the leading English dictionaries were based on the British National Corpus (BNC) of 100 million words, which had been collected at great expense over three years (cf. Kilgarriff and Kosem, Reference Kilgarriff, Kosem, Granger and Paquot2012: 31).
It was not until the start of the twenty-first century that it became feasible, at relatively low cost, to collect, annotate and store corpora which included billions of words rather than millions. In keeping with Rundell (Reference Rundell, Granger and Paquot2012: 19), the ‘cost of data storage became negligible, processing power of ordinary computers increased enormously, software for automatically encoding the corpus data (lemmatizers, taggers) became fast and reliable, [and] text in digital form became available through the Internet in almost infinite quantities’. No serious dictionary project can do without a corpus these days (cf. Granger, Reference Granger, Granger and Paquot2012: 3), and lexicographers now routinely work with corpora including several billion words and more (cf. Rundell, Reference Rundell, Granger and Paquot2012: 19).Footnote 3 The internet, which is increasingly being used as a corpus, is estimated to provide over 10,000 billion words of English language text (Kilgarriff and Kosem, Reference Kilgarriff, Kosem, Granger and Paquot2012: 31). Even though the world wide web has been criticised as ‘dirty’, with spelling mistakes and ungrammaticalities, ‘the signal of correct language is much greater than the noise that these errors generate’ (Grefenstette, Reference Grefenstette, Fjeld and Torjusen2012: 201). Its potential as a corpus for lexicographers is unquestionable, even if it may take a few more years to master all the technical issues involved.
Corpus tools for lexicographers
The immense increase in size of corpora has brought along many benefits but also a few challenges for dictionary makers. There is no doubt that the abundance and diversity of data provided by corpora - together with electronic tools capable of handling the data efficiently and quickly - allow more precise lexicographic descriptions than in pre-corpus times (cf. Granger, Reference Granger, Granger and Paquot2012: 3). As a result of the enormous size of today's corpora, however, lexicographers need to abandon raw corpus data as their primary resource and rely instead on semi-processed data. Such data have already been pre-analyzed and sorted into lexicographically meaningful patterns with the help of software tools (cf. Granger, Reference Granger, Granger and Paquot2012: 3). Granger finds that this transfer of various tasks from humans to computers is both a blessing and a curse for dictionary makers. The costs of corpus creation have been reduced significantly, yet the sheer dimensions of today's corpora entail greater editorial costs. ‘This has in turn triggered the development of new technologies that are progressively changing the lexicographers’ task from that of writing entries from scratch to editing provisional entries generated by the computer’ (Granger, Reference Granger, Granger and Paquot2012: 6).
This development marks, indeed, a paradigm shift for corpus analysis and the compilation of dictionaries. ‘Word sketches’, as they are called, have now become one of the main mechanisms for corpus analysis. For any given headword, a word sketch provides the lexicographer with a succinct summary of the most significant patterns of lexical and grammatical co-occurrence. An unintended - but certainly most welcome - side effect of word sketches is that they have largely resolved the problem of data overload (Rundell, Reference Rundell, Granger and Paquot2012: 22). No matter how big a corpus is, lexicographers can see a convenient short summary of a headword's behaviour. Their main task is then to validate the decisions made by the software. Rundell has emphasized that lexicographers are currently still required for tasks such as word sense disambiguation and definition writing, but he predicts that sophisticated software tools might one day take over these tasks entirely (Reference Rundell, Granger and Paquot2012: 28).
Current corpus tools provide well-known basic features such as KWICFootnote 4 concordances and collocational patterns but also advanced features such as the aforementioned word sketches and corpus query language (CQL) searches. Kilgarriff and Kosem (Reference Kilgarriff, Kosem, Granger and Paquot2012: 34) have attempted to classify corpus tools as follows: a) computer-based vs. online tools: WordSmith is perhaps the best-known example of a computer-based tool, requiring both the corpus and the software to be stored on the user's computer. Online tools, such as the Sketch Engine and Mark Davies's tools (available at http://corpus.byu.edu), can be accessed via any computer connected with the internet; b) corpus-related vs. corpus independent tools: Some tools, e.g. SARA, can only be used with the corpora for which they were originally programmed (in this case the British National Corpus), whereas the Sketch Engine and Word Smith, for instance, can be used with any corpus; c) prepared corpus vs. web as corpus: Most corpus tools are used to analyse corpora compiled with linguistic research in mind. Google and other web search engines can also function as corpus tools, allowing their users to access and interpret the vast amount of data available on the internet. Kilgarriff and Kosem (Reference Kilgarriff, Kosem, Granger and Paquot2012: 35) further distinguish between d) simple and advanced tools, the latter offering more sophisticated functionality, and discuss e) typical users, i.e. lexicographers, linguists, students of linguistics, language teachers and learners. In the following, some important corpus tools currently employed by dictionary compilers are discussed briefly.
The Sketch Engine
Since its launch in 2003, the Sketch Engine (created by Adam Kilgarriff and Pavel Rychlý) has become a leading tool for dictionary making. Aimed particularly at lexicographers, it can be used with corpora of all languages, types and sizes. Like other corpus tools, the Sketch Engine provides concordances in the KWIC format and gives word frequencies. Amongst its unique features are the so-called ‘word sketches’, which Kilgarriff et al. define as ‘one-page automatic, corpus-based summaries of a word's grammatical and collocational behaviour’ (Reference Kilgarriff, Rychlý, Smrz, Tugwell, Williams and Vessier2004: 105). Word sketches use a parser to identify, for example, verb-object pairs, subject-verb pairs, modifier-modifiee pairs, etc. and then apply statistical filtering to provide a result list (Kilgarriff, Reference Kilgarriff and Jackson2013: 84). First used for the Macmillan English Dictionary (MEDAL1, Reference Rundell2002), word sketches have become the preferred starting point for lexicographers who need to analyse complex headwords (Kilgarriff and Kosem, Reference Kilgarriff, Kosem, Granger and Paquot2012: 44).
The Sketch Engine provides further functions which have proven invaluable for dictionary makers. Perhaps most importantly, GDEX (Good Dictionary EXamples) ‘attempts to automatically sort the sentences in a concordance according to how likely they are to be goodFootnote 5 dictionary examples’ (Kilgarriff and Kosem, Reference Kilgarriff, Kosem, Granger and Paquot2012: 46). GDEX uses heuristics for readability and informativeness. With regard to readability, the heuristics ensure that sentences with infrequent words, proper names, unusual sentence or word length are ranked lower; informativeness heuristics favour sentences including typical collocates of the expression (Kilgarriff and Kosem, Reference Kilgarriff, Kosem, Granger and Paquot2012: 47). According to Kilgarriff (Reference Kilgarriff and Jackson2013: 91), GDEX enables lexicographers to find a suitable example in the top five suggestions, rather than, on average, within the top 20 to 30.Footnote 6 More recently, the Sketch Engine has been developed further to produce thesauruses as well as synonym and antonym comparisons. And, as Baisa et al. have shown (Reference Baisa, Jakubíček, Kilgarriff, Kovář, Rychlý, Abel, Vettori and Ralli2014: 505ff), lexicographers can now access bilingual word sketches, which provide the translation equivalent of a word sketch on the basis of parallel corpora, comparable corpora and bilingual dictionaries. For a more thorough discussion of the functions of the Sketch Engine, see Kilgarriff and Kosem (Reference Kilgarriff, Kosem, Granger and Paquot2012).
FrameNet and WordNet
The FrameNet database is another well-known corpus-basedFootnote 7 tool with the potential to support and accelerate dictionary making. Created largely by Charles Fillmore and initiated in 1997, FrameNet can be regarded as the practical implementation of Frame SemanticsFootnote 8 (Fillmore, Reference Fillmore and Zampolli1977). Amongst other tasks, it is employed for meaning analysis, sense differentiation and valence description. FrameNet primarily records information about the combinatorial requirements of the keyword in question, i.e. its syntagmatic relations (Atkins, Fillmore & Johnson, Reference Atkins, Fillmore and Johnson2003: 272). For a more detailed description of the relevance of FrameNet to practical lexicography, see Atkins, Rundell & Sato (Reference Atkins, Rundell and Sato2003).
Word sketch for linguist in the Sketch Engine
WordNet, called ‘the mother of all semantic networks’ by Fontenelle (Reference Fontenelle2012: 439), also plays a role in English lexicography. Superficially, it might be said to resemble a thesaurus in that it groups semantically related words. However, WordNet interconnects not just word forms, but specific senses of words, disambiguating terms that occur in close proximity to one another in the network (Fellbaum, Reference Fellbaum2014: 379). WordNet's relations are mainly paradigmatic, focussing on words from the same lexical class. A discussion of the strengths and limitations of WordNet for lexicographers is provided by Fellbaum (Reference Fellbaum2014).
Corpus tools for dictionary users
Some dictionaries - particularly English learners’ dictionaries - are not only corpus based, but actually include self-contained corpora on their CD-ROM or DVD supplements. The first English learners’ dictionary to offer direct corpus access to users was COBUILD5 (Reference Sinclair, Macaulay and Seaton2006). It included a small sample from the Bank of English - small in the sense that it represented less than 1 % of the entire corpus.Footnote 9 The Wordbank tool, as it was called, contained approximately five million words from different text types, differentiating between written and spoken as well as UK and US English. It also functioned as a concordance, and users could search not only for single lexemes but also for phrases.
Searching the corpus tool in COBUILD5
Other English publishers have followed Collins’ approach, e.g. Longman in the 5th edition of the Longman Dictionary of Contemporary English (LDOCE5, Reference Mayor2009). Unlike Collins, Longman integrated the corpus into the microstructure of the dictionary, i.e. as part of each lexical entry (cf. Paquot, Reference Paquot, Granger and Paquot2012: 168). On the other hand, the dictionary provided no concordance facility, making it impossible to use the corpus as a stand-alone tool. Paquot (Reference Paquot, Granger and Paquot2012: 169) has rightly emphasized that dictionaries should ideally not just follow one of these two approaches, but aim for a combination.
LEAD
The LEAD (Louvain English for Academic Purposes Dictionary), an online reference tool aimed at non-native speaker students and researchers who want to write academic English texts, offers precisely such a blend. It is a combined dictionary-corpus product that offers two ways of using its integrated corpora: as an additional example bank or via an independent corpus-query tool integrated into the dictionary (Paquot, Reference Paquot, Granger and Paquot2012: 170). Remarkably, users are only given examples pertaining to the discipline they have preselected - be it linguistics, medicine or business; alternatively, they can look up words in the academic component of the British National Corpus. LEAD has an integrated open source Web-based corpus tool named CQPweb. It allows users not only to search for any word form but also to retrieve exact word sequences (e.g. the pattern the + NOUN + of). Collocations are automatically identified by the software (Paquot, Reference Paquot, Granger and Paquot2012: 180). The most innovative aspect of LEAD, however, is the fact that the corpora it provides can be customized according to users’ needsFootnote 10. Future electronic dictionaries will hopefully imitate this approach.
Conclusion & outlook
As stated by Hanks (Reference Hanks, Granger and Paquot2012a: 63), ‘the impact of corpus data on synchronic lexicography since 1987 (the date of publication of COBUILD1) has been overwhelming’. Lexicographers now have sufficient evidence to make generalisations about the English language with reasonable confidence. Their attention is drawn to aspects of words that are likely to escape intuition altogether (cf. L'Homme & Cormier, Reference L'Homme and Cormier2014: 332). Corpora support various aspects of dictionary creation, from developing headword lists to identifying key features of lexemes to providing examples. In particular, word sketches have become an invaluable tool for lexicographers, who usually find it much more efficient to start from the sketch than from the concordance (Kilgarriff, Reference Kilgarriff and Jackson2013: 85). Word sketches show the various patterns of meaning and usage that a headword enters into, primarily through collocationsFootnote 11. Incidentally, ‘the unexpected consequence [of word sketches] was that the lexicographer's methodology changed, from one where the technology merely supported the corpus-analysis process, to one where it pro-actively identified what was likely to be interesting and directed the lexicographer's attention to it’ (Kilgarriff, Reference Kilgarriff and Jackson2013: 85). The lexicographer's task has thus shifted towards validating and completing the data gained from corpora (cf. Kosem et al., Reference Kosem, Gantar, Logar, Krek, Abel, Vettori and Ralli2014: 355). The ever-growing size of current corpora - originally a problem for those interpreting the data - makes the results even more reliable.
The potential of dictionary-cum-corpus tools such as COBUILD5, LDOCE5 and LEAD for dictionary users in general and learners in particular can hardly be overstated. The additional corpus data can serve as a syntactic model for correct usage or as the basis for lexical analysis. As discussed above, corpus data needs to be pre-modified to make it suitable for the intended user group. Frankenberg-Garcia (Reference Frankenberg-Garcia2012: 289) has argued that learners would probably benefit from separate example sections related to decoding (i.e. language comprehension) and encoding (i.e. language production). Her studies have shown that ‘it is quite possible for learners to understand new words and correct their own frequent mistakes by looking at examples alone’ (Frankenberg-Garcia, Reference Frankenberg-Garcia2012: 289), an approach known as data-driven learning. Apart from their primary linguistic function, corpora also play an important didactic role for dictionary users. By employing these tools, users become linguists themselves, and their interest in the language and its structures is thus fostered.
This discussion has sought to demonstrate that corpora have become an indispensable tool for both dictionary compilers and users. Dictionary making has become more accurate, efficient and consistent (Kilgarriff, Reference Kilgarriff and Jackson2013: 94). And the inclusion of corpora within dictionaries has made dictionary use more reliable, versatile and motivating.
Dr. Reinhard Heuberger is an Assistant Professor in the Department of English at the University of Innsbruck. His research focuses on lexicography (learners' dictionaries and online dictionaries), ecolinguistics (and human-animal studies), and English dialectology. He was the co-director of the government-funded project SPEED (2006–2010), and now co-directs its follow-up project EDD Online, both concerned with the digitization and investigation of Joseph Wright's English Dialect Dictionary. reinhard.heuberger@uibk.ac.at
