1. Language teachers and corpora

Many language teachers in the EFL context have been using corpus-based dictionaries, grammars and textbooks for some time now without actually knowing what a corpus is. Indeed, understanding corpora and how to use them is not strictly necessary when the language teaching community resorts to off-the-peg corpus-based materials designed by experts. The picture changes when teachers are encouraged to handle corpora themselves, to promote data-driven learning in classroom activities such as those put forward by Johns & King (Reference Johns and King1991), Tribble & Jones (Reference Tribble and Jones1997), Aston (Reference Aston2001), Sinclair (Reference Sinclair2004), Bennet (Reference Bennet2010), Reppen (Reference Reppen2010), and many others.

In spite of the enthusiastic acceptance of the few language teachers who have managed to break the barrier and have put their hands on corpora, only a very small number of teachers have actually tried using corpora directly. In the survey that Tribble circulated on the Linguist List (www.linguistlist.org) and on Corpora (http://torvald.aksis.uib.no/corpora/) ten years ago, only 52.8% of the 89 respondents declared they had used corpora in their teaching (Tribble Reference Tribble2001). This percentage is extremely low if we bear in mind that the readers of these lists are far more likely to know about corpora than the average language teacher. In what seems to be the only available well-documented survey on the use of corpora in actual schools, Mukherjee interviewed 248 secondary school teachers in Germany and found that nearly 80% had never even heard of corpora (Mukherjee Reference Mukherjee, Connor and Upton2004). And although the number of free, online corpora that can be used by anyone with access to computers and the Internet has taken a giant leap over the past few years (Anderson & Corbett Reference Anderson and Corbett2009), the fact that several scholars continue to show concern over the still very limited use of corpora beyond a small community of experts reinforces the notion that there is a long way to go before corpora can be understood and used by language teachers in general (Mukherjee Reference Mukherjee, Connor and Upton2004, Reference Mukherjee, Braun, Khon and Mukherjee2006; Braun Reference Braun2005; Römer Reference Römer2006, Reference Römer and Aijmer2009; Boulton Reference Boulton2009; Breyer Reference Breyer2009; Frankenberg-Garcia Reference Frankenberg-Garcia2010; Gilquin & Granger Reference Gilquin, Granger, O'Keeffe and McCarthy2010).

Yet corpora are far from being a secret closely guarded by a restricted community of corpus linguists. On the contrary, there is a wealth of information on the specific uses of corpora in language teaching and a growing body of literature that strives to make corpus resources in general accessible to non-experts. This includes: (1) books about using corpora in language teaching (for example, Johns & King Reference Johns and King1991; Tribble & Jones Reference Tribble and Jones1997; Burnard & McEnery Reference Burnard and McEnery2000; Aston Reference Aston2001; Kettemann & Marko Reference Kettemann and Marko2002; Aston, Bernardini & Stewart Reference Aston, Bernardini and Stewart2004; Sinclair Reference Sinclair2004; Hidalgo, Quereda & Santana Reference Hidalgo, Quereda and Santana2007; O'Keeffe, McCarthy & Carter Reference O'Keeffe, McCarthy and Carter2007; Aijmer Reference Aijmer2009; Anderson & Corbett Reference Anderson and Corbett2009; Bennet Reference Bennet2010; Reppen Reference Reppen2010; Frankenberg-Garcia, Flowerdew & Aston Reference Frankenberg-Garcia, Flowerdew and Aston2011; Kübler forthcoming); (2) online introductions to corpora in language teaching (see examples in appendix); (3) corpus-specific tutorials (see examples in appendix); and (4) countless conference papers and journal articles.

One of the generally acknowledged reasons why the direct use of corpora in language teaching has not caught on is that the majority of corpus resources are neither pedagogically oriented nor user friendly. In spite of the recent development of a few remarkable projects like the SACODEYL corpus (Widmann, Khon & Ziai Reference Widmann, Kohn, Ziai and Frankenberg-Garcia2011) and IFAConc (Kaszubski Reference Kaszubski and Frankenberg-Garcia2011), most corpora that can be used in the classroom were originally conceived for research rather than for teaching purposes. In this presentation I would like to argue that another major drawback is that so far little attention has been paid to training teachers in basic corpus skills.

2. Novice corpus users

With a limited number of teachers using corpora, it comes as no surprise that there do not seem to be any studies of this particular kind of user behaviour. Some of the difficulties encountered by novice corpus users in general are, however, described by Bernardini (Reference Bernardini, Burnard and McEnery2000), Kennedy & Miceli (Reference Kennedy and Miceli2001), Frankenberg-Garcia (Reference Frankenberg-Garcia2005), Bianchi & Manca (Reference Bianchi and Manca2006) and Santos & Frankenberg-Garcia (Reference Santos and Frankenberg-Garcia2007). Although these studies differ quite substantially from one to another, they all converge to suggest that corpus skills that come as second nature to experts are not at all obvious to the untrained. Apart from corpus-specific difficulties in handling different search interfaces and query languages – and the human–computer interaction issue should not be overlooked – these studies bring to light a number of very basic problems that novice users encounter, no matter which corpus or concordancer they use.

To begin with, novice users do not always know how to choose between different types of corpora. They tend to overlook factors as seemingly obvious as provenance and publication dates, what text types are represented and how big different corpora are. In classroom observations of undergraduates using corpora in applied translation, for example, I have come across students choosing the British National Corpus (BNC), a general language corpus, to search its concordances for highly technical terms such as electrostatic precipitator, or for looking up neologisms such as Bluetooth technology, which were not yet in use when the BNC was compiled in the early nineties. In her description of language students exploring corpora for the first time, Bernardini (Reference Bernardini, Burnard and McEnery2000) noted that the students were content to use the full BNC instead of narrowing it down to the specific sub-corpora that were relevant to their particular queries. Paradoxically, in our analysis of the log files of COMPARA – a much smaller, three-million-word parallel corpus of English and Portuguese (Frankenberg-Garcia & Santos Reference Frankenberg-Garcia, Santos, Zanettin, Bernardini and Stewart2003) – Diana Santos and I were able to observe that certain users tended to limit their searches to minute sub-corpora that did not return any hits for their queries, although they would have found relevant and useful hits if they had chosen to use less restrictive sub-corpora (Santos & Frankenberg-Garcia Reference Santos and Frankenberg-Garcia2007).

Novice users also appear to experience considerable difficulties in formulating corpus queries. For example, in the same study of log files from the COMPARA corpus (Santos & Frankenberg-Garcia Reference Santos and Frankenberg-Garcia2007), we found records of queries reflecting serious misconceptions about the kind of information that can be retrieved from a corpus, including queries as absurd as the string this still did not give me the happiness I thought it would or for which I sought. Logs with queries such as water shining, bill quantities and like a manor also suggest that people who are new to corpora have very little idea of the way chunks of words behave. Indeed, an in-depth analysis of 1000 queries returning null results in this corpus shows that only around 20% had actually been plausible queries. Some of the problems identified could be traced back to users applying dictionary and web browsing strategies to corpora, but most of them seemed to reflect a complete failure to grasp the very basics of corpus exploration (Santos & Frankenberg-Garcia Reference Santos and Frankenberg-Garcia2007).

Existing studies on corpus-user behaviour also report that many of the queries by newcomers to corpora are either too general or too restrictive, and that apprentice users will often give up searching instead of trying to reformulate their queries. For example, in their classroom observations of undergraduates using corpora, Kennedy & Miceli (Reference Kennedy and Miceli2001) noted that the students rarely carried out follow-up queries in order to refine their searches. Likewise, when analysing log files from COMPARA to check what happened after a user obtained a null result, we found that in almost half the user sessions we examined, people simply ended the session instead of trying out a modified or an altogether different query (Santos & Frankenberg-Garcia Reference Santos and Frankenberg-Garcia2007).

Another important point to bear in mind is that newcomers to corpora may find it difficult to interpret corpus data. My classroom observations of students using corpora directly show that they usually have no qualms about accepting as positive evidence results returning just one or two hits. If they see something in the British National Corpus, they tend to assume it corroborates typical English usage, even when there is not enough evidence to attest this. It also seems very easy for novice users not to take corpus size and composition into account when interpreting their results. In an end of term exam, I asked a group of Portuguese undergraduate translation students to compare the distribution of the misspelled Portuguese verb *caiem ‘fall’ in CETEMPúblico, a Portuguese newspaper corpus of 180 million words and in DiaCLAV, a smaller comparable corpus of 6.7 million words (both available online at www.linguateca.pt). The larger corpus had 44 instances of this spelling mistake and the smaller one had six. However, only one out of sixteen students managed to conclude that CETEMPúblico contained comparatively fewer mistakes. All other students made the simplistic assumption that the larger corpus was less accurate.

In a study where I examined how students used dictionaries, corpora, terminology databases and other language resources during a translation task, I noted that the students found it difficult to deal with unedited language and felt more comfortable using resources mediated by lexicographers and terminologists than corpora, which require autonomous user interpretation (Frankenberg-Garcia Reference Frankenberg-Garcia2005). In this same study, I compared the students’ perceptions of the usefulness of corpora with the effects of corpus lookups on their translations. The few students who had used corpora to assist them with their translations mistakenly felt that corpora had helped them more than they had actually done. Similarly, Kennedy & Miceli (Reference Kennedy and Miceli2001) wrote that their students were frequently lured by misleading near-matches in their results and tended to make a summary analysis of the concordance lines retrieved, forgetting to pay attention to both co-texts and contexts. Bianchi & Manca (Reference Bianchi and Manca2006) also observed this behaviour and reported that their students often failed to enlarge a concordance word span in order to arrive at a more significant analysis, or even to select a relevant concordance line for analysis in the first place.

3. Developing basic corpus skills

The findings and observations I have described suggest that language teachers who are new to corpora may find it difficult to grasp that corpora do not work in the same way as the familiar language learning resources – such as dictionaries, grammar books and textbooks – that they are accustomed to using. Although most corpora provide detailed information about their composition, non-experts cannot be expected to understand the full significance of this without being explicitly taught how to do so.

To begin with, there is the question of selecting an appropriate corpus. While most people are aware of the differences between large and small dictionaries – indeed, they come in different sizes and weights – the distinction between a corpus of 100 thousand words and a corpus of 100 million words is not readily noticeable, nor are the implications of such a distinction obvious. Awareness that different corpora use different criteria for text selection is also important. Teachers are familiar with what distinguishes monolingual from bilingual from specialized language dictionaries, but the differences between corpora of newspapers, fiction, business letters, parliamentary debates and general language may not be immediately visible to them. Likewise, the distinctions between corpora of spoken and written texts or between monolingual and multilingual corpora can be just as elusive.

The ability to devise reasonable corpus queries is also something that cannot be taken for granted. Language teachers who feel comfortable looking up headwords in dictionaries and are familiar with using search engines on the Web – where it is actually very difficult not to come up with some sort of search result – may not be aware that the principles underlying corpus exploration are different. They may need help in developing the basic skills required to formulate meaningful and plausible queries.

And, of course, there is the question of interpreting corpus results. Teachers who are used to dealing with language learning materials that have been carefully selected and edited by experts cannot be assumed to know how to handle raw data samples containing language mistakes and idiosyncrasies, too many or not enough hits, and unforeseen findings that can go against their intuitions. We cannot presume that teachers will be automatically able to derive reasonable conclusions from all the information provided by corpora without any training. As corpus output is very different from the polished materials normally used in the classroom, teachers may need some help to decode the results of their initial corpus searches.

4. Task-based, consciousness-raising corpus exercises

In the remaining part of this presentation I would like to propose a few consciousness-raising exercises that can help language teachers assess different corpora, develop basic corpus-searching strategies, and begin to learn how to interpret corpus data. The exercises proposed are task-based and, unlike most corpus tutorials available, they are not corpus-specific. Rather, they were conceived for different corpora that are available online and do not require users to pay subscription fees, so they are easily accessible to anyone with a computer and an Internet connection. Although similar exercises can be devised for practically any corpus, those presented here make use of the following English language corpora and multilingual corpora containing English: the BNC via the Simple Search service provided by the British Library and Mark Davies’ BYU interface (Davies Reference Davies2004), the Corpus of Contemporary American English (COCA), also developed by Mark Davies (Davies Reference Davies2008), Collins Wordbanks Online (CWO), also known as the COBUILD Concordance and Collocation Sampler, the Business Letter Corpus (Someya Reference Someya2000), COMPARA (Frankenberg-Garcia & Santos Reference Frankenberg-Garcia, Santos, Zanettin, Bernardini and Stewart2003) and the EuroParl Corpus via the OPUS interface conceived by Jörg Tiedemann (Tiedemann Reference Tiedemann, Nicolov, Bontcheva, Angelova and Mitkov2009). The exercises proposed focus on some of the fundamental principles of corpus querying in general, rather than on the capacities and the query languages of the search interfaces of each corpus in particular.

4.1 Understanding different corpora

Perhaps the most basic way of helping novice corpus users to become aware of the implications of using different types of corpora is to ask them to take a few different types of corpora, read the details about the texts they contain, and compare them. Explicit questions on the size and composition of each corpus may suffice to help people who are not familiar with corpora to understand that, just as there are different types of dictionaries, corpora too may – and often do – differ. I provide an example of this very simple consciousness-raising exercise in Table 1.

Table 1

Comparing corpora I

Another way of helping novice users to assess different types of corpora is to ask them to repeat the same query in a number of different corpora and compare the results obtained. The task-based exercise I present in Table 2 is an example of how this can be achieved. The different types of keywords selected for comparison were deliberately chosen so that they (a) would highlight important differences between the different corpora in analysis and (b) were single words so that novice users would not, at this stage, be confused by different query languages.

Table 2

Comparing corpora II

Just as dictionary users might want to try out a number of different look-ups to get a feel for a given dictionary, new corpus users can be encouraged to try out different queries in a specific corpus in order to obtain a measure of its capacities and limitations. In Table 3 I give an example of a task-based exercise aimed at helping new corpus users become familiar with the contents of a specific corpus.

Table 3

Assessing a specific corpus

4.2 Formulating corpus queries

Apart from the need to become aware of what different corpora are composed of, novice users can also benefit from being explicitly taught how to formulate basic corpus queries.

4.2.1 Single words

People who are learning to use corpora should perhaps realize first of all that some corpus-browsing software, unlike many search engines and search facilities in word processors, is by default sensitive to upper and lower case distinctions, while others may require users to ignore diacritics and capital letters altogether. For example, in his tutorial on the use of the CWO corpus, Thomas (Reference Thomas2002) prompts novice users to try out a search for BBC (in capital letters) and then explains that it will only work if we counterintuitively search for bbc (in lower case). In contrast, in the OPUS interface to EuroParl and in the COMPARA corpus, both of which use the CPQ query language (Evert Reference Evert2010), users will find hits for BBC only if they use capital letters.

Diacritics can also be important. Users will get no results if they look up café in CWO, but if they type in cafe (without an accent) they will get concordances for both café and cafe. In contrast, in COMPARA and the OPUS interface to EuroParl, cafe and café are processed separately.

When users are trying out a corpus for the first time, they should be encouraged to experiment with capital letters and accents in order to understand how the software treats diacritics and upper and lower case distinctions. They should come to realize that different corpus tools tend to deal with these non-trivial details differently.

Another point that must be made explicit is that corpora are sensitive to word inflections. People who are used to employing dictionary strategies should learn that looking up uninflected forms in corpora will only retrieve uninflected forms. A very simple, task-based exercise to make new corpus users aware of this is to ask them to carry out queries for uninflected forms (e.g. look) and different inflections (e.g. looks, looked, looking), and then discuss the results obtained. Having done this, they can then be encouraged to read the information about the query language associated with a specific corpus in order to find out if there is a shortcut to capturing the different inflections of a word in a single query.

4.2.2 Strings of words

Once novice users are familiar with the basics of single-word queries, they can begin to get acquainted with queries involving strings of words. Perhaps the first point that apprentice users need to bear in mind is that, unlike search engines, corpora require people to retain words such as the, of and it, which when entered into search engines are treated as ‘stopwords’ and ignored, to speed up searches. In the OPUS interface to EuroParl, for example, there are eighty-five hits for European Year of Languages. However, if we leave out the word of and enter the expression European Year Languages instead, we get absolutely no hits at all. Novice users who are encouraged to carry out these two queries will be able to conclude for themselves that, when using corpora, words such as articles and prepositions should not be ignored.

Once apprentice users realize that corpus searches need not be limited to single-word queries, they also need to understand that while it is usually possible to retrieve frequent combinations of words from corpora, strings of words put together in ways that have not been used before or that have only been used in very limited contexts are very unlikely to be found. In Tables 4 and 5 I propose two different task-based exercises to help novice users understand this.

Table 4

Expansion and reduction exercise

Table 5

The trigram exercise

The exercise in Table 4 shows learners that there are no instances of the string It was fine as far as I could see in the whole of the BNC. From that, novice users can be led to conclude that, even though it is a perfectly well-formed English sentence, it is a unique utterance which is not represented in any of the texts that make up the BNC. In contrast, they will also be able to see that English speakers often use clusters of words like it was fine and as far as I could see in that precise combination and order. They should also be able to notice that as the word strings get shorter, the frequencies get higher. The idea is for them to deduce that the shorter the sequence of words, the more likely it is that other people have used it before.

When novice corpus users do the exercise in Table 5, they should realize that although all clusters are made up of exactly three words each, there is a dramatic difference between them with regard to the frequency with which they are represented. The tri-grams as a rule, a rule of, rule of thumb, you need a, a litre of and square metres of are clusters of words often used by native speakers, while sequences like of thumb you, of paint to, every 12 square, and so on are not. This should help people who are new to corpora understand that it is not just the length of a string of words that matters: it is also the actual words chosen, together with the exact order in which they occur. Apprentice users should realize that when using corpora to look up strings of more than one word, we can only realistically expect to find sequences that occur in clusters that represent conventional uses of the language appearing in the corpus.

4.2.3 Reformulating queries

Another important point to introduce to novice corpus users is that queries often have to be reformulated. They should find it helpful to be explicitly reminded that they might have to broaden word queries so as to capture words with alternative spellings such as generalise and generalize. More commonly, however, single-word queries may have to be narrowed down. For example, newcomers to corpora should realize that homographs – such as look (V) and look (N) – need to be filtered if they want to capture just one of the meanings of the word. Using COCA or the BYU-BNC, both of which have user-friendly, drop-down part-of-speech (POS) annotation options, it is quite simple to ask new users to run queries with and without POS annotation in a series of short, hands-on exercises to draw attention to these intricacies.

Multiple-word queries, in contrast, are often too restrictive. Apprentice users can be shown that a multiple-word query that produces no results at first can be made more flexible by replacing certain elements of the string with wildcards or, if the corpus is grammatically tagged, with POS categories. For example, they can be asked to look up a search string like give Barbara a hand in COCA or the BNC, which will return no results. They can then be asked to replace Barbara with a wildcard and run the query again, which will give them plenty of analogous hits which sanction the use of the string give Barbara a hand. As a follow-up, they can then be shown how to replace Barbara with a noun or pronoun.

Many of the intricacies of single and multiple-word queries are difficult to anticipate, and even experienced corpus users may fail to foresee them. Novice users must learn that corpus querying is often a recursive process, and that the unsatisfactory results we get from initial queries can teach us how to improve our subsequent attempts to extract the information we are after.

5. Concluding remarks

Several of the studies I have referred to here indicate that there is a long way to go before corpora can be understood and used by language teachers in general. A closer look at novice-user behaviour suggests that a major hurdle on the way to popularizing corpora is that so far little attention has been paid to training newcomers in basic corpus skills. Experts who use corpora routinely usually take these skills for granted, but language teachers who are not yet very familiar with corpora may need help in order to understand how to choose between different types of corpora, how to retrieve information from a corpus and how to evaluate that information.

I have presented a series of task-based, consciousness-raising exercises aimed at helping language teachers understand the basics of corpora. Exercises such as those proposed may be too easy for experts, but the ideas underlying them are not necessarily self-evident for people who have never used corpora before.

I have only had time to describe a limited number of exercises addressing just a handful of issues that are important to bear in mind when using corpora. Many more straightforward, hands-on, consciousness-raising exercises can be created. Of course, if you have ever tried doing so, you will know that it is actually quite hard to produce pedagogic materials that will make complex corpus concepts and procedures look simple to newcomers.

Yet exercises such as these can be very important if we are to make the direct use of corpora in the classroom more popular. After all, corpora do not work in the same way as the familiar language-learning resources – such as dictionaries, grammar books and textbooks – that language teachers are accustomed to using. Helping teachers who are new to corpora to take their first steps in using corpora autonomously will hopefully encourage them to want to find out more about using corpora in the classroom. And there is no lack of follow-up material available: as I mentioned at the beginning of this paper, there is a growing body of literature that strives to make corpus resources in general accessible to non-experts, including general introductions to corpora, corpus-specific tutorials, and books and articles on how to use corpora in language teaching.