1.1 Types of data-driven learning and corpus-informed textbooks

Despite the research supporting DDL, a recent trend over the last decade in developing corpus-informed grammar textbooks, has not been the incorporation of an inductive approach to grammar learning, but rather the presentation of information from corpora and corpus-based research about grammar in an explicit, rule-centered framework which runs counter to the inductive principles inherent in DDL (cf. Conrad & Biber, Reference Conrad and Biber2009; McCarthy, McCarten & Sandiford, Reference McCarthy, McCarten and Sandiford2005; Reppen, Bunting, Diniz, Blass, Iannuzzi & Savage, Reference Reppen, Bunting, Diniz, Blass, Iannuzzi and Savage2012). While there are examples of corpus-driven grammar books that are not purely rule-based (e.g., Thornbury, Reference Thornbury2004), they remain rare. The apparent contrast between DDL research on grammar instruction and corpus-informed grammar textbooks could, perhaps, be dismissed as an effect of the fundamental difference between a textbook and an in-class approach to instruction based around learner-centered, computer-based activities.

However, over the history of DDL, there has been a range of approaches to learning from corpora as outlined in Mukherjee (Reference Mukherjee2006: 12), varying in the degree of learner autonomy and the degree to which computers and concordancing software have been used directly by learners. Since Johns’ early work in DDL, a popular instructional strategy has been to have learners interact directly with corpora using concordancers to conduct their own searches of language features and patterns. However, as an example of rethinking traditional computer-based DDL, Boulton (Reference Boulton2010b) demonstrates that DDL can still be effective with prepared, preselected inductive activities based on corpora, and that this type of less autonomous, paper-based DDL may actually be more effective with certain groups of learners.

In fact, interacting directly with corpora is not a necessary part of DDL, nor is it always desirable. Hands-on DDL faces several hurdles, including helping learners understand what a corpus is and why it would be a useful tool in language learning, familiarizing learners with a concordancer (often using a software interface not in their L1), and helping learners decide what language features can be used with the software. A teacher must carefully consider these potential challenges when considering implementing DDL in a grammar classroom. Boulton (Reference Boulton2010b) suggests that when working with lower-level learners especially, using computers for DDL is possibly an unnecessary complication when activities with pre-prepared DDL materials can accomplish many of the same goals. There are other situations, additionally, where using prepared DDL materials instead of having learners interact directly with corpora may be more beneficial. For learners who are unfamiliar with using software in language learning and for those who are unaccustomed to inductive or learner-centered activities, the challenge of directly interacting with corpora may have the unintended result of inhibiting learning instead of being a benefit.

So DDL, in its use over the last few decades, is not necessarily characterized by direct interaction with language corpora or by total learner autonomy. Rather, I propose that two particular characteristics define DDL:

1. 1) real language data are used as sources of language learning materials or reference resources;

2. 2) learning activities are student-centered and focus on language discovery.

DDL, under this definition, is not based on having learners work semi-autonomously with concordancing software, but rather relies on carefully designed and scaffolded activities to help learners discover language structures through working with real language samples, whether on computer or not. In this sense, explicit presentations of rules in corpus-informed grammar textbooks and discovery learning through DDL are not solely distinguished from one another by mode of delivery, but also in their understanding of how grammar is learned. Only the first characteristic listed above, using language from authentic texts as a tool for learning, has been a consistent link between DDL research and the recent emergence of corpus-informed textbooks. In fact, the benefits of using corpus-based language samples and language descriptions have been the foundation of both DDL research and the development of corpus-informed grammar textbooks.

In both hands-on and paper-based DDL the pedagogical approach focuses on induction as a learning strategy (Flowerdew, Reference Flowerdew2009; Mukherjee, Reference Mukherjee2006). The assumption is that there is an inherent connection between inductive language discovery and corpora as learning tools, and this assumption is not shared by the aforementioned corpus-informed grammar textbooks like the Touchstone series (McCarthy et al., Reference McCarthy, McCarten and Sandiford2005), Real grammar (Conrad & Biber, Reference Conrad and Biber2009), or Grammar and beyond (Reppen et al., Reference Reppen, Bunting, Diniz, Blass, Iannuzzi and Savage2012). Rather, these texts present information about language deductively with explicit information about grammar rules and patterns. The existence of textbooks in parallel with DDL-based approaches to grammar instruction outlines an important question, in the current research on learning with corpora, of whether instruction benefits from an inductive or a deductive approach. When the current research on DDL has shown positive learning outcomes, are these gains attributable to the use of corpus-informed teaching materials and tools, or to the inductive approach of DDL, or both?

1.2 Guided induction in data-driven learning

At the less autonomous end of Mukherjee’s (Reference Mukherjee2006) cline of DDL, learning through preselected concordance materials does not necessarily entail a purely inductive approach. Rather, the language discovery activities are guided and facilitated by an instructor (Boulton, Reference Boulton2010b; Flowerdew, Reference Flowerdew2009; Stevens, Reference Stevens1991). Research on the pedagogical approach of guided induction has sought to examine best practices of teacher-facilitated discovery learning in second language teaching. Sinclair (Reference Sinclair2003), in particular, models how facilitated discovery learning can be accomplished through the use of concordancing tools. In L2 grammar pedagogy, the question of inductive or deductive approaches is especially relevant, as explicit grammar teaching has historically first lost and then regained favor among language teachers and researchers over the last few decades (Ellis, 2006; Hammerly, 1975; Herron & Tomasello, Reference Herron and Tomasello1992), resulting in no clear agreement on the best ways to teach grammar, regardless of the source or type of teaching materials – or even if grammar should be taught at all (Haight, Herron & Cole, Reference Haight, Herron and Cole2007). While there is no one universally approved form of explicit instruction (whether deductive or inductive), there is some consensus among language learning researchers that explicit instruction in grammar does benefit language learners (Herron & Tomasello, Reference Herron and Tomasello1992; Norris & Ortega, Reference Norris and Ortega2000).

Guided induction, as an approach that provides a structured, scaffolded framework for inductive learning, places the learner at the center of the learning task, with the learner seeking to discover the nature of the grammar structure through interacting with the language. Many early inductive approaches were criticized for simply having learners work with examples first before the teacher presented the grammar rules; instead, as Herron and Tomasello (Reference Herron and Tomasello1992: 709) observe, the learner’s involvement in discovering the language may help him/her to internalize the target items. Guided induction is a particular evolution of the inductive approach where learners are first presented with language samples in the form of an interactive task that guides them to discover the language structure they contain; the teacher has an active role in facilitating these tasks, but typically does not present explicit grammar rules. Learners are then guided to produce the language structure in meaningful communicative tasks (Haight et al., Reference Haight, Herron and Cole2007; Herron & Tomasello, Reference Herron and Tomasello1992; Shaffer, Reference Shaffer1989).

Proponents of guided induction contend that this approach allows learners to develop generalizable abilities to formulate and make their own observations about language. An additional benefit is that through inductive learning tasks, which are potentially more interactive than traditional deductive tasks, learners engage more and receive direct feedback from their language teachers as they seek to construct and refine their hypotheses about the targeted language structures (Herron & Tomasello, Reference Herron and Tomasello1992: 716). Critics of discovery-oriented learning, however, point out that the preponderance of educational research indicates that more learning occurs when learners are given greater guidance (e.g., Kirschner, Sweller & Clark, Reference Kirschner, Sweller and Clark2006). Minimally guided approaches to instruction, according to Kirschner et al., compared with more guided approaches, do not lead to similar gains in long-term retention or skills development. The relevant recommendations, based on a survey of the last half-century of research on various forms of minimal guidance instruction by these researchers, are that learners need to be provided with adequate guidance and scaffolding, and that educators should not confuse the general practices of inquiry and research in a discipline as being the same as what learners need to learn about the field.

In the case of language instruction, Flowerdew (Reference Flowerdew2009) outlines a guided approach to DDL where the instructor helps learners through the language discovery process without explicitly providing them with grammar rules, but with accommodation for varying levels of teacher guidance and scaffolding. This approach, first outlined as three steps for corpus-based research enquiries by Carter and McCarthy (Reference Carter and McCarthy1995), has been updated to include an intermediary intervention step by Flowerdew (Reference Flowerdew2009):

1. 1. Illustration: looking at data.

2. 2. Interaction: discussion and sharing observations and opinions.

3. 3. Intervention: optional step to provide learners with hints or clearer guides for induction.

4. 4. Induction: making one’s own rule for a particular feature.

Consequently, corpus-based activities in this guided inductive approach would begin with learners looking at pre-selected examples of language data. They are then guided by the teacher to make observations, typically through group- or pair-based problem-solving activities, to identify patterns or trends about the language data. At this point, the teacher can if necessary intervene with careful hints or suggestions for the learners to help them accomplish the problem-solving activities. Finally, the learners are guided to complete subsequent activities using the patterns of the grammar feature under review.

1.3 The range of corpora in DDL for grammar instruction

An additional methodological distinction between the emergence of corpus-informed grammar textbooks and the development of DDL grammar teaching materials and tools lies in the differences in the types of corpora used. Whereas corpus-informed textbooks are often based around large general corpora designed to represent broad areas of language use (cf. Conrad & Biber Reference Conrad and Biber2009; Reppen et al., Reference Reppen, Bunting, Diniz, Blass, Iannuzzi and Savage2012), classroom research into using DDL as an instructional approach has drawn from a much wider range of corpora.

Some studies of teaching grammar with hands-on DDL have employed large, general corpora that are accessible through web interfaces. These include corpora like COBUILD (Estling Vannestål & Lindquist, Reference Estling Vannestål and Lindquist2007; Yoon, Reference Yoon2008), along with other large, web-based corpus tools (Gaskell & Cobb, Reference Gaskell and Cobb2004; Hanafiyeh & Keshi, Reference Hanafiyeh and Keshi2013). Other studies have used corpora built to focus on specific areas of language use or even with a teaching focus, such as using parallel corpora that present matching corpus lines in the target language and the language of instruction (Chujo, Anthony & Oghigian, Reference Chujo, Anthony and Oghigian2009; Chujo & Oghigian, Reference Chujo and Oghigian2012; Schmied, Reference Schmied2006). Belz and Vyaktina (2005a, 2005b) had learners build their own corpus and work on both hands-on and paper-based learning activities; in other studies, researchers themselves have built smaller corpora for specific pedagogical purposes (C. Chang & Kuo, 2011; P. Chang, 2012; Hadi & Alibakshi, 2012).

More specifically, in studies where the focus has been on paper-based DDL grammar instruction, the range of corpora is similarly broad, but here it is the teacher who interacts with the corpora. Whereas in hands-on studies the learners interact directly with corpora, in paper-based studies, the researchers use corpora as a tool for developing instructional materials (most often in the form of concordance lines or activities derived from concordances). Boulton (Reference Boulton2008, Reference Boulton2010b) used the British National Corpus for developing instructional materials in paper-based DDL; Sripicharn (Reference Sripicharn2003) similarly used the Bank of English to produce concordance data for DDL activities; and Tian (Reference Tian2005a, Reference Tian2005b) developed paper-based instructional materials using online news sources.

2.1 Participants

Volunteer participants were recruited from three advanced grammar classes in an intensive English program at a regional public American university. The learners were L2 learners of English who had completed, on average, 5.4 years of English study, and had a mean age of 22. In addition to the grammar class, the learners also took courses in listening and speaking, composition, and reading and vocabulary. Of the 49 participants, 38 were male and 11 were female. All participants were from one of two L1 language backgrounds: Chinese was the first language for 12 participants and Arabic for the remaining 37. Neither sex nor the L1 were controlled for, as pre-existing classes were used for the study. However, the three groups (DDL, DCI, and TGI) had similar distributions of Chinese and Arabic speakers and of males and females (Table 1).

Table 1

Participant information by treatment group

Prior to the instructional treatment, all participants completed a general test of grammar knowledge based on the structure section of the paper-based TOEFL (i.e., section 2) (Table 2). A one-way ANOVA of the participants’ mean scores indicated no significant differences between the three treatment groups, F (2, 43)=.988, p=.38.

Table 2

Mean scores on general test of grammar ability

The researcher was also the classroom instructor for all three groups. This decision was made in order to maintain as consistent instruction as possible across the three groups and to control for variation in teaching style and related effects. In order to reduce the possibility of influencing the outcome of the instructional treatments (i.e., via a Hawthorne effect), the teacher kept a reflective journal of his own behavior in each of the classes, both prior to and during the instructional treatments, as a way to monitor interactions with the treatment groups. Each of the three classes met twice weekly for one-hour sessions, and the study was conducted over a three-week period, with four class periods (i.e., four hours) dedicated to instruction on the active and passive voice.

2.2 Instructional treatments

Each grammar class received instruction on the passive voice using a different instructional approach. For the two corpus-informed treatment groups (DDL and DCI), teaching materials were developed using available corpus-based descriptions of the passive voice (Biber, Johansson, Leech, Conrad & Finegan, Reference Biber, Johansson, Leech, Conrad and Finegan1999; Conrad & Biber, Reference Conrad and Biber2009); an example of a data-driven activity is provided in Appendix A. Grammar teaching materials for the two approaches to corpus-informed instruction were developed using an operational framework for describing the passive voice based on Celce-Murcia and Larsen-Freeman’s (Reference Celce-Murcia and Larsen-Freeman1999; also Larsen-Freeman Reference Larsen-Freeman2001) three-dimensional model of Form – Meaning – Use. Information on use included simple register differences in use (between speech and academic writing) as well as differences in use between ‘short’ and ‘long’ passives.

Authentic language samples were also used in developing materials, the samples being taken from academic prose and news articles from two publicly available online corpora: the TIME Magazine corpus (Davies Reference Davies2007) and the Corpus of Contemporary American English (Davies Reference Davies2008). Whenever possible, the language samples were used for materials for both corpus groups.

In the DDL group, the instructional approach was based on the principles of guided induction and followed the ‘four I’s outlined in Flowerdew (Reference Flowerdew2009): illustration, interaction, intervention, and induction. Specifically, the participants received preselected language samples (as printed concordance lines) that were designed to illustrate specific differences in form, meaning, and use of the passive voice. The participants worked in small groups and in jigsaw activities to identify and explain what they found in the examples. Following the discovery activities, they shared their findings with the class in a discussion activity, revised their findings according to feedback from classmates, and then completed additional productive writing activities where they applied their rules.

The DCI group used corpus-informed materials in a deductive instructional approach. The instruction followed the widely used instructional approach of Presentation – Practice – Production (PPP). This group first received corpus-informed grammar rules about the passive voice, including information about form, meaning, and use. They then practiced different forms of the passive voice in different contexts (in exercises designed using sentences taken from the corpora), and then produced the target structures in the same short writing activities as the DDL group.

As a point of comparison, the TGI group received grammar instruction following the same deductive instructional approach as the DCI group (i.e., PPP), but used traditional grammar teaching materials instead of corpus-informed descriptions of rules and activities. This group first received instruction on the form, meaning, and use of the passive voice based on widely used grammar textbooks, and then completed practice activities from the textbooks. Following the practice, they also produced the passive voice in short writing tasks.

2.3 Instruments

Prior to the instructional period, the participants in all three groups were given a short pre-test of their grammar ability related to the active and passive voice in English. Immediately following the two weeks of instruction, the participants completed another test with the same test tasks. Then, two weeks following the instructional period, participants completed a delayed post-test, again with the same test tasks. Given the short duration of each class period (60 minutes), the tests were designed to be short demonstrations of the participants’ overall grammar knowledge in relation to the active and passive voice in English through different tasks.

The pre-test, post-test, and delayed post-test all consisted of three tasks (see sample test in Appendix B) that were designed to measure ability related to form, meaning, and use (Table 3). The first task was to determine if sentences contained errors in the use of either the active or passive voice, and if so, to correct the error. This task was designed to measure the participants’ knowledge of the form and meaning of the active and passive voice. The second task was to rewrite active sentences into the passive voice or vice versa; and in the third task, participants had to decide which type of sentence would be more appropriate based on the register of use (i.e., speech or academic writing).

Table 3

Test task descriptions

The goal of the register awareness task was to measure participants’ immediate ability to distinguish which use of a particular voice was appropriate in one of the two major registers of speech and academic writing (Biber, Reference Biber1988). Understanding how registers differ based on language structure is a valuable area of language ability for learners (Aguado-Jiménez, Pérez-Paredes & Sánchez, Reference Aguado-Jiménez, Pérez-Paredes and Sánchez2012), and while simplistic, the task was designed to provide some feedback on how the instructional treatments may have affected participants’ awareness of register differences.

2.4 Analysis

The participants’ performance on the three tests were scored and mean scores were calculated across the groups (Table 4). As the data met the assumptions of normality for parametric analysis, ANOVA tests were conducted to determine whether there was improvement during the study among the three groups. Initially, a one-way ANOVA was conducted on the participants’ pre-test scores to determine if there were differences between groups prior to the instructional treatment. Subsequently, Repeated Measures ANOVAs were conducted for each treatment group to measure their performance on the three assessment tasks over time. Additionally, scores on individual test tasks were analyzed to compare performance across the three treatment groups. However, as these individual task scores were based on low point values and did not individually meet the assumptions of normality or sphericity, non-parametric Kruskal Wallis H tests were used in this stage of the analysis.

Table 4

Mean scores for active/passive voice tests

3.1 Individual test tasks

Following the analysis of the overall tests of the active/passive voice, analyses of the three test tasks were conducted individually. The first task, error correction, measured participants’ receptive and productive grammar ability associated with the form of the active/passive voice construction. Scores on the pre-test, post-test, and delayed post-test tasks are reported in Table 9. The Kruskal-Wallis analysis failed to show any differences between the three treatment groups on the pre-test performance for this task (H (2)=.412, p=.814). On the post-test, the scores did differentiate significantly between the three groups (H (2)= 11.083, p=.004), with a mean rank of 13.15 for TGI, 25.05 for DDL, and 14.85 for DCI. Analysis of the delayed post-test also indicated a significant difference between the treatment groups (H (2)=7.863, p=.02), with a mean rank of 12.77 for TGI, 23.45 for DDL, and 17.10 for DCI. The DDL group outperformed the other two groups on both the post-test and the delayed post-test for this task.

Table 9

Mean scores for error correction task

On the second task, rewriting sentences in the active/passive voice, the analysis showed significant difference between the three treatment groups on the pre-test (H (2)= 11.897, p=.003), with a mean rank of 15.00 for TGI, 22.73 for DDL, and 15.00 for DCI. While the previously reported analysis of overall scores for the active/passive voice did not show a significant pre-test difference between the three treatment groups, this analysis suggests that the participant groups did not have the same ability level related to this particular task at the beginning of the instructional treatments. Specifically, the mean scores for the TGI and DCI groups were lower than the DDL group on the pre-test (Table 10). The TGI and DCI participants both had a mean score of 0.00 on this task, with the DDL participants scoring only slightly higher (M=.45). Analysis of the post-test scores for this task also indicated a significant difference between treatment groups (H (2)= 6.354, p=.042), with a mean rank of 18.04 for TGI, 22.05 for DDL, and 11.80 for DCI. Analysis of the delayed post-test scores showed an even greater (and statistically) significant difference between groups (H (2)=12.543, p=.002), with a mean rank of 14.27 for TGI, 25.41 for DDL, and 13.00 for DCI. The differences between the DDL group and the other two groups that were present at the pre-test were continued through the post-tests, with DDL scoring higher on all three tasks.

Table 10

Mean scores for sentence rewriting task

For the third, register awareness task, the mean results of group scores by test are reported in Table 11. Analysis of these scores indicated that there was no significant difference between treatment groups on the pre-test (H (2)=.949, p=.622), on the post-test (H (2)=.180, p=.949), or on the delayed post-test (H (2)= 3.427, p=.180). Although this task was designed to distinguish between the corpus-derived instructional treatments and the traditional instruction as a measure of language performance directly related to they type of information corpora can provide, it did not provide any meaningful discrimination between groups.

Table 11

Mean scores for register awareness task

4.1 Limitations and further research

While these results provide some limited evidence that this combination of inductive instruction with corpus-informed teaching materials can lead to greater outcomes, caution must be used in generalizing the findings. The study focused primarily on rather concrete ‘rules’ for the grammatical differences between the active and passive voice, taking a narrow view of what a grammar structure is and also, due to the constraints of using real grammar classrooms with immediate curricular goals, taking only limited advantage of the possibilities of using corpus-informed teaching materials for providing more robust insight into register and use differences between the two voices in English. Further study of this type of instructional intervention may be better addressed by integrating these instructional treatments into a teaching approach that incorporates communicative language learning as the basis for learning about register and uses of the target language feature. Additionally, the passive voice, as a grammatical construction, may be simpler to learn using DDL than other, more complex grammar structures (in terms of form, meaning, and use).