In recent years, corpus linguistics has greatly benefited from rapid technological advances that have provided increased access to large sets of authentic language data and enhanced computational power and storage capacity. At the same time, new quantitative and statistical techniques have broadened the analytical possibilities of large datasets, expanding the ways in which language data can be explored. As a result, corpus linguistics today has become a much more diverse and multifaceted field than it was only a few decades ago.
These developments have also prompted scholars to re-examine some of the fundamental principles of corpus linguistics as a methodological approach to language study. In fact, it could be argued that this re-examination has been constantly ongoing, and different types of challenges have been discussed in a number of edited volumes, such as Viana et al. (Reference Viana, Zyngier and Barnbrook2011), Hiltunen et al. (Reference Hiltunen, McVeigh and Säily2017) and Kaunisto & Schilk (Reference Kaunisto and Schilk2024), to name but a few. Interestingly, these collections themselves differ in terms of their aims and scope, which further highlights the diversity of the field with its wide range of tools, methods and theoretical orientations. The discipline has become significantly enriched, but one can also perceive an increased challenge in maintaining shared methodological perspectives within the research community, which is why continued discussion and reflection on such matters is essential.
The volume edited by Ole Schützler and Julia Schlüter, comprising an introduction and eleven chapters distributed into four parts, focuses on the critical evaluation of the use of different datasets and methods of analysis, with the aim of exploring what pros and cons are involved with such choices with regard to the study of different types of linguistic phenomena. The volume brings together contributions written by experts for experts or advanced graduate or doctoral-level students, and the chapters assess the advantages and limitations involved in the use of different datasets, tools and analytical approaches. Each of the chapters involves the analysis of at least one specific linguistic question, and the studies illustrate how the methodological choices influence the level of detail with which the data can be scrutinized. Very often the authors summarize the benefits and drawbacks of different approaches in tabular form, and the overall message that is conveyed is the importance of applying multiple approaches and sets of data, if possible.
One of the commendable features of the volume is the authors’ sometimes explicit, sometimes implicit understanding and acknowledgement that many readers are perhaps quite unfamiliar with the different analysis methods described, or not well versed as regards their programming skills or knowledge of statistical computing with R. This is reflected in the meticulous and patient way in which the authors describe the different methods and the practical problems involved in the execution of the studies. Probably much of the credit for setting the overall tone goes to the editors of the volume, who in the excellent Introduction (pp. 1–14) describe the starting point and motivation of the book project. The idea of adopting a comparative approach – that is, applying at least two datasets or methodological frameworks to examine the same research question – is especially well suited to fulfilling the broader aims of the volume. As is often the case with thematically arranged collections of research articles, it is not always possible to predict or control the exact range of topics and subthemes that will appear in the final volume. Nevertheless, the editors’ efforts in shaping the collection have undoubtedly been considerable, and the chapters have been organized into parts in a way that is both logical and coherent.
The studies in the first part of the volume, titled ‘Corpus dimensions and the viability of methodological approaches’, draw attention to the characteristics of corpora themselves, and to the viability of using corpora or even less structured large text databases for different types of studies. In chapter 1 (‘Comparing standard reference corpora and Google Books Ngrams’, pp. 17–45), Lukas Sönning and Julia Schlüter investigate variation in the occurrence of onset [h] sounds in British and American English. The study examines phonological variation through written data via the use of indefinite articles a or an before h-initial words. Drawing on the British National Corpus (BNC), the Corpus of Contemporary American English (COCA) and the Google Books Ngrams database, the authors compare findings from datasets that differ markedly in size and composition. While the vast scale of Google Books provides a greater number of tokens and types, the limitations in its metadata raise serious questions about its useability and reliability. Importantly, the chapter offers a clear and detailed discussion of the practical challenges involved in analysing such materials, presented in a manner that is remarkably transparent and reader-friendly. Overall, the chapter provides valuable insights into how the reliability of results can be assessed across corpora and databases with differing sizes and genre profiles.
Chapter 2 (‘Comparing approaches to phonological and orthographic corpus formats’, pp. 46–72), by Sabine Arndt-Lappe and Sebastian Hoffmann, examines how the tendency toward regular alternation of stressed syllables in English – known as ‘The Principle of Rhythmic Alternation’ – can be studied through both spoken and written corpus data. While written corpora have been used as a proxy for studying prosodic features, spoken data obviously allows more direct investigation of rhythm. However, orthographic transcriptions of speech rarely include stress annotations, and the limited size of spoken corpora poses challenges for reliable analysis. The authors conduct an auditory analysis of potential stress shifts using the 5-million-word Audio BNC and provide a detailed discussion of the strengths and constraints of different data sources. The study of how metrical factors influence the choice between lexical and grammatical alternants extends to considerations of how the spontaneity of speech production relates to the distribution of stress shift patterns. Although many observations rely on instances of numerals ending in -teen – a result of the relatively small dataset – the chapter breaks new ground and points the way toward multi-pronged studies of rhythmic alternation in the future.
The three chapters in the second part of the volume (‘Selection, calibration and preparation of corpus data’) address the practical and methodological challenges involved in the selection and necessary readjustments of corpus data, emphasizing the impact that these choices can have on the reliability and interpretability of research findings. A key example is provided in chapter 3 by Fabian Vetter, who examines the problem of ensuring true comparability between datasets when conducting cross-corpus analyses of the same register (‘Comparing approaches to (sub-)register variation’, pp. 75–100). Focusing on the press editorial sections of the British, Canadian and Jamaican components of the International Corpus of English (ICE-GB, ICE-CAN, ICE-JA), Vetter shows how differences in corpus composition – in this case, the varying proportions of different types of editorial texts – can distort comparative findings. In the study, supplementary texts are compiled in order to examine the possible regional differences between the language choices of this register more efficiently as well as to investigate the level of distortion caused by the imbalanced nature of the original datasets. The description of the various steps taken in the study is very rich in detail, and the chapter ultimately highlights the crucial role of sufficiently detailed metadata in corpora for comparative studies.
The common practice of referring to frequencies of linguistic features relative to the total word count of a corpus – nowadays typically as occurrences ‘per million words’ – has for some time been criticized for failing to express how often a feature appears in contexts where it could actually occur. In chapter 4 (‘Comparing baselines for corpus analysis’, pp. 101–26), Sean Wallis and Seth Mehl make a strong case for abandoning the ‘per million words’ baseline in favour of measures that more accurately reflect the true opportunity of use. As an illustration of this argument, they analyse the occurrences of get- and be-passives in the ICE-GB component, taking into account the mutual substitutability of the two passive constructions as well as the presence or absence of an agentive by-phrase. Comparing three alternative baselines, the authors conclude that baselines relying on verified alternation not only add to the accuracy of one’s findings, but they also encourage researchers to engage in a deeper and more critical fashion with their research questions.
The issue of possible distortions caused by insufficient metadata is further explored in chapter 5 by Lukas Sönning and Manfred Krug, who emphasize the importance of reliably linking search hits to individual sources (‘Comparing study designs and down-sampling strategies in corpus analysis’, pp. 127–60). They illustrate their point through a study on the distribution of actually in the 1994 and 2014 spoken components of the BNC. The core problems stem from the limited access to detailed metadata and from the uneven word counts contributed by speakers, both of which make it difficult to examine the uses of actually in its different functions by speakers from different socio-demographic groups. In a detailed and practically oriented analysis, the authors describe the possibilities of down-sampling strategies to overcome such problems, and they conclude by presenting recommendations for designers of corpus interfaces, including greater accessibility to detailed metadata as well as more down-sampling options during searches.
Part 3 of the volume, ‘Perspectives on multifactorial methods’, presents examinations of alternative approaches for assessing the simultaneous effects of multiple factors on linguistic choices. The suitability of different statistical methods varies depending on the phenomenon under study, and the chapters illustrate how methodological choices influence the interpretability and reliability of the results. Chapter 6 (pp. 163–93) by Tobias Bernaisch compares three methodological approaches – generalized linear mixed-effects models, generalized linear mixed-effect model trees and random forests – applied to the study of filled and unfilled pauses attested in the spoken components of the British, Indian and Sri Lankan components of ICE (ICE-GB, ICE-IND, ICE-SL). After describing the annotation of relevant instances across factors such as structure, genre and speaker characteristics, Bernaisch describes the main findings obtained with each approach and evaluates their main advantages.
Somewhat similar goals are sought in chapter 7 by Matthew Fahy, Jesse Egbert, Benedikt Szmrecsanyi and Douglas Biber, which examines a linguistic scenario involving three possible variants – the ’s-genitive, the of-genitive and a semantically comparable noun+noun combination (‘Comparing logistic regression, multinomial regression, classification trees and random forests applied to ternary variables’, pp. 194–223). In their analysis of the occurrences of the three variants in data from three registers in the ARCHER corpus, they compare three statistical approaches – binary logistic regression analyses, a multinomial regression model and a classification tree tool modified into a random forest. The authors make an important point in being careful in their characterizations of the properties of the different approaches, emphasizing that the approaches have beneficial aspects depending on various factors instead of suggesting that one method is categorically preferable to the others across the board.
Chapter 8 by Natalia Levshina (‘Comparing Bayesian and frequentist models of language variation’, pp. 224–58) and chapter 9 by Ole Schützler (‘Comparing methods for the evaluation of clusters in multidimensional analyses’, pp. 259–88) extend the comparative discussion of methodological approaches into areas which may still be less familiar to many readers. Levshina contrasts frequentist and Bayesian approaches to regression analyses, noting the growing interest in Bayesian statistics within studies of language variation. Using data from the Magazines section in COCA, she examines the factors influencing the choice between bare and to-infinitive complements following the verb help. While the Bayesian framework may seem novel and complex to many readers, Levshina’s clear and methodical exposition demonstrates the rationale behind its use and its advantages. Schützler, in turn, discusses the ways in which techniques such as multidimensional scaling, unrooted phenograms and hierarchical dendrograms can be applied to examine non-tree-like linguistic relationships. He argues, however, that visualization methods such as NeighborNets are not necessarily used to their full potential when limited to visual inspection, and that in some cases their interpretation can be difficult. Through the analysis of concessive constructions with although, though and even though, Schützler compares visual and distance-based quantitative approaches, concluding that multidimensional approaches alone can lead to observations that are perhaps too abstract, but that they can be used more efficiently when complemented by more objective, distance-based metrics.
The final part of the volume, titled ‘Applications of classification-based approaches’, comprises two chapters that explore this theme. In chapter 10, Gerold Schneider examines how a corpus-driven approach can be applied to the study of diachronic variation (‘Comparing corpus-driven and corpus-based approaches to diachronic variation’, pp. 291–322). After presenting a number of arguments in favour of adopting corpus-driven approaches and encouraging the move away from hypothesis- or rule-driven methods, Schneider describes his study, which examines automated feature profiles of texts from two diachronic corpora, the ARCHER corpus and the Corpus of Historical American English (COHA). By focusing on a variety of features based on part-of-speech tagging, parsing and processes of pattern and document classification typically used in computational linguistics, he examines patterns of overuse and underuse between different subperiods of the data. Observing the potential application of the findings further, the chapter offers new perspectives on corpus-driven approaches and challenges some of the commonly held views on them.
In the insightful concluding chapter of the volume (chapter 11, ‘Comparing annotation types and n-gram sizes’, pp. 323–52), Volker Gast examines translation equivalents of pragmatic markers that can be identified in parallel corpora, focusing on the German modal particles ja and doch and their English renderings. As English does not have a directly corresponding category of modal particles, the identification of corresponding English expressions presents a particular challenge. Gast discusses a range of methodological options for tackling this issue and analyses a set of different types of n-grams drawn from the Europarl-Direct corpus to determine which kinds of cues in the English translations are the best indicators of the presence of a modal particle in German.
Overall, the chapters in the volume provide richly detailed insights, and given the many directions in which corpus linguistics is expanding – along with the growing influence of related disciplines – the collection is a timely and valuable contribution. Scholars are likely to benefit from the book’s comparative perspective, which illustrates the distinct insights that different methodological approaches can yield, and it contributes toward an understanding of which techniques are most appropriate in various research contexts. Making informed choices between alternative options requires consideration of multiple factors, relating to both the linguistic phenomena themselves as well as the characteristics of available datasets.
Other notable strengths of the volume include the helpful lists of further readings provided at the end of each chapter and the inclusion of links to supplementary online materials.
