This book is dedicated to a heterogeneous set of linguistic constructions that has generated recurrent interest in natural language processing (NLP) and correlated areas throughout the years: Collocations. These have also been referred to as multiword expressions and phraseological units, for example, and include among them idioms (in 1), phrasal verbs (in 2 and 3) and compound nouns (in 4):
(1) To make ends meet he (2) ended up (3) relying on a (4) monthly bus pass
Collocations are discussed in this book in the context of a framework for their automatic identification using syntactic information. In particular, the author examines the hypothesis that parse-based identification, using the syntactic proximity between words, produces improvements in performance, in relation to standard window methods, using linear text proximity. A parse-based approach looks for recurrent combinations of words that are syntactically related and satisfy morphological and syntactic constraints (e.g., not being an auxiliary verb and having a subject–verb or verb–object relation), like prepare and exam in a verb–object relation. Window-based methods focus on recurrent pairs of words found in a short window of text, including interruptible and adjacent sequences (n-grams). A parse-based method would, for instance, be able to extract combinations even on the face of morphological and syntactic transformations and long-distance dependencies, and would be less susceptible to data sparseness, as it would collapse syntactically varied occurrences of a combination into a canonical form. This hypothesis is presented and evaluated in this book organized in six chapters and six appendices, which include a comprehensive overview of the literature in the area from an NLP perspective.
In Chapter 1, the book starts with a brief introduction to collocations and motivation for the research through examples. It also defines the aims and structure of the book.
Chapter 2 provides a historical overview of the interest in collocations, starting with their role in first and second language learning, as language chunks that need to be treated as units and require memorization. Therefore, they are associated with high levels of proficiency and often seen as the privilege of native speakers for lending fluency to speech. Their importance has long been recognized in lexicographic work, reflected by the many on-going initiatives for compiling collocation dictionaries for many languages (some of them listed in Appendix A). However, one of the basic difficulties for work in this area is a lack of consensus of what actually constitutes a collocation, as can be seen in the survey of definitions presented in Section 2.2. These can be roughly divided according to the prominence given to considerations about frequency and/or linguistic characteristics, often resulting in the inclusion of different sets of linguistic constructions. Acknowledging this division of statistically based versus linguistically based collocation work, the author analyzes common features among the different definitions, narrowing down to five that are considered relevant to a core collocation concept. These are as units that are (a) pre-fabricated, (b) arbitrary, (c) unpredictable, (d) recurrent and (e) of unrestricted length. From a linguistic perspective, this complex interplay of lexical, syntactic, semantic, pragmatic and statistical interdependences has been formalized in different theoretical frameworks and this chapter reviews some of these accounts.
Turning to a more computational perspective, Chapter 3 provides an overview of approaches proposed for collocation identification and how they incorporate the core features of collocations. A common architecture for collocation extraction from corpora includes a first step of identification of candidates from the text that is subsequently ranked in a second step, usually according to an Association Measure (AM). AMs include the t-score, chi-square and mutual information, and some of these are presented in Section 3.2.4. AMs have been used for approximating notions like recurrence and have been used as indicators of the degree of ‘collocationness’ of a candidate with varying degrees of success; this chapter discusses the performance of these methods on different collocation types and languages. Another factor that influences the performance is the level of linguistic pre-processing of the corpus used for candidate extraction. Approaches for collocation extraction vary to the extent to which linguistic pre-processing, such as POS tagging and parsing of the candidates, is adopted in combination with AMs, but this decision is often restricted by the availability of the necessary tools for a given language. A direct comparison between the performance of AMs and a consequent agreement on the suitability of a given measure for particular collocation types and languages is often prevented by a lack of common benchmarks for evaluation, as the author remarks. The chapter finishes with a review of the state-of-the-art in extraction for different languages, including English, German, French and Korean, and how their characteristics, like word order and morphology, affect the applicability of these approaches.
Given the main hypothesis that the use of syntactic proximity, instead of linear proximity for collocation identification, combined with AMs, can substantially improve the quality of the results obtained, Chapter 4 analyses the use of syntactic information for identification in related work. The author then defines a set of requirements, like robustness and coverage that need to be met in a successful collocation extraction system, before a detailed presentation of the materials and methods used for assessing the main hypothesis. In order to do that, two main experiments are proposed: the first uses a monolingual French corpus, while the second employs the parallel corpora of Europarl for English, French, Spanish and Italian. The evaluation includes a window-based method as baseline, and compares the performance of the parser- and window-based approaches in each of these languages. Several evaluation scenarios are discussed, and the author adopts one that relies on human annotation of some samples. The results are quantitatively and qualitatively evaluated in terms of precision, grammaticality, position in rank and frequency among others, complemented with an error analysis of each of the methods. The strengths of both methods are discussed and backed by a detailed examination of the results. The two methods seem to agree considerably on the candidate set and on their ranking (with Spearman's rank correlation), with the results obtained by the parse-based approach being of substantially better quality and less susceptible to low frequency counts than those of the window-based method.
Chapter 5 extends the proposed approach even further investigating three further scenarios: (a) the identification of complex collocations, (b) the automatic induction of syntactic patterns and (c) the extraction of corpus-based collocation translations. The first extension addresses the focus of extracting collocations of exactly two words, often adopted in related work, given the difficulties of extending AMs for longer sequences. A straightforward extension of the parse-based approach is proposed that takes previously extracted collocations as the basis for longer sequences, allowing any two collocations to be combined if they have elements in common. This is evaluated for French. The second extension looks at the automatic induction of patterns as a way to increase the recall of the method further than allowed by the initial set of patterns defined, and this is evaluated for English and French. The last scenario examines the application of the proposed approach for the identification of translations for the extracted collocations from parallel corpora. After extracting a collocation, the proposed approach selects some sentences surrounding the collocation in the source corpus and aligns them with the target corpus. The collocations in the target sentences are extracted using the parse-based approach and potential translations are selected from these after applying some filters. Evaluation is performed for the 12-language pairs involving English, French, Spanish and Italian.
Chapter 6 finalizes with a discussion of the main contributions of the book, in relation to the aim of proposing a collocation extraction approach based on syntactic information. In addition, suggestions for future investigations are discussed.
The appendices present supplemental material, including
• a list of collocation dictionaries in several languages, in Appendix A,
• a list of collocation definitions, in Appendix B,
• mathematical notes on the AMs, in Appendix C,
• monolingual evaluation material for experiments 1 (Appendix D) and 2 (Appendix E) and
• output comparison for the parse- and window-based approaches, in Appendix F.
From a research perspective, the book makes a convincing case for syntactically based approaches, backed by evaluation data. Window-based methods are popular as they are fast, robust and mostly language-independent. However, the results clearly show the superiority of the parse-based approach over the window-based approach: while the latter results in many false positives, it also misses candidates found by the former. In addition, the missing entries may also affect the frequency signature that is attributed to collocations by the window-based approach, producing shifts in the ranking of candidates. On the other hand, even if syntactic-based identification requires robust and accurate language-dependent tools that may not be readily available for many languages, it produces a smaller and more targeted set of candidates of better quality that can be subsequently ranked and filtered with an AM.
This book provides a much-needed overview of the state of the art on collocations and their identification from a NLP perspective. It is well written and provides clear explanations for some of the core techniques adopted in the area. Given the sizeable and interdisciplinary body of research in the field, some of the discussion of related work is understandably kept at a superficial level and will still require further reading for a more in-depth understanding. For instance, Section 3.2.4.2, which explains some typical AMs, may need to be complemented with additional background material on statistical measures (e.g., Manning and Schütze, 1999), depending on the background of the reader. Likewise, for a more in-depth analysis of the linguistic characteristics of collocations, the reader may want to look at references, such as Dehé et al. (2002), Nunberg, Wasow, and Sag (1994) and Moon (1998), for particular collocation types and languages. But the main pointers are there.
The book can also serve as a step-by-step guide to identification in practice, discussing methodological choices and the impact of different levels of pre-processing in terms of performance. It also discusses evaluation issues and possible alternatives on the face of a lack of benchmarks or agreement about standard procedures, and exemplifies a thorough and in-depth quantitative and qualitative evaluation regime.
In sum, the book can serve as both a good introductory text for anyone starting on the field and an up-to-date compilation of the main references and recent advances in context for the expert reader. Although far from being a solved problem, many advances in collocation identification have been made, as advances in NLP technology have enabled further and deeper analysis and provided the grounds for more precise and targeted results as discussed in this book. Overall, this book is a good outcome from an area that has attracted considerable theoretical and practical interest in research ranging from second language acquisition to machine translation.
