Natural Language Processing with Python is designed as a textbook that provides a practical introduction to Natural Language Processing (NLP). It targets a wide audience: people with no prior background in Python programming or programming in general, people with no prior background in linguistics, or people with no background of either. Even readers with some background in both can benefit from studying the more advanced algorithms covered in this book. A unique feature of the book is that it is written in a tutorial style that allows the reader to try out hundreds of code examples throughout the book to see how language tasks, large and small, work in practice. The tutorial style is supported by the Natural Language ToolKit (NLTK), a publicly available Python library that consists of 120,000 lines of code and is still growing (http://www.nltk.org).
The book has 11 chapters, plus a preface and an afterword.
Chapter 1 attempts to get the reader interested in using the NLTK to perform language tasks right off the bat. After a brief tutorial on how to set up Python and the NLTK, the chapter goes on to demonstrate how to use Python to produce a concordance list, to find similar words, and to plot word usage chronologically using the presidential inaugural speeches. The chapter then moves on to the problem of discovering the most representative lexical items for a text, and in the process demonstrates how to calculate word frequency distributions, and how to find words with specific properties. The chapter ends with a high-level description of a few natural language understanding tasks such as word sense disambiguation and machine translation in an attempt to paint a broad picture of the field of NLP. The chapter essentially frames the starting point and the eventual goals in natural language processing, inviting the reader to fill what is in between.
Chapter 2 covers the manipulation of natural language corpora and linguistic resources with the NLTK. It starts by demonstrating how a uniform set of interfaces can be used to access natural language corpora in a wide range of different formats and how to set up ones own corpus with these interfaces. The chapter also shows how to use lexical resources to perform a variety of simple language processing tasks.
WordNet (Fellbaum Reference Fellbaum1998) is one of the most important lexical resources in NLP and the last section of the chapter explores the different lexical relations (synonymy, antonymy, hyponymy, entailment, etc.) represented in WordNet.
The first two chapters deal with data that has already been cleaned up and organized in structures that allow easy access and manipulation. Chapter 3 takes a step back and describes procedures to process data in their raw format: web pages in HTML format, RSS feeds, files on local disks, etc.
The chapter also briefly discusses Unicode, and the Python support for processing multilingual documents encoded in Unicode and other character encoding schemes. The chapter devotes considerable space, quite appropriately, to the use of regular expressions in NLP tasks ranging from finding word stems to word tokenization. Word segmentation for languages such as Chinese where there are no natural delimiters is also discussed.
After introducing Python in bits and pieces in the context of language processing tasks throughout the first three chapters, Chapter 4 attempts to tie up the loose ends and provide a more systematic introduction to Python. The chapter reviews some basic Python programming constructs: variable assignment, equality and identity tests, flow control, data structures that model sequences and operations on sequences. The review of functions is rather detailed, reflecting their role as the fundamental building blocks in Python programming. The chapter also touches on Python coding style, effective programming techniques, as well as selected topics in algorithm design that are important to language processing: divide-and-conquer, recursion, space–time considerations, and dynamic programming. The chapter ends with a description of a few Python libraries that can be used in conjunction with the NLTK.
Chapter 5 takes up the task of part-of-speech (POS) tagging, a very basic natural language processing task that operates on word tokens. The chapter starts with a discussion of POS-tagged corpora and how they are represented as Python objects within the NLTK. The focus of this chapter is on assigning POS tags to words automatically. The chapter provides a fairly easy-to-follow presentation of increasingly more accurate POS-taggers that use increasingly more sophisticated techniques.
The chapter ends with a discussion of the Brill tagger (Brill Reference Brill1995), a transformation-based algorithm that assigns a tag to each token with an initial tagger and then goes back to correct the errors with an ordered list of transformation rules. The chapter does a great job of bringing to bear all sources of information pertinent to the POS-tagging problem in an incremental fashion so that the contribution of each source of information is isolated.
Many NLP problems can be formulated as classification tasks, and this topic is covered in Chapter 6. The chapter focuses on supervised classification, which uses a training set already annotated with correct labels to train a model that can be used to classify new input. The chapter demonstrates how to represent and extract features, a crucial step in any statistical machine learning-based classification task. The chapter also discusses sequence classification, where the classification of one input depends on the result of the classification of another, but the chapter stops short of getting into the details of more advanced machine learning techniques such as Hidden Markov Models, Maximum Entropy Markov Models and Linear-Chain Conditional Random Fields that are typically used for sequence classification. The chapter ends with a fairly intuitive presentation of three basic classification techniques: Decision Trees, Naïve Bayes and the Maximum Entropy Model.
Chapter 7 deals with the topic of Information Extraction (IE). The chapter starts by presenting a basic IE pipeline, where raw data is first segmented into sentences, and then tokenized and POS-tagged. The chapter then devotes considerable space to the task of NP chunking and carefully develops an NP chunker that starts with using just POS-tag information, and gradually adds lexical, contextual and morphological information to the mix. Having described NP chunking, the chapter then takes up Named Entity Recognition (NER), which is simply the problem of identifying NP chunks that are meaningful named entities (persons, organizations, locations, etc.) The chapter ends with a brief description of Relation Extraction, the task of detecting relations between named entities. As a chapter on Information Extraction, there are some obvious omissions in the chapter, such as the distinction between mentions and types, which motivates the coreference task that groups mentions into types.
Chapter 8 covers syntactic parsing. Using a context-free grammar as a formal language description device, parsing is the process of producing a syntactic representation of a sentence in the form of a parse tree. The central part of the chapter is devoted to the development of a few fundamental parsing algorithms. The chapter details a top-down (recursive descent) and a bottom-up (shift-reduce) parsing strategy, and points out the weaknesses of each. A hybrid parsing strategy, left-corner parsing, is shown to address some of the weaknesses using top-down parsing combined with bottom-up filtering. A well-formed substring table that uses dynamic programming to recognize acceptable sentences given a grammar is also presented, providing a precursor for more sophisticated algorithms such as chart parsing. After briefly touching on dependency structure parsing, the chapter turns to the astronomical rate that the number of parse trees grows as the size of the grammar and the length of the sentences increases, motivating the use of a probabilistic grammar as the solution to select the desired parses from all possible parses.
Chapter 9 starts by using morphological agreement between the subject and the verb to motivate the need for enhancing context-free grammars with features in order to avoid generating ungrammatical sentences. Since features can be used to represent any kind of information, they provide an elegant way of preventing a CFG from overgenerating while keeping the grammar compact. Features can have atomic or complex values, and groups of features can bundle together to form a feature structure. The chapter goes on to discuss feature subsumption, where a general feature structure subsumes a more specific feature structure, and feature unification, where the values of two features are combined. The final section of the chapter demonstrates how to use features to represent the subcategorization frames of verbs, inversion involving auxiliary verbs, unbounded dependencies between a dislocated element and the gap it leaves behind and case and gender in German.
Chapter 10 starts by motivating the use of formal logic as a generic way to represent the meaning of natural language sentences. The chapter describes how to use propositional logic to represent certain aspects of natural language meaning (mostly connectives) and shows that this form of representation is not expressive enough because the internal structures of the propositions cannot be manipulated. This shortcoming is addressed to some extent by first-order logic, which keeps all the Boolean operators of propositional logic while adding other formal mechanisms that allow a much richer representation of natural language semantics. The chapter goes into quite a bit of detail about implementing a model that assigns truth values to first-order logic formulas, assigning values to variables, handling quantified formulas and creating models given some first-order logic formulas. The chapter ends with discussing the translation of English sentences to a logic representation via the mediation of a context-free grammar guided by the principle of compositionality. Considerable space is devoted to the discussion of lambda-calculus and its central role in building semantic representations for natural language sentences.
Chapter 11, the final chapter of the book, discusses a wide range of issues in developing a corpus: balance and coverage, quality control, data collection and data format conversion, common standards versus shared tools, as well as issues in working with endangered languages. The chapter also discusses in detail how to work with the Extensible Markup Language (XML) and the Toolbox data. The chapter ends with a discussion of using the Open Language Archive Community (OLAC) META data to describe linguistic resources. Compared with other chapters, this chapter is somewhat of an anomaly. It departs from the tutorial style characteristic of other chapters where an NLP task is discussed. This chapter feels like an add-on chapter that is not closely tied to other chapters of the book.
Writing a book that teaches both Python and NLP is an incredible challenge. There is no lack of textbooks on either Python or on NLP. However, most Python books focus on programming and the exercises provided have no relevance to NLP. The dominant NLP textbooks on the market, such as Jurafsky and Martin (Reference Jurafsky and Martin2008), and Manning and Schütze (Reference Manning and Schütze1999), target an audience who is already proficient in programming, and are therefore not accessible to beginning programmers. A few books that are designed to help linguists acquire Python programming skills are still works in progress, and cannot compare to this book in either scope or depth. This book thus carves out a unique place in the NLP textbook space. This book and the NLTK behind it make it possible to cover substantially more ground in a typical semester than was previously possible.
There are two possible approaches of blending programming and NLP. One is to organize the book by NLP topics, and present Python programming topics in the context of language processing tasks. The other alternative is to organize the book based on the logical progression of a programming language, and present language processing problems to practice Python programming skills. The book primarily adopts the former approach. This approach does run the risk of severing the natural logical order of Python programming topics. Although the authors claim to assume no prior background in programming, whether this approach is completely successful needs to withstand the test in classrooms. From my own experience at least, students with some programming experience tend to fare better, while students with no background at all tend to get lost initially. It is advisable to precede the material in the book with a couple of weeks of Python primers.
Throughout the book, the data-driven approach is obvious, representing the current trend in the field of NLP and drawing a stark contrast with similar books preceding this such as Allen (Reference Allen1995). The book starts with three chapters on data manipulation and ends with a chapter on language data creation and management. Chapters 5–7 do not manipulate text directly, but the tasks presented in those chapters require the use of non-trivial natural language data sets. The POS-tagging task presented in Chapter 5 requires n-gram statistics gathered from a corpus tagged with POS tags. The classification tasks presented in Chapter 6 are trained on data sets already annotated with correct class labels. The same is true for the chunking and information extraction tasks presented in Chapter 7. The lone exceptions are Chapter 4, which deals exclusively with Python programming, and Chapters 8–10, which deal with syntactic parsing, building feature-based grammars and analysing the meaning of sentences, even though they could potentially be made more data-oriented.
The book usually does not present NLP algorithms formally in pseudo-code, as other NLP books do. Instead, all language processing tasks and algorithms are actually implemented in the NLTK, thus encouraging the reader to learn from examples by getting their hands dirty. The problems described throughout the text and exercises presented at the end of each chapter are very well thought out. They vary in scope and difficulty, and do a good job of reinforcing the basic concepts discussed in each chapter in interesting and rewarding ways. More difficult problems provide an opportunity to experiment with the NLTK and encourage creative solutions. Going through this material will be a good preparation for a more advanced course in NLP using the two textbooks mentioned earlier. However, many important computational concepts are glossed over, especially in earlier chapters, and supplementary reading material on introductory computer science concepts may be necessary for students with little programming experience.
As an introductory book, a good balance is maintained between the breadth of topics covered and the depth at which each topic is handled, providing exposure to a variety of NLP topics that are explored in more detail in other textbooks. Some of the chapters could serve as an entry point to a particular area of NLP that the reader can explore further. This is particularly true for the text classification (Chapter 6), Information Extraction (Chapter 7), syntactic parsing (Chapter 8) and semantic analysis (Chapter 10) chapters. While I fully acknowledge the enormous effort the authors have made in developing this book and the NLTK behind it, it does not have everything on my wish list. For example, I would love to see a chapter on machine translation, which I think can be written in a similar style, and maybe this can be added in future editions. All in all, it is a greater beginners' book that provide a gentle introduction to the field of NLP, and I think more students will get into the field because of it. Together with the extensive code base of the NLTK, this book represents a tremendous achievement.
