Frequency and the emergence of linguistic structure, edited by Bybee and Hopper, lays the foundations for a proper empirical inquiry into usage-based functional linguistics. The distinctions langue versus parole and competence versus performance underpin the assumption pervading much of linguistic theory that language structure is independent of language use. However, this longstanding view is now being challenged by functional linguists and psychologists who hold that linguistic representations are highly influenced by a speaker's experience of language. These usage-based approaches to language and language development view grammar, phonology, and lexis alike as dynamic systems that emerge from frequently occurring patterns of language use rather than as fixed, autonomous, closed, and synchronic. The chapters in this volume document this case with empirical investigations of distributions in corpora of natural conversation and written language, diachronic change, variability, child language acquisition, and psycholinguistic experimentation. All of the chapters are rich in data and sophisticated in their methodologies; their hypotheses are carefully operationalized and their detailed predictions closely measured against objective data. The theoretical coverage of the 20 chapters includes phonological and morphological variation, language change, child language acquisition, argument structure, subjectivity, phonotactic knowledge, lexis, formulae, and a wide range of phrases and constructions. The common conclusion is that frequency has a significant effect on the emergence of linguistic structure in each of these domains. The distribution and frequency of the units of language are governed by the content of people's interactions. These interactions consist of a preponderance of subjective, evaluative statements, dominated by the use of pronouns, copulas, and intransitive clauses. This discourse is the linguistic evidence that shapes language acquisition: This is the usage that tunes the emergence of the speaker's cognitive representations of language; this is the parole that determines sociolinguistic variation and diachronic change. The frequency with which certain items and sequences of items are used profoundly influences the way language is broken up into chunks in memory storage, the way such chunks are related to other stored material, the ease with which they are accessed, and the way linguistic regularities, abstract schematic constructions, default patterns, and central tendencies emerge. This excellent collection shows how patterns of social interaction shape discourse and how this usage shapes language acquisition, the tuning and emergence of cognitive representations of language, and language itself. Each chapter is a clear exemplar of how to measure patterns of usage and their statistical distributions and how to assess their effect on acquisition, structure, or representation. Frequency and repetition affect and ultimately bring about form in language.

Probabilistic linguistics, edited by Bod, Hay, and Jannedy, is a handbook that provides the necessary background for scholars wanting to further explore the probabilistic nature of language and its effects but whose learning style favors tutorial review rather than focused exemplars. Because probability theory is not the normal stuff of traditional linguistics curricula, it begins with a tutorial by Bod that overviews elementary probability theory and probabilistic grammars. The first half of Jurafsky's chapter presents a thorough and balanced review of the use of probabilistic knowledge in language comprehension and production processes and the influence of conditional probabilities in the processing of lexical items, multiword structures, syntactic subcategorization frames, and sentence comprehension. The second half presents a tutorial review of the architectures used for modeling these frequency effects: constraint-based connectionist models, the Competition model, Rational analysis, Markov models, stochastic context-free grammars, and Bayesian networks. Mendoza-Denton, Hay, and Jannedy outline probabilistic sociolinguistics and illustrate the ways that the negotiations of interaction transmit linguistic structures and social identities both, and they illustrate how these dynamic processes of language variation can be modeled using VARBRUL and CART (classification and regression tree) methods. In the next chapter, Zuraw analyzes diachronic language change as a probabilistic process, illustrating how probabilistic methods are used to identify language relatedness and how simulation studies are being used to understand these processes. Two particular simulations are chosen for detailed consideration: the connectionist model of Tabor, where frequency drives syntactic change, and the exemplar-based model of Pierrehumbert, which simulates the frequency-driven erosion processes of Bybee's model of phonological change and grammaticization. What is important here is that these models close the circle from interaction to usage to mental representation, and on again in the continuous dynamic interrelations of social selves, their linguistics, communication, psychology, and cognition.

Pierrehumbert's own chapter reviews empirical evidence that implicit knowledge at all levels of phonological representation is probabilistic, with the phonetic encoding system and word-forms being acquired from the learner's exposure history to speech and the phonological grammar then abstracted over the lexicon. Pierrehumbert describes how statistical classification models explain how different languages utilize the phonetic space in different ways and how the incremental experience of word-forms during acquisition determines the organization of the empirical distributions in this space and the fine-grained discriminations that can be made as a result. The fact that people with different language exposures can end up with essentially the same grammar—that is, that language acquisition is resistant to outliers and is statistically robust, in conjunction with the amount of language exposure that underpins the eventual classification—places important constraints on the nature of the phonological system. Thus, a probabilistic approach enables us to make inferences about the utilization of the phonetic space and the possible constraint set in phonology in ways that are not possible in a purely categorical approach. Baayen's chapter illustrates the probabilistic nature of morphology, how the graded phenomenon of morphological productivity is grounded in junctural phonotactic and parsing probabilities, and how morphological regularities are intrinsically probabilistic in nature and are readily captured in connectionist models involving spreading activation. Baayen clearly illustrates how nonprobabilistic theories of morphology are inadequate in that they cannot handle graded phenomena in an insightful way. His approach to modeling these phenomena is again exemplar based, with probabilities developing dynamically over time as analogies made on the basis of both frequency of (co-)occurrence and form similarity. Manning's chapter on probabilistic syntax begins with the lessons of corpus linguistics that language description should be based on proper empirical foundations and that, when they are, it becomes clear that syntax is noncategorical: Categories and well-formedness are gradient. Categorical linguistic theories place a hard categorical boundary of grammaticality where there really is a fuzzy edge that results from many conflicting constraints and conventionality versus creativity competitions. In contrast, probabilistic syntactic models allow an understanding of how such soft constraints explain variable expression and variable understanding. A key lesson here comes from the analysis of the formal learnability of stochastic grammars: The assumption of probabilistic grammar actually makes languages easier to learn. Such a grammar puts probability distributions over sentences; any phenomenon given a high probability in the grammar will show up in a corpus, and absence of a sentence from a corpus thus gives implicit negative evidence. The result, under reasonable assumptions, is that stochastic grammars are learnable from positive evidence alone. Finally, Cohen's chapter on probabilistic approaches to semantics shows how probabilistic notions can usefully augment traditional truth-conditional semantics.

Probability pervades language. Linguistic representations at multiple levels of representation are probabilistic, as are linguistic constraints and well-formedness rules. Probabilities are operative in acquisition, perception, and production. They permeate and unite sociolinguistics, language change, psycholinguistics, phonology, morphology, lexis, syntax, and semantics. It is not just that the chapters in these books reflect interesting new developments, it is that they are coming in significant numbers, and the implications for theories of language acquisition are profound. Acquisition involves the gathering of the relevant probabilistic information. Usage-based construction grammar accounts fit best with such perspectives, alongside frequency-tuned exemplar-based models of the extraction of linguistic regularities—the ways that language emerges from such usage are too complicated to imagine, which is why simulation is an essential part of theory building.

What aspects of language are missing from these two books? SLA is conspicuous in its absence. Language transfer, prior perceptual tuning and first language (L1) entrenchment, L1-learned selective attention, and the ways explicit knowledge interfaces on implicit learning all conspire to complicate the ways that probabilities operate in SLA. However, frequency effects pervade SLA as well, and our ultimate goal is the understanding of the bounded rationality of second language learners. How does learners' processing of exemplars derive the subjective probabilities that underpin interlanguage abstraction from usage-based and form-focused SLA?