The “Now-or-Never” bottleneck presents a real challenge. Rather than the solution presented by Christiansen and Chater (C&C), however, an alternative – one that is both simpler and more economical – is possible. They do allude to the solution I want to present, but they apply it locally rather than globally. The solution in question is prediction. One explanation for why this globally applied solution is not presented is that C&C adopt a traditional view of cognition, according to which inputs come in, get processed, and passed on. Adherence to this view is evidenced in talk of “Chunk-and-Pass” processing. Inputs “come in” and get “chunked” and “passed.” Within a predictive processing framework, on the other hand, the direction, if anything, is reversed. “Processing” constitutes inputs having been successfully predicted from the top down. What gets “passed” is prediction error, not some honed incoming product. What does get honed, in light of incoming prediction error, is predictions. Indeed,
If we adopt a wholesale predictive processing approach, according to which prediction is not an aid to processing as traditionally construed but is rather its fundamental principle, then we overcome the Now-or-Never bottleneck in an evolutionarily, biologically, and computationally plausible way, and end up with all of the same consequences for how to understand language that the authors are at pains to point out.
Firstly, all of the presented solutions to the Now-or-Never bottleneck need not be seen, as C&C present them, as separate, but rather may be viewed as different facets of predictive processing. In other words, (a) “eager recoding and compression” of input, and (b) hierarchical levels of “representation” become consequences of, and not additions to, a need to (c) “deploy all available information predictively.” Let me explain why.
Predictive processing is a concrete implementation of a Bayesian strategy. Incoming signals are noisy and ambiguous, and so the brain uses Bayesian inference (it takes into account not only the “fit” of the hypothesis with the input, but also its “prior probability”) to settle on one hypothesis rather than another. Thus, a hypothesis can have a really good fit but such a low prior probability that it isn't selected (or it can have a poor fit, but such a high prior probability that it is selected).
This Bayesian strategy gets implemented in the brain as follows. The selection of a hypothesis determines a set of predictions about subsequent inputs, namely, inputs that are compatible with the hypothesis. If the hypothesis does a bad job of predicting inputs, it will be tweaked or abandoned altogether in favour of another hypothesis. These hypotheses are hierarchically arranged, with the hypotheses of one level providing the inputs (prediction error) for the next. “Higher” parts of the hierarchy are, roughly, those parts that are further away from the sensory stimulus. These tend to operate at longer timescales, and at higher levels of abstraction. “Lower” parts of the hierarchy are closer to the sensory stimulus. These tend to be at shorter timescales, and at low levels of abstraction. These, for example, correspond to early stages of visual processing: your brain's early statistically driven attempts to make sense of (predict) noisy inputs.
Predictive processing is time and energy efficient, and it involves compression (more or less “lossy” depending on the occasion). You save on bandwidth by passing on only what is newsworthy. What counts as “newsworthy” is simply what the receiver of the message hasn't already predicted, namely, prediction error. To sum up, then, predictive processing in the brain always involves (1) compression and (2) hierarchical arrangement of hypotheses (which, to use C&C's terminology, can be thought of as “representations”).
Now I'd like to gesture towards some of the consequences that predictive processing has for how we think about language. It has all of the same nine consequences that C&C enumerate, but, again, they are (at least for the most part) facets of each other rather than separate consequences. Let me illustrate this with two seemingly distant consequences: the multilevel organization of language (Consequence 1), the nature of what is learned during language acquisition (Consequence 5).
The hierarchical arrangement of hypotheses in predictive processing clearly suggests that language processing has a “multilevel organization.” Of course, our processing of other, nonlinguistic, stimuli has a similar organization, but that structure is not so clearly delineated since nonlinguistic worldly items themselves lack that structure. As theorists, we tend to use rough-and-ready descriptions in natural language (e.g., “light comes from above” or “This is a face”) when talking about neurally encoded hypotheses, but there is nothing intrinsically linguistic about the hypotheses themselves. The same applies when that which is being processed is linguistic (e.g., a communicative utterance or a written sentence). Very schematically put, from the “bottom” to the “top” it goes like this: One's brain can be initially “uncertain” about the shapes seen, or the sounds heard. Having resolved that, it can be uncertain about the letters or phonemes, and then the words used, and then what they mean, and then what is meant by them, or by the whole utterance, and so on. Within this picture, the way in which hypotheses are hierarchically arranged, and priors are updated and can become entrenched, developing a sensitivity to deep, below-the-surface structured statistical regularities in the world, suggests (in line with C&C's suggestion) that acquisition is indeed learning to process. Gone is the need for innate linguistic knowledge (although some of our priors – or our propensity to form them – may be, in some sense, innate). However, learning to process is learning to predict, where this involves being attuned to the dynamic statistical structure of the world of which language, and language users, are an important part.
In conclusion, although I am sympathetic to the spirit of what C&C present, a more wholesale predictive processing account yields very similar consequences but casts things in a different (and arguably more plausible) light.
You have
Access
The “Now-or-Never” bottleneck presents a real challenge. Rather than the solution presented by Christiansen and Chater (C&C), however, an alternative – one that is both simpler and more economical – is possible. They do allude to the solution I want to present, but they apply it locally rather than globally. The solution in question is prediction. One explanation for why this globally applied solution is not presented is that C&C adopt a traditional view of cognition, according to which inputs come in, get processed, and passed on. Adherence to this view is evidenced in talk of “Chunk-and-Pass” processing. Inputs “come in” and get “chunked” and “passed.” Within a predictive processing framework, on the other hand, the direction, if anything, is reversed. “Processing” constitutes inputs having been successfully predicted from the top down. What gets “passed” is prediction error, not some honed incoming product. What does get honed, in light of incoming prediction error, is predictions. Indeed,
an expected event does not need to be explicitly represented or communicated to higher cortical areas which have processed all of its relevant features prior to its occurrence. (Bubic et al. Reference Bubic, von Cramon and Schubotz2010, p. 10, quoted in Clark Reference Clark2013)
If we adopt a wholesale predictive processing approach, according to which prediction is not an aid to processing as traditionally construed but is rather its fundamental principle, then we overcome the Now-or-Never bottleneck in an evolutionarily, biologically, and computationally plausible way, and end up with all of the same consequences for how to understand language that the authors are at pains to point out.
Firstly, all of the presented solutions to the Now-or-Never bottleneck need not be seen, as C&C present them, as separate, but rather may be viewed as different facets of predictive processing. In other words, (a) “eager recoding and compression” of input, and (b) hierarchical levels of “representation” become consequences of, and not additions to, a need to (c) “deploy all available information predictively.” Let me explain why.
Predictive processing is a concrete implementation of a Bayesian strategy. Incoming signals are noisy and ambiguous, and so the brain uses Bayesian inference (it takes into account not only the “fit” of the hypothesis with the input, but also its “prior probability”) to settle on one hypothesis rather than another. Thus, a hypothesis can have a really good fit but such a low prior probability that it isn't selected (or it can have a poor fit, but such a high prior probability that it is selected).
This Bayesian strategy gets implemented in the brain as follows. The selection of a hypothesis determines a set of predictions about subsequent inputs, namely, inputs that are compatible with the hypothesis. If the hypothesis does a bad job of predicting inputs, it will be tweaked or abandoned altogether in favour of another hypothesis. These hypotheses are hierarchically arranged, with the hypotheses of one level providing the inputs (prediction error) for the next. “Higher” parts of the hierarchy are, roughly, those parts that are further away from the sensory stimulus. These tend to operate at longer timescales, and at higher levels of abstraction. “Lower” parts of the hierarchy are closer to the sensory stimulus. These tend to be at shorter timescales, and at low levels of abstraction. These, for example, correspond to early stages of visual processing: your brain's early statistically driven attempts to make sense of (predict) noisy inputs.
Predictive processing is time and energy efficient, and it involves compression (more or less “lossy” depending on the occasion). You save on bandwidth by passing on only what is newsworthy. What counts as “newsworthy” is simply what the receiver of the message hasn't already predicted, namely, prediction error. To sum up, then, predictive processing in the brain always involves (1) compression and (2) hierarchical arrangement of hypotheses (which, to use C&C's terminology, can be thought of as “representations”).
Now I'd like to gesture towards some of the consequences that predictive processing has for how we think about language. It has all of the same nine consequences that C&C enumerate, but, again, they are (at least for the most part) facets of each other rather than separate consequences. Let me illustrate this with two seemingly distant consequences: the multilevel organization of language (Consequence 1), the nature of what is learned during language acquisition (Consequence 5).
The hierarchical arrangement of hypotheses in predictive processing clearly suggests that language processing has a “multilevel organization.” Of course, our processing of other, nonlinguistic, stimuli has a similar organization, but that structure is not so clearly delineated since nonlinguistic worldly items themselves lack that structure. As theorists, we tend to use rough-and-ready descriptions in natural language (e.g., “light comes from above” or “This is a face”) when talking about neurally encoded hypotheses, but there is nothing intrinsically linguistic about the hypotheses themselves. The same applies when that which is being processed is linguistic (e.g., a communicative utterance or a written sentence). Very schematically put, from the “bottom” to the “top” it goes like this: One's brain can be initially “uncertain” about the shapes seen, or the sounds heard. Having resolved that, it can be uncertain about the letters or phonemes, and then the words used, and then what they mean, and then what is meant by them, or by the whole utterance, and so on. Within this picture, the way in which hypotheses are hierarchically arranged, and priors are updated and can become entrenched, developing a sensitivity to deep, below-the-surface structured statistical regularities in the world, suggests (in line with C&C's suggestion) that acquisition is indeed learning to process. Gone is the need for innate linguistic knowledge (although some of our priors – or our propensity to form them – may be, in some sense, innate). However, learning to process is learning to predict, where this involves being attuned to the dynamic statistical structure of the world of which language, and language users, are an important part.
In conclusion, although I am sympathetic to the spirit of what C&C present, a more wholesale predictive processing account yields very similar consequences but casts things in a different (and arguably more plausible) light.
ACKNOWLEDGMENT
This research was supported by a Wellcome Trust Strategic Award (WT098455MA).