The intuition that our brains harbor a predictive (forward) model linking visual percepts to their probable external causes (Helmholtz Reference Helmholtz1876) has been fleshed out over recent decades by sophisticated models (Friston Reference Friston2005; Mumford Reference Mumford1992; Rao & Ballard Reference Rao and Ballard1999), inspiring the view that Clark puts forward in the target article, that predictive coding is a cardinal principle of neural systems (cf. Friston Reference Friston2010; Hawkins & Blakeslee Reference Hawkins and Blakeslee2004). While this perspective offers elegant post-hoc explanations for a wide array of behavioral and neural phenomena, empirical studies directly testing the basic biological assumptions of predictive coding remain scarce. Specifically, the core empirical hypotheses derived from the predictive coding scheme are the presence of separable and hierarchically organized visual expectation and surprise computations (and associated neural units/signals) in the posterior brain (Friston Reference Friston2005). These predictions are provocative, because they differ drastically from traditional views of visual neurons as mere bottom-up feature detectors (Hubel & Wiesel Reference Hubel and Wiesel1965; Riesenhuber & Poggio Reference Riesenhuber and Poggio2000). But what is the empirical evidence directly supporting these claims? We first address results from macroscopic, human neuroimaging studies, followed by microscopic data from invasive animal experiments.
At the macroscopic level of inquiry provided by whole-brain functional neuroimaging, there are at present modest but promising lines of empirical support for predictive coding's core propositions. Most firmly established is the finding of robust occipital responses evoked by the surprising presence or absence of visual stimuli, presumably attributable to the computation of prediction error (e.g., Alink et al. Reference Alink, Schwiedrzik, Kohler, Singer and Muckli2010; den Ouden et al. Reference den Ouden, Friston, Daw, McIntosh and Stephan2009; Egner et al. Reference Egner, Monti and Summerfield2010). Similarly, “repetition suppression,” the attenuated neural response to a repeated stimulus that predictive coding attributes to a decrease in prediction error (Friston Reference Friston2005), has repeatedly been shown to be modulated by expectations, including in human functional magnetic resonance imaging (fMRI) (Summerfield et al. Reference Summerfield, Trittschuh, Monti, Mesulam and Egner2008), electroencephalographic (EEG) (Summerfield et al. Reference Summerfield, Wyart, Johnen and De Gardelle2011), and magnetoencephalographic (MEG) (Todorovic et al. Reference Todorovic, van Ede, Maris and de Lange2011) recordings. However, although evidence for visual surprise signals at the neural population level is fairly abundant, the attribution of these signals to local prediction error computations is not unequivocal, in that they could instead be argued to reflect attentional highlighting of unexpected stimuli (cf. Pearce & Hall Reference Pearce and Hall1980) driven by predictive processing elsewhere in the brain. In fact, the precise role that attention plays in the predictive coding machinery is currently under debate (Feldman & Friston Reference Feldman and Friston2010; Summerfield & Egner Reference Summerfield and Egner2009) and represents an important line of recent (Kok et al. Reference Kok, Rahnev, Jehee, Lau and de Lange2011; Wyart et al. Reference Wyart, Nobre and Summerfield2012) and future investigations into the predictive brain hypothesis.
In contrast to this support for the existence visual surprise signals, the proposition that there are simultaneous computations of prediction and prediction error signals carried out by distinct neural populations in visual cortex is presently only poorly substantiated. One recent fMRI study showed that neural population responses in the ventral visual stream can be successfully modeled as reflecting the summed activity of putative prediction and prediction error signals (Egner et al. Reference Egner, Monti and Summerfield2010; Jiang et al. Reference Jiang, Schmajuk and Egner2012). Similarly, a recent computational model can account for a wide array of auditory EEG responses by supposing co-existing prediction and prediction error neurons (Wacongne et al. Reference Wacongne, Changeux and Dehaene2012). However, neither of these studies demonstrates unambiguously the simultaneous operation of distinct neural sub-populations coding for expectations and surprise, a finding that would greatly bolster the biological feasibility of predictive coding models. Finally, the purported hierarchical nature of the interplay between expectation and surprise signals has garnered indirect support from a handful of fMRI studies. For instance, Murray and colleagues demonstrated the “explaining away” of activity in lower-level visual regions by activity in higher-level visual cortex when presenting a coherent visual object compared to its dissembled constituent parts (Murray et al. Reference Murray, Kersten, Olshausen, Schrater and Woods2002). Other investigators have employed effective connectivity analysis of fMRI data to probe how dynamic interactions between different brain regions may mediate prediction and surprise signals (den Ouden et al. Reference den Ouden, Friston, Daw, McIntosh and Stephan2009; Reference den Ouden, Daunizeau, Roiser, Friston and Stephan2010; Kok et al. Reference Kok, Rahnev, Jehee, Lau and de Lange2011; Summerfield & Koechlin Reference Summerfield and Koechlin2008; Summerfield et al. Reference Summerfield, Egner, Greene, Koechlin, Mangels and Hirsch2006). Nevertheless, a comprehensive demonstration of predictive coding “message passing” across several adjacent levels of the visual processing hierarchy remains lacking from the literature.
Perhaps most importantly, microscopic or cellular level data addressing the core tenets of the predictive coding hypothesis have been particularly scarce. In part, this may be for methodological reasons: For example, neurons with proposed “predictive fields” might be excluded from recording studies where cells are screened according to their bottom-up sensitivity. Moreover, the dynamics of the reciprocal interaction within the hierarchy might give rise to complex neural responses, making it hard to segregate prediction and error signals. Nevertheless, recent work has supplied some promising data. First, Meyer and Olson (Reference Meyer and Olson2011) have recently described single neurons in monkey inferotemporal cortex that exhibit surprise responses to unexpected stimulus transitions, thus possibly documenting visual prediction error neurons in the ventral visual stream. Two other recent studies, one in monkeys (Eliades & Wang Reference Eliades and Wang2008) and one in mice (Keller et al. Reference Keller, Bonhoeffer and Hubener2012), assessed neuronal activity in the context of sensorimotor feedback (e.g., the integration of movement with predicted changes in visual stimulation), observing putative prediction error signals in primary sensory cortices (for alternative interpretations, see Eliades & Wang Reference Eliades and Wang2008). Importantly, in Keller et al. (Reference Keller, Bonhoeffer and Hubener2012), these surprise signals co-occurred with both pure motor-related and sensory-driven signals, thus providing initial evidence for the possibility of co-habiting prediction and prediction error neurons in early visual cortex. Moreover, the putative prediction error neurons were found in supra-granular layers 2/3, which house precisely the superficial pyramidal cells that have been posited to support prediction error signaling by theoretical models of predictive coding (Friston Reference Friston2008; Mumford Reference Mumford1992).
In conclusion, we submit that the extant data from studies that directly aimed at testing core tenets of the predictive coding hypothesis are few but generally supportive. Looking to the future, additional demonstrations of simultaneous prediction and surprise computations within a single processing stage (in particular from single-neuron electrophysiology), as well as evidence for hierarchical interactions with adjacent stages, are required. We hope that over coming years, neuroscientists will be inspired to collect these data.
The intuition that our brains harbor a predictive (forward) model linking visual percepts to their probable external causes (Helmholtz Reference Helmholtz1876) has been fleshed out over recent decades by sophisticated models (Friston Reference Friston2005; Mumford Reference Mumford1992; Rao & Ballard Reference Rao and Ballard1999), inspiring the view that Clark puts forward in the target article, that predictive coding is a cardinal principle of neural systems (cf. Friston Reference Friston2010; Hawkins & Blakeslee Reference Hawkins and Blakeslee2004). While this perspective offers elegant post-hoc explanations for a wide array of behavioral and neural phenomena, empirical studies directly testing the basic biological assumptions of predictive coding remain scarce. Specifically, the core empirical hypotheses derived from the predictive coding scheme are the presence of separable and hierarchically organized visual expectation and surprise computations (and associated neural units/signals) in the posterior brain (Friston Reference Friston2005). These predictions are provocative, because they differ drastically from traditional views of visual neurons as mere bottom-up feature detectors (Hubel & Wiesel Reference Hubel and Wiesel1965; Riesenhuber & Poggio Reference Riesenhuber and Poggio2000). But what is the empirical evidence directly supporting these claims? We first address results from macroscopic, human neuroimaging studies, followed by microscopic data from invasive animal experiments.
At the macroscopic level of inquiry provided by whole-brain functional neuroimaging, there are at present modest but promising lines of empirical support for predictive coding's core propositions. Most firmly established is the finding of robust occipital responses evoked by the surprising presence or absence of visual stimuli, presumably attributable to the computation of prediction error (e.g., Alink et al. Reference Alink, Schwiedrzik, Kohler, Singer and Muckli2010; den Ouden et al. Reference den Ouden, Friston, Daw, McIntosh and Stephan2009; Egner et al. Reference Egner, Monti and Summerfield2010). Similarly, “repetition suppression,” the attenuated neural response to a repeated stimulus that predictive coding attributes to a decrease in prediction error (Friston Reference Friston2005), has repeatedly been shown to be modulated by expectations, including in human functional magnetic resonance imaging (fMRI) (Summerfield et al. Reference Summerfield, Trittschuh, Monti, Mesulam and Egner2008), electroencephalographic (EEG) (Summerfield et al. Reference Summerfield, Wyart, Johnen and De Gardelle2011), and magnetoencephalographic (MEG) (Todorovic et al. Reference Todorovic, van Ede, Maris and de Lange2011) recordings. However, although evidence for visual surprise signals at the neural population level is fairly abundant, the attribution of these signals to local prediction error computations is not unequivocal, in that they could instead be argued to reflect attentional highlighting of unexpected stimuli (cf. Pearce & Hall Reference Pearce and Hall1980) driven by predictive processing elsewhere in the brain. In fact, the precise role that attention plays in the predictive coding machinery is currently under debate (Feldman & Friston Reference Feldman and Friston2010; Summerfield & Egner Reference Summerfield and Egner2009) and represents an important line of recent (Kok et al. Reference Kok, Rahnev, Jehee, Lau and de Lange2011; Wyart et al. Reference Wyart, Nobre and Summerfield2012) and future investigations into the predictive brain hypothesis.
In contrast to this support for the existence visual surprise signals, the proposition that there are simultaneous computations of prediction and prediction error signals carried out by distinct neural populations in visual cortex is presently only poorly substantiated. One recent fMRI study showed that neural population responses in the ventral visual stream can be successfully modeled as reflecting the summed activity of putative prediction and prediction error signals (Egner et al. Reference Egner, Monti and Summerfield2010; Jiang et al. Reference Jiang, Schmajuk and Egner2012). Similarly, a recent computational model can account for a wide array of auditory EEG responses by supposing co-existing prediction and prediction error neurons (Wacongne et al. Reference Wacongne, Changeux and Dehaene2012). However, neither of these studies demonstrates unambiguously the simultaneous operation of distinct neural sub-populations coding for expectations and surprise, a finding that would greatly bolster the biological feasibility of predictive coding models. Finally, the purported hierarchical nature of the interplay between expectation and surprise signals has garnered indirect support from a handful of fMRI studies. For instance, Murray and colleagues demonstrated the “explaining away” of activity in lower-level visual regions by activity in higher-level visual cortex when presenting a coherent visual object compared to its dissembled constituent parts (Murray et al. Reference Murray, Kersten, Olshausen, Schrater and Woods2002). Other investigators have employed effective connectivity analysis of fMRI data to probe how dynamic interactions between different brain regions may mediate prediction and surprise signals (den Ouden et al. Reference den Ouden, Friston, Daw, McIntosh and Stephan2009; Reference den Ouden, Daunizeau, Roiser, Friston and Stephan2010; Kok et al. Reference Kok, Rahnev, Jehee, Lau and de Lange2011; Summerfield & Koechlin Reference Summerfield and Koechlin2008; Summerfield et al. Reference Summerfield, Egner, Greene, Koechlin, Mangels and Hirsch2006). Nevertheless, a comprehensive demonstration of predictive coding “message passing” across several adjacent levels of the visual processing hierarchy remains lacking from the literature.
Perhaps most importantly, microscopic or cellular level data addressing the core tenets of the predictive coding hypothesis have been particularly scarce. In part, this may be for methodological reasons: For example, neurons with proposed “predictive fields” might be excluded from recording studies where cells are screened according to their bottom-up sensitivity. Moreover, the dynamics of the reciprocal interaction within the hierarchy might give rise to complex neural responses, making it hard to segregate prediction and error signals. Nevertheless, recent work has supplied some promising data. First, Meyer and Olson (Reference Meyer and Olson2011) have recently described single neurons in monkey inferotemporal cortex that exhibit surprise responses to unexpected stimulus transitions, thus possibly documenting visual prediction error neurons in the ventral visual stream. Two other recent studies, one in monkeys (Eliades & Wang Reference Eliades and Wang2008) and one in mice (Keller et al. Reference Keller, Bonhoeffer and Hubener2012), assessed neuronal activity in the context of sensorimotor feedback (e.g., the integration of movement with predicted changes in visual stimulation), observing putative prediction error signals in primary sensory cortices (for alternative interpretations, see Eliades & Wang Reference Eliades and Wang2008). Importantly, in Keller et al. (Reference Keller, Bonhoeffer and Hubener2012), these surprise signals co-occurred with both pure motor-related and sensory-driven signals, thus providing initial evidence for the possibility of co-habiting prediction and prediction error neurons in early visual cortex. Moreover, the putative prediction error neurons were found in supra-granular layers 2/3, which house precisely the superficial pyramidal cells that have been posited to support prediction error signaling by theoretical models of predictive coding (Friston Reference Friston2008; Mumford Reference Mumford1992).
In conclusion, we submit that the extant data from studies that directly aimed at testing core tenets of the predictive coding hypothesis are few but generally supportive. Looking to the future, additional demonstrations of simultaneous prediction and surprise computations within a single processing stage (in particular from single-neuron electrophysiology), as well as evidence for hierarchical interactions with adjacent stages, are required. We hope that over coming years, neuroscientists will be inspired to collect these data.