To construct an up-to-date, dynamic picture of the outside world, brains have to overcome a problem: different sensory signals are processed at different speeds in different parts of the brain. As Nijhawan emphasizes, all this processing takes time – time during which the outside world continues to change. One clever method by which nervous systems can reduce this problem is by predicting, whenever possible, the next state of the world (see Hawkins & Blakeslee Reference Hawkins and Blakeslee2004; Nijhawan Reference Nijhawan1994; Suddendorf & Corballis Reference Suddendorf and Corballis2007; and the present target article). Nijhawan has long championed the idea that prediction happens at the level of the perceptual system (e.g., Nijhawan Reference Nijhawan1994), mitigating temporal delays at early, premotor stages. My comments here reinforce an emphasis on the brain's need to compensate for delays, but I offer some additional methods that the brain may use beyond prediction.
Prediction and postdiction: Two frameworks with the goal of compensation
In conjunction with any prediction it attempts, the brain also appears to implement a postdictive strategy (Eagleman & Sejnowski Reference Eagleman and Sejnowski2000). This term simply means that the brain continues to collect information after an event (such as a flash) while settling on its best estimate about events and their timing in the outside world. Postdiction is easily illustrated by apparent motion: When the first stimulus disappears, one cannot perceive the motion until after the second stimulus has appeared (in an unpredicted location). Nonetheless, one has the illusion of seeing it move on the fly. Obviously, this can only happen when perception is generated retrospectively (Eagleman Reference Eagleman2001).
In the context of the flash-lag effect (FLE), three empirical findings seem to support postdiction: the flash-initiated (Khurana & Nijhawan Reference Khurana and Nijhawan1995), flash terminated (Nijhawan Reference Nijhawan1992), and motion-reversing (Whitney & Murakami Reference Whitney and Murakami1998) conditions – all of which illustrate that the motion information after the flash, not before it, drives the effect (Eagleman & Sejnowski Reference Eagleman and Sejnowski2000). Nijhawan suggests that these challenges might be accounted for by a rapidly arbitrated competition between rivaling neural representations. Although he avoids the term postdiction, a biased competition model nonetheless requires it. The competition between a predictive model and new incoming information cannot be tipped until the new information is collected after the flash. No matter how rapidly the arbitration occurs, it still is postdictive.
There are competing explanations for the FLE that do not require the predictive component at all. For example, one alternative draws on the fact that motion signals shift localization judgments (for a review, see Whitney Reference Whitney2002). We have shown that the motion signals collected after an event are the ones that bias perceived position (Eagleman & Sejnowski Reference Eagleman and Sejnowski2007). Our evidence supports the framework that incoming motion information updates localizations so that the data do not become stale (for details, see Eagleman & Sejnowski Reference Eagleman and Sejnowski2007). This is a purely postdictive explanatory framework for the FLE.
To understand why postdiction is necessary, one only has to examine the tasks and resources of the visual system. As one of its tasks, the visual cortex tries to get the timing of outside events correct; for its resources, it has to deal with the foibles of the populations that feed it. The retina and visual thalamus have circuitries that cause incoming signals to temporally spread from the first stages of the visual system based, for example, on their luminance (Gawne et al. Reference Gawne, Kjaer and Richmond1996; Lee at al. Reference Lee, Williford and Maunsell2007; Maunsell et al. Reference Maunsell, Ghose, Assad, McAdams, Boudreau and Noerager1999). For the visual brain to correctly align the timing of events in the world, it may have to wait ~100 msec for the slowest information to arrive – thereby allowing the visual system to discount different delays imposed by the early stages.
The reason for the vigorous debate between prediction and postdiction is that both sides enjoy strong support: On the one hand, brains use every available opportunity to predict what is coming next, thus saving valuable processing time; on the other hand, experiments show that perceptions can be changed retrospectively, as may be necessitated by an unpredictably changing world.
The important thing to note is that both the predictive and postdictive frameworks have the same goal: to perceptually place the location of a moving object closer to its real-world position. Prediction does this by guessing ahead; postdiction does this by allowing incoming motion signals to advantageously adjust location judgments. Both techniques lead to the perception of a moving object closer to its real-world position. The advantage of postdiction is its ability to naturally account for sudden motion reversals such as bounces and ricochets; however, it is possible that both mechanisms are implemented in different contexts or in different areas of the nervous system. Future experiments are required to determine these details.
Perceptual compensation for temporal delays
Finally, I want to expand on part of Nijhawan's argument. He correctly points out that compensation can take place at the level of the motor output or the perceptual input (or anywhere in between). He argues on the side of perceptual compensation, a position that has traditionally received little attention, in part because of the paucity of direct experimental support. However, in his target article he leaves out a critical example that supports perceptual compensation: the recalibration of motor sensory systems when exposed to delayed feedback.
The recalibration story begins with a mystery: Given that multisensory signals arrive in the brain at different times, how can the brain decide which events were supposed to be simultaneous in the outside world? We have proposed that the brain perceptually recalibrates its expectations about the arrival times of signals by employing a simple assumption: when the motor system executes an act (such as knocking on a door), all the resulting feedback should be assumed to be simultaneous, and any delays should be perceptually adjusted until simultaneity is perceived (Stetson et al. Reference Stetson, Cui, Montague and Eagleman2006). Although this recalibration is normally adaptive, note a strange consequence: Imagine we repeatedly inject a tiny delay (e.g., 100 msec) between a subject's key-press and a subsequent flash for a dozen trials, and then we suddenly remove the delay. Because the perceptual systems have recalibrated to compensate for the delay, the subject now perceives that the flash occurred before the key-press: an illusory reversal of action and sensation (Cunningham et al. Reference Cunningham, Billock and Tsou2001a; Stetson et al. Reference Stetson, Cui, Montague and Eagleman2006). Note that this recalibration is no mere party trick – it is critical to solving the problem of causality, which requires, at bottom, temporal order judgments. The only way causality can be accurately determined in a multisensory brain is to keep the temporal delay of signals calibrated in the face of different sensory pathways operating at different speeds. This example serves to buttress Nijhawan's general argument that compensation for delays can take place at very early, perceptual levels.
