Nijhawan makes a clear case for the need to compensate for delays arising from processing and transmission times. The evidence for compensation in perceptual decision and visual awareness appears somewhat controversial (Eagleman & Sejnowski Reference Eagleman and Sejnowski2000; Krekelberg & Lappe Reference Krekelberg and Lappe2001), but the evidence for compensation for motor reactions to a rapidly changing sensory stimulus is uncontroversial. Typical visuomotor delays in ballistic interception of fast targets (such as in catching or hitting in ball games) are about 200 msec – at least an order of magnitude longer than the temporal accuracy required for interception (about ±10 msec). Unless the nervous system has built-in mechanisms to compensate for such delays, the interception program would be based on obsolete visual information about target motion, and, as a consequence, the target would be badly missed.

Nijhawan proposes a mechanism for neural compensation of delays that is based on a spatial extrapolation linearly related to the time delay. According to his hypothesis, visual prediction would be concerned primarily with horizontal processes, which transmit neural information between two neighbouring retinotopic sites. The speed of neural transmission and the distance between neighbouring neurons along the horizontal direction would jointly determine the amount of spatial and temporal extrapolation. Another mechanism could consist in a shift of the receptive field in response to moving stimuli. Sundberg et al. (Reference Sundberg, Fallah and Reynolds2006) found that neurons in monkey area V4 exhibit such a shift in response to a particular type of moving stimuli. The direction of the receptive field shift was opposite to the direction of target motion, as if the cell had been recruited by a wave of activity preceding the target. Ferrera and Barborica (Reference Ferrera and Barborica2006) argued that a moving target would leave a trail of refractory neurons in its wake so that spiking activity would be shifted toward the leading edge.

Interestingly, mechanisms of visual prediction based on spatial extrapolation, such as those mentioned above, work only for targets moving at constant speed (uniform motion), because the spatial shifts co-vary with the time samples in a fixed manner. Most targets, however, accelerate or decelerate to a variable extent. Let us consider a very common situation – that of motion affected by Earth's gravity, such as free-fall, ballistic, pendulum, or wave motion. Although all objects are accelerated downward by gravity at the same rate, the corresponding acceleration of the retinal image is not at all constant, being inversely related to the apparent viewing distance of the object. The question then is how the central nervous system (CNS) compensates for delays in the case of accelerating or decelerating motion. Here we show that temporal extrapolation rather than spatial extrapolation can more easily do the job.

Figure 1A depicts space–time plots similar to those of Figures 3 and 4 of Nijhawan, but for an object moving at constant acceleration (when the spatial variable decreases from right to left) or deceleration (when the spatial variable increases from left to right). The dashed curve depicts the physical trajectory, and the dotted curve depicts the corresponding trajectory “seen” by a neuron with a fixed visual delay. Clearly, the spatial shifts required to compensate for the visual delay (solid line segments connecting the two curves) are not constant anymore, as they were in the spatial extrapolation scheme proposed by Nijhawan.

Figure 1.

Figure depicts space-time plots similar to those of Figures 3 and 4 in the target article, but for an object moving at constant acceleration (when the spatial variable decreases from right to left) or deceleration (when the spatial variable increases from left to right). The dashed curve depicts the physical trajectory and the dotted curve depicts the corresponding trajectory “seen” by a neuron with a fixed visual delay.

In theory, a first-order model might be used to approximate a second-order motion. One such model is provided by the tau function, tau=x(t)/v(t), where x(t) is the spatial position of the target and v(t) is the corresponding velocity (Lee Reference Lee1976). However, it can be shown that, in case of free-fall motion from relatively short drop heights, such an approximation would imply significant temporal errors in interception (>50 msec), corresponding to the difference between the time-to-contact predicted by tau and the actual time-to-contact of the ball accelerated by gravity (Zago & Lacquaniti Reference Zago and Lacquaniti2005). In fact, we know that unless taken by surprise, people can easily intercept targets descending along the vertical accelerated by gravity (Lacquaniti & Maioli Reference Lacquaniti and Maioli1989; Zago et al. Reference Zago, Bosco, Maffei, Iosa, Ivanenko and Lacquaniti2004); they generally intercept such targets much better than similar targets descending at constant speed (McIntyre et al. Reference McIntyre, Zago, Berthoz and Lacquaniti2001; Zago et al. Reference Zago, Bosco, Maffei, Iosa, Ivanenko and Lacquaniti2004). The inescapable conclusion, therefore, is that neural compensation uses second-order information. However, direction, velocity, and acceleration are probably encoded together at the level of single neurons both in retinal ganglion cells (Thiel et al. Reference Thiel, Greschner, Eurich, Ammermuller and Kretzberg2007) and in visual motion area MT (Lisberger & Movshon Reference Lisberger and Movshon1999; Price et al. Reference Price, Ono, Mustari and Ibbotson2005; Schlack et al. Reference Schlack, Krekelberg and Albright2007). It may not be an easy task for neural populations to disentangle acceleration estimates independently of velocity and to extrapolate target trajectory in a time-varying fashion, as required by the schematic graph of Figure 1A.

An alternative solution, however, is to extrapolate trajectories in time, rather than in space. This is shown in Figure 1B, where the same spatio-temporal trajectories of an accelerating or decelerating object as those of Figure 1A are plotted. Here, however, the compensation for the visual delay is implemented by means of a temporal shift (black segments connecting the two curves). The resulting correction at each sampled position of the target is constant throughout the trajectory. Needless to say, time extrapolation works equally well for uniform motion.

Behaviourally, it has been shown that when subjects must intercept novel types of vertical target motion, they learn to improve their performance by shortening or lengthening central processing time, with the direction of change depending on the task (Senot et al. Reference Senot, Zago, Lacquaniti and McIntyre2005; Zago & Lacquaniti Reference Zago and Lacquaniti2005; Zago et al. Reference Zago, Bosco, Maffei, Iosa, Ivanenko and Lacquaniti2004; Reference Zago, Bosco, Maffei, Iosa, Ivanenko and Lacquaniti2005). This is equivalent to modulating the time shifts in Figure 1B. We do not know where in the brain a time extrapolation mechanism might be implemented, though it could occur at the interface between sensory and motor systems. There is some evidence that neurons in posterior parietal cortex (area 7a and LIP) may encode elapsed time by means of ramping activity (Leon & Shadlen Reference Leon and Shadlen2003; Maimon & Assad Reference Maimon and Assad2006; Merchant et al. Reference Merchant, Battaglia-Mayer and Georgopoulos2004).