Introduction
Color and motion direction are basic visual features. Both are essential for perceptual organization of the environment and can also capture visual attention with ease (for a review, see Wolfe, Reference Wolfe and Pashler1998). In natural visual environments, color and motion direction of visual elements can be correlated as different parts of a moving object may be of similar color. In spite of this, many studies do not examine the processing of color–motion correlations in the visual system, often choosing to vary color and motion independently (Bartels & Zeki, Reference Bartels and Zeki2000). This approach is supported by the fact that color and motion are processed in anatomically segregated systems that are sparsely interconnected (Shipp & Zeki, Reference Shipp and Zeki1995). Interactions between color and motion processing systems are thought to be limited to responsiveness of some motion-selective cells to isoluminant parvocellular stimuli (Logothetis et al., Reference Logothetis, Schiller, Charles and Hulbert1990); parvocellular signals are also enhanced during smooth pursuit eye movements, benefiting the processing of moving stimuli (Schütz et al., Reference Schütz, Braun, Kerzel and Gegenfurtner2008). Time courses of color and motion processing also differ—Color is analyzed prior to the processing of local motion signals (Moutoussis & Zeki, Reference Moutoussis and Zeki1997), which in turn become combined in order to perceive global motion (i.e., motion of larger objects). Due to these temporal differences, color information could act as a cue to segment the image before local motion signals are integrated. This would lead to more efficient processing of scenes containing motion–color correlations.
However, previous studies indicate that this prediction does not hold universally. In many instances, the human motion system does not seem to use color–motion correlations automatically, and a segmentation of a dynamic scene by color does not necessarily precede the extraction of motion. In the experiments by Li and Kingdom (Reference Li and Kingdom2001), the random dot kinematograms (RDKs) contained an equal number of signal and noise dots, and all signal dots (i.e., dots that move in the same direction) were associated with one color, and all noise dots (i.e., dots that move in random directions) were associated with another color; the performance was not better than if no color–motion association existed. Under certain conditions, however, observers can exploit the association between color and motion signals. Croner and Albright (Reference Croner and Albright1997) reported improved performance when the number of same-color coherent dots fell to about 10% of total dots. Likewise, an improvement was brought about by cueing in the study of Li and Kingdom (Reference Li and Kingdom2001): If the observer had prior knowledge of the color that is associated with the signal dots, his or her performance in the motion task improved even if signal and noise dots were evenly distributed.
What could be the neural basis for the improvements due to color–motion correlations described by Croner and Albright (Reference Croner and Albright1997) and Li and Kingdom (Reference Li and Kingdom2001)? The processing advantage could have occurred through the operation of a neural module sensitive to both the color and the motion direction. Alternatively, a feedback from color processing areas could have led to the segregation (i.e., grouping) of local motion signals associated with the same color, an option suggested by Li and Kingdom (Reference Li and Kingdom2001). Using a chromatic motion paradigm that allows us to dissociate color and motion information, we can eliminate the possibility of joint color–motion processing, leaving grouping by color as the only possible mechanism that leads to improvements due to color–motion correlations. Croner and Albright (Reference Croner and Albright1997) and Li and Kingdom (Reference Li and Kingdom2001) used two-color RDKs where all the dots carried a large but constant luminance signal and, in addition, the signal and noise dots differed along the red–green dimension (i.e., had different L-M-cone inputs). It has been demonstrated that unless specific stimulus parameters are used (large blob size and fast motion speed; Ruppertsberg et al., Reference Ruppertsberg, Wuerger and Bertamini2003, Reference Ruppertsberg, Wuerger and Bertamini2007), global motion extraction in the isoluminant plane only operates on local motion signals that contain an L-M component (Fig. 1a); if the RDK consists of local motion signals defined exclusively along the S-cone isolating axis, only the color of the signal is processed but no global motion direction can be extracted (i.e., motion always appears random). As summarized in Fig. 1a, differences in the S-cone signals affect only color appearance, while different L-M inputs affect not only color appearance but can also be used to extract global motion (Ruppertsberg et al., Reference Ruppertsberg, Wuerger and Bertamini2003). In the experiment reported here, we used the fact that the S-cone signals are “invisible” to the motion system for a wide range of spatial and temporal parameters. We generate a stimulus with a constant ΔLM signal (Fig. 1a), allowing for global motion to be extracted. Then, we can manipulate color appearance by adding an S-cone component (ΔS; increment or decrement—see Fig. 1), which will have no input into the motion mechanism, as ΔLM for both colors would remain the same. Therefore, selective color-based enhancement of the local motion signal is not possible, and the only mechanism that can lead to improvements in performance is grouping by color.
(Color online) (a) The chromaticities of colors C1 and C2 in the isoluminant DKL plane. Along the L-M axis, only the difference between L- and M-cone varies, while keeping L + M constant (ΔLM). Along the S-axis, only the S-cones vary. Colors C1 (magenta) and C2 (orange) only differ in their S-cone component (ΔS). (b) In this example, a RDK is presented, wherein all signal blobs are of color C1 and all noise blobs are of color C2. C1 and C2 only differ in their S-cone component (ΔS), which changes the color appearance but does not contribute anything to the motion signal. Any facilitation must therefore occur through grouping based on the S-cone component.
Li and Kingdom (Reference Li and Kingdom2001) have already hypothesized that grouping by color is the crucial process that underlies improvements due to color–motion correlations. They also argued that attentional selection is crucial if such grouping is to lead to an improvement in performance; attentional selection could operate in both an exogenous (Croner & Albright, Reference Croner and Albright1997) and an endogenous (Li & Kingdom, Reference Li and Kingdom2001) fashion. More precisely, Li and Kingdom (Reference Li and Kingdom2001) argued that the difference in the proportion of signal-color and noise-color blobs had introduced an exogenous cue that created a pop-out effect in the study by Croner and Albright (Reference Croner and Albright1997), capturing the attention of observers and thus improving performance. The effect by Croner and Albright (Reference Croner and Albright1997) is thus derived from exogenous (“bottom-up,” stimulus-driven) attention; its main characteristic is that it is automatically drawn by an abruptly appearing or salient stimuli. The time course of such attentional enhancement is rapid and transient. Li and Kingdom (Reference Li and Kingdom2001) have argued that the effect they obtained is due to endogenous (“top-down,” goal-driven) attention; here, attention is deployed voluntarily to a certain location or attribute. Endogenous attention has a slow and sustained time course (Müller & Rabbitt, Reference Müller and Rabbitt1989). Exogenous and endogenous attention interact with each other, although they are subserved by partially segregated networks of brain areas (Corbetta & Shulman, Reference Corbetta and Shulman2002). Our experiments will provide us with a possibility to further investigate the potential role of attention in performance improvements due to color–motion correlations.
The novelty of our approach when compared to Croner and Albright (Reference Croner and Albright1997) and Li and Kingdom (Reference Li and Kingdom2001) lies in the fact that by choosing the appropriate spatial and temporal parameters (Ruppertsberg et al., Reference Ruppertsberg, Wuerger and Bertamini2003), we can study global motion extraction using chromatic modulations that produce a strong color difference (e.g., Wuerger et al., Reference Wuerger, Atkinson and Cropper2005) between signal and noise blobs without affecting the local motion signal. To that end, we generated isoluminant RDKs where signal blobs (i.e., coherent motion) and noise blobs (random motion) differed only in the S-cone component, while both signal and noise blobs had the same L-M component (ΔS; Fig. 1). This will allow us to look at the effect of grouping by color in isolation.
Another aim of our study was to identify the neural markers in the human electroencephalogram (EEG) that would accompany such grouping by color driven by S-cone signals when the signal color is cued (as in Li & Kingdom, Reference Li and Kingdom2001). The high temporal resolution of EEG makes it well suited for the study of specific neural markers of early visual processing. In fact, there is a large body of EEG work, wherein overlapping surfaces consisting of rotating dots are formed through color–motion correlations and exogenous cues are used to direct attention to either one of them (Valdes-Sosa et al., Reference Valdes-Sosa, Bobes, Rodriguez and Pinilla1998, Reference Valdes-Sosa, Bobes, Rodriguez, Acosta, Perez, Iglesias, Borrego, Kanwisher and Duncan2004; Pinilla et al., Reference Pinilla, Cobo, Torres and Valdes-Sosa2001). Exogenous cueing of the surface containing the motion translation affects early sensory components P1 and N1 of the human event related potential (ERP). Valdes-Sosa et al. (Reference Valdes-Sosa, Bobes, Rodriguez and Pinilla1998), for instance, found that color-based attention suppresses P1 and N1 components elicited by the translation of the unattended surface. In other studies, the amplitude of the N1 was affected by attentional cueing (Pinilla et al., Reference Pinilla, Cobo, Torres and Valdes-Sosa2001; Lopez et al., Reference Lopez, Rodriguez and Valdes-Sosa2004), similarly to N1 effects observed in studies with spatial cueing tasks (e.g., Eimer, Reference Eimer1993, Reference Eimer1994). In line with these findings, we expected to obtain a modulation of the N1 in our EEG experiment, with larger amplitude for cued stimuli. We did not expect that the P1 component will be present in our data as this component is usually absent when isoluminant chromatic displays are used (Porciatti & Sartucci, Reference Porciatti and Sartucci1999; Khoe et al., Reference Khoe, Mitchell, Reynolds and Hillyard2005).
General methods
Rationale
The purposes of our experiments are (1) to investigate whether the visual system can use correlations between color and motion to segregate motion noise from coherent motion using only color as a grouping cue and (2) to determine the neural correlates of color cueing using EEG.
In the main experiments, we therefore used isoluminant two-color RDKs, wherein one color, the “signal” color, is associated with coherent motion, and the other color, the “noise” color, is associated with random motion. Observers had to discriminate between random and coherent motion. The signal color and the noise color differed only along the S-cone axis (Fig. 1) but not in their L-M coordinate. The advantage of defining the color difference only by S-cone modulation is that any performance improvement can only be driven by the color difference and not by a direct gain in the color–motion signals per se since the S-cone component is invisible to the motion mechanism.
In the following, we describe three experiments. In the baseline experiment, we used single-color RDKs to confirm that the colors C1 and C2 (Fig. 1) are equally effective in a global motion discrimination task and that S-cone modulation per se cannot be used effectively in this task. In Experiment 1, we tested whether S-cone signal–defined correlations between the color and the direction of the local motion signals are exploited automatically (Croner & Albright, Reference Croner and Albright1997) or whether this requires a top-down cue (Li & Kingdom, Reference Li and Kingdom2001). In Experiment 2, we used EEG to characterize the neural markers of color cueing.
Apparatus
Our experiments were run on a standard PC with a VSG2/5 graphics card (32 MB memory; Cambridge Research Systems, Ltd., Kent, UK). The stimulus presentation was controlled using Matlab (Mathworks, Natick, MA), and the stimuli were presented on a 21-inch cathode ray tube (CRT) monitor (SONY GDM-F500, Berkshire, UK). The chromatic and luminance output of the monitor were calibrated using the Colour CAL (Cambridge Research Systems, Ltd.) and checked with a spectroradiometer (Photo Research PR650; Glen Spectra, Ltd., Middlesex, UK). The monitor had been switched on for at least 30 min before any experiment. Participants responded via a button box (CT3; Cambridge Research Systems, Ltd.) and were seated 140 cm from the screen in an otherwise dark cubicle.
The psychophysics and the EEG laboratory are equipped with the same visual display apparatus (DELL PC with a VSG2/5 graphics card; SONY GDM-F500 monitor). The visual displays in both laboratories were calibrated such that they produced identical maximum outputs.
Color space
To describe the chromatic properties of our stimuli, we used the Derrington, Krauskopf, & Lennie (DKL) color space (Derrington et al., Reference Derrington, Krauskopf and Lennie1984; Brainard, Reference Brainard, Kaiser and Boynton1996), which is an extension of the MacLeod–Boynton chromaticity diagram (MacLeod & Boynton, Reference MacLeod and Boynton1979). In this space, any color is defined by modulations along three different “cardinal” axes: Along the achromatic axis, all three cone classes (L, M, and S) are modulated such that the contrast is identical, that is, ΔL/ΔL BG = ΔM/ΔM BG = ΔS/ΔS BG, where ΔL, ΔM, and ΔS denote the incremental cone excitations in three cone classes, respectively. L BG,, M BG, and S BG indicate the L-, M-, and S-cone excitations of the background. By definition, the maximum contrast that can be achieved in each cone class is 1 if stimuli are modulated along the achromatic direction. The second direction refers to a modulation along a red–green axis; modulations in this direction leave the excitation of the S-cones constant (i.e., ΔS = 0), and the excitation of the L- and M-cones covaries as to keep their sum constant. Therefore, this axis if referred to as a “constant S-cone axis” (Kaiser & Boyton, Reference Kaiser and Boyton1996, p. 306) or a “red–green isoluminant” axis (Brainard, Reference Brainard, Kaiser and Boynton1996). Along the third axis, only the S-cones are modulated, and ΔL = ΔM = 0. Therefore, this axis is often referred to as a “constant L & M cone” axis (Kaiser & Boyton, Reference Kaiser and Boyton1996, p. 306) or as an “S-cone isoluminant” axis (Brainard, Reference Brainard, Kaiser and Boynton1996) or as a “tritanopic confusion line.”
Instead of defining the chromatic properties of a stimulus by their respective L-, M-, and S-cone modulations, the stimulus can be defined in terms of the responses of a set of hypothesized mechanisms that are isolated by these cardinal color modulations (Derrington et al., Reference Derrington, Krauskopf and Lennie1984; Brainard, Reference Brainard, Kaiser and Boynton1996; Eskew et al., Reference Eskew, McLellan, Guilianini, Gegenfurtner and Sharpe1999; Wuerger et al., Reference Wuerger, Watson, Ahumada, Rogowitz and Pappas2002). The three corresponding mechanisms are two cone-opponent color mechanisms and a luminance mechanism. One of the two cone-opponent mechanisms is a red–green mechanism that takes the weighted difference between the differential L- and the M-cone excitations. The second cone-opponent mechanism is a yellowish–violet mechanism that takes the weighted difference between the differential S-cone and the summed differential M- and L-cone excitations. The luminance mechanism sums the weighted differential M- and L-cone signals. For the sake of simplicity, these mechanisms are referred to as “L + M,” “L − M,” and “S − (L + M)” (Derrington et al., Reference Derrington, Krauskopf and Lennie1984).
In the present study, we only used stimuli in an isoluminant plane (at 50 cd/m2), which is shown in Fig. 1a. The Commission Internationale de l’Eclairage (CIE) coordinates of the gray background are as follows: x = 0.289, y = 0.301, and Lum = 50.1 cd/m2; the corresponding cone coordinates are L = 32.43, M = 17.68, and S = 1.10. Along the “L − M” direction, red is indicated by 0 deg and at a radius (r) of 1, its absolute cone coordinates are L = 34.74, M = 15.76, and S = 1.09; 180 deg refers to green (at r = 1, L = 30.33, M = 19.47, and S = 1.09). Along the “S − (M + L)” direction, 90 deg represents violet (at r = 1, L = 32.15, M = 17.54, and S = 1.86), and 270 deg indicates yellow–green (at r = 1, L = 32.42, M = 17.47, and S = 0.33). Therefore, for a radius of 1, the resulting L-, M-, and S-cone contrasts along the L-M axis are as follows: for modulations from gray to red: 0.07 (L), −0.11 (M), and 0.0 (S); for modulations from gray to green: −0.065 (L), 0.10 (M), and 0.0 (S). For positive S-cone modulations (between gray and violet), the L-, M-, and S-cone contrasts corresponding to a radius of 1 are: 0.0 (L), 0.0 (M), and 0.7 (S); negative S-cone modulations (between gray and yellow–green) of radius 1 yield a negative S-cone contrast of −0.7 and zero L- and M-cone contrasts. When cone contrast is defined as vector length in L-, M-, and S-cone contrast space, a radius of 1 for L-M modulations corresponds to a cone contrast of 0.13 and to a cone contrast of 0.7 for S-cone modulations.
Stimuli
We used RDKs with 300 colored Gaussian blobs. Half of the blobs were defined by an S-cone increment (Fig. 1; C1, indicated by 45 deg, at r = 1, L = 33.93, M = 16.37, and S = 1.63) and the other half by an S-cone decrement (C2, indicated by 315 deg, at r = 1, L = 33.76, M = 16.04, and S = 0.55). Colors C1 and C2 differed only in their S-cone component (ΔS; Fig. 1) and therefore looked very different (C1: magenta and C2: orange).
Each single Gaussian blob subtended a visual angle of 0.22 deg. The blobs moved at a speed of 1 deg/s, and the array of the RDK was 7.9 × 6 deg on an otherwise gray background. The monitor refresh rate was 120 Hz. Each individual blob moved along a trajectory during a 350-ms interval (i.e., their lifetime was not limited). No overlap was allowed between the trajectories of individual blobs. A separate RDK was generated for each stimulus interval. The coherence level, that is, the proportion of the Gaussian blobs in the RDK that shared the same motion direction, was 50% (Fig. 2a) in order to prevent differences in the number of blobs of a particular color.
(Color online) (a) RDK, with 0, 50, or 100% coherence. (b) Trial outlook—The number of blobs is reduced for the sake of clarity. In this example, the condition is cued correlated, with signal color being C1 (magenta). This is reflected by the color of the cue.
The chromatic, spatial and temporal parameters of the chosen stimuli closely matched the study by Ruppertsberg et al. (Reference Ruppertsberg, Wuerger and Bertamini2003) since under these conditions, S-cone modulations do not contribute to global motion perception. This means that with S-cone isolating stimuli (i.e., no L-M-cone input), observers are not able to perform the task.
Observer isoluminance
There are individual differences in the luminous efficiency (V(λ); Wyszecki & Stiles, Reference Wyszecki and Stiles2000), which may result in a small luminance signal being present in the nominally isoluminant chromatic signals. In order to adjust for observers’ individual point of isoluminance, we used heterochromatic flicker photometry (HCFP; Walsh, Reference Walsh1958) for each individual observer, following the same procedure as described in Ruppertsberg et al. (Reference Ruppertsberg, Wuerger and Bertamini2003).
The display altered between a red and a green stimulus at a 20-Hz frequency. By varying the luminance of red and green in opposite directions, it is possible to find a setting for which the perception of flicker is minimal. HCFP uses the fact that the chromatic system is too slow to follow fast temporal changes, but the luminance system is able to detect the fast changing luminance differences between red and green. Therefore, if the perception of flicker is minimized, the luminance difference is minimized as well. Since temporal and spatial factors may affect the individual isoluminance point, we used stimuli as similar as possible to the temporal and spatial layout of our global motion task—We used a screen with 300 Gaussian blobs and same display settings as described in “Stimuli” section. All observers reported that they could find a setting where the flicker almost completely disappeared. Each observer repeated this selection five times reliably, and the average was taken.
A second factor that may influence performance in a motion task is chromatic aberration. Chromatic aberration can result in luminance artifacts, which then could provide an effective input to motion processing or could act as a cue for segmentation. These luminance artifacts covary with S-cone modulation. To minimize the effect of chromatic aberration, the RDKs in our study consisted of Gaussian blobs instead of uniform discs, hence removing high–spatial frequency information. Furthermore, in the baseline experiment (discussed in the next section), we confirm that the colors (C1, defined by S-cone increment, and C2, S-cone decrement) used in the main experiment did not produce significantly different performances when used in single RDKs. This makes an effect of chromatic aberration unlikely in the present experiment, which is also consistent with previous findings using S-cone isolating stimuli in a motion extraction task under comparable spatial and temporal parameters (Wuerger & Landy, Reference Wuerger and Landy1993).
Baseline experiment
The choice of our stimulus colors (C1 and C2; Fig. 1) is based on the results of Ruppertsberg et al. (Reference Ruppertsberg, Wuerger and Bertamini2003) who showed that for the given spatial and temporal parameters, S-cone modulations did not provide an effective input to global motion processing and the performance in the global motion task was mediated by the projections onto the L-M axis (i.e., there is no significant difference in contrast thresholds for colors with the same L-M component). We therefore first replicated a subset of experiments by Ruppertsberg et al. (Reference Ruppertsberg, Wuerger and Bertamini2003) with a new set of observers, using the same spatial and temporal parameters as in the original study.
We used single-color RDKs to establish how effective different color modulations are when viewed in isolation. To determine chromatic contrast thresholds for global motion discrimination for each of the two color directions (modulations from gray background to C1 or C2; Fig. 1a), we used a two-interval forced-choice (2IFC) design in which participants had to decide whether the first or the second interval contained coherent motion irrespective of its direction. RDKs contained 300 blobs of a single color; both C1 and C2 contrasts were controlled by their own interleaved staircases. In one interval, the direction of coherent motion was either leftward or rightward; in the other interval, all blobs moved randomly. A trial started with a fixation cross for 500 ms, presented in the center of the screen, then the first motion interval was displayed for 350 ms, followed by another fixation cross for 500 ms, and finally the second motion interval appeared for 350 ms. Participants were required to respond after the second motion interval by pressing a button to indicate the interval in which they thought coherent motion had appeared. The next trial started after the participant had responded. Participants were instructed to give a correct answer, not a fast answer, and they were provided with acoustic feedback. The participant’s responses guided an adaptive adjustment of the signal contrast (Quest procedure; Watson & Pelli, Reference Watson and Pelli1983). To estimate the color contrast threshold from the relative frequency of a correct response, we fitted a Weibull function to obtain the threshold contrast (defined as the 75% correct point on the psychometric function). Every threshold was measured five times for each participant. The means of these measurements are shown in Fig. 3 in a polar plot corresponding to the colors depicted in Fig. 1a. A radius of 1 indicates a cone contrast of 0.13 along the L-M axis, and a contrast of 0.7 along the S-cone axis (for details, see “General Methods” section, Color space). Contrast thresholds in color directions C1 and C2, defined by hue angles of 45 and 315 deg, respectively, were approximately symmetric around the L-M axis. Using a paired t-test, we confirmed that there were no significant differences in color contrast thresholds, C1: 1.05 ± 0.05, C2: 1.14 ± 0.07, t(11) = −1.84, n.s. Modulations along the S-cone axis only (90 and 270 deg) did not yield above-chance motion discrimination performance; that is, we could not generate an S-cone contrast large enough to yield reliable motion discrimination performance. The measured contrast thresholds shown in Fig. 3 show that colors C1 and C2 are approximately equally effective in our motion discrimination task. These measurements replicate published results by Ruppertsberg et al. (Reference Ruppertsberg, Wuerger and Bertamini2003; Fig. 2). Our baseline experiment therefore confirms that only the L-M component in colors C1 and C2 provides a useful input to a motion mechanism.
(Color online) Results from the baseline experiment using single-color RDKs. The diamonds indicate the contrast at which observers can reliably discriminate coherent (50%) from random motion. A radius of 1 corresponds to a cone contrast of 0.13 along the L-M axis and to a cone contrast of 0.7 along the S-cone axis (for details, see “General Methods” section and Fig. 1). Dotted lines indicate the average L-M coordinate. Two color directions (C1: 45 deg and C2: 315 deg) were tested. Observer names are indicated next to the data through a four-letter code (three letters for initials and last letter for gender).
The purpose of the baseline experiment was to select isoluminant chromatic stimuli, modulated simultaneously along the L-M- and the S-cone axis (C1 and C2 in Fig. 1) such that (1) the S-cone component in isolation cannot be used for motion discrimination and (2) modulations between the gray background and the particular color contrast (C1 and C2) are equally effective in our global motion task. In subsequent experiments, these contrast thresholds (Fig. 3) are used to generate the two-color RDKs for each of the participants.
Experiment 1: Correlating color and motion information—psychophysics
The purpose of this experiment was to evaluate whether observers can use the correlation between the color and the direction of local motion signals in a global motion task. In other words, we tested whether the color of the local motion signals can act as a cue to segment the stochastic motion stimulus into coherent motion and motion noise and thereby improve performance.
Participants
The study included 12 participants, all of whom had normal or corrected-to-normal vision. We tested participants’ color vision with the Cambridge Colour Test (Cambridge Research System, Ltd.). The participants received a small honorarium to compensate for their time. Each participant performed four 1-h sessions, following the 1.5-h session that consisted of control measurements (color test and HCFP), the baseline experiment, and several practice blocks to prepare them for the main experiment (criterion: d′ > 1). Individual written informed consent was obtained, and the study conformed to the ethical guidelines of the University of Liverpool.
Stimuli and procedure
Participants were presented with two-color RDKs that contained 150 magenta (C1; cf. Fig. 1) and 150 orange (C2) blobs. Performance was measured for the following three conditions:
(1) The “uncorrelated” condition: Direction of each local motion signal is assigned randomly. Fifty percent of the signal blobs (i.e., coherent motion) are of color C1, and the other half is of color C2; similarly for the noise blobs. Thus, there is no information about the motion direction associated with a particular color.
(2) The “correlated, but uncued” condition: Motion and color are correlated; all blobs that move coherently (“signal blobs”) are of color C1 (C2); all blobs that move randomly (“noise blobs”) are of color C2 (C1). The observer has no prior knowledge which color is the signal color (cf. Fig. 1b).
(3) The “correlated cued” condition: Motion and color are correlated, and the observer is cued to the signal color.
Conditions were run blocked, with each block containing 90 trials (half of them coherent and the other half random motion). The task of the observer was to determine whether the stimulus contained random or coherent motion (always 50% coherence; Fig. 2a) and respond by pressing an assigned button on the response box. We used a coherent–random global motion judgment instead of a forced-choice paradigm (2IFC, as in the baseline experiment) in order to facilitate comparisons with the EEG study (Experiment 2) and with study by Li and Kingdom (Reference Li and Kingdom2001), which also used a single-trial motion task and examined at measures of accuracy instead of changes of threshold.
Depending on the condition, they were asked to selectively attend to both colors (magenta: C1 and orange: C2) or solely to magenta (C1) or orange (C2). Each trial started with a fixation target. After a 100-ms period of fixation, the color cue was presented for 500 ms; the cue was followed by a variable period of fixation (750–950 ms), after which the participants observed a single 350-ms interval of global motion (for trial outlook, see Fig. 2b). Participants were instructed to fixate the cross and to avoid any eye movements during the motion interval. The color cue consisted of a static image containing a random field of 300 blobs, depicted in the to-be-attended color(s); it was presented in every trial in order to remind the participants of the to-be-attended color(s). In the “uncorrelated motion” and the “uncued correlated motion” blocks, the cue was half orange (C2) and half magenta (C1) and thus not informative about the signal color; in “cued correlated motion” blocks, the trials were preceded by a fully magenta or a fully orange cue (depending on whether C1 or C2 was to be cued in that particular block) and thus was informative about the signal color.
The first 10 trials of each block were considered to be practice and discarded from the final analysis. The participants performed a total of three blocks per condition, producing 240 trials each for the final analysis. The order of the blocks was different for each participant; it was balanced across the sample, so that each condition occupied the same position an equal number of times.
Behavioral data analysis
We computed d′ for each of the three different conditions by pooling the responses from different blocks and then averaging across colors. To determine the effects of color–motion correlations, we analyzed the d′ values with a repeated measures analysis of variance comprising the factor of color–motion correlation with three levels (uncorrelated, “uncued correlated,” and “cued correlated”).
The task in Experiment 1 required a judgment of motion within a single interval, which is somewhat different from the 2IFC procedure that was used to equate the chromatic contrasts of C1 and C2 in the baseline experiment. Therefore, we also conducted an additional control condition, wherein the participants performed the single-interval judgment on single-color RDKs depicted fully in C1 or C2 (80 trials; 40 for C1 and 40 for C2). If the performance in this control experiment is not different for the two colors C1 and C2, this verifies that the two colors are approximately equally effective in the global motion task. This was tested by contrasting the d′s for C1 and C2 against each other as well as against the uncorrelated condition using paired t-tests. Means and s.e. are reported throughout. Planned post hoc tests were performed using paired t-tests.
Results
Fig. 4a shows performance (d′; Macmillan & Creelman, Reference Macmillan and Creelman1991) for the three experimental conditions (uncorrelated, “correlated-uncued,” and “correlated cued”) and also for the single-color RDK (“control”) averaged over all observers. The data for the two-color RDK are shown with bipartite symbols; the d′ for the single-color RDKs (CONTROL) is indicated by the filled symbol. Comparing the d′ for the three main conditions shows that correlating the color and the local motion signals has an effect on global motion performance, F(2, 22) = 6.48; P < 0.01. When there was a correlation between color and motion and the observer was cued to the signal color (i.e., the color that is associated with 100% coherent motion; CORR-C), then the improvement in performance compared to the uncorrelated case (UNCORR) was highly significant, cued d′ = 1.75 ± 0.14; t(11) = −4.29; P < 0.001. There was a strong trend for improvement in performance when color and motion of the local signals were correlated but the observer was not cued to the signal color, uncorrelated d′: 1.49 ± 0.13; uncued correlated d′: 1.63 ± 0.11; t(11) = −2.07, P = 0.06. The difference between uncued and cued correlated conditions itself was not significant, t(11) = −1.37, n.s. There were no significant differences in performance for single-color RDKs, C1: d′ = 1.49 ± 0.11, C2: d′ = 1.30 ± 0.12, t(10) = 1.51, n.s., and the difference between single-color RDKs (filled symbol) and uncorrelated two-color RDKs (bipartite symbol; “uncorr” condition) was also not significant.
Behavioral findings from Experiment 1. (a) Motion discrimination sensitivity (d′) showed a significant difference between correlated uncued (CORR-U) and correlated cued (CORR-C) conditions. (b) Data are replotted as differences in d′. The effect of color–motion correlation is demonstrated by subtracting uncorrelated from correlated uncued; the effect of cueing is shown by subtracting correlated uncued from correlated cued. Mean values are plotted with 2 s.e.
Fig. 4b summarizes the effects of color–motion correlation that occur automatically or through cueing by replotting the data from Fig. 4a and showing the difference in performance (d′) due to a correlation between color and motion only [“CORRELATION”; diff d′ = d′(correlated, but uncued) − d′(uncorrelated)] or due to cueing of the signal color [“CUEING”; diff d′ = d′(correlated and cued) − d′(correlated, but uncued)] or due to correlation and cueing [“CORR + CUEING”; diff d′ = d′(correlated and cued) − d′(uncorrelated)]. On their own, correlation and cueing result in a performance improvement of about 0.1–0.15 d′. The combined effect of cueing and correlation is shown in the last column and is about 0.25 d′ difference and highly significant.
In summary, the behavioral experiment showed that observers can use an S-cone difference to segment motion noise and coherent motion; this grouping by color (defined by an S-cone difference) is highly significant when the observer is cued to the signal color.
Experiment 2: Neural markers of color cueing revealed with EEG
The purpose of this experiment was to determine the neural markers of the cueing effect demonstrated in Experiment 1. We therefore used similar stimuli as in the psychophysical experiment (Experiment 1) while recording EEG for uncued and cued RDKs.
Participants
Eight participants were recruited; four of them had previously participated in Experiment 1. They had normal or corrected-to-normal vision. We tested participants’ color vision with the Cambridge Colour Test (Cambridge Research System, Ltd.). The participants received a small honorarium to compensate for their time. The experiment consisted of one initial hour and a half session (color test, HCFP, and baseline experiment) and a 1-h EEG experiment. Individual written informed consent was obtained, and the study conformed to the ethical guidelines of the University of Liverpool.
Stimuli and procedure
We aimed to keep the experimental stimuli and procedure as similar to Experiment 1 as possible. However, due to the necessities of adapting our psychophysical approach to EEG methods, several significant changes had to be introduced.
First, the chromatic contrast levels for each participant were not fixed at threshold but scaled from C1 and C2 thresholds to a mean radius of 1.4 (see “General Methods” section, Color space) according to the following formula: (C1_scaled = 1.4 × C1_threshold/((C1_threshold + C2_threshold)/2)), similarly for color C2. This was done in order to increase the amount of measurable electrical activity because the amplitude of ERPs to isoluminant colored stimuli is contrast dependent (Porciatti & Sartucci, Reference Porciatti and Sartucci1999). The value of 1.4 was selected as it was clearly above threshold but still within the gamut of our monitor. While above threshold chromatic contrasts may lead to ceiling performance in the behavioral task, attentional effects on the early ERP components are likely to persist (Valdes-Sosa et al., Reference Valdes-Sosa, Bobes, Rodriguez and Pinilla1998), which validates our approach.
Second, we changed the task so that instead of judging whether motion was coherent or random, participants had to discriminate the direction (up or down, left or right) of coherent motion and indicate it by pressing the appropriate button on the response box. This was again done in order to increase the quality of the signal; the waveforms elicited by random and coherent motion cannot be averaged together, so that would lead to an effective loss of one half of recorded trials. With a motion direction task, all correctly answered trials could be taken into analysis, which greatly enhances our signal-to-noise ratio.
Third, we did not include the uncorrelated motion condition and focused instead on differences between uncued and cued correlated. This was done due to time constraints; with our EEG setup, a recording of more than 1 h is not viable. As the physical stimuli are identical in uncued and cued conditions, any differences between them would be clearly interpretable as arising due to cueing and not to a difference in the stimulus itself. Furthermore, as previously mentioned, attentional effects are manifest in early ERP components even if there is no difference in performance (e.g., Valdes-Sosa et al., Reference Valdes-Sosa, Bobes, Rodriguez and Pinilla1998).
And finally, in this experiment, we asked for a speeded and accurate response, as EEG experiments usually rely on time-limited trials. In Experiment 1, accuracy was demanded as this is optimal when d′s are computed. However, in EEG, it is necessary to speed up the experiment as it needs to finish within a single recording; also, one must avoid long periods of visual inactivity, which can lead to high levels of alpha activity in the EEG. Variable levels of alpha activity before trials would introduce a large amount of noise to early visual ERPs (for an overview, see Becker et al., Reference Becker, Ritter and Villringer2008).
Participants were instructed to minimize eye movements and blinking during the display of a stimulus or the fixation cross and to try and remain relaxed and refrain from body or head movements throughout the experiment. At the end of each trial, an “X” would replace the fixation cross for 900 ms—Participants were instructed that they are allowed to blink during the presentation of the X.
Behavioral data analysis
We measured performance as percentage of correct responses and reaction times (RTs) for data collapsed across color. To determine the effects of color cueing, we compared accuracies and RTs from correlated uncued and correlated cued motion with paired t-tests. Accuracies were calculated instead of d′ as a more appropriate measure since data were based on four alternative responses.
EEG data acquisition and analysis
Continuous EEG was recorded with an EGI (Electrical Geodesics, Inc., Eugene, OR) 129-electrode array. Vertical and horizontal eye movements were monitored with a subset of the electrodes. Electrode impedances were kept below 50 kΩ as recommended by the system manufacturer, and all scalp electrodes were referenced to the vertex during the recording. All signals were bandpass filtered online from 0.1 to 100 Hz and digitized at 250 Hz. EEG was segmented into epochs starting 500 ms prior and lasting 700 ms following stimulus onset. EEG data processing was performed using the EEGlab toolbox (Delorme & Makeig, Reference Delorme and Makeig2004) combined with self-written procedures running under Matlab (The Mathworks, Inc., Natick, MA). Artifact removal was performed using an established procedure [statistical correction of artifacts in dense array studies (SCADS); Junghoefer et al., Reference Junghoefer, Elbert, Tucker and Braun2000]. When artifact rejection was completed, the average rejection rate was 32.8%, resulting in an average of approximately 44 remaining trials per condition for each subject. Further analyses were performed using the average reference.
We also assessed the number of trials with eye movements prior to artifact rejection as eye movements are known to play a significant role in motion perception. We used a criterion of 50-μV voltage deflection on either vertical electrooculograms (VEOGs) or horizontal electrooculograms (HEOGs), which is an accepted procedure (for a review, see Croft & Barry, Reference Croft and Barry2000). The computations were performed on subtracted signal recorded above and below the eyes (VEOGs) or at the left and right outer canthi of the eyes (HEOGs) between 200 ms before and 350 ms after motion onset.
A 25-Hz low-pass filter was applied to the data before all ERP analyses. We assessed the N1 component of the ERP. Regional means (shown in Fig. 5; see “Results” section) were assigned based on which electrodes exhibited maximal activity when data were collapsed across conditions. Average amplitudes across the electrodes at these sites in the time window of maximal activity (170–290 ms) were then computed, and the mean amplitude during the 100-ms period prior to stimulus onset (baseline) was subtracted (these are all common procedures in the analyses of ERPs; Handy, Reference Handy2005). Mean latencies were not analyzed as there were no hypotheses concerning them. Differences in amplitude between uncued and cued correlated motion were analyzed with a paired t-test.
(Color online) EEG findings from Experiment 2. (a) Scalp topography of N1 from grand mean data averaged across all conditions. Boxes indicate electrode sites included in the regional mean. (b) Grand mean baseline-corrected ERP waveforms at the regional mean. Shaded area indicates the N1 component. (c) Source localization of the N1 (170–290 ms): Statistical parametric maps of the inverse solution component based on a Hotelling T 2-test of the uncued (CORR-U) and cued (CORR-C) conditions, collapsed across color. Montreal Neurological Institute (MNI) coordinates of the center of gravity in the occipital lobe are x = 49, y = −69, and z = −1 (MT area) in the cued condition. The Y-coordinate represents the location of the coronal slice in MNI space. Significant voxels are depicted in black.
To reconstruct the estimated sources of the observed effects the VARETA (Variable Resolution Electromagnetic Tomography) method was used (for a detailed description, see Bosch-Bayard et al., Reference Bosch-Bayard, Valdes-Sosa, Virues-Alba, Aubert-Vazquez, John, Harmony, Riera-Diaz and Trujillo-Barreto2001). In brief, VARETA estimates the spatially smoothest generator estimates compatible with the observed scalp topographies and places anatomical constraints on the allowable solutions. The generators of the EEG data inside the brain are mapped using a three-dimensional grid of points (or voxels) that represent possible sources of the signal and for which the probability for gray matter is different from 0 (based on the average probabilistic brain atlas produced by the Montreal Neurological Institute; Evans et al., Reference Evans, Collins, Mills, Brown, Kelly and Peters1993). In order to reveal possible differences in activation due to the signal color being cued, sources were analyzed for both cued and uncued conditions. Statistical parametric maps were constructed based on voxel by voxel Hotelling T 2-tests against zero. Activation threshold was set at a significance level of P < 0.01; correction to account for spatial dependencies between voxels was calculated by means of Random Field Theory (Worsley et al., Reference Worsley, Marrett, Neelin and Evans1996).
Results
The purpose of the EEG experiment was to identify the neural marker of cued color–motion correlations. In the EEG measurements, we observed a prominent N1 component at central parietooccipital sites (Fig. 5a). This posterior N1 showed an effect of color cueing, t(7) = 3.32, P < 0.05, being enhanced when the signal color had been cued (Fig. 5b). Cued and uncued conditions showed differing levels of activation in a broad area of extrastriate visual cortex, encompassing the lateral occipitotemporal, lingual, and superior occipital gyrus, with cued colors also causing significant levels of activity in middle temporal (MT) area (Fig. 5c; BET Viewer, Neuronic, S.A., Cuba).
During the EEG experiment, we also collected behavioral data (performance and RTs). Performance (percent correct) in the cued condition is slightly better than in the uncued condition, uncued: 80.5% correct with a s.e. of 3.7%; cued: 82.8% correct with a s.e. of 3.7%; t(7) = −0.99, n.s. Similarly, RTs are slightly faster in the cued condition, but this difference is again not significant, uncued: RT = 0.82 s with a s.e. of 0.028 s; cued: RT = 0.80 s with a s.e. of 0.032 s; t(7) = 0.58, n.s. Since the EEG experiment was not run at threshold but at suprathreshold contrast levels, we did not necessarily expect significant behavioral results due to ceiling effects (Valdes-Sosa et al., Reference Valdes-Sosa, Bobes, Rodriguez and Pinilla1998). Moreover, significant differences between cued and uncued correlated motion were not obtained in Experiment 1 either.
Eye movement analysis was conducted in six of eight participants, as in the remaining two participants, SCADS had identified excessive artifacts in one of the HEOG channels. Eye movements had occurred on a very small number of trials for each of the six participants whose data were analyzed: 0.1, 0.2, 0.2, 0.3, 0.3, and 0.23% for uncued motion, and 0.1, 0.3, 0.3, 0.5, 0.12 and 0.13% for cued motion. There was no significant difference between conditions, t(5) = 0.32, n.s. Thus, participants generally followed the instructions to maintain central fixation and avoid eye movements during the motion interval.
Discussion
We studied whether the human visual system uses correlations between color and motion to segregate coherent motion from motion noise. The novelty of our approach is that we dissociated the information available to the motion system from the information available to a color processing module using coherent motion and motion noise that differed from each other only in their S-cone component. Any segregation of coherent motion from motion noise must therefore rely on the S-cone difference, and any additional facilitation can only act on this S-cone–driven color difference. Furthermore, since the S-cone signal is invisible to the motion system under the parameters we have used, any improvement due to color–motion correlation cannot be due to sensory gain in the motion signal and has to operate through grouping by color.
Our main result is that color–motion correlations can improve performance, in particular if cued. This improvement can only be explained by grouping the local motion signals according to their color and thereby aiding segmentation of motion signal and motion noise. Such facilitation requires that the S-cone–driven koniocellular pathway can affect the processing of local motion information before it is analyzed in MT. Time courses of color and motion processing support this possibility: Color is analyzed prior to the processing of local motion signals (Moutoussis & Zeki, Reference Moutoussis and Zeki1997). Our EEG experiment focused on differences between cued and uncued correlated motion, which did not emerge in the behavioral experiment. Enhancements for cued color–motion correlations were found at approximately 240 ms, in the time range of the early visual component N1. N1 reflects the discrimination process within attentional focus, and enhancements have been previously reported for spatial-based attention (Vogel & Luck, Reference Vogel and Luck2000). We now report enhancements that may be due to nonspatial attention.
Grouping of local motion signals based on S-cone information
We found a small and automatic improvement in performance (about 0.15 d′) when color and motion direction of local motion signals were correlated (Fig. 4). This is consistent with evidence that color and local motion are bound automatically and at an early stage of processing (Blaser et al., Reference Blaser, Papathomas and Vidnyanszky2005). The small improvement for correlated RDKs (Fig. 4a) would fit within this approach, which supposes joint processing of color and motion if they form a surface. Surfaces have the properties of an emergent object; it is considered that such emergent objects can capture attention (Wolfe, Reference Wolfe and Pashler1998). Thus, it is conceivable that observers rapidly extract some information about the signal color (i.e., the color of all coherently moving blobs), which then acts like a cue and facilitates subsequent motion processing (within the same trial) by grouping the local motion signals carrying the signal color. On the contrary, it is not likely that information about motion coherence would be extracted prior to identifying the blobs’ common color and used to drive the color–motion binding process, as isoluminant motion stimuli are not available preattentively and thus cannot capture attention (Lüschow & Nothdurft, Reference Lüschow and Nothdurft1993). However, the binding of local motion signals possessing the signal color is not efficient, as the surface is emerging in the unfavorable context of other colocalized randomly moving blobs. Whether coherent local motion signals are bound into one global surface depends strongly on the spatial configuration of the local signals (Lorenceau & Alais, Reference Lorenceau and Alais2001); the random distribution of blobs has a negative effect on the formation of local borders (Wolfe, Reference Wolfe and Pashler1998) and thus hinders the emergence of the surface. This could explain why the observed increases were small.
Li and Kingdom (Reference Li and Kingdom2001) failed to show a significant effect of color–motion correlation for red–green RDKs without cueing. In their study, a strong luminance signal was available for motion processing, and coherent motion and motion noise were associated with different colors (red and green). The fact that we find a trend for an improvement through color–motion correlation may be due to very different experimental conditions; we investigated whether S-cone signals can modulate global motion processing, whereas Li and Kingdom (Reference Li and Kingdom2001) used RDKs with strong luminance signals and additional chromatic information. In our experiment, no luminance information was provided, and the motion information was carried exclusively by a chromatic (L-M) signal. It is conceivable that the relative effect of grouping based on color carries larger weight when the motion signal is poor. This is supported by the findings of Thiele et al. (Reference Thiele, Dobkins and Albright1999) that chromatic contrast can influence direction responses in area MT only when luminance contrast of the motion stimulus is very low (smaller than 5–8% contrast). Otherwise, the response is dominated by the luminance component.
In accordance with Li and Kingdom (Reference Li and Kingdom2001), we found that cueing the signal color leads to significantly improved performance (Fig. 4). Li and Kingdom (Reference Li and Kingdom2001) remarked that the enhancements introduced by endogenous attentional focusing in a global motion task were smaller and less consistent than those brought about by color-driven pop-out in the study by Croner and Albright (Reference Croner and Albright1997). In the study by Li and Kingdom (Reference Li and Kingdom2001), only two of a total of five observers exhibited significant, albeit small, increases in performance when color-based attentional selection was used. Unlike Li and Kingdom (Reference Li and Kingdom2001), in our behavioral experiment, we used color contrasts at threshold for each observer (determined in the baseline experiment; Fig. 3) in our much larger sample. In doing so, we discovered that although the effect of cueing is indeed small, it is highly significant, which clearly demonstrates that such improvements do occur in chromatic global motion integration. Indeed, our finding parallels the results of Khoe et al. (Reference Khoe, Mitchell, Reynolds and Hillyard2005) who obtained small effects of surface-based exogenous attention in a paradigm that relied on two surfaces of moving dots. An improvement of 0.68 of d′ was observed when surfaces were segregated by two cues (color and direction of rotary motion) but an improvement of only 0.23 of d′ occurred when surfaces were segregated through a single cue.
Therefore, our findings seem to be in accordance with a large body of existing literature and provide further support for the assertion by Li and Kingdom (Reference Li and Kingdom2001) that color–motion correlations are effective because of attentional involvement. However, caution also needs to be taken when interpreting our results along these lines. The slight improvement in performance brought about by uncued color–motion correlations could have indeed been due to the ability of uniformly colored surfaces to capture attention, as discussed above. Still, exogenous attention was not tested directly so its involvement cannot be definitely asserted. In addition, although cueing the signal color in a color–motion correlated RDK did produce an improvement in performance in relation to an uncorrelated display (Experiment 1), no behavioral difference was observed between cued and uncued RDKs in either of the two experiments. This requires caution when interpreting the improvement in discrimination to be due to endogenous attention.
The neural processing of cued color–motion correlations
EEG allows us to look at subtler modulations that may occur between the cued and the uncued correlated motion. Modulations of EEG responses indicative of attention may be found even when performance improvements are lacking (e.g., Valdes-Sosa et al., Reference Valdes-Sosa, Bobes, Rodriguez and Pinilla1998; Müller et al., Reference Müller, Andersen, Trujillo, Valdes-Sosa, Malinowski and Hillyard2006). In our global motion direction task, we identified a difference between cued and uncued correlated motion at approximately 240 ms, at the level of the early visual N1 component. This is the name given to the first negative component of the ERP waveform evoked by a visual stimulus. The information-processing correlates of this component are not yet fully understood, although it is generally considered to reflect a visual discrimination process and its modulation by attention. The visual discrimination may involve color, form, or motion, demonstrating that N1 effects are generally similar for both ventral and dorsal stream information (for an overview, see Vogel & Luck, Reference Vogel and Luck2000). Using a stimulus display of two differently colored rotary surfaces, a previous study has found an enhancement of N1 amplitude through exogenous surface–based attention (Khoe et al., Reference Khoe, Mitchell, Reynolds and Hillyard2005). This enhancement occurred at around 200 ms and showed a topography that was distributed with left and right symmetrical occipital foci. Khoe et al. (Reference Khoe, Mitchell, Reynolds and Hillyard2005) remarked that this cueing effect on the N1 was similar to that reported by the Valdes-Sosa group (Valdes-Sosa et al., Reference Valdes-Sosa, Bobes, Rodriguez, Acosta, Perez, Iglesias, Borrego, Kanwisher and Duncan2004; Rodriguez & Valdes-Sosa, Reference Rodriguez and Valdes-Sosa2006) and localized to a ventral–lateral occipital source in the general region of areas MT/middle superior temporal area (MST) and V4. Our N1 occurred somewhat later, at 240 ms, and exhibited a midline occipital topography (Fig. 5a). The differences in the topography are slight and may originate in the fact that we used a fully isoluminant stimulus display, with sole stimulus modulations exciting the S-cones and thus being carried along the koniocellular pathway. Indeed, our ERP waveforms are consistent with those for S-cone isolating stimuli, obtained in the normative study of Porciatti and Sartucci (Reference Porciatti and Sartucci1999). Source localization has indicated that activity in our experiment was generated by widespread sources, including a range of visual areas (e.g., lateral occipital gyrus, lingual gyrus, superior occipital gyrus). After thresholding the activations, it was revealed that an important difference arose in area MT, which was significantly active when the signal color was cued (Fig. 5c). We conclude that the N1 effect may reflect a more effective discrimination process possibly due to endogenous attentional modulation. This process distinguishes itself from automatic color–motion correlation processing by a significant involvement of the high-level visual area MT/MST, which indicates that it must occur after the binding of color and local motion.
A contribution of eye movements to the observed effects has to be considered as well, as dot lifetime was unlimited and trial duration of 350 ms allowed a possibility of tracking eye movements being initiated. Smooth pursuit eye movements play a very significant role in motion perception; motion direction judgments and pursuit eye movements correlate with each other and both have a neural substrate in area MT (for a review, see Spering & Gegenfurtner, Reference Spering and Gegenfurtner2008). Such eye movements are initiated within 80–150 ms in humans, although their latency is delayed by 50 ms when isoluminant stimuli are used (Braun et al., Reference Braun, Mennie, Rasche, Schutz, Hawken and Gegenfurtner2008). While the early part (0–40 ms) of smooth pursuit eye movements seems independent of stimulus features and reliant on luminance signals, the late component that follows a catch-up saccade is dependent on the properties of the stimulus (Tychsen & Lisberger, Reference Tychsen and Lisberger1986; Wilmer & Nakayama, Reference Wilmer and Nakayama2007). Can a modulation of smooth pursuit eye movements be responsible for the motion–color correlation effects obtained in this study? As our motion stimulus differed only along the S-cone dimension, it is unlikely that our findings are influenced by any of the early presaccadic components of pursuit eye movements that are driven by luminance changes. In order to assess the possibility of postsaccadic influences, we performed additional analyses on electrooculogram (EOG) signals collected in Experiment 2, using more stringent criteria for detecting eye movements. We found that large eye movements were present on a very few number of trials (above 10% overall only for one of the participants), indicating that the instructions to fixate were followed successfully. Most importantly, there were no differences in eye movements between cued and uncued motion. One must consider that EOGs are not an ideal tool in assessing eye movements unless they are calibrated for each participant (Croft & Barry, Reference Croft and Barry2000). Still, the robustness of our findings between conditions indicates that cueing of S-cone–defined motion does not automatically induce prominent postsaccadic smooth pursuit eye movements in participants who are instructed to fixate during a brief motion interval.
The role of S-cone signals in global motion processing
Whereas the role of the parvocellular pathway in motion processing has been demonstrated by numerous physiological and behavioral studies (for a review, see Cropper & Wuerger, Reference Cropper and Wuerger2005), the role of the S-cone–driven koniocellular pathway for global motion processing is less clear. Recent neuroanatomical results are intriguing since they show direct projections from the nucleus geniculatus lateralis dorsalis (LGN) to extrastriate areas (MT/MST) that are traditionally considered as motion processing areas (Morand et al., Reference Morand, Thut, Grave de Peralta, Clarke, Khateb, Landis and Michel2000; Sincich et al., Reference Sincich, Park, Wohlgemuth and Horton2004; Sincich & Horton, Reference Sincich and Horton2005); interestingly, these projections arise from the (S-cone driven) koniocellular layers of LGN. There is plenty of evidence that it is possible to extract global motion from S-cone isolating stimuli (Ruppertsberg et al., Reference Ruppertsberg, Wuerger and Bertamini2007; Michna & Mullen, Reference Michna and Mullen2008). At the same time, behavioral data show that S-cone isolating stimuli are very poor at mediating effective motion extraction (Ruppertsberg et al., Reference Ruppertsberg, Wuerger and Bertamini2003) and S-cones support global motion perception only at low spatial frequencies (Ruppertsberg et al., Reference Ruppertsberg, Wuerger and Bertamini2007). These behavioral data in conjunction with imaging and neurophysiological data (Gegenfurtner & Hawken, Reference Gegenfurtner and Hawken1996; Wandell et al., Reference Wandell, Poirson, Newsome, Baseler, Boynton, Huk, Gandhi and Sharpe1999; Gegenfurtner, Reference Gegenfurtner2001) suggest that the role of the koniocellular pathway for low-level motion processing is probably not very significant.
In our behavioral experiment, we eliminated any residual direct input of the S-cone system to global motion processing, and we demonstrate a processing advantage when observers are cued to the S-cone–driven signal color (i.e., the color carrying coherent motion). Since this effect of color–motion correlations cannot be explained by a gain in the local motion signals, it must be reliant on a koniocellular feedback into or before the motion analysis systems, once color and local motion have already been bound into a single surface. The EEG experiment suggests that the effect of cueing is at least partly localized in area MT, a part of the brain whose neurons have relatively large receptive fields (5 deg at the fovea) and respond to planar global motion (Movshon et al., Reference Movshon, Adelson, Gizzi and Newsome1986). Our results therefore suggest that the role of the koniocellular pathway in motion processing may not lie in providing a direct input but in selecting the information to be used for further analysis based on the parvo- or magnocellular input.
The importance of this study lies in its finding that the koniocellular pathway provides a significant input to global motion analysis via grouping of local motion signals. We found both an automatic improvement in performance brought about by color–motion correlations and a further benefit due to cueing of the signal color. The first type of improvement could have occurred due to an exogenous attentional process, wherein a salient or new stimulus captures attention involuntarily; the second type of improvement could be explained by an additional effect of endogenous attention. Both these attentional effects were previously suggested by Li and Kingdom (Reference Li and Kingdom2001) in their attempt to explain the processing of color–motion correlations, though further studies are needed if this attentional involvement is to be demonstrated unequivocally. At this stage, we can conclude that the koniocellular pathway plays an important role in high-level motion analysis by guiding grouping by color and that this role may be linked to the functioning of both exogenous and endogenous attentional processes.
Acknowledgments
This research was funded by a Wellcome Trust Project grant (WT080205) awarded to S.M.W. The Matlab colour functions incorporated in many routines were written by Alexa Ruppertsberg and Sue Harding. We thank Thomas Gruber and Neil Harrison for help with the EEG analysis. Thanks also to Thomas Wooley for assistance with behavioral data collection and Nicola Williams for assistance with the EEG recordings. We also thank Frederick A.A. Kingdom for valuable comments on an earlier draft of the manuscript.
