Introduction
In life, we engage in different activities more or less regularly, for example, shopping for groceries, riding on a bus or eating out in a restaurant. These activities are composed of single events that proceed in a very typical temporal order. Hence, they can be considered event sequences that are represented in long-term memory in script structures (Schank & Abelson, Reference Schank and Abelson1977). Upon accessing the script, the events are activated in their temporal order (cf., Barsalou, Reference Barsalou2008; Collins & Loftus, Reference Collins and Loftus1975), which allows us to predict which event will take place next. The detection of errors in the event sequence, which may occur for example due to temporal violations, is a prerequisite to alter one's behavior and adapt to the new situation (Bar, Reference Bar2007; Zacks, Speer, Swallow, Braver, & Reynolds, Reference Zacks, Speer, Swallow, Braver and Reynolds2007). Therefore, it is vital to efficiently take into account available information that might facilitate error detection. Event properties, such as events’ temporal positions within an event sequence, are crucial sources of information. In the present study, we extend the current literature by combining behavioral and pupillometric analyses with eye movement recordings. Eye movements are suitable for analyzing information acquisition strategies and enable inferences about the processes of accessing temporal event information.
Just as objects are stored with their properties like size, color or weight, events of event sequences may be stored with their properties like their temporal position and order within a sequence. This “temporal property” codes the temporal order of events and whether a specific event occurs early or late during a given event sequence (Barsalou & Sewell, Reference Barsalou and Sewell1985; Nottenburg & Shoben, Reference Nottenburg and Shoben1980). In the event sequence “eating out in a restaurant”, the event enter the restaurant may be labeled as “early” indicating that it is from the beginning of the sequence, whereas the event pay the bill may be labeled as “late” since it is from the end. The labels that are included in the representation of the events as discrete codes (Kosslyn, Reference Kosslyn1980) have been shown to support conceptual processing (Nottenburg & Shoben, Reference Nottenburg and Shoben1980; Pohl, Reference Pohl1990). Therefore, it is feasible that error detection could also benefit from accessing these codes.
While we could not find direct support for this assumption in a previous study, we, nevertheless, showed that early and late events led to processing differences regarding cognitive resource consumption (Raisig, Hagendorf, & van der Meer Reference Raisig, Hagendorf and van der Meer2011). Resource consumption was measured with the pupillary response (Beatty, Reference Beatty1982; Beatty & Lucero-Wagoner, Reference Beatty and Lucero-Wagoner2000; Just, Carpenter, & Miyake, Reference Just, Carpenter and Miyake2003). Its validity has been shown in a variety of tasks, for example in analogical reasoning tasks (van der Meer et al., Reference van der Meer, Beyer, Horn, Foth, Bornemann, Ries and Wartenburger2010), event sequencing tasks (Raisig, Welke, Hagendorf, & van der Meer, Reference Raisig, Welke, Hagendorf and van der Meer2007, Reference Raisig, Welke, Hagendorf and van der Meer2009), or the Stroop task (Brown et al., Reference Brown, Kindermann, Siegle, Granholm, Wong and Buxton1999; Siegle, Ichikawa, & Steinhauer, Reference Siegle, Ichikawa and Steinhauer2008). Crucially, the degree of pupil dilation reflects task difficulty: the pupil increases more in difficult tasks reflecting an increase of cognitive resource consumption that is required to solve the task (Landgraf, van der Meer, & Krueger, Reference Landgraf, van der Meer and Krueger2010; van der Meer et al., Reference van der Meer, Beyer, Horn, Foth, Bornemann, Ries and Wartenburger2010). We found that an event with the temporal position property “late” that was presented as the first event of a sequence of three, elicited a greater pupillary response than an event with the temporal property “early” (Raisig et al., Reference Raisig, Hagendorf and van der Meer2011). That is, when event sequences begin with an unexpected, late event, processing costs increase. This does not seem surprising: when do we ever experience an event sequence that begins with a late event? The unexpected beginning is, therefore, a reasonable explanation for this effect. Although we did not find more long-lasting effects of temporal position on the generation of predictions and error detection, the interesting question is whether the cognitive system uses temporal information strategically to subsequently solve the task in a more efficient way.
Recording eye movements allows identifying effective information acquisition strategies. In healthy individuals, eye movement strategies are spontaneously adapted to task demands leading to better behavioral performance (Araujo, Kowler, & Pavel, Reference Araujo, Kowler and Pavel2001; Galey & Galey, Reference Galey and Galey1999; Sprague & Ballard, Reference Sprague and Ballard2004). Effectiveness of strategies may depend on task characteristics but can be characterized by longer fixation durations and more fixations if local processing demands are higher (Hogeboom & van Leeuwen, Reference Hogeboom and van Leeuwen1997; Joos, Rötting, & Velichkovsky, Reference Joos, Rötting and Velichkovsky2003). Furthermore, eye movements have been shown to differentiate, for example, between low and high cognitive ability (IQ) individuals (Vigneau, Caissie, & Bors, Reference Vigneau, Caissie and Bors2006), between older and younger participants (Green, Lemaire, & Dufau, Reference Green, Lemaire and Dufau2007), and between healthy and psychotic individuals (Landgraf, Amado, Bourdel, Leonardi, & Krebs, Reference Landgraf, Amado, Bourdel, Leonardi and Krebs2008; Landgraf, Amado, Purkhart, et al., Reference Landgraf, Amado, Purkhart, Ries, Olie and van der Meer2011). Furthermore, Landgraf, Amado, Brucks, et al. (Reference Landgraf, Amado, Brucks, Krueger, Krebs and van der Meer2011) differentiated between a more efficient (faster reaction time, less errors) problem solving strategy (“constructive matching”) and a less efficient problem solving strategy (“response elimination”) (for a detailed description of the strategies, see also Bethell-Fox, Lohmann, & Snow, Reference Bethell-Fox, Lohmann and Snow1984; Vigneau et al., Reference Vigneau, Caissie and Bors2006). The authors found that a less efficient strategy is more often used with increasing task difficulty. With regard to event sequences, this means that eye movement recordings can be used to disambiguate how the processing of temporal information in terms of order and position of individual events is associated with strategic mechanisms.
To this end, we presented participants with event pairs from different event sequences (e.g., going to a restaurant, buying groceries, Table 1). Each pair consisted of either two early or two late occurring events of an event sequence. For example, an early event in the event sequence “going to the restaurant” would be “sit down at the table”; a late event would be “to pay the bill.” Each event pair was presented either in the correct (chronological) or incorrect (inverse) temporal order (e.g., “sit down at the table” followed by “read the menu” was a chronological pair whereas “pay the bill” followed by “ask for the bill” was an inverse pair). Participants had to decide whether the temporal order of the presented event pair was correct or not. Presenting two events has the advantage that processing of the temporal position and processing of the temporal order of the events can be dissociated. When reading the first event, the temporal position (early, late) can be evaluated while the temporal order (chronological, inverse) of the event sequence cannot. Only after reading the second event, temporal order can be assessed (Figure 1). For each of these processes there are specific hypotheses:
Examples for the four event pair conditions
Schematic description of one trial in the temporal judgment task.
1. Processing the temporal position (i.e., the first event): We expected the pupil to differentiate between early and late events. More specifically, a late event should elicit a greater pupillary response than an early event. Supporting our assumption of an expectancy effect, we predicted longer average fixation duration and more fixations during late compared to early events. In other words, late events should elicit higher information acquisition and processing demands because it is unusual being confronted with the end of a sequence without “experiencing” its beginning. In contrast, early events should be processed with more ease resulting in average shorter fixation duration and less fixations than late events.
2. Processing the temporal order (i.e., the second event): During processing of the second event, the presented temporal order is compared to the temporal order that is stored in the script structure to decide whether there is a temporal violation or not. Temporal order and temporal position can influence this process. Based on previous findings we expected that inverse (possibly inverse/late) event pairs would elicit a greater pupil response than chronological (possibly chronological/early) event pairs. Furthermore, fixation duration should be longer and number of fixations should be higher for inverse (possibly inverse/late) event pairs compared to chronological (possibly chronological/early) event pairs.
3. Influence of temporal position and temporal order on processing strategies: Extending former results, we distinguished two strategies recording eye movements during the task: Reading each event of an event sequence only once (“reading once” – strategy, “RO”) and reading the events more than once (“reading more than once” – strategy, “RMO”). We hypothesized that the “RO” – strategy would be used more often when task demands were low (early, chronological events) compared to when task demands were high (late, inverse events). Furthermore, this strategy was hypothesized to be more effective than the “RMO” – strategy due to the uninterrupted temporal judgment processes. Hence, the “RO” – strategy should lead to shorter reaction times and possibly lower error rates. Finally, when using the “RO” – strategy, the evaluation of the temporal order is resolved after processing (reading) each event only once. The evaluation of temporal order, however, might, for different reasons, get interrupted with the “RMO” – strategy. Therefore, since temporal order judgment only takes place when reading the second event, we expected that eye movement parameters would distinguish between strategies during processing of the second but not during processing of the first event.
Materials and Methods
Participants
Twenty-three students (mean age = 24.39 years; SD = 5.55; 17 females) with normal or corrected-to-normal vision participated in the experiment to obtain course credit. All participants were German native speakers. Testing took place in the eye movement laboratory at the Institute of Psychology at Humboldt-Universität zu Berlin, Germany. Human data included in this manuscript was obtained in compliance with the Helsinki Declaration.
Stimuli and Procedure
Participants sat comfortably in a weakly illuminated room (luminance ≈ 300 lux) 60 cm in front of the computer screen with their heads stabilized on a head and chin rest. Written instructions on the computer screen explained the task and asked for quick but accurate responses. Four practice trials followed during which participants received feedback about whether their response was correct or not. After the practice trials, the experimental block began with the calibration of the eye position. Here, a fixation cross appeared on the screen and the participant was instructed to fixate it. When the position of the eye was recorded, the cross moved to a new position. After the 13-point calibration was successfully completed, a short notice appeared on the screen informing that the experiment was now going to start. The experimental block was initiated via a mouse click.
During the experiment, event pairs, which consisted of two component events taken from event sequences that had been produced in a script generation task (Raisig, Welke, Hagendorf, & van der Meer, Reference Raisig, Welke, Hagendorf and van der Meer2009), were administered. One trial proceeded as follows (Figure 1): A fixation cross appeared at the middle of the screen for 2000 ms. At time point zero, the title of an event sequence (e.g., going to a restaurant) appeared for 1000 ms. When it disappeared from the screen, the first event of the event pair appeared left of the center. A total of 1500 ms after the appearance of the first event, the second event appeared right of the center while the first event remained visible. Reading both events required participants to cross the midline of the screen at least once with their eyes. Participants could now make a response deciding as to whether the presented order of the events corresponded to the chronological order of the events in real life or not. Participants responded with the left and right control key on a standard computer keyboard. Key response assignment was randomized across participants. After the response, a mask consisting of Xs appeared for 2000 ms at the places where the two events were shown before. The mask was used as a refraction period for the pupil. Reaction times, pupil size and eye movements were recorded from the presentation of the title of the event sequence onward. The experiment lasted approximately 30 min.
Design
The experiment was a 2 × 2 design. The temporal order of the events was manipulated (chronological vs. inverse) as well as the temporal position of events during the event sequence (early vs. late). Event pairs occurred either early in an event sequence (chronological: A-B; inverse: B-A) or late in an event sequence (chronological: Y-Z; inverse Z-Y). Mixed conditions between early and late events in one single pair (e.g., Y-B or A-Z) were not allowed. Sixty event pairs were constructed that could be presented in the chronological or inverse order. To ensure that a participant saw an event pair either in the chronological or the inverse order (but not both), different lists were created. Each list contained 15 chronologically related early event pairs, 15 chronologically related late event pairs, 15 inversely related early event pairs, and 15 inversely related late event pairs. In 50% of the cases the chronological order was presented, in 50% it was not, resulting in an around even number of yes- and no-responses. Examples for “early” and “late” event pairs are depicted in Table 1.
Pupil and Eye Movement Recordings
The experimental software Presentation (Version 12.1) was used to administer the stimuli. A compatible 240 Hz iView X system (SensoMotoric Instruments) recorded changes in pupil size and eye movements mono-ocularly (right eye) and with an accuracy above .03 mm. Pupil size was calibrated directly before the beginning of the experiment by placing a black circular sticker with a fixed diameter of 5 mm onto the closed eyelid of the participant. This allowed converting pupil diameter from pixels into millimeters for each participant. Fixation duration had to be between 60 and 2000 ms. Furthermore, eye movements (in between fixations) were defined as having an acceleration of more than 60°/s2. Blinks and other artifacts were excluded (interpolated) using the systems own software and manual inspection of the data.
Data Preparation and Analysis
Data were normally distributed (Kolgomorov-Smirnov, homogeneity tests) unless otherwise specified. If the assumption for an analysis of variance was violated (test of sphericity), degrees of freedom were corrected using the Greenhouse-Geisser estimates of sphericity. Significance level for all effects was set at p < .05.
Data of the pupil curves was smoothed using a computer algorithm developed in-house (box filter-technique). Missing data in the curves due to blinks (sudden large changes in pupil diameter) were corrected by interpolation. All data correction was done with Matlab. The pupil curves were averaged stimulus-locked to the presentation of the title of the event sequence (time point = 0). Pupil dilation was expressed as a millimeter deviation from the baseline, which was calculated as the mean pupil size during a 200 ms time window before presentation of the title of the event sequence (baseline correction). To test the effect of the temporal position of the event on cognitive resource consumption, we obtained the mean pupil dilation within the time window in which the first event of the event pair was presented taking into account the fact that the pupil can lag behind stimulus presentation by up to 800 ms (cf., Zimmer, Reference Zimmer1984). Because we wanted to compare processing differences between early and late events at this point, we computed overall means for early and late items regardless of sequence condition. To test the effect of temporal order on cognitive resource consumption, we analyzed the maximal pupil dilation that is elicited by the second event until the end of the trial. We submitted the maximal pupil dilation to a 2 (temporal order) × 2 (temporal position) repeated-measures analysis of variance (ANOVA).
As mentioned above, in the present paradigm events were administered on the left (first event) and right side (second event) of the screen. Hence, crossing the midline with the eyes indicates the disengagement of attention from the first event and the engagement of attention toward the second event. One midline crossing per trial suggests an effective evaluation strategy since each event is read only once and their temporal relation compared accordingly. Crossing the midline more than once during a trial suggests that at some stage the evaluation and comparison process was interrupted. The percentage of strategies (reading once – “RO” – vs. reading more than once – “RMO”) was compared between conditions (early/chronological, late/chronological, early/inverse, late/inverse) using a repeated-measures ANOVA. Furthermore, reaction time and error rate were compared between conditions and strategies.
The cognitive processes differ before compared to after the first midline crossing. Reading the first event in an event pair does not allow to compare the temporal order of events. However, evaluating whether the first event belongs to the beginning or the end of an event sequence is already possible, yet not obligatory at this point. Therefore, eye movement parameters (fixation duration, number of fixations, time to midline crossing) were compared between the two conditions (early, late) and the two strategies (“RO”, “RMO”) using repeated-measures ANOVAs for the time before the first midline crossing.
Reading the second event of the event pair initiates the temporal judgment process. Here, both temporal position and temporal order can be evaluated. Consequently, eye movement parameters (fixation duration, number of fixations, time to midline crossing/end of trial) were compared between the four conditions (early/chronological, late/chronological, early/inverse, late/inverse) and the two strategies (“RO” vs. “RMO”) using repeated-measures ANOVAs for the time after the first midline crossing. Note that eye movement parameters can only be compared between strategies before the first and after the first midline crossing because the “RO” – strategy ends by definition after that. Before the first midline crossing, we compared time to midline crossing between strategies. After the first midline crossing we compared time to end of trial (“RO” – strategy) and time to second midline crossing (“RMO” – strategy).
Results
The results section is structured according to the hypotheses stated in the introduction. There were no gender effects for any of the reported parameters (p's > .1).
Processing of Temporal Position (i.e., During the First Event)
To investigate the effect of the temporal position of the event on cognitive resource consumption and strategic processing mechanisms, we analyzed the pupil dilation and eye movement parameters in response to the presentation of the first event. Figure 2 displays the mean pupil curves collapsed over early and late items. The mean pupil dilation was compared in the time window 2000–3500 ms (please refer to the figure caption for further information). The t test revealed no significant difference between early and late events (t = 1.06; p = .30) upon processing the first event.
Pupil dilation curves for correct items in the early and late conditions. Note. The pupil is a rather slow reacting parameter and can lag behind stimulus presentation by up to 800 ms (Zimmer, Reference Zimmer1984). The title of the event sequence was presented at time point 0 (first vertical line), where the pupil initially decreased in size. At time point 1000 ms (second vertical line), the first event was presented for 1500 ms. Due to the slow reaction of the pupil, it only begins to increase at time point 2000 ms. Therefore, we obtained the mean pupil size in the time window 2000–3500 ms. At time point 2500 ms (third vertical line), the second event appeared on the screen. Again, the response of the pupil is delayed and it only begins to increase in response to the second event with a delay of approximately 800–1000 ms. It reaches its final peak after approximately 4.2 s into the trial.
Regarding the eye movement parameters during processing of the first event, there were no effects for fixation duration, number of fixations, and time to midline crossing (F's < 1; p’s > .1) during first event processing.
Processing of Temporal Order (i.e., During the Second Event)
To assess the influence of temporal order processing, we looked at the maximal pupil dilation and eye movements during the processing of the second event. Figure 3 shows the maximal pupil dilations for the four conditions. There was a main effect of temporal order (F(1,22) = 13.06; p = .002; d = 1.07) whereby inverse event pairs elicited a greater response than chronological event pairs. There was no main effect of temporal position (F < 1; p > .1) and no interaction effect (F < 1; p > .1).
Maximal pupil dilations for all four conditions in millimeter from baseline.
Regarding eye movements during processing of the second event, there were effects for all parameters. The repeated-measures ANOVA for fixation duration showed a main effect for temporal order (F(1,21) = 5.94; p = .002; d = .77). The post hoc t test showed that fixation duration was longer for inverse than for chronological event pairs (t(21) = −2.05; p = .048; d = .60). There was also a significant temporal order × temporal position interaction (F(1,21) = 6.15; p < .05) showing that fixation duration was shortest for early, chronological items compared to all other conditions (late/chronological: t(21) = 2.93; p = .008; d = .80; early/inverse: t(21) = 2.91; p = .008; d = .86; late/inverse: t(21) = 2.70; p = .001; d = .80), which did not differ from each other (t's < 1.5; p's > .2).
The repeated-measures ANOVA for number of fixations revealed a main effect for temporal order (F(1,21) = 9.73; p = .005; d = .92). The post hoc t test showed that the number of fixations was higher for inverse than for chronological event pairs (t(21) = 6.70; p < .001; d = 1.98). There was no effect for temporal position and no interaction.
The repeated-measures ANOVA for time to midline crossing/end revealed a main effect for temporal order (F(1,21) = 19.22; p < .001; d = 1.29). The post hoc t test showed that the time to midline crossing/end was longer for inverse than for chronological event pairs (t(21) = −4.47; p < .001; d = 1.32). There was no effect for temporal position and no interaction.
Influence of Temporal Position and Temporal Order on Processing Strategies
To assess the influence of temporal position and order on processing strategies, we took into account participant's eye movement strategies. The repeated-measures ANOVA concerning the percentage of strategies indicated a main effect for temporal position only (F(1,21) = 26.69; p < .001; d = 1.52). The subsequent Tukey's t test revealed that the “RO”– strategy was used more often during early (70.7%) than during late items (61.8%) (t(21) = 4.19; p = .001; d = 1.24) irrespective of whether items were in the chronological or inverse order.
We then looked at how the chosen strategy would influence behavioral measures. Regarding reaction times, displayed in Table 2, the repeated-measures ANOVA showed main effects for temporal order (F(1,21) = 23.02; p < .001; d = 1.41), temporal position (F(1,21) = 6.43; p = .019; d = .75), and strategy (F(1,21) = 84.91; p < .001; d = 2.72). Chronological (t(21) = −3.54; p = .002; d = 1.04) and early event pairs (t(21) = −2.03; p = .045; d = .60) were answered faster than inverse and late event pairs, respectively. As hypothesized, reaction time was lower in the “RO”– strategy compared to the “RMO” – strategy (t(21) = −8.17; p < .001; d = 2.41). There were no interactions.
RT and ER for the four event pair conditions separately for each eye movement strategy
Note. RT = reaction time; ER = error rate; ms = milliseconds; % = percentage; SD = standard error. Reaction time, and as a tendency error rate, was lower in the “RO”-strategy than in the “RMO”-strategy. When using the “RMO”-strategy, error rate was lower in the early/chronological condition, compared to all three other conditions.
Regarding error rate, the repeated-measures ANOVA showed a main effect for temporal order (F(1,21) = 10.25; p = .004; d = .94) and, as a tendency, for strategy (F(1,21) = 3.63; p = .071; d = .56). Error rate was lower for chronological compared to inverse event pairs (t(21) = −3.05; p < .006; d = .90). Error rate was also, as a tendency, lower for the “RO”– strategy compared to the “RMO” – strategy (t(21) = −1.84; p = .079; d = .54). There was also a significant three-way interaction for temporal order × temporal position × strategy (F(1,21) = 5.00, p = .004, d = .66). The Tukey's t tests revealed that when using the “RMO” – strategy, error rate was lower in the early/chronological condition compared to all three other conditions (late/chronological: t(21) = −2.52; p = .019; d = .74; early/inverse: t(21) = −2.95; p = .008; d = .87; late/inverse: t(21) = −4.08; p = .001; d = 1.20).
Finally, we looked at how eye movement parameters would differ between strategies. As hypothesized, there were no differences between strategies for any of the eye movement parameters during processing of the first event (Table 3a). During processing of the second event, the repeated-measures ANOVAs for fixation duration, number of fixations, and time to midline crossing/end showed a main effect for strategy (FFixation duration(1,21) = 14.32; p = .001; d = 1.12; Fnumber of fixations (1,21) = 34.02; p = .001; d = 1.67; Ftime to midline crossing/end (1,21) = 53.50; p = .001; d = 2.16). Fixation duration was longer (t(21) = −5.11; p < .001; d = 1.51), the number of fixations was higher (t(21) = −2.59; p = .017; d = .76), and the time to midline crossing/end was longer (t(21) = 8.11; p < .001; d = 2.39) for the “RO” – strategy compared to the “RMO” – strategy (Table 3b).
Eye movement parameters of the two strategies and the four conditions BEFORE the first midline crossing
Note. RT = reaction time; ER = error rate; ms = milliseconds; % = percentage; SE = standard error. There were no differences between the two strategies in any of the eye movement parameters or conditions.
Eye movement parameters of the two strategies and the four conditions AFTER the first midline crossing
Note. RT = reaction time; ER = error rate; ms = milliseconds; % = percentage; SE = standard error. Fixation duration was longer, the number of fixations was higher, and the time to midline crossing/end was longer for the “RO”-stragey compared to the “RMO”-strategy.
Discussion
By administering event pairs in the present study, we were able to illustrate temporal position and temporal order processing and their interaction during a temporal judgment task of event sequences. Our results can be summarized as follows: We found that pupil dilation and eye movement parameters were not sensitive to the processing of temporal position. Furthermore, we showed that upon processing the temporal order of events to detect temporal violations, inverse event pairs elicited the greatest changes in overall pupil size. This indicated that inverse event pairs consume more cognitive resources than chronological event pairs. With respect to eye movement parameters, we found an analogous picture: Fixation duration was shortest in the early/chronological condition, and chronological items elicited less fixations compared to inverse ones. Finally, by dissociating two information acquisition strategies, we found that the percentage of strategy use was influenced by temporal position but not by temporal order of the event sequence. In accordance with our hypothesis, applying the “RO” – strategy resulted in shorter reaction times compared to the “RMO” – strategy. Error rate, on the other hand, was lower in the “RMO” – strategy only if an early, chronological event pair was evaluated. Eye movements could best distinguish between strategies during the evaluation of the second event suggesting that changes in eye movement strategies are most effective when processing temporal order.
During the processing of the first event, only the temporal position but not temporal order can be evaluated. Former results indicated that event sequences are not expected to begin with a late event resulting in increased resource consumption if they do (Raisig et al., Reference Raisig, Hagendorf and van der Meer2011). This was not directly supported by either our pupil data or by the eye movements results. The lack of differences in the pupil dilation data might be due to the paradigm used because expectations about how a sequence begins might not come into play in event pairs. As we have argued elsewhere (Raisig et al., Reference Raisig, Welke, Hagendorf and van der Meer2007), event pairs are not necessarily considered event sequences. Therefore, longer sequences (e.g., three-event sequences) may be needed to reveal processing differences that are accounted for by expectancy. This interpretation receives support from earlier findings in semantic judgment tasks where events from event pairs were processed equally regardless of their temporal position (Galambos & Rips, Reference Galambos and Rips1982; Nottenburg & Shoben, Reference Nottenburg and Shoben1980; Pohl, Reference Pohl1990). This interpretation is further confirmed by our eye movement results. Eye movement parameters did not differ between early and late events during the processing of the first event. In addition, eye movements are strongly sensitive to time constraints, that is, for example, fixation duration is adjusted to available inspection time (Hogeboom & van Leeuwen, Reference Hogeboom and van Leeuwen1997; Joos et al., Reference Joos, Rötting and Velichkovsky2003; Velichkovsky, Sprenger, & Pomplun, Reference Velichkovsky and Sprenger1997). The first event, in contrast to the second event, had a fixed time frame of 1500 ms, after which the second event appeared on the screen. After these 1500 ms the second event appeared making it likely that participants abandoned to read the first event and started to read the newly appearing second event. Processing of the first event was, therefore, limited. In contrast, eye movement parameters differed between early and late events during processing of the second event, indicating that processing of the first event could have been, indeed, homogeneous regardless of temporal position of the event. Finally, task demands did not explicitly state to evaluate temporal position but instead required a temporal order judgment. The explicit processing of temporal position information might actually affect eye movement and also pupil dilation parameters. Since this was not the case in the present experiment, it might be investigated in a separate experiment. Nevertheless, one can generalize that spontaneous processing of temporal position of an event does neither affect pupil dilation nor eye movement parameters.
Processing the second event during the present task suggested, first, that temporal order of the event pair could be evaluated and, second, that temporal order and temporal position processing could interact. We found support for our hypotheses that the pupillary response as well as the eye movement parameters reflected differences in processing demands between chronological and inverse event pairs. Inverse event pairs elicited a greater pupillary response than chronological event pairs. Furthermore, fixation duration was longer, number of fixations was higher, and time to midline crossing/end was longer for inverse than for chronological event pairs. Of interest, we found that at this stage the temporal position of the events also seemed to influence temporal order processing. This became manifest in the eye movements where fixation duration was shortest for chronological, early event pairs compared to the three other conditions. This suggests that while eye movement parameters appear to be most sensitive to temporal order processing, the temporal position “early” additionally allows a shortening of required information processing. We argue that early and chronological events are the simplest task demands in our paradigm. Previous studies have already documented that the chronological order is processed with greater ease, for example, in sentence processing (van der Meer, Beyer, Heinze, & Badel, Reference van der Meer, Beyer, Heinze and Badel2002), in the judgment of object-adjective pairs where the adjective describes an object state (Nuthmann & van der Meer, Reference Nuthmann and van der Meer2005), in simple event pairs (van der Meer, Krueger, Strauch, & Kuchinke, Reference van der Meer, Krueger, Strauch and Kuchinke2006) as well as in more complex event sequences (Raisig et al., Reference Raisig, Welke, Hagendorf and van der Meer2007; Raisig, Welke, Hagendorf, & van der Meer, Reference Raisig, Welke, Hagendorf and van der Meer2010). The study by Raisig et al. (Reference Raisig, Welke, Hagendorf and van der Meer2010) additionally showed that violations of the chronological order were detected more successfully when more cognitive resources were invested: healthy subjects who solved the temporal judgment task successfully showed a greater pupillary response than subjects who were less successful in the task. Insufficient cognitive effort might account for the problems with temporal order processing in populations with impairments to the frontal lobes (e.g., Sirigu et al., Reference Sirigu, Zalla, Pillon, Grafman, Agid and Dubois1995, Reference Sirigu, Zalla, Pillon, Grafman, Agid and Dubois1996) and respective training might help reducing these problems.
At this point, we briefly would like to comment on the influence of frontal areas on decision making in our temporal order task and how this relates to pupillometry. As outlined in the framework by Aston-Jones and Cohen Reference Aston-Jones and Cohen2005 there is a close connection between frontal systems and the pupillary system. The authors suggest projections between frontal areas (more specifically the orbitofrontal cortex, OFC, as well as the anterior cingulate cortex, ACC) and the locus coeruleus (LC), an area in the brainstem that releases the neurotransmitter norepinephrine (NE). Crucially, pupil diameter is considered to index LC-NE activity (Gilzenrat, Nieuwenhuis, Jepma, & Cohen, Reference Gilzenrat, Nieuwenhuis, Jepma and Cohen2010). As we have outlined elsewhere (Raisig et al., Reference Raisig, Welke, Hagendorf and van der Meer2010) the pupillary response is usually related to the cognitive workload and the cognitive resources that are consumed during a task (e.g., Just et al., Reference Just, Carpenter and Miyake2003). In the present as well as in previous studies, we have found that temporal violations elicited a large pupillary response. We concluded that the temporal violation was perceived as a mismatch or conflict, which increased the need for investing cognitive resources. A brain area repeatedly associated with conflict monitoring and error detection is the anterior cingulate cortex (ACC) (cf., Botvinick, Braver, Barch, Carter, & Cohen, Reference Botvinick, Braver, Barch, Carter and Cohen2001). It has been suggested that the ACC becomes active when tasks are difficult. Informational conflicts make a task cognitively demanding thereby indexing the demand for cognitive resources and mental effort (Rushworth, Walton, Kennerley, & Bannerman, Reference Rushworth, Walton, Kennerley and Bannerman2004; Walton, Bannerman, Alterescu, & Rushworth, Reference Walton, Bannerman, Alterescu and Rushworth2003). Hence, the ACC may be “importantly involved in linking mental effort to the autonomic changes that typically accompany it” (Botvinick, Cohen, & Carter, Reference Botvinick, Cohen and Carter2004). Relating our findings to the assumption of a connection between LC-NE and the autonomic pupillary response (Aston-Jones & Cohen, Reference Aston-Jones and Cohen2005) as well as ACC functioning (Botvinick et al., Reference Botvinick, Cohen and Carter2004), our present and previous work can be related to work emerging in this field that assumes a connection between frontal systems (ACC), LC and the pupillary system. However, this line of research may be pursued more carefully in future studies to provide valid conclusions.
In the present study, we further demonstrated that the preference for the chronological order is complemented by the event's temporal position. It can be concluded that chronological/early event sequences are favored as they are encountered in real life which is in turn preserved in the long-term memory representation (e.g., Barsalou, Reference Barsalou1999; Schank & Abelson, Reference Schank and Abelson1977).
Although the temporal position initially neither affected eye movement parameters nor the pupil dilation, it may have had, however, a more long-lasting effect on processing. On the one hand, this is seen in the result discussed above. On the other hand, the temporal position led to different strategies in processing the temporal order. Distinguishing between the “reading once” and the “reading more than once” strategies, the “RO” – strategy was used more often than the “RMO” – strategy when early rather than late events were processed. This means that during processing items with the temporal property “late”, participants frequently jumped back to the first event, which can be interpreted as a kind of re-analysis due to unexpectedness that is present in late events. This is in line with our hypothesis that the process of comparing the presented temporal order with the expected temporal order is interrupted more often for late than for early items. This result shows that the event's temporal property “position” is accessed thereby influencing processing strategies as reflected by eye movement parameters (when time constraints are absent). This in turn directly affects processing: The “RO” – strategy was the more effective strategy in terms of reaction time and as a tendency in terms of error rate. Of interest, the lowest error rate was observable for the “RMO” – strategy for early and chronological items. This shows that error rate can be minimized using a less efficient (slower) strategy in these tasks. Possibly in this case, the “RMO” – strategy served as a safe option to solve the task, which can be interpreted as a security strategy used to reassure whether an initial answer tendency had been correct or not. Future studies could investigate, if specific individual characteristics (e.g., conscientiousness, neuroticism) and/or specific circumstances (time constraints, task instructions) would be associated with such a response option in event sequence judgment tasks.
The effective “RO” – strategy was also marked by higher fixation duration, more fixations, and a longer time to midline crossing/end than the “RMO” – strategy when processing the second event. This suggests a deeper level of processing when applying the “RO” – strategy enabling participants to solve this task in a more efficient way (Hogeboom & van Leeuwen, Reference Hogeboom and van Leeuwen1997; Landgraf, Amado, Brucks, et al., Reference Landgraf, Amado, Brucks, Krueger, Krebs and van der Meer2011; Vigneau et al., Reference Vigneau, Caissie and Bors2006). Hence, when processing temporal position as well as temporal order, a certain depth of processing is required to solve the task efficiently. Therefore, we can conclude that there is an expectancy effect: a late event at the beginning of the trial is unexpected and shows its after-effects in a higher rate of the “RMO” – strategy compared to early events. Hence, even though basic eye movement parameters as well as the pupillary response does not appear to be sensitive to temporal position of the event, as reported above, it nevertheless impacts information acquisition strategies and the processing of temporal order. Therefore, we concluded that temporal position leads to a shift in information acquisition strategies upon processing of the temporal order.
In this study, we investigated temporal position and temporal order processing with the help of an event sequence judgment task. Pupil recordings were sensitive to temporal order processing. Eye movement recordings were more sensitive to temporal order processing and also showed an interaction with temporal position processing. This means that processing different aspects of the temporal properties of events has different behavioral outcomes that can be assessed by combining eye movement and pupil dilation data. Further research is granted regarding the investigation of the overall centrality, length, and associated motor complexity of single events in a certain script.
Acknowledgments
This work was supported by the Konsul-Karl-und-Dr-Gabriele-Sandmann Stiftung, Berlin Germany, and the Université Pierre et Marie Curie, EC3C No. 158, Paris VI, France (to S.L.). This work was also supported by the German Science Foundation (DFG, research grant ME 1362/13-2 to E.v.d.M.). There are no conflicts of interest.
This research benefited from conversations with H. Picard. The authors wish to acknowledge the assistance of M.-C. Bourdel, H. Lüdicke, A. Bellon, M. Brucks, and D. Meshi, as well as the participating individuals. Drs. Landgraf and Raisig contributed equally to this work.
