Fleming and Michel highlight the functional limitations of consciousness imposed by lengthy processing delays: A lower limit of 400 ms to form a conscious visual percept means many of our everyday sensorimotor activities must be performed without conscious intervention.

What’s taking so long? We propose that the processing delay for conscious vision arises from the need to create a unified percept over visual features processed with different, flexible time constants.

Information is integrated over time in different functionally specialized brain areas. Temporal windows of integration appear to extend in duration across the cortex, forming a hierarchy from low-level sensory regions to prefrontal and motor regions (Murray et al., Reference Murray, Bernacchia, Freedman, Romo, Wallis, Cai and Wang2014; Spitmaan et al., Reference Spitmaan, Seo, Lee and Soltani2020). Sensory regions analyzing elementary features integrate information over tens of milliseconds (Runyan et al., Reference Runyan, Piasini, Panzeri and Harvey2017), while information can be sustained for several seconds in the prefrontal cortex (Zylberberg & Strowbridge, Reference Zylberberg and Strowbridge2017). Functional anatomical architecture can be revealed from neural time courses alone (Bartels & Zeki, Reference Bartels and Zeki2004), suggesting a relationship between functionality and temporal integration.

There is also a clear relationship between integration time and conscious visual experience. Stimuli presented for longer durations appear more intense (Nachmias, Reference Nachmias1967; Kitterle, Reference Kitterle1975). Integration time is not fixed, but flexible. Integration is extended for low contrast stimuli (Stromeyer & Martini, Reference Stromeyer and Martini2003). Inversely, increasing luminance speeds up processing, allowing for fine discrimination of duration (Nilsson, Reference Nilsson1979). Processing is faster in the periphery (Carrasco et al., Reference Carrasco, McElree, Denisova and Giordano2003), and here, events are perceived earlier (Jovanovic & Mamassian, Reference Jovanovic and Mamassian2020). Attention can also speed up processing (Tünnermann, Petersen, & Scharlau, Reference Tünnermann, Petersen and Scharlau2015), and objects in attended locations are perceived earlier (Jovanovic & Mamassian, Reference Jovanovic and Mamassian2019).

The dependence of conscious visual experience on processing time introduces a challenge: How do we create a unified percept when different features of the same world are processed with different time constants? Occasionally, we observe errors in synchronization – color change is perceived before changes in motion direction, leading to misbinding of the two attributes in a single visual object (Moutoussis, & Zeki, Reference Moutoussis and Zeki1997). We propose that conscious vision might be delayed due to the need to stitch together features perceived with different time constants (Hogendoorn, Reference Hogendoorn2022): conscious vision could only be as fast as the slowest visual process.

Still, this is not enough time to account for the lower limit of 400 ms proposed by Fleming and Michel. For this, we need to go beyond the visual cortex. Several theories propose that the purpose of vision is not indiscriminate information processing; rather, vision is intentional (Cisek, Reference Cisek2007; Clark, Reference Clark1997; Shadlen et al., Reference Shadlen, Kiani, Hanks and Churchland2008). Visual information is further processed in higher-order brain regions to test propositions (e.g., how to act?). This intentional processing of visual information invokes the longer integration windows of parietal (Shadlen & Newsome, Reference Shadlen and Newsome2001), frontal (Heekeren et al., Reference Heekeren, Marrett, Ruff, Bandettini and Ungerleider2006), and even motor regions (Heitz & Schall, Reference Heitz and Schall2012). A greater challenge for conscious vision would be to account not only for the distinct time constants across these intentional processes but also for different delays in information transfer across the cortex. If conscious vision were only as fast as the slowest intentional visual process, this may account for the 400-ms lower limit.

But, as integration windows increase up the hierarchy, so too does their flexibility. Similar to early visual processes, stimulus intensity decreases parietal integration time (Shadlen & Newsome, Reference Shadlen and Newsome2001). Integration time can also be voluntarily adjusted according to time pressure: Hanks, Kiani, & Shadlen (Reference Hanks, Kiani and Shadlen2014) showed monkeys would (nearly) double their reaction times when encouraged, and this was accompanied by slower integration, evident from the build-up rate of lateral intraparietal neural firing. This also demonstrates how integration must be adjusted to meet functional demands. We need to be able to make fast (100 ms) adjustments to our movements when a target changes location (Brenner & Smeets, Reference Brenner and Smeets1997), while also slowing down to improve decisions under less time pressure.

It is here that Fleming and Michel define “Hamlet’s problem” of when to stop integrating. While more information improves accuracy, it also incurs a cost in terms of time and neural resources (Drugowitsch et al., Reference Drugowitsch, Moreno-Bote, Churchland, Shadlen and Pouget2012). Several computational models propose the use of choice-confidence (or uncertainty) for controlling integration (Drugowitsch, Moreno-Bote, & Pouget, Reference Drugowitsch, Moreno-Bote and Pouget2014; Balsdon, Wyart, & Mamassian, 2020; Reference Balsdon, Mamassian and Wyart2021; Balsdon & Philiastides, Reference Balsdon and Philiastides2024; Bénon et al., Reference Bénon, Lee, Hopper, Verdeil, Pessiglione, Vinckier and Daunizeau2024; Lee & Daunizeau, Reference Lee and Daunizeau2021). Fleming and Michel propose a conscious metacognitive mechanism, suggesting conscious vision might have some use in solving “Hamlet’s problem” for perceptual decision-making.

Rather than framing this as a problem of when to stop integrating, decision-time could be moderated by the flexible adjustment of integration windows. We (Balsdon & Philiastides, Reference Balsdon and Philiastides2024) have proposed a confidence-control mechanism that coordinates temporal integration across the cortex: choice-confidence at the level of a primary integrator shortens integration windows at the motor level, speeding up the decision. Importantly, this adjustment happens during decision-making, according to the flux of incoming sensory information. The time courses described by our model map onto electroencephalographic activity across visual, parietal, motor, and frontal regions, and are theoretically in line with the autocorrelations in single neuron activity supporting the hierarchy of temporal integration windows (Murray et al., Reference Murray, Bernacchia, Freedman, Romo, Wallis, Cai and Wang2014).

This continual adjustment of integration windows is too fast for the proposed 400 ms to conscious vision, suggesting it may rely on an unconscious uncertainty signal that only later informs explicit metacognitive judgments. Speculatively, the same uncertainty signal that controls temporal integration could also be used to stitch together information across cortical processes for the purpose of creating a unified conscious percept. In this case, the timing for conscious vision would follow the flexibly adjusted integration windows: there is no need for a strict lower limit.

To understand the timing of conscious vision, we need to understand how information is integrated at different timescales across the brain, and the flexibility with which these timescales are implemented to meet functional demands. Given the multitude of integration timescales, one challenge is how to obtain a unified percept of the visual world while maintaining flexibility, an ability that might be altered in psychiatric populations (Giersch, Reference Giersch2024). A critical question is whether the flexibility of processing time may also apply to conscious vision.