Clarifying the main points

In the target article, I argued that visual attention is in a Kuhnian crisis for a number of reasons. First, a number of supposedly attentional phenomena can instead be explained by the rich but lossy encoding of peripheral vision in terms of summary statistics. Due to the quirks of peripheral vision, one must not only worry about the observer’s eye movements (a point amplified by Godwin, Hart, & Barnhart), but also about the details of the stimulus and the task. Still other supposedly attentional phenomena might be predicted by an ideal observer. These results are problematic for an influential theory, Feature Integration Theory (FIT), and necessitate rethinking what we have learned about visual attention. Second, this points to problems with the methodologies of the visual attention paradigm not yielding the promised results. Third, this situation has facilitated a proliferation of types of attention and often vague proposed mechanisms. In short, the signs that visual attention is in crisis (Box A, target article) extend beyond merely non-attentional explanations for some supposedly attentional phenomena (Chapman, Addleman, & Störmer).

Nonetheless, a number of phenomena seem to require explanation by an attention-like mechanism. I proposed a list of such phenomena (Box B, target article) and, based on that analysis, suggested two candidates for a new theory. My preferred option reframes all perception as requiring performing a task, with limits on task complexity. Making sense of this requires rethinking tasks. One may be unaware of the task, and rather than corresponding exactly with one’s nominal task, it may often involve a sort of multitasking. If a task exceeds the complexity limit, the observer’s brain must simplify it, leading to poorer performance. One might want to use “attention” to refer to the flexible mechanism that chooses and implements a task, subject to this limit.

In the end, the target article does not outright dismiss a mechanism one might want to call “attention” (Ma et al.), and, in fact, suggests a new way to think about an attention-like limit and the mechanisms for dealing with that limit. I agree that inattentional blindness necessitates an additional attention-like mechanism (Most); that is why it appears in Box B. Similarly, as the target article states, search tasks can encounter task limits. In this sense, I agree with Wolfe that “any plausible model of visual search must include visual selective attention,” though I remain unclear what Wolfe or other commentators mean by “selective attention.” Wolfe also raises the topic of attention-like limits on perception even at the fovea; this already appears in the list in Box B under “Dual task.” I am not surprised that these errors are stochastic, given the noisy nature of vision. Perhaps Wolfe would like to add other phenomena to Box B, which seems reasonable. I do see, however, several difficulties. It is not clear to me at this point what lessons we can learn about attention from visual search. I am particularly wary of deriving mechanisms from the subjective experience of searching item-by-item. Considerable processing in peripheral vision seems to happen without awareness. Even if this processing proceeds in parallel, the different items might accumulate evidence at slightly different rates, leading to awareness of their identities at different times, a bit like popcorn popping.

A number of commentators point out that attention is not just about vision (Posner, Dudarev & Enns, Chapman, Addleman, & Störmer, and Prasad & Hommel). Prasad & Hommel, in particular, see this as a critical error, overly subdividing the field. I largely agree and merely attempted to stick to what I know and limit the scope of the target article. See Rosenholtz (2020), for instance, where I relate task complexity limits to working memory. Similarly, I wholeheartedly agree with commentators who point out that we need to worry about both sensory and attentional aspects of perception over time, not just space (Dux, Dell’Acqua, & Wyble; Junker & Huber; Belledonne & Yildirim; Denison). Again, this was not a theoretical division, merely limiting the scope to aspects I better understand. In the same vein, Koevoet & Van der Stigchel rightly protest my focus on classification tasks; this is simply easiest for me to conceptualize and easiest to explain to a broad audience. (I disagree that the target article is limited to categorical classification tasks; see Figure 8).

The nature of the visual representation would fundamentally determine the complexity of a given task. Stimuli with evolutionary importance might have developed more advantageous representations (Capparini, To, & Reid). Here, I have far more agreement with Chapman, Addleman, & Störmer than they suggest. Under limited task complexity, capacity limits in visual processing would be “predicted by the organization of the visual representations themselves,” plus the nature of the limit. If the visual representation makes two classes well separated by a linear classifier, one would imagine that discriminating between those classes would be simple by most definitions of task complexity. If one were limited in the number of neurons one could use, then performance would depend on how well the best N neurons discriminated between cats and dogs, rather than how well the entire brain could discriminate between the two. In addition, I have argued for the utility of thinking of the visual system as almost never merely doing tasks like cat vs. dog discrimination. Particularly in the real world, but also in the laboratory, vision at the bare minimum attempts to monitor the situation and keep track of the general layout of the scene. It is this compound task that potentially encounters complexity limits. Performance would also depend on the usual suspects, such as the strength of the observer’s prior information and the effort they exert (Koevoet & Van der Stigchel).

In a related note, Most describes the role of attentional set as opposed to task limits in inattentional blindness. On the surface, task limits might merely be a constraint and attentional set the implementation of task requirements subject to that constraint. However, in rethinking the lessons from FIT, we must rethink whether attentional set might include far more sophisticated features (Kristjánsson & Chetverikov). Most points to experimental results in which observers better react to a blue motorcycle than a gold one when following blue arrows; surely, he suggests, this is feature-based attention. As a description of results, sure. As evidence that the visual system can only attend to simple features, I would wager not. The observer likely simultaneously gives preference to processing features relevant to the driving task. The task set might appear to include simpler features because monitoring blue stuff generally requires less effort than monitoring blue arrows.

Agreement on a number of issues

Commentaries expressed general agreement with a number of the target article’s main points. This includes the importance of peripheral vision and ideal observers as potential explanatory factors, the state of Feature Integration Theory, and the proliferation of ill-defined or underspecified attentional mechanisms. In addition, several commentators resonated with the second half of the target article: the suggested list of attentional phenomena, thoughts about tasks, and task limits.

Vanunu & Ratcliff broadly agree that peripheral vision accounts in part for change blindness and inattentional blindness. Wolfe concurs that at least some search puzzles may have been due to peripheral vision confounds. Posner agrees broadly that the field has sometimes confounded sensory and attentional effects. Carrasco, Chapman, Addleman, & Störmer, and Kristjánsson & Chetverikov reiterate the need to control for both perceptual factors and what can be explained by an ideal observer. Carrasco rightly points out that her laboratory has spent decades controlling for perceptual factors in order to study attention.

In spite of Wu’s prediction that “Commentators will likely debate her crossing out moves…,” essentially no essays obviously defended Feature Integration Theory. Kristjánsson & Chetverikov and Denison describe FIT as having been in trouble for a long time. From the commentaries, we might infer that others agree; Carrasco, for instance, concurs with past shortcomings in interpreting search experiments. Posner, in fact, cites evidence that a lack of binding can occur even with visual attention. To me, this looks like a Goldilocks effect, as described in the target article. Despite this lack of disagreement, FIT lives on in the field in the form of the assumptions that often go unquestioned. We should be alert to ideas about automatic processing, the availability of only simple features without attention, the need for attention for binding, and FIT-like notions of the purpose and mechanisms of selection.

A number of commentaries agreed with the focus of previous essays on the detrimental effects of the proliferation of ill-defined (Carrasco), underspecified (Chapman, Addleman, & Störmer; Wu) types of attention or attentional mechanisms (Anderson, 2011; Hommel et al., 2019; Zivony & Eimer, 2021; Chun, Golumb, & Turk-Browne, 2011). But authors also argued for the correctness of their favorite distinctions, for example: Posner for alerting, orienting, and executive function; Dudarev & Enns for the correspondence between multiple reasons to prioritize and multiple kinds of selectivity; Carrasco for the exogenous (involuntary)/endogenous (voluntary) and feature/spatial/temporal distinctions; and Denison for multiple mechanisms generally. The sections on “Paradigm Shift = Bad?” and “Paradigm Shift Good” debate whether this precludes a unifying theory.

Several commentaries found the list of critical attentional phenomena valuable (Box B). Belledonne & Yildirim appreciate the connection between the phenomena listed and a fundamental part of attention: how goals impact our perception. Denison agrees that more modern theories of attention, for example, the normalization model and extensions thereof (Reynolds & Heeger, 2009; Jigo, Heeger, & Carrasco, 2021), do not attempt to capture all of the attentional phenomena one might want, and the list provides “a useful guide to theory development.” However, in some cases, reading between the lines, a number of commentaries seem to want to add phenomena to that list. I will address this along the way, and in more detail in the subsection on “Paradigm Shifts and Ambitious Science.”

Some of the more interesting agreement concerns how we might think about tasks, task-based selection, and even task limits. First, in terms of tasks and complex templates, Kristjánsson & Chetverikov cite evidence for summary-statistic-based, complex attentional templates, a welcome change from FIT’s claim that one can only select simple features. Tomasello resonates with complex tasks in discussing attention to the situation, rather than to objects. Morsella et al. describe what I would consider a task-based view of attention, which they call “priority-based,” in which one is likely unaware of the complex tasks performed by the system. Duncan & Theeuwes concur that their results show that one is not in full conscious control of the task set but rather processes information outside of the nominal task (see Box A, target article). Similarly, Chapman, Addleman, & Störmer describe attentional effects without an explicit nominal task.

In terms of formulating attention in terms of task-based selection and task limits, we can see agreement on several points. Wu finds it reasonable to think of task performance and selection as inseparable. Koevoet & Van der Stigchel stress the importance of considering effort if one is to study decision complexity, and I wholeheartedly agree. Belledonne & Yildirim describe a theory with a moment-to-moment evolution of attention over time, consisting of mini tasks or “targetable singleton computations,” each of which takes unit time. One might think of this time requirement as a different form of complexity limit. Poth & Poth point to number of end effectors as an important additional limit, and I agree that this may have guided the development of the attentional system. Along these lines, several commentators from the domain of perception and action resonate with a number of the claims in the target article, from thinking of attention as selecting tasks (O’Bryan & Song) to notions of decision complexity; Prasad & Hommel describe this as “old news,” which I will take as a form of agreement. Krauzlis independently arrived at a task-based selection proposal based on physiology research, but cautions that those results also support the flexible pooling account.

The fact that similar ideas have arisen from thinking about perception for action (Dudarev & Enns, Prasad & Hommel), consciousness (Morsella et al.), evolution (Tomasello), and physiology (Krauzlis) gives me hope that we are on the right track.

Suggested extensions

A number of commentaries went beyond agreement with ideas in the target article to suggest extensions to those ideas. This includes the role of perceptual factors, the utility of banning “attention,” and thinking about task complexity. Many such suggestions lie outside of my expertise. However, I will speculate about attentional blink, the topic of three commentaries.

Francis & Thunell, for instance, mention that their concerns about statistical power apply to other attentional paradigms, not just object-based attention. They point in particular to studies that examine modulations of already small effects, e.g., looking for some sort of group difference in a spatial cueing effect.

Capparini, To, & Reid report that both peripheral vision and attention change a great deal during development. They suggest it is important to extend understanding of peripheral vision to development, where many studies look at attention but far fewer examine peripheral vision. One needs to understand possible confounds in the same way as in the target paper.

Martelli, Mancuso, & Zoccolotti note that allocentric neglect leads to errors that sound a bit like crowding. They ask whether there might be a peripheral vision component. This question has particular relevance since search tasks are a popular experimental paradigm for studying the topic.

In an interesting collection of commentaries, Mattei and Junker & Huber suggest that attentional blink (AB) might be due at least in part to temporal crowding. Dux, Dell’Acqua, & Wyble, on the other hand, describe AB as obviously non-perceptual and teaching us something about the time course of attention.

I do think that Mattei and Junker & Huber have a valid point that some AB phenomena, such as lag-1 sparing, may have perceptual explanations. Experimenters create stimuli by presenting a sequence of discrete symbols, but the visual system inputs not a list of symbols but a spatiotemporal signal. Early in vision, it represents this as energy in multiple bands, not as items for selection. We know far less about temporal crowding than about spatial, but it seems likely that vision pools over significant portions of time as well as over space, even at the fovea. This could lead to temporal crowding with familiar signatures. In fact, the temporal phenomena of AB can sound eerily like spatial crowding: The stimulus appears to contain separate items, and yet, puzzlingly, it can be difficult to individuate and report them under certain circumstances. Crowding has well-known dependence on stimulus similarity, which could conceivably make discerning two letter targets separated by a more distinct number harder than discerning two adjacent letter targets. Even without crowding, sequential, similar target letters, such as a K, H, or B, may be grouped by motion into a K-H morph, or even a K-H-B morph, like a flip book animation. Retrospectively, the observer may be able to parse these chunks into a reasonable guess as to the individual letters, whereas a K-6-H sequence with more complex motions may not create such a comprehensible chunk.

That said, I agree with Dux, Dell’Acqua, & Wyble that better performance when one has to report only the second target is a clear sign of an attention-like mechanism, and probably deserves to be listed in Box B. It does seem like we learn something about the timing of some kind of attentional or task-switching mechanism, though it might be helpful to understand temporal crowding to disentangle the timings of perceptual as opposed to attentional mechanisms.

Hertzmann, meanwhile, takes note of a different point in the target article: a summary-statistic encoding has still-unrealized implications for understanding much of later visual processing. In particular, he rethinks 3D vision across eye movements through the lens of both a lossy encoding in peripheral vision and preferentially preserving ecologically valid information across fixations. This seems like a promising approach.

Most finds value in the exercise of banning the word “attention” when studying individual biases and anxiety disorders, though for a somewhat opposed reason. He suggests that researchers in that field may be finding a confusing pattern of results because they are examining different kinds of attention. One might also want to do Francis & Thunell’s analysis on what power is necessary to use a dot-probe task to study what sounds like individual differences or impact of training.

Finally, Mattei deserves a shout-out for reasoning about task complexity and about perception occurring as the result of a task. He focuses on ensemble perception and, by extension, whether attention is needed for action. Vanunu & Ratcliff also discuss ensemble perception, and correctly suggest that the ability to report the mean size of a set of circles cannot serve as an alternative to attention nor as a paradigm shift. However, this confuses the rich image statistic representation in my peripheral vision model with the task of reporting ensemble properties; see also Kristjánsson & Chetverikov, who clarify the difference between simple ensemble statistics and the summary statistics in the model.

Two other sources of confusion worth discussing

Several commentators associate attention with awareness. Dux, Dell’Acqua, & Wyble, as an example, describe the observer as faced with far too much information “for it all to be processed up to the level of consciousness.” The possible association between attention and awareness has been much debated (see Koch & Tsuchiya, 2007; Cohen, Dennett, & Kanwisher, 2016; Rosenholtz, 2020). I would presume that consciousness faces limits beyond those that govern attention. We know from perception for action that quite a lot of perception can occur without awareness. Along these lines, Hertzmann might be cautious about assuming that when observers fail to notice redirected walking in VR that is because their visual systems have thrown away inconsistent information; apparently researchers in that field note that even observers who fail to notice the trick appear more likely to experience motion sickness (N. Williams, personal communication, March 15, 2025). Personally, I would not attempt to fix the mismatch between awareness and action by proposing two separate systems (Most), as this adds considerable complexity to the theory.

More benignly, Duncan & Theeuwes have mistaken my point about the 40-ms effects in their distraction paradigm. They say that in the real world, 40 ms can matter a lot. I wholeheartedly agree, particularly in applications such as vision in driving (Wolfe et al., Reference Wolfe, Kosovicheva and Stent2021). The target article did not actually say that I thought that 40 ms is not important, but rather that the observer might be unaware of or unconcerned by that additional time. In other words, the effect might depend on the effort the observer was willing to put in to ignore the distraction (Koevoet & Van der Stigchel). Along these lines, O’Bryan & Song report that, in real-world tasks, observers may not be distracted by irrelevant salient stimuli, and, in fact, sometimes show the opposite effect.

“All models are wrong, some are useful”

This famous quote from George Box (Box, Reference Box, Launer and Wilkinson1979) is apt for several of the commentaries. These ask both about the quality of the peripheral vision model and the falsifiability of task complexity.

First, Bornet, Herzog, & Doerig provocatively ask whether we should “start a revolution with a refuted model.” Work by these researchers has demonstrated that the Texture Tiling Model (TTM) for peripheral vision appears not to preserve enough information to predict performance on some Mooney face tasks. They have also argued that TTM cannot explain grouping effects in Vernier acuity tasks. I argue that Vernier acuity is a special hyperacuity task, and quibble with whether they have performed the definitive test, but I necessarily leave this detailed discussion for another venue. Their suggestion echoes the concern that attention influences crowding (Carrasco), though those results are consistent with cueing improving performance rather than changing the peripheral representation. Bornet et al. also state that TTM is unlikely to be a good model of vision in general. TTM is not intended as a model of vision in general; one would say the same of a model of the retina, or of V1.

Even if we take as a given that TTM is wrong in these and no doubt other ways, it is nonetheless useful. In particular, the claim that visual attention is in crisis relies predominantly on predictions for visual search, scene perception, and dual-task performance. We have backed up all of these predictions with peripheral vision experiments, plus, in some cases, intuitions from TTM to help us understand how the sparse dual-task displays relate to the crowded search ones. Hulleman, together with Olivers (2017), provide corroborating evidence. FIT is in trouble – and attentional mechanisms need rethinking – without predictions from TTM. What TTM does provide is the glue that allows us to see that a single model can predict all of these results; in other words, that peripheral vision more parsimoniously explains that entire set of phenomena than attention does. Furthermore, a model can clearly be “wrong” and support a paradigm shift; just look at Copernicus’ version of heliocentrism.

Ma et al. ask whether the peripheral vision explanation for some phenomena gives us “greater precision in understanding cognition and behavior” than the old attentional explanation. They suggest asking “how much variance in [supposedly] attentional tasks can be accounted for by peripheral factors alone.” I certainly agree that we should not accept a new handwavy theory in place of the old, but TTM is a useful model in just this way. Unlike old word-model predictions for things like visual search, with ill-defined attention mechanisms, TTM can make quantitative predictions. Zhang et al. (2015), for instance, show that peripheral mechanisms explain 75% of the variance for the 10 search conditions studied. (We should not, I think, presume that all the remaining 25% of the variance is attention.) In addition, Chang & Rosenholtz (2016) showed the model could predict in advance the results for five new pairs of search conditions.

On the other hand, Koevoet & Van der Stigchel and Chapman, Addleman, & Störmer worry that task complexity remains ill-defined, and the latter ask whether that theory is testable. I agree that we would need to nail down how to operationalize task complexity, though I have found it useful to get intuitions even without that precision. Rosenholtz (2017) discusses testability in more detail. In brief, for a given measure of complexity, and a candidate visual representation, such as from a pre-trained neural network, one would want to ask whether, for example, hard dual tasks correspond to higher complexity than easy dual tasks. Here, Koevoet & Van der Stigchel have, I think, a critical point: one should do one’s best to normalize the effort exerted by the observers across conditions.

Paradigm shift = Bad?

Overwhelmingly, commentators most agreed with each other either in declaring visual attention not in crisis or not in need of a paradigm shift, and/or in suggesting that such a paradigm shift would be pointless or detrimental to science. Here, I review those arguments and respond briefly. In Paradigm Shift Good, I argue for the feasibility and benefits of a paradigm shift.

Paradigm shift good

Having gone through a multi-year exercise of rethinking visual attention, I understand that it looks like a significant amount of work. Discarding old understandings can be difficult and even painful, especially if we have put substantial effort into the old paradigm. The reader might question whether the rewards justify the effort. Having seen the benefits in understanding a broad array of phenomena and making predictions, I believe the effort is well worthwhile. Let us start by addressing the question of whether there might yet be hope for a unifying theory. I argue that we should not worry too much about the apparent diversity of mechanisms.