The target article presents a laudable effort to depart from conceptual analyses of visual search that are based primarily on discrete stimulus items. By using fixations as the central unit, the authors propose a significant paradigm shift for core research on visual search. As the authors note, this idea has been proposed before. In fact, it is central to most computational theories and models of visual attention. Indeed, the standard test for saliency map models consists of measuring their ability to predict human fixation locations in a wide range of tasks (Borji et al. Reference Borji, Sihite and Itti2013), including visual search (e.g., Ehinger et al. Reference Ehinger, Hidalgo-Sotelo, Torralba and Oliva2009). A framework based on fixations promises an understanding of visual search behavior that is more focused on perceptual and motor mechanisms than on external stimulus composition.

Computational models of attention – applicable not only to visual search but also to virtually any visually guided behavioral task – often distinguish among at least three facets of visual attention (Itti & Borji Reference Itti, Borji, Nobre and Kastner2013):

1. 1. Guidance: What computations are involved in deciding where or what in a scene to attend to next? This includes bottom-up or stimulus-driven guidance signals, such as visual salience, and top-down guidance signals, for example related to spatial or feature-based cueing.

2. 2. Selection: How is a fraction of the visual input separated out from other incoming visual signals so that it can be processed in more detail?

3. 3. Enhancement: How are selected (attended to) visual items processed differently – usually better by some measure such as enhanced contrast sensitivity or better discrimination threshold – than nonselected (unattended) items?

All three aspects have been studied extensively in the electrophysiology, psychophysics, and modeling literature (for reviews, see, e.g., Allport et al. Reference Allport, Meyer, Kornblum, Meyer and Kornblum1993; Borji & Itti Reference Borji and Itti2013; Carrasco Reference Carrasco2011; Desimone & Duncan Reference Desimone and Duncan1995; Driver & Frith Reference Driver and Frith2000; Itti & Koch Reference Itti and Koch2001; Reynolds & Desimone Reference Reynolds and Desimone1999; Robertson Reference Robertson2003). The target article authors deliberately sidestep guidance in their simulations and focus on selection, in particular through the concept of functional field of view (FVF), which is akin to an attention spotlight (Crick Reference Crick1984), and they omit enhancement.

The framework could be strengthened by clearly embracing the idea of separate functional mechanisms (although possibly overlapping and using shared neural components) for guidance, selection and enhancement. Indeed, with a single FVF, the concepts of scene gist and contextual guidance already seem to be a struggle for the framework (sect. 6.2): Presumably, when starting a new search over a natural scene, one would first need a wide FVF for scene identification and to establish semantic guidance, then switch to a smaller FVF during search. Thus, as the authors concede, the FVF size is unlikely to be fixed as hypothesized in the simulations, and may instead change before and possibly during search. Even with a dynamic FVF, it is unclear whether phenomena such as contextual guidance or cueing of search – in natural or artificial scenes (Chun & Jiang Reference Chun and Jiang1998; Torralba et al. Reference Torralba, Oliva, Castelhano and Henderson2006) – would require the same FVF processing characteristics as would be required to select and analyze items at the current fixation. An FVF that can rapidly change size and functional form presents little conceptual value as it is an unknown time-varying entity. It becomes a liability for the framework because any unexplained phenomenon could be attributed to some sudden change in FVF size or processing form that one would be hard-pressed to measure in real time.

Thus, one may need to split the concept of FVF into at least two or three: possibly a broader FVF operating coarser or more statistical processing might be necessary to compute a saliency map around the current fixation and to integrate contextual information broadly, to provide effective guidance of search. But a narrower and more selective FVF may sometimes be needed for selection, for example in difficult compound search. Enhancement may require a different FVF as well, possibly even more tightly wrapped around the selected items to reject background noise within the selected region. All three might be required in complex search scenarios (e.g., compound search on a noisy yet semantically meaningful natural scene background). Positing a single FVF unnecessarily conflates guidance, selection and enhancement. This is already evident in the second component of the framework (sect. 5.2), where a target may be easy to detect peripherally because of a distinctive feature but hard to identify because of a more complex response-defining attribute. The authors assume that a broad FVF should be deployed for guidance of each fixation, followed by a switch to a narrow FVF for selection and identification of items. As with contextual guidance, this dynamic resizing essentially amounts to interlacing over time two different FVFs, one for guidance and the other for selection. Because in addition to different sizes (which may themselves be dynamic), these two FVFs might also have different processing characteristics (e.g., with respect to crowding or lateral masking), I suggest that decoupling the two into a guidance mechanism and a selection mechanism might enhance the long-term usefulness and robustness of the framework (and likewise for an enhancement mechanism).

Electrophysiological and neuroimaging evidence supports, at least in part, distinct mechanisms for guidance, selection, and enhancement. In primates, including humans, guidance likely recruits saliency mechanisms in the dorsal visual processing stream (e.g., Itti & Koch Reference Itti and Koch2001; Kusunoki et al. Reference Kusunoki, Gottlieb and Goldberg2000), and scene-level contextual mechanisms, for example in the lateral occipital complex (area LOC; e.g., Preston et al. Reference Preston, Guo, Das, Giesbrecht and Eckstein2013). The frontal eye fields (FEF) and ventral prearcuate (VPA) areas of the prefrontal cortex are thought to provide goal-directed spatial and feature-based modulation of guidance (Bichot et al. Reference Bichot, Heard, DeGennaro and Desimone2015). Shrink-wrapping of receptive fields in the ventral processing stream including inferotemporal cortex may serve selection and enhancement of selected information (Desimone & Duncan Reference Desimone and Duncan1995; Zhang et al. Reference Zhang, Meyers, Bichot, Serre, Poggio and Desimone2011).

Although the authors of the target article wish to remain at a conceptual rather than algorithmic level, I believe that the best route to avoiding possible conceptual ambiguities or conflating between what should be distinct entities will have to be through detailed computational implementation of the components of this exciting framework.