Hulleman & Olivers (H&O) help move the study of visual search in the right direction by promoting the idea of active vision (Findlay & Gilchrist Reference Findlay and Gilchrist2003), which emphasizes the role of eye movements in visual search over a traditional emphasis on covert shifts of attention; this also reflects a shift to eye movement measures rather than response times as the relevant dependent variables. Their paper incorrectly suggests, however, that a consideration of visual objects as discrete psychological objects is problematic. The problem is not that items are involved, but rather that most current theories assume a simplistic relationship between number of items and reaction time that misrepresents how visual capabilities are actually used in the task. Therefore, for many stimuli and tasks such as searching for icons on a computer screen (e.g., Fleetwood & Byrne Reference Fleetwood and Byrne2006; Kieras & Hornof Reference Kieras and Hornof2014), the goal is to identify a specific display object (e.g., by clicking on it) which implies that the search must be item-based at some level, even though the subject might be processing multiple visual objects simultaneously.

We believe that the actual theoretical problem that underlies the issues pointed out by H&O is an impoverished theoretical framework for how human perceptual, cognitive, and motor mechanisms operate together in performing visual search. Although their model demonstrates an approach that improves on the status quo, the problem can be more thoroughly alleviated by the use of an explicit computational cognitive architecture foundation.

In an analogy with computer architecture, a computational cognitive architecture such as ACT-R (Anderson & Lebiere Reference Anderson and Lebiere1998) or EPIC (Meyer & Kieras Reference Meyer and Kieras1997) proposes basic "hardware" mechanisms for perception, cognition, and action, with task-specific "software" in the form of production rules that provide a strategy for how to perform the task with the architectural mechanisms. In EPIC, the visual modules in fact already incorporate the functional viewing field (FVF) concept that H&O promote. These modules use functions from the psychophysical literature for the detectability of visual properties based on object eccentricity and size (e.g., Anstis Reference Anstis1974; Gordon & Abramov Reference Gordon and Abramov1977; Virsu & Rovamo Reference Virsu and Rovamo1979). Certain visual search results characterize the memory stores for visual information (e.g., Henderson & Castelhano Reference Henderson, Castelhano and Underwood2005; Kieras Reference Kieras2011; Peterson et al. Reference Peterson, Kramer, Wang, Irwin and McCarley2001), and the oculomotor module incorporates results from eye movement studies of the speed and accuracy of saccades (e.g., Abrams et al. Reference Abrams, Meyer and Kornblum1989; Harris Reference Harris1995). The production rules comprising the cognitive strategy specify how the human uses the perceptual, motor, and memory resources to accomplish the task. For example, the strategy determines which object should be fixated next based on the task requirements and the available perceptual and memory information, and what is to be done if the fixation has failed, such as missing the targeted object. EPIC's cognitive strategy represents explicitly the cognitive decisions that must be executed to complete visual search tasks, the same sort of decisions that H&O summarize in the “flow diagram of the conceptual framework” shown in their Figure 2.

Unlike simple ad hoc models, a computational cognitive architecture serves to synthesize many empirical effects in a stable, reusable fashion in the architecture components, and because the strategy is represented separately, it is easy to build a model for a different task, or to explore the consequences of different strategies for the same task, all based on the same architectural assumptions. The resulting simulation models provide a computational version of the active vision concept itself and can be applied to practical problems such as analyzing and improving the design of computer interfaces and multimodal interaction (Halverson & Hornof Reference Halverson and Hornof2011; Hornof Reference Hornof2004; Kieras & Hornof Reference Kieras and Hornof2014; Kieras & Meyer Reference Kieras and Meyer1997).

However, a lesson of cognitive architecture modeling is that even in simple tasks, subjects can adopt subtle strategies that can obscure the underlying mechanisms unless taken into account (Kieras & Meyer Reference Kieras, Meyer, Schraagen, Chipman and Shalin2000; Meyer & Kieras Reference Meyer, Kieras, Gopher and Koriat1999; Zhang & Hornof Reference Zhang and Hornof2014). In our models of visual search, the task strategy plays a role in determining the object to be next fixated and, when there are no clear candidates for the next saccade, there are a variety of possible decisions that could be made, and these decisions affect the predicted search performance.

Unless some effort has been made to control subject strategies in the experiment, the data themselves may be an arbitrary mixture of idiosyncratic performance by each subject – and this strategy indeterminism can happen even in seemingly simple tasks (Meyer & Kieras Reference Meyer, Kieras, Gopher and Koriat1999; Kieras & Meyer Reference Kieras, Meyer, Schraagen, Chipman and Shalin2000; Schumacher et al. Reference Schumacher, Seymour, Glass, Fencsik, Lauber, Kieras and Meyer2001). Modeling such data in the aggregate typically requires either ad hoc or unrepresentative strategies, but the alternative of identifying individual subject strategies in normal-size data sets is extremely difficult. Instead, providing explicit incentives to the subjects can lead to more stable strategy choices and thus data that reflect underlying mechanisms much more clearly (e.g., Thompson et al. Reference Thompson, Iyer, Simpson, Wakefield, Kieras and Brungart2015). Thus, we believe that task strategy needs to be considered more explicitly both in experiments and models about visual search.

We applaud the authors for their effort to further promote gaze fixations and the FVF as first-class entities in models and theories of visual search. We believe that the development of computational cognitive architecture models provides a promising pathway to achieve the goals they have expressed. We encourage other researchers to embrace these positive developments but to also go further and (a) more explicitly consider the role of cognitive strategy in visual search and (b) as hinted by H&O, collect the empirical data needed to describe more completely and parametrically how visual properties are detected based on object eccentricity, size, and density (building on Anstis Reference Anstis1974; Bouma Reference Bouma1970; Engel Reference Engel1977; Gordon & Abramov Reference Gordon and Abramov1977; Virsu & Rovamo Reference Virsu and Rovamo1979). Both are needed for a comprehensive predictive model of visual search.