We share many of Rosenholtz’s concerns about attention as a gateway to perception and a limited-capacity mental resource. However, we argue that this crisis follows from adhering to a see → decide → act model of perception (Nakayama, Moher, & Song, Reference Nakayama, Moher and Song2023). This framework prevents her analysis from leveraging the emerging view that attention functions as an outcome—a consequence of multiple constraints on perception and action—rather than a singular, causal entity (Enns, Brennan, & Whitwell, Reference Enns, Brennan and Whitwell2017; Ristic & Enns, Reference Ristic and Enns2015; Kozik & Enns, Reference Kozik and Enns2021).
Our first point is that myriad factors shape visual selectivity, including the nonuniform resolution of the retina, neuronal selectivity in numerous cortical maps, recent experience with similar events, immediate goals, and stable individual dispositions (Kozik & Enns, Reference Kozik and Enns2021). Labeling only some of these factors as “attention” is as problematic as applying the term to all of them. Allport (Reference Allport, Heuer and Sanders1987) clarified this complexity by distinguishing between selecting objects in the environment and selecting actions to interact with them. Even simple actions, like reaching for a single orange among others in a bowl, require selectivity across both dimensions. Cisek (Reference Cisek2012) has updated this perspective, proposing that perception and action emerge from a dynamic competition between action possibilities. Rao (Reference Rao2024) predictive coding model of the sensory-motor cortex also distinguishes between state predictions (modeling the environment) and action predictions (modeling possible actions). Evidence from multiple species and across diverse brain regions suggests that this is the default design feature of the brain (Rao, Reference Rao2024). When a cortical network becomes selective for an object or an action, it is therefore not the result of a centralized decision-maker. Rather, distributed circuits continuously integrate sensory and motor information to guide behavior toward the most viable perception and action.
These dynamics unfold even when viewers passively inspect a scene or a visual artwork. The nonuniform resolution of the retina requires that the viewer inspects the scene in a series of fixations. These fixations predictably focus on certain regions before others, guided by many factors, including the viewer’s self-identified purpose (Hayhoe & Ballard, Reference Hayhoe and Ballard2005), their personal history (Tanaka & Philbert, Reference Tanaka and Philibert2022), momentary physiological states (Liu et al., Reference Liu, Sun, Li, Jiang, Yang, Li and Liu2013), and of course low-level features of the image medium (DiPaola, Riebe, & Enns, Reference DiPaola, Riebe and Enns2010; DiPaola,Riebe, & Enns, Reference DiPaola, Riebe and Enns2013). “Attention” does not cause this selective experience unless we accept as many attention mechanisms as there are influencing factors. Instead, attention emerges as the outcome of all these forces combined.
Our second point is that it is a mistake to ignore the role of action, a term mentioned only twice in the target article (ms pp. 21 and 25), with the actions considered in the target article referring mostly to button presses. There is now a wealth of research showing an intimate coupling between action and perception. Merely planning an action toward an object increases the neural activity in human early visual and auditory cortices linked to those objects (Gallivan et al., Reference Gallivan, Chapman, Gale, Flanagan and Culham2019; Gale et al., Reference Gale, Areshenkoff, Honda, Johnsrude, Flanagan and Gallivan2021). Neurons in the premotor and motor brain regions implicated in self-action also respond when observing others perform the same actions (Rizzolatti, Fogassi, & Gallese, Reference Rizzolatti, Fogassi and Gallese2001). Neurons in the lateral interparietal cortex represent environmental locations in anticipation of an action toward the location, during perception of the location, and even in memory of it (Colby, Reference Colby1998), maintaining a representation that can guide the eyes and limbs. These action-perception links seen in neurophysiology are directly translated into behavioral abilities. People are able to respond more rapidly to sensory events that are near one’s hand (Reed, Grubb, Steele, Reference Reed, Grubb and Steele2006; Perry, Amarasooriya, & Fallah, Reference Perry, Amarasooriya and Fallah2016) and to spatial locations that are the target of upcoming actions (Andersen & Mountcastle, Reference Andersen and Mountcastle1983; Snyder, Reference Snyder2000). In real-world settings, rock climbers, bird watchers, and automotive enthusiasts, all see and act on objects in their domain at a finer level of detail than novices (Tanaka & Philibert, Reference Tanaka and Philibert2022).
Actions influence perception even in the limited scenario of passively viewing a visual display. DiPaola et al. (Reference DiPaola, Riebe and Enns2010, Reference DiPaola, Riebe and Enns2013) and Enns & MacDonald (Reference Enns and MacDonald2013) demonstrated that subtly blurring some parts of a display while enhancing the clarity of other regions directs fixations toward the higher resolution areas, as if the eyes adjust to the expected properties of the percept based on the non-uniform resolution of the retina. The knock-on effects of these predictable variations in gaze reliably shape personal attributions made about depicted individuals and even affect the physical proximity one is willing to maintain during simulated interactions with them (Enns, Lin-Yang, & Dudarev, Reference Enns, Lin-Yang and Dudarev2023).
Consider also the influence that selective gaze and limb actions have on the cognitive appraisal of objects. Acting on an emotionally neutral object enhances its appeal in subsequent evaluations (Peck & Shu, Reference Peck and Shu2009; Chapman, Gallivan, & Enns, Reference Chapman, Gallivan and Enns2015; Wispinski et al., Reference Wispinski, Lin, Enns and Chapman2021). Even regular involuntary interaction with objects that are initially appraised as negative contributes to their increased emotional appeal over time (Dudarev et al., Reference Dudarev, Manaligod, Enns and Todd2022). And arbitrarily assigning some objects as “mine” and then moving them toward one’s body leads to subsequent improved memory of those objects, compared to objects assigned as “yours” or moved away from one’s body (Truong et al., Reference Truong, Chapman, Chisholm, Enns and Handy2016). All of this evidence points to iterative and bi-directional relationships between our sensory experiences and the actions we take based on those experiences.
Taking action into account would certainly give Rosenholtz’s limited discussion of task complexity more concrete shape, but it would still leave important questions unanswered. For example, it remains important to distinguish between the computational complexity of a visual problem (Ullman, Reference Ullman1984; Neumann, Reference Neumann, Heuer and Sanders1987) from the constraints imposed by a particular physical body (Gibson, Reference Gibson1997; Wolpert & Flanagan, Reference Wolpert and Flanagan2001). Whereas the complexity of image segmentation may be similar in a computational sense for a human, an octopus, and a large language model, their differing action capabilities will constrain their selective perception in unique ways. This embodied aspect of attention tempts us to propose replacing Rosenholtz’s claim that “selectively attending is the observer’s task” (ms pp. 11) with “selectively attending is what happens when someone prepares, performs, or remembers an action (including the act of looking), thereby privileging an event or object for perception.”
We share many of Rosenholtz’s concerns about attention as a gateway to perception and a limited-capacity mental resource. However, we argue that this crisis follows from adhering to a see → decide → act model of perception (Nakayama, Moher, & Song, Reference Nakayama, Moher and Song2023). This framework prevents her analysis from leveraging the emerging view that attention functions as an outcome—a consequence of multiple constraints on perception and action—rather than a singular, causal entity (Enns, Brennan, & Whitwell, Reference Enns, Brennan and Whitwell2017; Ristic & Enns, Reference Ristic and Enns2015; Kozik & Enns, Reference Kozik and Enns2021).
Our first point is that myriad factors shape visual selectivity, including the nonuniform resolution of the retina, neuronal selectivity in numerous cortical maps, recent experience with similar events, immediate goals, and stable individual dispositions (Kozik & Enns, Reference Kozik and Enns2021). Labeling only some of these factors as “attention” is as problematic as applying the term to all of them. Allport (Reference Allport, Heuer and Sanders1987) clarified this complexity by distinguishing between selecting objects in the environment and selecting actions to interact with them. Even simple actions, like reaching for a single orange among others in a bowl, require selectivity across both dimensions. Cisek (Reference Cisek2012) has updated this perspective, proposing that perception and action emerge from a dynamic competition between action possibilities. Rao (Reference Rao2024) predictive coding model of the sensory-motor cortex also distinguishes between state predictions (modeling the environment) and action predictions (modeling possible actions). Evidence from multiple species and across diverse brain regions suggests that this is the default design feature of the brain (Rao, Reference Rao2024). When a cortical network becomes selective for an object or an action, it is therefore not the result of a centralized decision-maker. Rather, distributed circuits continuously integrate sensory and motor information to guide behavior toward the most viable perception and action.
These dynamics unfold even when viewers passively inspect a scene or a visual artwork. The nonuniform resolution of the retina requires that the viewer inspects the scene in a series of fixations. These fixations predictably focus on certain regions before others, guided by many factors, including the viewer’s self-identified purpose (Hayhoe & Ballard, Reference Hayhoe and Ballard2005), their personal history (Tanaka & Philbert, Reference Tanaka and Philibert2022), momentary physiological states (Liu et al., Reference Liu, Sun, Li, Jiang, Yang, Li and Liu2013), and of course low-level features of the image medium (DiPaola, Riebe, & Enns, Reference DiPaola, Riebe and Enns2010; DiPaola,Riebe, & Enns, Reference DiPaola, Riebe and Enns2013). “Attention” does not cause this selective experience unless we accept as many attention mechanisms as there are influencing factors. Instead, attention emerges as the outcome of all these forces combined.
Our second point is that it is a mistake to ignore the role of action, a term mentioned only twice in the target article (ms pp. 21 and 25), with the actions considered in the target article referring mostly to button presses. There is now a wealth of research showing an intimate coupling between action and perception. Merely planning an action toward an object increases the neural activity in human early visual and auditory cortices linked to those objects (Gallivan et al., Reference Gallivan, Chapman, Gale, Flanagan and Culham2019; Gale et al., Reference Gale, Areshenkoff, Honda, Johnsrude, Flanagan and Gallivan2021). Neurons in the premotor and motor brain regions implicated in self-action also respond when observing others perform the same actions (Rizzolatti, Fogassi, & Gallese, Reference Rizzolatti, Fogassi and Gallese2001). Neurons in the lateral interparietal cortex represent environmental locations in anticipation of an action toward the location, during perception of the location, and even in memory of it (Colby, Reference Colby1998), maintaining a representation that can guide the eyes and limbs. These action-perception links seen in neurophysiology are directly translated into behavioral abilities. People are able to respond more rapidly to sensory events that are near one’s hand (Reed, Grubb, Steele, Reference Reed, Grubb and Steele2006; Perry, Amarasooriya, & Fallah, Reference Perry, Amarasooriya and Fallah2016) and to spatial locations that are the target of upcoming actions (Andersen & Mountcastle, Reference Andersen and Mountcastle1983; Snyder, Reference Snyder2000). In real-world settings, rock climbers, bird watchers, and automotive enthusiasts, all see and act on objects in their domain at a finer level of detail than novices (Tanaka & Philibert, Reference Tanaka and Philibert2022).
Actions influence perception even in the limited scenario of passively viewing a visual display. DiPaola et al. (Reference DiPaola, Riebe and Enns2010, Reference DiPaola, Riebe and Enns2013) and Enns & MacDonald (Reference Enns and MacDonald2013) demonstrated that subtly blurring some parts of a display while enhancing the clarity of other regions directs fixations toward the higher resolution areas, as if the eyes adjust to the expected properties of the percept based on the non-uniform resolution of the retina. The knock-on effects of these predictable variations in gaze reliably shape personal attributions made about depicted individuals and even affect the physical proximity one is willing to maintain during simulated interactions with them (Enns, Lin-Yang, & Dudarev, Reference Enns, Lin-Yang and Dudarev2023).
Consider also the influence that selective gaze and limb actions have on the cognitive appraisal of objects. Acting on an emotionally neutral object enhances its appeal in subsequent evaluations (Peck & Shu, Reference Peck and Shu2009; Chapman, Gallivan, & Enns, Reference Chapman, Gallivan and Enns2015; Wispinski et al., Reference Wispinski, Lin, Enns and Chapman2021). Even regular involuntary interaction with objects that are initially appraised as negative contributes to their increased emotional appeal over time (Dudarev et al., Reference Dudarev, Manaligod, Enns and Todd2022). And arbitrarily assigning some objects as “mine” and then moving them toward one’s body leads to subsequent improved memory of those objects, compared to objects assigned as “yours” or moved away from one’s body (Truong et al., Reference Truong, Chapman, Chisholm, Enns and Handy2016). All of this evidence points to iterative and bi-directional relationships between our sensory experiences and the actions we take based on those experiences.
Taking action into account would certainly give Rosenholtz’s limited discussion of task complexity more concrete shape, but it would still leave important questions unanswered. For example, it remains important to distinguish between the computational complexity of a visual problem (Ullman, Reference Ullman1984; Neumann, Reference Neumann, Heuer and Sanders1987) from the constraints imposed by a particular physical body (Gibson, Reference Gibson1997; Wolpert & Flanagan, Reference Wolpert and Flanagan2001). Whereas the complexity of image segmentation may be similar in a computational sense for a human, an octopus, and a large language model, their differing action capabilities will constrain their selective perception in unique ways. This embodied aspect of attention tempts us to propose replacing Rosenholtz’s claim that “selectively attending is the observer’s task” (ms pp. 11) with “selectively attending is what happens when someone prepares, performs, or remembers an action (including the act of looking), thereby privileging an event or object for perception.”
Financial support
V.D. is supported by a NARSAD Young Investigator Grant from the Brain & Behavior Research Foundation (32562). J.T.E. is supported by a Natural Sciences and Engineering Research Council of Canada Discovery Grant (RGPIN-2024-04400).
Competing interests
V.D. None.
J.T.E. None.