What is the general resource?

We formalize the limited resource rationed by attention as targetable singleton computations, denoted $C$ . Such computations form the individual operations underlying the algorithm approximating the percept $Pr(S|\;X)$ (e.g., a unit step of the MCMC algorithm, a feedforward pass in a neural network). These perceptual computations must be (1) targetable in that they can be applied to update the approximation of particular object or latent k, ${C_k}\left( S \right)\; \to S'$ where $\;S'\; = S\;/\;\left\{ {{s_k}} \right\} \cup \{ {s_k}'\;\} $ , and (2) singleton in that any such application of said computation takes nearly identical amounts of time regardless of the object or latent, $O\left( {{C_k}} \right)\; = \;O\left( {{C_j}} \right)$ , where $O\left( \cdot \right)$ indicates computational complexity.

Critically, a targetable singleton computation has two consequences – one on perception and one on decision making. First, each ${C_k}$ converges the current approximation of a percept, for example, by improving the estimate of the object’s position and velocity. Importantly, some representations will require more computation to converge, with the rate of convergence denoted ${\delta _k}S$ . Second, computations over some representations will, in turn, differentially impact decision-making, depending on how changes in the representation inform decision outcomes, denoted ${\delta _k}\pi $ .

What is task-relevance and how can it be tractably estimated?

A novel formulation of task-relevance follows from this concrete specification of the general resource attention may allocate: The simple product of the two consequences of a targetable singleton computation, ${\Delta _k} = {\delta _k}\pi \; \cdot {\delta _k}S$ , with $\vec \Delta $ denoting the task-relevance vector of all representations (Fig. 1A). Thus, a representation is only task-relevant if applying additional computations can readily improve its approximation and doing so informs decisions.

Figure 1.

(Belledonne et al. Reference Belledonne, Butkus, Scholl and Yildirim2025). (A) Adaptive computation ratios perceptual computations to approximate object representations in a world model $S = \left[ {{s_1}, …,{s_N}} \right]$ , which in turn informs decision-making outcomes $\pi $ . Adaptive computation quantifies the task-relevance of a given object ${s_k}$ with a general formulation: ${\Delta _k} = {\delta _k}\pi \cdot {\delta _k}S$ , that integrates the impact of applying perceptual computation to that object ( ${C_k}$ ) on decision outcomes ( $\delta \pi $ ) and improvements in the world model ( ${\delta _k}S$ ). Thus, an object is task-relevant if rationing additional perceptual computations can both readily improve the object’s representation and doing so further inform decision-making. (B) Applied to a case study of MOT, adaptive computation generates intuitive patterns of object-level attention, with subsecond temporal resolution – all without the need for attention to explicitly access low-level details of perception or decision making (e.g., the distance between targets or distractors). (C) Probe detection adaptation of the standard MOT task to measure attention patterns throughout a single trial. In addition to the main task of target tracking, the motion phase of a trial also includes probes that briefly appeared on an object, with detection rates providing a measure of momentary attention. The patterns of task-relevance generated by adaptive computation strongly explained variability in human probe detection rates.

In order to be useful, attention must tractably estimate $\vec \Delta $ to ration imminent perceptual computations. The formulation of $\vec \Delta $ affords a simple estimation procedure with little overhead: aggregating the impact of recently applied computations across representations.

Accordingly, in a three-stage cycle, adaptive computation continually updates task-relevance to ration perceptual computations across objects and moments. The first stage involves a small amount of (i.e., quick) preattentive processing to initialize perceptual representations and decisions. In the second stage, adaptive computation uses the current $\vec \Delta $ to ration the total number of computations to use at that cycle (based on $\left| {\left| {\vec \Delta } \right|} \right|$ ) and to distribute these computations across representations (based on softmax $\left( {\vec \Delta } \right)$ ), yielding a goal-conditioned approximation of $S$ . The third stage aggregates the impacts of the rationed computations to update $\vec \Delta $ for the next cycle.

Every trial is an experiment unto itself

In Belledonne et al., (Reference Belledonne, Butkus, Scholl and Yildirim2025), we implemented and applied an adaptive computation architecture to the domain of MOT, explaining novel aspects of attentional dynamics.

Although adaptive computation only has access to the abstract measures of $\delta \pi $ (how much decisions change) and $\delta S$ (how much object representations change), the architecture can recover rich and intuitive patterns of processing. For example, a target approaching a distractor (Fig. 1B, bottom left) will be task relevant ( ${\Delta _k} \gt0$ ) as additional computations will resolve the object’s representation ( ${\delta _k}S\; \gt \;0$ ) and such updates will in turn alter confidence over selecting targets ( ${\delta _k}\pi \; \gt \;0$ ). However, two lone targets approaching each other (Fig. 1B, top right) would not warrant additional computations, as doing so would rarely alter target designation confidence (swapping object identities between targets does not impact expected performance). Critically, these object-level patterns of adaptive computation (determined by softmax $\left( {\vec \Delta } \right)$ ) match the subsecond attentional dynamics in humans (measured via probe detection during the tracking phase; Fig. 1C).

Moreover, as difficulty fluctuates (e.g., via increasing object counts and speeds), the total number of computations exerted (determined by $\left| {\left| {\vec \Delta } \right|} \right|$ ) provides a computational account for why some trials feel more effortful than others: we find that those trials that humans subjectively judge more effortful are those in which adaptive computation rations more perceptual computation.