Why and how animals increase energy reserves in uncertain environments has been overdue wider attention. We applaud the effort of Anselme & Güntürkün in synthesizing a novel theory seeking to unify several distinct literatures. We comment along three frontiers: the normative, the controllable, and the algorithmic.
How Foraging Works (HFW) is predicated on evolutionary assumptions, whereby energetic uncertainty should promote foraging policies that minimize starvation risk. This line of theorizing is welcome, but would benefit from more rigor. Their definition of uncertainty (sect. 1, para. 2) is that a trial or food acquisition attempt is rewarded on a random basis, independent of the time spent in that environment. They state that this uncertainty yields an inability to predict whether the next trial is rewarded (sect. 1, para. 2) and that this uncertainty is maximal when that probability is 0.5 (sect. 3.2, para. 1). First, the time independence is a questionable clause to insert into a definition of uncertainty, in that in many settings, foraging actions do interact with future probabilities of food acquisition (e.g., as finite resources are depleted). Perhaps this aspect of the definition is better considered a simplifying assumption than a definition. Second, the deeper issue we have with this treatment of uncertainty is that it is too narrow. Ecologies are causal networks with hierarchical structure, evolving over multiple timescales. The intelligent agent has to evolve, grow, and update its own hierarchical generative model of this environment to deploy the foraging decisions necessary for it to stay alive. For instance, to maximize reward rates, such an agent needs to compute expectations, variances, and labilities of reward variables over multiple timescales. Suppose patch #1 affords a mean energetic rate of 5 calories per hour and the probability of reward is 0.5 on any single foraging encounter (e.g., with a conditioned stimulus). The agent will be unable to predict the next outcome of the next encounter, yielding uncertainty over trials as defined above. Despite this trial-level uncertainty in the outcome, the intelligent agent can still compute precise expectations with respect to the mean energetic rates over longer timescales. This would be a form of expected uncertainty, in which reward rates are predictable over longer timescales, but unpredictable over short timescales. By contrast, consider patch #2, which comprises highly predictable cues, that currently delivers the same mean energetic rate but with a reward probability of 0.9 (i.e., by delivering less energy per reward). Suppose that the agent has a less precise estimate of the mean energetic rate, which, because of its volatility, jumps above and below the agent's minimal metabolic needs. This would be a form of unexpected uncertainty, in which outcomes are relatively predictable over short timescales but unpredictable as energetic rates over longer timescales. Given that patch #1 yields a higher cumulative survival probability, it has a higher fitness value than patch #2. Thus, all else being equal, the agents should normatively prefer to occupy patch #1. If the agent was unable to control which patch it occupies, what should it do with respect to their foraging vigor and consumption while within these patches? Because of its volatility, the agent should increase its foraging efforts and consumption in patch #2, but not in patch #1, even though patch #1 is the more uncertain and less predictable patch, under the HFW definition. The problem is that unless one is more nuanced about the hierarchical structure and multiple timescales of ecological uncertainty, one becomes easily tangled in these apparent contradictions. Ultimately, this hierarchical and multiscale uncertainty must be scored by something deeper than the narrow uncertainty articulated in HFW. This exercise is potentially important, because without a deeper conception of uncertainty, the model presented in HFW is vulnerable to being “falsely falsified” by empirical observations such as these, even though, at the deepest level, we think it merits remaining as a candidate theory for the field.
Another fundamental dimension missing from HFW is the inference of environmental controllability. Increasing foraging effort is only advantageous if the effort is likely to be profitable over life-relevant timescales; otherwise it wastes energy and shortens the odds on starvation. Complex organisms are typically able to estimate whether their foraging policies are hopeless, and thus whether they are better off conserving energy through inaction, until greater control is (hopefully) restored.
The functional forms of the HFW model appear to be ad hoc, or at least no deeper justification is provided. So for instance, linking wanting to energy level is described in Equation 1 (Online Supplement) as a power law. But why a power law? The concavity of this function is biologically implausible. For instance, for each unit loss of energy, the marginal increase in wanting decreases. This is the opposite of what should be expected if motivation is to optimize survival. As you get closer to the boundary of death, you need motivation to marginally increase, not decrease, in line with the fact that the fitness costs of losing each unit of energy are marginally increasing as energy reserves decrease. Likewise, in Equation 4 (Online Supplement), there is no specific justification for its functional form, above the plausible justification that wanting, uncertainty and scarcity all matter to incentive hope. Without justification, the reader is left wondering whether the observed behaviors are specific to this seemingly ad hoc formulation or whether they are general to a particular class of function. We are not against using simplified assumptions where necessary to make a model work; however, there are several equations that seem fundamental to the meaning and rationale of the theory, but that seem to be arbitrarily contrived or without deeper justification. This speaks to the broader point of designing the model to optimize survival. Consideration of survival optimality could provide crucial constraints in choosing the functional forms of the equations, because for any given survival probability function linking energy state to survival, there will be specific (potentially unique) wanting and foraging motivation functions that are survival optimal. Indeed, the HFW model has approximated a very simple environment in which this normativity analysis could be quite tractable. We look forward to seeing future foraging models inspired by HFW that can address these challenges.
Why and how animals increase energy reserves in uncertain environments has been overdue wider attention. We applaud the effort of Anselme & Güntürkün in synthesizing a novel theory seeking to unify several distinct literatures. We comment along three frontiers: the normative, the controllable, and the algorithmic.
How Foraging Works (HFW) is predicated on evolutionary assumptions, whereby energetic uncertainty should promote foraging policies that minimize starvation risk. This line of theorizing is welcome, but would benefit from more rigor. Their definition of uncertainty (sect. 1, para. 2) is that a trial or food acquisition attempt is rewarded on a random basis, independent of the time spent in that environment. They state that this uncertainty yields an inability to predict whether the next trial is rewarded (sect. 1, para. 2) and that this uncertainty is maximal when that probability is 0.5 (sect. 3.2, para. 1). First, the time independence is a questionable clause to insert into a definition of uncertainty, in that in many settings, foraging actions do interact with future probabilities of food acquisition (e.g., as finite resources are depleted). Perhaps this aspect of the definition is better considered a simplifying assumption than a definition. Second, the deeper issue we have with this treatment of uncertainty is that it is too narrow. Ecologies are causal networks with hierarchical structure, evolving over multiple timescales. The intelligent agent has to evolve, grow, and update its own hierarchical generative model of this environment to deploy the foraging decisions necessary for it to stay alive. For instance, to maximize reward rates, such an agent needs to compute expectations, variances, and labilities of reward variables over multiple timescales. Suppose patch #1 affords a mean energetic rate of 5 calories per hour and the probability of reward is 0.5 on any single foraging encounter (e.g., with a conditioned stimulus). The agent will be unable to predict the next outcome of the next encounter, yielding uncertainty over trials as defined above. Despite this trial-level uncertainty in the outcome, the intelligent agent can still compute precise expectations with respect to the mean energetic rates over longer timescales. This would be a form of expected uncertainty, in which reward rates are predictable over longer timescales, but unpredictable over short timescales. By contrast, consider patch #2, which comprises highly predictable cues, that currently delivers the same mean energetic rate but with a reward probability of 0.9 (i.e., by delivering less energy per reward). Suppose that the agent has a less precise estimate of the mean energetic rate, which, because of its volatility, jumps above and below the agent's minimal metabolic needs. This would be a form of unexpected uncertainty, in which outcomes are relatively predictable over short timescales but unpredictable as energetic rates over longer timescales. Given that patch #1 yields a higher cumulative survival probability, it has a higher fitness value than patch #2. Thus, all else being equal, the agents should normatively prefer to occupy patch #1. If the agent was unable to control which patch it occupies, what should it do with respect to their foraging vigor and consumption while within these patches? Because of its volatility, the agent should increase its foraging efforts and consumption in patch #2, but not in patch #1, even though patch #1 is the more uncertain and less predictable patch, under the HFW definition. The problem is that unless one is more nuanced about the hierarchical structure and multiple timescales of ecological uncertainty, one becomes easily tangled in these apparent contradictions. Ultimately, this hierarchical and multiscale uncertainty must be scored by something deeper than the narrow uncertainty articulated in HFW. This exercise is potentially important, because without a deeper conception of uncertainty, the model presented in HFW is vulnerable to being “falsely falsified” by empirical observations such as these, even though, at the deepest level, we think it merits remaining as a candidate theory for the field.
Another fundamental dimension missing from HFW is the inference of environmental controllability. Increasing foraging effort is only advantageous if the effort is likely to be profitable over life-relevant timescales; otherwise it wastes energy and shortens the odds on starvation. Complex organisms are typically able to estimate whether their foraging policies are hopeless, and thus whether they are better off conserving energy through inaction, until greater control is (hopefully) restored.
The functional forms of the HFW model appear to be ad hoc, or at least no deeper justification is provided. So for instance, linking wanting to energy level is described in Equation 1 (Online Supplement) as a power law. But why a power law? The concavity of this function is biologically implausible. For instance, for each unit loss of energy, the marginal increase in wanting decreases. This is the opposite of what should be expected if motivation is to optimize survival. As you get closer to the boundary of death, you need motivation to marginally increase, not decrease, in line with the fact that the fitness costs of losing each unit of energy are marginally increasing as energy reserves decrease. Likewise, in Equation 4 (Online Supplement), there is no specific justification for its functional form, above the plausible justification that wanting, uncertainty and scarcity all matter to incentive hope. Without justification, the reader is left wondering whether the observed behaviors are specific to this seemingly ad hoc formulation or whether they are general to a particular class of function. We are not against using simplified assumptions where necessary to make a model work; however, there are several equations that seem fundamental to the meaning and rationale of the theory, but that seem to be arbitrarily contrived or without deeper justification. This speaks to the broader point of designing the model to optimize survival. Consideration of survival optimality could provide crucial constraints in choosing the functional forms of the equations, because for any given survival probability function linking energy state to survival, there will be specific (potentially unique) wanting and foraging motivation functions that are survival optimal. Indeed, the HFW model has approximated a very simple environment in which this normativity analysis could be quite tractable. We look forward to seeing future foraging models inspired by HFW that can address these challenges.