Hoerl & McCormack (H&M) argue that mature humans rely on a temporal reasoning system to represent particular times, temporal order, and tense. By contrast, they claim that nonhuman animals and human infants rely exclusively on a temporal updating system, which represents change by replacing one atemporal representation with another. As H&M note, this entails that human infants and nonhuman animals only represent the world as present. We find their dual systems approach promising. But we are apprehensive about their claim that temporal updating utilizes only atemporal representations; this claim generates puzzles for the phenomenology of temporal experience, and it conflicts with the most promising models of behavioral learning.
There are difficult philosophical questions about how the experience of temporality or flow is related to the structures that produce such experiences (Ismael Reference Ismael, Bouton and Huneman2017). One common way to account for the temporality of conscious experience is by arguing that each momentary phase includes a trace of the previous experience and an anticipation of what is about to occur (Husserl Reference Husserl and Brough1917/1991). It is claimed that these retentive and anticipatory features make the unification of conscious experience and the continuity of experienced objects possible. If a moving object were experienced only as present, each momentary state would feature a static object, with nothing to bind these states together as an experience of ongoing motion (Gallagher Reference Gallagher, Dyke and Bardon2013). Likewise, if the temporal parts of an object were viewed sequentially and statically, there would be nothing to integrate them into an experience of a unified object. Finally, if an infant could not track its recent behavior, while orienting toward a future state, their sensorimotor capacities (e.g., hand-mouth coordination) would go unexplained (Gallagher Reference Gallagher and Callendar2011). But if traces of the immediate past and anticipations of the immediate future are intentionally available in our present experience (Thompson Reference Thompson2007, p. 320), it becomes easier to see how sequences of momentary experiences are organized to yield the experience of succession and duration. Temporally extended objects are experienced as persisting across multiple phases of a conscious process. Change occurs when there is variation across a process. And we experience surprise when a process fails to unfold as we expect it to.
Focusing on perceptual phenomena, Hoerl (Reference Hoerl2013a; Reference Hoerl2013b) argues that experienced movement and change can be explained by successive relations between an organism and worldly events, because worldly events already have the requisite temporal structure. Philosophical considerations and data from psychophysics provide some support for this extensionalist proposal. However, considerations from the literature on reinforcement learning suggest that goal-directed behavior also requires cognitive processes that track the temporal dynamics of the world (cf. Petter et al. Reference Petter, Gershman and Meck2018). Sometimes, animals track evidence that now is a good time to forage; sometimes they recognize that a current situation is risky; and sometimes, they track the succession of different actions that will eventually lead to the acquisition of a reward. And in each of these cases, expectations about which actions will be followed by reward are fundamental to survival. And this means that animals need some way to contextualize momentary models of the world within dynamic and unfolding cognitive processes.
As H&M argue, a raven could represent an apparatus as something that “can be opened with a tool,” and a tool as useful for a non-present task, without engaging in temporal reasoning. But doing so would require representing these objects as persisting, representing these actions as part of temporally extended events, understanding their relative value, and deciding whether to act on the basis of this information. Over the past two decades, several solutions to this representational problem have emerged. Model-free systems compute forward-looking predictions, track discrepancies between experienced and predicted rewards, and adjust future predictions to accommodate such discrepancies. Model-based strategies store a model of the world that specifies when a sequence of actions is expected to yield reward and compute decisions on this basis. Finally, recent models relying on successor representations suggest an intermediate class of systems that cache “long-term predictions about the states it expects to visit in the future” (Momennejad et al. Reference Momennejad, Russek, Cheong, Botvinick, Daw and Gershman2017, p. 681). Following H&M, each of these capacities can be implemented by a system that dynamically updates its models of the world in light of new information. But each kind of system relies on forward-looking expectations about which actions are likely to be rewarded, as well as retained representations of what has worked in the past (Niv Reference Niv2009); moreover, given the dynamics of our world, these expectations must be sensitive to temporal differences, as well as temporal relations between stimuli (Gershman et al. Reference Gershman, Moustafa and Ludvig2014; Luzardo et al. Reference Luzardo, Alonso and Mondragón2017; Petter et al. Reference Petter, Gershman and Meck2018).
Precisely how violations of expectations are experienced is a difficult matter, which we cannot address here. However, on the assumption that cognition frequently unfolds in the service of guiding action, most animals should possess the capacities required to track where they are in various ongoing processes, anticipate the evaluatively significant aspects of their actions, and adjust their behavior where things do not go as planned. Furthermore, when they update their models of their world, they should do so in ways that evoke changes in retentive content as well as changes in expectations. H&M could accept this form of temporal directionality by adopting a position midway between the phenomenologically grounded perspectives discussed above and the hypothesis articulated in the target article. Such an approach would permit a gap between mature forms of temporal reasoning and temporal updating; but instead of decomposing representations of processes into representations of states, it would focus on cognitive representations of events. And because events have a minimal temporal structure, it would be well poised to organize and integrate these representations in a way that would yield an experience of temporal flow. Consequently, this would yield a dual systems hypothesis that could account for considerations regarding both experience and behavior. This seems like a minimal amendment, as every cognitive state that spans a short interval is “embedded in a psychological context with very long scope, both in the forward and backward direction” (Ismael Reference Ismael, Bouton and Huneman2017, p. 25).
Hoerl & McCormack (H&M) argue that mature humans rely on a temporal reasoning system to represent particular times, temporal order, and tense. By contrast, they claim that nonhuman animals and human infants rely exclusively on a temporal updating system, which represents change by replacing one atemporal representation with another. As H&M note, this entails that human infants and nonhuman animals only represent the world as present. We find their dual systems approach promising. But we are apprehensive about their claim that temporal updating utilizes only atemporal representations; this claim generates puzzles for the phenomenology of temporal experience, and it conflicts with the most promising models of behavioral learning.
There are difficult philosophical questions about how the experience of temporality or flow is related to the structures that produce such experiences (Ismael Reference Ismael, Bouton and Huneman2017). One common way to account for the temporality of conscious experience is by arguing that each momentary phase includes a trace of the previous experience and an anticipation of what is about to occur (Husserl Reference Husserl and Brough1917/1991). It is claimed that these retentive and anticipatory features make the unification of conscious experience and the continuity of experienced objects possible. If a moving object were experienced only as present, each momentary state would feature a static object, with nothing to bind these states together as an experience of ongoing motion (Gallagher Reference Gallagher, Dyke and Bardon2013). Likewise, if the temporal parts of an object were viewed sequentially and statically, there would be nothing to integrate them into an experience of a unified object. Finally, if an infant could not track its recent behavior, while orienting toward a future state, their sensorimotor capacities (e.g., hand-mouth coordination) would go unexplained (Gallagher Reference Gallagher and Callendar2011). But if traces of the immediate past and anticipations of the immediate future are intentionally available in our present experience (Thompson Reference Thompson2007, p. 320), it becomes easier to see how sequences of momentary experiences are organized to yield the experience of succession and duration. Temporally extended objects are experienced as persisting across multiple phases of a conscious process. Change occurs when there is variation across a process. And we experience surprise when a process fails to unfold as we expect it to.
Focusing on perceptual phenomena, Hoerl (Reference Hoerl2013a; Reference Hoerl2013b) argues that experienced movement and change can be explained by successive relations between an organism and worldly events, because worldly events already have the requisite temporal structure. Philosophical considerations and data from psychophysics provide some support for this extensionalist proposal. However, considerations from the literature on reinforcement learning suggest that goal-directed behavior also requires cognitive processes that track the temporal dynamics of the world (cf. Petter et al. Reference Petter, Gershman and Meck2018). Sometimes, animals track evidence that now is a good time to forage; sometimes they recognize that a current situation is risky; and sometimes, they track the succession of different actions that will eventually lead to the acquisition of a reward. And in each of these cases, expectations about which actions will be followed by reward are fundamental to survival. And this means that animals need some way to contextualize momentary models of the world within dynamic and unfolding cognitive processes.
As H&M argue, a raven could represent an apparatus as something that “can be opened with a tool,” and a tool as useful for a non-present task, without engaging in temporal reasoning. But doing so would require representing these objects as persisting, representing these actions as part of temporally extended events, understanding their relative value, and deciding whether to act on the basis of this information. Over the past two decades, several solutions to this representational problem have emerged. Model-free systems compute forward-looking predictions, track discrepancies between experienced and predicted rewards, and adjust future predictions to accommodate such discrepancies. Model-based strategies store a model of the world that specifies when a sequence of actions is expected to yield reward and compute decisions on this basis. Finally, recent models relying on successor representations suggest an intermediate class of systems that cache “long-term predictions about the states it expects to visit in the future” (Momennejad et al. Reference Momennejad, Russek, Cheong, Botvinick, Daw and Gershman2017, p. 681). Following H&M, each of these capacities can be implemented by a system that dynamically updates its models of the world in light of new information. But each kind of system relies on forward-looking expectations about which actions are likely to be rewarded, as well as retained representations of what has worked in the past (Niv Reference Niv2009); moreover, given the dynamics of our world, these expectations must be sensitive to temporal differences, as well as temporal relations between stimuli (Gershman et al. Reference Gershman, Moustafa and Ludvig2014; Luzardo et al. Reference Luzardo, Alonso and Mondragón2017; Petter et al. Reference Petter, Gershman and Meck2018).
Precisely how violations of expectations are experienced is a difficult matter, which we cannot address here. However, on the assumption that cognition frequently unfolds in the service of guiding action, most animals should possess the capacities required to track where they are in various ongoing processes, anticipate the evaluatively significant aspects of their actions, and adjust their behavior where things do not go as planned. Furthermore, when they update their models of their world, they should do so in ways that evoke changes in retentive content as well as changes in expectations. H&M could accept this form of temporal directionality by adopting a position midway between the phenomenologically grounded perspectives discussed above and the hypothesis articulated in the target article. Such an approach would permit a gap between mature forms of temporal reasoning and temporal updating; but instead of decomposing representations of processes into representations of states, it would focus on cognitive representations of events. And because events have a minimal temporal structure, it would be well poised to organize and integrate these representations in a way that would yield an experience of temporal flow. Consequently, this would yield a dual systems hypothesis that could account for considerations regarding both experience and behavior. This seems like a minimal amendment, as every cognitive state that spans a short interval is “embedded in a psychological context with very long scope, both in the forward and backward direction” (Ismael Reference Ismael, Bouton and Huneman2017, p. 25).