Learning about the causal structure of the world is a fundamental problem for human cognition, and causal knowledge is central to both intuitive and scientific theories. Over the last twenty years or so, cognitive scientists have applied advances in our formal understanding of causation in philosophy and computer science to understand human causal learning. In particular, researchers have used the Causal Bayes Net formalism, which relates directed acyclic causal graphs to patterns of conditional probability, interventions, and counterfactuals in systematic ways (Pearl Reference Pearl2000 (Reference Pearl2009); Spirtes et al. Reference Spirtes, Glymour and Scheines2001). The formalism assumes an “interventionist” account of causation—roughly, variable X is causally related to variable Y if, holding all else constant, an intervention changing the value of X would lead to a change in the value of Y. It provides a natural way to describe both causal models and the patterns of data they generate. This allows the reverse Bayesian inference, at least in principle—from the pattern of data it should be possible to infer the causal model that was most likely to have generated that data. This approach to causal learning is part of a broader Bayesian approach to learning in developmental cognitive science (for a recent review see Ullman and Tenenbaum Reference Ullman and Tenenbaum2020). Empirically, causal learning in children is remarkably well captured by this formalism (for reviews see Gopnik and Wellman Reference Gopnik and Wellman2012; Gopnik and Bonawitz Reference Gopnik and Bonawitz2015; Goddu and Gopnik Reference Goddu and Gopnik2024). Given a pattern of data, even preschool children infer the correct causal hypothesis.

From a computational perspective, however, this kind of inference, like many other kinds of Bayesian inference, proves to be intractable—the space of possible hypotheses is too large. Various computational techniques have been used to try to deal with this problem, particularly sampling methods, and there is some evidence that young children use similar methods (Bonawitz et al. Reference Bonawitz, Denison, Gopnik and Griffiths2014a, 2014b). These methods have had some success, but the search problems remain challenging. Another approach might be to suggest that potential causal models are strongly constrained by prior innate “core knowledge” (e.g., Spelke Reference Spelke2022). Although this might help address the search problem, it also undermines one of the major advantages of causal learning. Causal learning is so valuable precisely because it allows us to go beyond innate knowledge and learn about causal structure that may be deeply counter intuitive—such as the causal structure of scientific theories.

Another challenge for Bayesian approaches involves the role of exploration, experimentation, and active learning. Children are not simply passive consumers of data. Instead from very early on they actively seek out evidence that is relevant to the causal problems they are trying to solve, and this exploration plays an important role in their solutions to those problems. There has recently been extensive and elegant work showing just how motivated children are to experiment and explore, and how intelligently they do so (e.g., Schulz Reference Schulz2012; Giron et al. Reference Giron, Ciranka, Schulz, van den Bos, Ruggeri, Meder and Wu2023). Similarly, in formal science, experimentation is the canonical way to discover causal structure. But, with a few exceptions, (e.g., Eberhardt and Scheines Reference Eberhardt and Scheines2007) there have not been computational accounts of how this kind of active intervention and experimentation takes place and how it allows causal inference. So, the cognitive science of causal learning seems to have reached a kind of impasse.

However, in parallel, an apparently quite different kind of learning mechanism—reinforcement learning (RL)—has become increasingly influential in neuroscience and computer science, particularly when combined with modern “deep” machine learning (Dayan and Balleine Reference Dayan and Balleine2002; Sutton and Barto Reference Sutton and Barto2018). For example, deep RL has been the key to Deep Mind’s accomplishments in mastering Go and Chess (Silver et al. Reference Silver, Hubert, Schrittwieser, Antonoglou, Lai, Guez and Lanctot2018).

As the philosopher James Woodward, among others, has pointed out, classical RL might be thought of as a very specific and narrow form of causal learning (Woodward Reference Woodward2007, Reference Woodward2021). In particular, it relates the specific actions an agent performs on the world—their interventions—to specific outcomes, in the form of external rewards. This basic structure is similar to the basic structure of causal relations on the interventionist account underlying the causal Bayes net formalism. Moreover, RL contrasts with other types of learning that are more purely associative, such as classical conditioning or many neural network learning mechanisms. Associative learning captures correlations between variables and allows predictions about those variables, but doesn’t allow a special role for interventions, and so is further removed from causal learning.

However, RL is much narrower in application than causal learning more generally, it typically applies only to the agent’s own actions and to the rewards that follow those actions. More importantly, the basic motivational structure of RL and causal learning are very different. Reinforcement learning is motivated by utilities—the attempt to maximize external rewards. Classically, causal learning is epistemically motivated—it involves approximating the true structure of the world. It is true that ultimately causal knowledge allows effective action and decision making and so increases utilities. But this is a long-term and indirect effect. In the Bayesian framework, decision making and utility calculations are layered on top of the fundamental epistemic project (as in “causal decision theory”). The reverse is true in RL; RL agents may try to learn about the environment but they only do so in service of the fundamental utility project—that is, they learn to maximize further rewards later.

Although RL methods can be very effective in relatively low-dimensional and fixed environments such as go or chess, especially with dense reward signals, they have much more difficulty in the high-dimensional and open-ended environments that are characteristic of human causal learning. In particular, RL systems face consistent difficulties in balancing exploitation—the fundamental utilitarian motivation of maximizing reward, and exploration—the more epistemic motivation of learning about the structure of the environment (e.g., Sutton and Barto Reference Sutton and Barto2018; Cohen et al. Reference Cohen, McClure and Yu2007). In the long run, exploration will lead to more effective action and reward, but it requires the agent to forego reward in the short run. These problems arise even in very simple “bandit tasks” where agents are only required to choose between actions with known or unknown outcomes. Choosing the known option allows an agent to be sure of reward, but the unknown option will be more informative and lead to more knowledge and so to more effective action in the long run.

One approach to solving these problems has been to propose systems that seek internal epistemic rewards in addition to the classical external rewards. These systems implement the intrinsic epistemic motivation of classical causal learning in an RL framework. Several types of intrinsic rewards have been proposed in the literature. They include a variety of curiosity-based rewards, particularly measures of information gain, entropy, and novelty (Oudeyer et al. Reference Oudeyer, Kaplan and Hafner2007; Schmidhuber Reference Schmidhuber2010; Pathak et al. Reference Pathak, Agrawal, Efros and Darrell2017). These rewards do seem to increase the exploratory efficacy of RL systems. Moreover, there is evidence that children and even infants also seek out such intrinsic rewards, particularly information gain (Schulz Reference Schulz2012; Kidd and Hayden Reference Kidd and Hayden2015).

However, it is still difficult to sufficiently constrain these systems. For example, a classic failure mode for systems that seek out information gain is the “noisy TV” problem (Schmidhuber Reference Schmidhuber2010). Such systems will be captured by random noise like TV static. Seeking novelty and information gain by themselves doesn’t seem to be adequate to understand the environment effectively in a way that supports action.

A different approach uses “empowerment” as an intrinsic epistemic reward (Klyubin et al. Reference Klyubin, Polani and Nehaniv2005; Salge et al. Reference Salge, Glackin and Polani2014). “Empowerment” originally emerged in the evolutionary computation and artificial life literature, but more recently it has been applied to a variety of RL problems (Du et al. Reference Du, Tiomkin, Kiciman, Polani, Abbeel and Dragan2020; Zhao et al. Reference Zhao, Abbeel and Tiomkin2021; De Abril and Kanai Reference De Abril and Kanai2018). In empowerment, an agent maximizes the mutual information between its actions and their outcomes, regardless of the reward value of those outcomes. In other words, the system is rewarded if variation in an action systematically leads to parallel variation in the outcome so that the value of the action predicts the value of the outcome. Simultaneously, the system is also rewarded for maximizing the variety of actions it takes, ensuring that it explores the widest range of possible actions and outcomes.

The result is an intrinsic motivation system that leads the agent to seek the maximum amount of control over its environment. By endowing RL agents with empowerment as an intrinsic reward those agents can explore and represent the environment more effectively. Rather than simply seeking out novelty or information, such a system will seek out exactly the relations in the world that involve the closest match between actions and outcomes—the most controllable relations. This means that it will discover the relations that will be most likely to be useful for a wide range of future goal-directed actions. If I discover, for example, that moving a stick systematically changes the position of objects that it contacts, I can later pick up the stick to draw an out of reach object toward me. In fact, infants seem to learn to use sticks in just this way (Uzgiris and Hunt Reference Uzgiris and Hunt1975).

Although they come from different traditions, I argue that causal learning and empowerment gain are intimately related. In particular, my hypothesis is that if an agent learns an accurate causal model of the world, they will necessarily increase their empowerment, and, vice versa, increasing empowerment will lead to a more accurate (if implicit) causal model of the world. This has implications for both cognitive science accounts of causal learning and for the metaphysics of causation.

This claim is rooted in the peculiar ontology of causation. A number of researchers have suggested that it is helpful to think of causal learning as an “inverse problem” (e.g., Gopnik et al. Reference Gopnik, Glymour, Sobel, Schulz, Kushnir and Danks2004). An inverse problem involves inferring the structure of the external world from the data that world generates. A classic example is the way a visual system infers the structure of the three-dimensional world from retinal images or pixels. Other examples might include the way that we infer beliefs and desires from behavior in “theory of mind,” infer neural structure from fMRI evidence in cognitive neuroscience, or infer atomic structure from cloud chamber traces. In all these cases we assume that there is, in fact, some single distinctive structure in the outside world that we are trying to reconstruct. A God’s-eye view of the universe would discover something corresponding to the 3-D structure or psychological structure or neural structure or atomic structure that we are trying to reconstruct. This structure is independent of the agents who are trying to understand it.

However, the most influential contemporary accounts of causation suggest a rather different ontological picture of causal relationships. The interventionist accounts that underlie causal Bayes nets (Woodward Reference Woodward2003; Pearl Reference Pearl2000 (Reference Pearl2009); Pearl and Mackenzie Reference Pearl and Mackenzie2018; Spirtes et al. Reference Spirtes, Glymour and Scheines2001) imply a different relation between agents and the world. The asymmetries between cause and effect that are central to interventionist accounts of causation are, notoriously, hard to be found in physics, at least at the micro level. Instead, our notions of causation are rooted in the idea that causal relations are precisely those external relations that support an agent’s interventions. Philosophers and computationalists in this tradition define causal relations as those relations such that intervening to alter the value of a cause variable will lead to a corresponding change in the effect variable—in short, those relations where actions predictably produce outcomes. The ideal interventions that underpin causal inference are not identical to the intentional actions of actual agents but they are closely related, and in many circumstances agents’ intentional actions will also serve as ideal interventions (see Woodward Reference Woodward2021) and so pick out causal relationships in the world. Discovering such relations between interventions and outcomes is also the fundamental idea behind empowerment. But there may not be any single God’s-eye view agent-independent causal structure analogous to, say, 3-D spatial structure or atomic structure.

The philosopher Peter Godfrey-Smith (Reference Godfrey-Smith2010), among others, has argued for “causal pluralism”—in which many ontologically disparate phenomena can all support causal interventions. On this view there is nothing analogous to the spatial structure of the 3-D world in the causal case. Rather there are many quite different relationships in the world, ranging from the intuitive physical relations that support “billiard-ball” causation to the belief-desire relationships of intuitive psychology, to the highly counterintuitive relations of relativistic physics, that all happen to systematically support causal interventions. Again, these are precisely the relations of control that would produce the greatest increases in empowerment. Maximizing empowerment will lead to the discovery of causal relations, and vice-versa.

A further distinctive feature of empowerment, with implications for the metaphysics of causation, is that it can have both “mind to world” and “world to mind” directions of fit. It is possible to maximize an agent’s empowerment by increasing its knowledge about how the world works (matching the mind to the world as in science). This is the classic picture of causal learning. However, it is equally possible to maximize empowerment by increasing an agent’s skill and control over the world (matching the world to the mind as in engineering). This is more like the orientation of classic RL.

These engineering cases point to other distinctive features of the metaphysics of causation. In these cases, we arguably create causation as well as discovering it. Human artifacts are a good example of this “world to mind” causal fit. Designing a gas pedal imposes a relation of mutual information between the pedal and the car acceleration that did not exist before the car was engineered. Arguably, also, it creates a causal relationship between the pedal and the acceleration that did not exist before. This is even true for simpler forms of tool use like using a stick to obtain an out of reach object. The stick lying on the forest floor might not have any causal relation to toy retrieval, but the stick as a tool in the hands of a toddler does.

Something similar happens in the relation between norms and “theory of mind.” By imposing a new social norm or convention we can introduce novel causal relations to the psychological as well as the physical world. By sending a single email that class starts at 4:00 in room 100 I can miraculously and precisely cause a hundred separate people to converge at just that place and time.

However, whether we think in terms of science or engineering, the basic structure of causal relations will be the same, intervene systematically on X to systematically change the value of Y. This is well captured by the notion of empowerment. Of course, in more abstract and conceptual cases, the interventions may be theoretical rather than actual—to say that the moon causes the tides is to say that if we could alter the position of the moon the tides would also alter, even if this intervention isn’t possible. But, as Woodward argues (Reference Woodward2003, Reference Woodward2021), if a relation is genuinely causal this sort of intervention should at least be conceptually possible. Moreover, this distinguishes causal relationships from other relationships such as spatial, geometric, or logical relationships. And, in practice, the test for whether we have discovered causal relations is to perform experiments—to determine whether experimentally varying one variable will systematically predict the value of another, that is, precisely to look for high mutual information between interventions and outcomes.

Thinking about empowerment might also help us understand the psychology of causal learning. A recent paper suggests that adult’s exploration of a video game-like environment can be well characterized by empowerment (Brändle et al. Reference Brändle, Lena Stocks, Gershman and Schulz2023) and we have shown that this is also true for children (Du et al. Reference Du, Kosoy, Dayan, Rufova, Abbeel and Gopnik2023; Yiu et al. Reference Yiu, Allen, Ginosar and Gopnik2026). But the empowerment approach more generally captures important features of early causal knowledge and learning and helps to explain a wide range of developmental findings.

Looking-time studies suggest that very young infants perceive some particular relations in intuitive physics that support causal inference, such as the relations of movement and collision in “billiard-ball” causality (Leslie Reference Leslie1982). However, the development of causal concepts more broadly is initially closely linked to actual goal-directed actions on the world and their outcomes. Goddu and Gopnik (Reference Goddu and Gopnik2024) following Woodward (Reference Woodward2007, Reference Woodward2021) suggest that both in phylogeny and ontology, causal understanding moves from a first-person perspective to a third-person perspective to an impersonal perspective. Reinforcement learning, which is found in almost all intelligent animals, represents a causal relation between the animal’s own actions and their outcomes. Imitation learning, which is found in some nonhuman animals in limited ways, but is ubiquitous in human infants, represents a causal relation between other animals’ actions and outcomes. The sort of impersonal causal understanding in science represents causal relations in the world independent of actual actions, though crucially supporting such actions in principal. Empowerment may be applied to all three types of relations.

In the 1970s, and interestingly in the context of thinking about operant conditioning, a series of papers suggested that even very young infants are indeed rewarded by something like empowerment. In classic studies of “conjugate reinforcement,” Rovee-Collier (1979) tied a ribbon from a crib mobile to the infant’s foot, so that kicking made the mobile move. Infants as young as three months old systematically acted to make the mobile move, varying their actions and observing the correlation between those actions and the behavior of the mobile. There were similar results in studies where infants could make a mobile move or activate a pattern of lights by turning their heads on a pressure sensitive pillow (Watson Reference Watson1972; Papousek and Papousek Reference Papoušek and Papoušek1975). Moreover, these actions could not simply be explained by classic RL with the mobile’s motion as a reward. Infants would learn to turn their heads or kick and would continue to act to do so, even though they no longer looked at the mobile or the lights, aside from a brief glance to check that their action was effective. Infants varied their actions and observed their results rather than simply converging on a single effective action. In addition, infants smiled and cooed when their actions consistently led to an effect, but not when that effect simply occurred independently of their actions (Watson 1972). A more recent study with this methodology shows that infants consistently alter their actions on the mobile in a way that increases the contingency of their actions, again unlike classical RL (Sloan et al. Reference Sloan, Jones and Kelso2023). In fact, Rovee-Collier described her results precisely as an empowerment reward: “The control which the infants have gained over the consequences of their own actions seems to have become the reward, rather than the specific consequences per se.” (Rovee-Collier and Gekoski Reference Rovee-Collier and Gekoski1979).

Later in infancy infants even more clearly act on intermediate objects to maximize empowerment—looking for contingencies between potential tools and outcomes, and at least implicitly appreciating the causal relations between them. Piaget (Reference Piaget and Cook1952) described “tertiary” circular reactions in infancy. In these behaviors, which emerge at around a year, infants systematically vary their actions on objects to produce varied outcomes. Again, Piaget explicitly described this behavior in terms of experimentation and suggested that it was a precursor to causal understanding, though for complicated reasons he denied that children had genuine causal understanding until well into the school-aged years.

In “conjugate reinforcement” infants are acting to maximize the empowerment of their own actions—their causal understanding has a first-person perspective. We know that from early in infancy children also represent the goal-directed actions of others and distinguish them from other kinds of events and movements (e.g., Woodward Reference Woodward1998, Reference Woodward2009). Moreover, they map the goal-directed actions of others on to their own actions (Meltzoff Reference Meltzoff2006). This is an important feature of human causal learning. It distinguishes it from other types of learning, such as classic RL, which only concern an agent’s own actions, and also distinguishes it from learning in other animals (Taylor et al. Reference Taylor, Cheke, Waismeyer, Meltzoff, Miller, Gopnik, Clayton and Gray2014). From early in life, then, children have the cognitive and conceptual structures in place to discover empowerment relations between actions and outcomes, both their own and those of others.

From at least twenty-four months and probably earlier, children make genuine causal inferences by observing the correlations between goal-directed actions—their own or others—and outcomes. However, until around age four, they do not make similar inferences from simple correlations between events (Waismeyer and Meltzoff Reference Waismeyer and Meltzoff2017; Bonawitz et al. Reference Bonawitz, Ferranti, Saxe, Gopnik, Meltzoff, Woodward and Schulz2010; Meltzoff et al. Reference Meltzoff, Waismeyer and Gopnik2012). Suppose a twenty-four-month-old sees a human hand repeatedly push a toy car against a block A, which causes a light to go on. Pushing the car against another block B does not have this effect. Now we ask the infant to make the light appear. Infants will reproduce the correct action on A to make the light go, but not the action on B. However, they will not do this if they simply see the car move on its own and cause the effect. This is true even though they will look toward the light in this condition, suggesting that they have learned the correlation between the motion of the car and the light (Meltzoff et al. Reference Meltzoff, Waismeyer and Gopnik2012). In short, toddlers appear to detect empowerment relations between actions and outcomes and use those relations to infer causal relationships that determine their own future interventions. They do not do this based on correlations among events that do not involve actions and outcomes. Four-year-olds do infer new interventions from correlations alone, but this ability seems to depend on their earlier learning through goal-directed action.

Empowerment also naturally applies to children’s early exploratory play (Chu and Schulz Reference Chu and Schulz2020). Even infants characteristically play by varying their actions on an object and observing the results—hence the perennial popularity of toys like rattles and busy boxes that afford such empowering actions. Empowerment-based RL, unlike Bayesian inference, also provides a natural way to characterize such experimental actions, they are precisely what you would expect from a system that was trying to act to maximize empowerment.

Thinking of causal learning in terms of empowerment may also help to resolve some of the search problems. Maximizing empowerment would not require the sort of search through a high-dimensional hypothesis space that is so challenging for Bayesian inference. Unfortunately, precisely calculating mutual information poses problems of tractability—but some very recent approximation methods make such calculations more feasible (e.g., Zhao et al. Reference Zhao, Abbeel and Tiomkin2020). Children might also begin by simply looking for correlations between actions, their own and others, and the outcomes that follow them, rather than fully calculating mutual information.

If children are maximizing empowerment, they would have a mechanism for independently discovering causal relations that are not specified innately, even without requiring the full apparatus of Bayesian causal inference. They might look for relations that have the feature of mutual information between their own actions and those of others and outcomes, like the relations between sticks and toys. This might then allow them to build up a repertoire of basic causal arrows that can then be combined to build more complex models.

In sum, I argue here that recent work on “empowerment” may help bridge Bayesian and RL approaches to learning and provide both empirical and theoretical insight into the crucial problem of learning the causal structure of the world.