Wilson et al. aim “to sketch a basic science of intentional change centered on evolution.” They warn that a critical issue is the lack of a conceptual unification among the involved disciplines. In the same spirit, we believe it is worthwhile drawing attention to a pair of unmentioned and related research areas that have addressed the issue of change in machine learning, artificial intelligence (AI), and developmental studies – namely, intrinsic motivations (IM) and hierarchical open-ended architectures (HOA). Both areas are closely related to the evolutionary psychology (EP) versus standard social science model (SSSM) debate and endorse the SSSM-related “open-ended capacity for change” in contrast to the “elaborate innateness” of EP.

To recap, the main connections between IM/HOA and intentional change are the following. First, thanks to their design-oriented approach, IM/HOA help us to understand the key notions of intentional change and open-endedness. Second, they clarify the details of the Darwin machine whose stability is crucial for “positive intentional change.” Third, likewise to Wilson et al., both IM and HOA address the issue of learning new motivations.

The notion of intrinsic motivation aims to model doing something because it is inherently interesting or enjoyable (Deci & Ryan Reference Deci and Ryan1985). In developmental psychology, there has been considerable interest in IM as a way to add new goals (Barto Reference Barto, Baldassarre and Mirolli2013; Mirolli & Baldassarre Reference Mirolli, Baldassarre, Baldassarre and Mirolli2013). Is intrinsically motivated behavior the result of innate metacognitive rules such as maximizing novelty, surprise, curiosity, and exploration (Berlyne Reference Berlyne1966; Dayan & Balleine Reference Dayan and Balleine2002; Dember & Earl Reference Dember and Earl1957; Dickinson & Balleine Reference Dickinson, Balleine and Gallistel2002)? Or the outcome of innate motivational open-endedness (Barto et al. Reference Barto, Singh and Chentanez2004; Manzotti & Tagliasco Reference Manzotti and Tagliasco2005)? Both options are currently being scrutinized (Georgeon et al. Reference Georgeon, Marshall and Gay2012; Manzotti Reference Manzotti and Poli2010; Oudeyer & Kaplan Reference Oudeyer and Kaplan2007). The objective of these approaches is to apply traditional learning paradigms (respondent and operant conditioning) to explain how an intentional cognitive agent may produce a new goal. Hence, there is a critical difference between being motivated (either intrinsically or extrinsically) and being able to develop new goals. The notion of IM encourages considering intentional change from a learning standpoint. Motivational theories address learning and decision making.

In both machine learning and AI, reinforcement learning (RL) has modeled with success motivational processes. RL “addresses how predictive values can be learned, it is naturally relevant to the study of motivation” (Barto Reference Barto, Baldassarre and Mirolli2013, p. 19). Yet, RL seldom considered the issues of generation and evolution of reward functions (Duda et al. Reference Duda, Hart and Stork2001; Sutton & Barto Reference Sutton and Barto1998). As a result, RL has not focused enough on the Kantorian key notions of evolution and devolution of stimulus functions (Kantor Reference Kantor1958). Usually, RL assumes that the reward function is given inside a module – the critic – whose nature is left unspecified. In RL, the theoretical point of departure is the study of IM. In fact, IM extend the original paradigm since intentional change may be seen as the result of teleological open-endness (Baldassarre & Mirolli Reference Baldassarre and Mirolli2013; Barto et al. Reference Barto, Singh and Chentanez2004). Furthermore, convincing experimental evidence about the neural underpinnings of IM has been put forward (Gottfried et al. Reference Gottfried, O'Doherty and Dolan2003; O'Doherty et al. Reference O'Doherty, Kringelbach, Rolls, Hornak and Andrews2001). In IM, a revised and open notion of RL's critic is the abstraction representing where and how new goals become part of the agent teleological structure. The notion of IM helps to complete a proper basic scientific foundation for an applied science of intentional change.

We have therefore reached the second area of research we want to draw attention to. To achieve IMs, an agent must be able to represent unexpected external stimuli and states of affairs. Hierarchical open-ended architectures (HOA) are a promising option (Dileep Reference Dileep2008; Kurzweil Reference Kurzweil2012; Manzotti et al. Reference Manzotti, Mutti, Lee, Gini, Chella, Pirrone, Sorbello and Jóhannsdóttir2012; Sendhoff et al. Reference Sendhoff, Körner, Sporns, Ritter and Doya2009). HOAs stand for architectures designed to represent and to interact with a potentially unlimited hierarchy of external stimuli. In fact, if a system could not develop a hierarchically organized manifold of concepts, how could it develop new motivations? Motivational openness must be flanked by conceptual and perceptual openness. A system must be able to extend its stimulus repertoire to use it as a reward. Borrowing von Uexküll's Umwelt, a process of intentional change modifies the agent's Umwelt (von Uexküll Reference von Uexküll and Schiller1957). From a neural perspective, there is evidence that this capability is a by-product of the interplay between the neocortex and the basal ganglia (Daw & Doya Reference Daw and Doya2006; Doya Reference Doya1999). Interestingly, Doya (Reference Doya1999) maintains that the CNS, although genetically geared toward ontogenetic development, is teleologically open. Moreover, the three main learning paradigms – supervised, unsupervised, and reinforcement learning – seem to be largely unbiased by evolutionary fine-tuning. According to Doya, “the learning modules specialized for these three kinds of learning can be assembled into goal-oriented behaving systems” (Doya Reference Doya1999, p. 961). This idea runs afoul of the massively modular view of the mind (as is often the case with EP). Two considerations follow. First, the discussion between EP and SSSM may benefit from these models and from the resulting interpretation of neural data. Second, the relation between learning and goal generation is emphasized.

In sum, these findings, together with the HOA and the IM, detail the necessary requirements of the Darwin machine that Wilson et al. advocate as the necessary stable core of any intentional change. In our commentary, we emphasize the relation between the proposed forthcoming science of intentional change and other selected approaches that share key fundamental insights – namely, the exploitation of some models for goal generations (open-endedness, teleological openness, intrinsic motivations). Furthermore, pace Barkow et al. (Reference Barkow, Cosmides and Tooby1992), domain-general learning does not appear any longer a theoretical impossibility. On the contrary, many scholars are working on domain-general cognitive architectures (Dileep Reference Dileep2008; Doya Reference Doya1999; Horton & Adams Reference Horton and Adams2005; Kurzweil Reference Kurzweil2012; Markram Reference Markram2006; Sendhoff et al. Reference Sendhoff, Körner, Sporns, Ritter and Doya2009). However, these models may shed a new light on why “our ability to change our behavioral and cultural practices lags far behind our ability to manipulate the physical environment” (sect. 1, para. 2). By integrating these approaches with their own, Wilson et at. may strengthen their case and gain a deeper understanding of the basic science of intentional change.