Ainslie provides a comprehensive account of willpower that integrates recent findings with his theory of recursive self-prediction. This commentary develops his account by incorporating three new developments from decision neuroscience, reinforcement learning, habit formation, and response inhibition. These integrate well into a framework I call neurodemocracy (Huang, Reference Huang2017). These developments highlight that the key constructs interact in ways that are more symbiotic and dynamical than he recognizes.

The first development is the drift-diffusion model, according to which a decision mechanism accumulates decision variables until a threshold is met. His account, although compatible with this model, has not fully taken on board the dynamical nature it entails. This has significant implications if we take into account a second development: For any given decision, the decision variables are generated by a “society of controllers” (influenced by Minsky, Reference Minsky1986) of different sophistication, speed, and capacity. For example, a response can be evaluated by multiple model-based controllers using different algorithms as well as a variety of model-free and Pavlovian controllers (Dayan & Berridge, Reference Dayan and Berridge2014; Dolan & Dayan, Reference Dolan and Dayan2013; Gershman, Daw, Rabinovich, Friston, & Varona, Reference Gershman, Daw, Rabinovich, Friston and Varona2012; Ito & Doya, Reference Ito and Doya2011). One reason that the dynamical nature of decision-making is crucial is that reliable evaluations can take more time to generate because of the nature of, say, model-based controllers. Hence, a lower decision threshold will result in decisions being overly influenced by faster, but potentially less reliable controllers. (Note that this point is orthogonal to that of hyperbolic delay discounting and visceral factors discussed by Ainslie.) The third development is that deliberation involves the basal ganglia, a sophisticated decision mechanism that handles complex and novel decisions better than cortical mechanisms. I will provide more details next as I explain the cortico-striatal models of the key constructs.

To begin with, recent findings suggest a model of routine habits as decision-making transferred from the basal ganglia to cortical mechanisms (Hélie, Ell, & Ashby, Reference Hélie, Ell and Ashby2015). According to this model, novel decisions are made in the basal ganglia, which provide the reinforcement learning capacity that allows good decisions to be learned quickly. Through choosing the same response repetitively, the basal ganglia train the cortical mechanism through the slower Hebbian learning, which is sensitive to repetition but not to rewards. When the cortical connection is strengthened sufficiently (i.e., a strong stimulus–response connection is established), the decision-making is taken over by the cortex and becomes habitual. This new model explains better than the existing model why routine habit is stimulus-bound and insensitive to reward.

In addition, the framework of “a society of controllers” suggests that resolve (as well as routine habit and suppression) are implemented by multiple controllers simultaneously. Each controller adopts a strategy of varying explicitness and cognitive sophistication. This subserves the different conceptions of resolve discussed by Ainslie. Moreover, good habits feel less effortful because consistent resolve maintains a “resolve-context,” in which SS (smaller, sooner reward) is evaluated increasingly less favorably and LL (larger, later reward) more favorably. This is because, among other reasons, at least some evaluations are performed by model-based controllers sensitive to the likelihood of obtaining a reward in a given context. Because LL is consistently selected (and obtained) in the resolve-context, but not SS, the evaluation for LL increases. This can also explain why failure of resolve (and failure of suppression or routine habit) can lead to more temptations: it increases the likelihood of obtaining SS and hence the evaluation of SS.

Finally, the cortico-striatal model of response inhibition points to two types of suppression (Wiecki & Frank, Reference Wiecki and Frank2013). One involves the hyperdirect pathway of the basal ganglia, which increases the decision threshold as a result of receiving signals of response conflict from cortical controllers. It can prevent the formation of intention for SS when the accumulated decision variable for SS cannot reach the increased threshold. The other type of suppression involves the competition between the basal ganglia's direct pathway (which carries the positive decision variable, Go signal, for SS) and indirect pathway (which carries the negative, NoGo signal). Because a small NoGo signal can cancel out a larger Go signal in the basal ganglia, even a small increase in a NoGo signal for SS can potentially prevent the accumulated decision variables from reaching the decision threshold.

These models draw attention to how the dynamical interactions between different controllers implementing suppression, resolve, and routine habit reinforce each other. We can see this by looking at the relationship between routine habit and resolve. Successful resolve trains routine habit by choosing LL repetitively (this is true for suppression as well). When this routine habit becomes well-formed, LL is produced automatically under the resolve-context – we may see this routine habit as an integral part of a good habit. Routine habit also contributes to resolve. Resolve is a complex psychological construct composed of many component cognitive responses, some of which can become routine habits. The more its components become habitual, the more effective it becomes (the same point also applies to suppression).

The significance of the temporal dimension is also illuminated by examining the interaction between suppression and resolve: a faster suppression (especially, when some of its component responses become habitual) can provide a longer temporal window for resolve to work. Suppression, by preventing the formation of intention for SS, allows more time for Go signals for LL to be generated by resolve-implementing controllers, some of which are relatively capacity-limited and slow. When the accumulated decision variables for LL pass the threshold, LL is recommitted. At the same time, resolve also contributes to suppression. Some faster resolve-implementing controllers may generate initial NoGo signals for SS, which facilitate timely suppression through competition between direct and indirect pathways. Good habits (including the well-formed resolve-context maintained by consistent resolve) may also help cortical controllers generate faster signals of response conflict, contributing to timely suppression through hyperdirect pathway.

In what precedes, I have shown that an appreciation of recent study suggests that the temporal nature of the interactions between resolve, suppression, and routine habits determines the extent to which they reinforce each other more than Ainslie recognizes.