Redish et al. advance a unified framework of decision making in which they explicitly conflate Pavlovian (stimulus-outcome) and instrumental (action-outcome) learning processes into a single “planning component.” They justify this by arguing that both processes “entail an expectancy of the outcome that must be evaluated to produce an expectancy of a value” (sect. 3, para. 8). However, it becomes clear that situational cues are assumed to be ultimately responsible for initiating action selection in the planning system; based on these cues, the agent “calculates the consequences of potential actions” and then “calculates the value of those consequences” (sect. 3.2, para. 2). This simplified view, although parsimonious, is at odds with a number of things we know about the role played by environmental cues in action selection.

First, the “unified framework” assumes that the evaluative processes that govern the performance of instrumental actions – for example, lever pressing for food – also control the performance of conditioned responses – for example, anticipatory approach to the location of food during a signal previously paired with food delivery. The authors are encouraged in this view by the fact that both instrumental actions and conditioned responses are sensitive to post-training manipulations of outcome (or unconditioned stimulus [US]) value. However, there is in fact considerable evidence that changes in outcome value affect these two categories of behavior through fundamentally different incentive processes (Balleine Reference Balleine, Mowrer and Klein2001; Reference Balleine, Whishaw and Kolb2004). One particularly effective way to reduce the value of a food is by pairing it with illness, which can be readily induced with an injection of lithium chloride. Rats given just one such pairing tend to immediately suppress their conditioned approach responses, but continue to lever press for the devalued food, at least when tested in extinction (Balleine & Dickinson Reference Balleine and Dickinson1991). However, giving rats the opportunity to actually consume the devalued outcome (either between training and testing or during the actual test session) is sufficient to reduce their lever press performance, suggesting that direct contact with the outcome is needed to learn that it is no longer an instrumental goal. The incentive processes controlling conditioned approach performance apparently do not require such explicit feedback. This is not just true of taste aversion learning but also of shifts in food deprivation (Balleine Reference Balleine1992), water deprivation (Lopez et al. Reference Lopez, Balleine and Dickinson1992), sexual motivation (Everitt & Stacey Reference Everitt and Stacey1987), thermoregulation (Hendersen & Graham Reference Hendersen and Graham1979), and drug-related states (Balleine et al. Reference Balleine, Ball and Dickinson1994), among other motivationally based shifts in incentive value. There is also evidence that the evaluative processes guiding Pavlovian and instrumental learning are neurally dissociable. Although orbitofrontal cortex (OFC) lesions have been shown to disrupt the effect of outcome devaluation on conditioned approach performance (Gallagher et al. Reference Gallagher, McMahan and Schoenbaum1999; Pickens et al. Reference Pickens, Saddoris, Setlow, Gallagher, Holland and Schoenbaum2003; Reference Pickens, Saddoris, Gallagher and Holland2005), we have reported that OFC lesions do not impair the sensitivity of instrumental lever pressing to outcome devaluation (Ostlund & Balleine Reference Ostlund and Balleine2007).

Second, the “unified framework” fails to distinguish between the roles played by instrumental discriminative stimuli and Pavlovian conditioned stimuli in action selection. Discriminative cues can act to signal particular action-outcome relationships; for example, in biconditional discrimination experiments rats learn that two discriminative stimuli signal different and opposite action-outcome relationships; that is, S1 signals R1→01 and R2→02, whereas S2 signals R1→02 and R2→01. In this situation, discriminative cues play a critical role in guiding devaluation performance, such that rats will tend to suppress their performance of an action when the prevailing stimulus signals that the devalued outcome is going to be earned (Rescorla Reference Rescorla1991). However, Redish et al. assign conditioned stimuli the same capacity to guide goal-directed action selection. It is true that conditioned stimuli can influence instrumental action selection. For instance, rats will tend to increase their performance of an action previously rewarded with a particular outcome when presented with a cue that has been independently paired with the same outcome relative to a cue paired with a different outcome (Kruse et al. Reference Kruse, Overmier, Konz and Rokke1983). However, it has more recently been shown that this Pavlovian-instrumental transfer effect is insensitive to manipulations of outcome value; that is, conditioned stimuli continue to selectively retrieve actions with which they share a common outcome even after that outcome has been devalued (Holland Reference Holland2004; Rescorla Reference Rescorla1994). Such findings suggest that conditioned stimuli do not play a significant role in guiding the evaluation of instrumental goals and instead appear to bias action selection through a separate associative priming process. This latter process likely reflects the information provided by these cues; Delamater (Reference Delamater1995) has shown that degrading the Pavlovian CS-US contingency is sufficient to abolish the selective influence of Pavlovian cues on instrumental performance.

Furthermore, we have found evidence that instrumental outcome devaluation and Pavlovian-instrumental transfer effects are neurally dissociable at the level of the prefrontal cortex. In rats, OFC lesions have no effect on the sensitivity of instrumental performance to outcome devaluation, as mentioned earlier, but they are effective in disrupting Pavlovian-instrumental transfer, at least when they are made after training (Ostlund & Balleine Reference Ostlund and Balleine2007). In contrast, lesions of the prelimbic region of the medial prefrontal cortex have been shown to disrupt instrumental outcome devaluation performance but leave Pavlovian-instrumental transfer intact (Corbit & Balleine Reference Corbit and Balleine2003).

This evidence that Pavlovian and instrumental conditioning are controlled by distinct incentive processes has implications for Redish et al.'s ultimate aim, which is to identify aspects of decision-making that are vulnerable to drug abuse. They describe Vulnerability 4 as the tendency for repeated drug exposure to produce “overvaluation in the planning system” (sect. 4, para. 8), and argue that “sensitization of motivational signals drives excess motivation for certain events” (Table 5). Although it has been shown that post-training amphetamine sensitization enhances the sensitivity of instrumental performance to Pavlovian-instrumental transfer (Wyvell & Berridge Reference Wyvell and Berridge2000), as we have argued, this effect is not likely to be due to an overvaluation of the instrumental reward. In fact, it has more recently been shown that rats sensitized to amphetamine after instrumental training are unimpaired in their ability to use outcome value to select between actions even though rats sensitized to amphetamine before training showed impaired outcome devaluation performance (Nelson & Killcross Reference Nelson and Killcross2006), consistent with the view that drug exposure resulted in excessive habit formation, something Redish et al. identify as Vulnerability 7.

We support the authors' attempt to apply concepts developed in associative learning to the study of addiction, but we believe that, rather than trying to blur the lines between Pavlovian and instrumental learning, it would accord better with the evidence to recognize that these two fundamentally different forms of learning provide unique potential determinants of the pathological drug seeking induced by drug addiction and, therefore, unique targets for its treatment.