Animals are faced with the necessity of seeking rewards in their environments. Whereas natural rewards such as food or mates motivate much goal-directed behavior, similar mechanisms appear to drive seeking for drugs of abuse such as cocaine (Parkinson et al., 2000a; Everitt et al., 2001; Robbins & Everitt, 2002). Further, through associations with the reward, environmental cues acquire motivational significance that can influence goal-directed behavior (Holland & Rescorla, 1975; Hyde, 1976; Rescorla, 1994; Arroyo et al., 1998). For example, food-related cues can induce feeding in rats that are completely sated, suggesting that such motivational cues have the ability to over-ride homeostatic satiety signals (Holland & Petrovich, 2005). Similarly, animal and humans will re-engage in drug-taking behaviors when presented with drug-associated cues after long periods of abstinence (Grimm et al., 2002; Kalivas & McFarland, 2003; Fuchs et al., 2004). These findings argue that Pavlovian cues provide powerful motivational features through their associations with various reinforcers. Given these common associative mechanisms, understanding the manner in which learning comes to guide goal-directed behavior for natural rewards can also provide insight into similar processes that become pathological in the drug-addicted state.
One setting in which these cues drive goal-directed behavior is in a task known as Pavlovian-to-instrumental transfer (PIT). In this behavioral model, previously learned Pavlovian cues are able to invigorate ongoing goal-seeking behavior (Estes, 1948; Rescorla & Solomon, 1967; Lovibond, 1983; Bray et al., 2008). Detailed studies have shown that this ‘PIT effect’ is dependent upon the associative value of the cue, and that this value can be of general motivational significance or specific to a single reinforcer (Blundell et al., 2001; Shiflett & Balleine, 2010). Indeed this paradigm has been proposed to model features of addiction as it highlights the importance of the conditioned aspects of drug-taking behavior (Everitt et al., 2001). Consistent with PIT as a model of addiction, microinfusions of amphetamine into the brain induced greater levels of PIT than in normal animals (Parkinson et al., 1999; Wyvell & Berridge, 2000), whereas repeated administration of drugs of abuse like amphetamine or heroin makes the PIT effect more sensitive during cue presentation (Wyvell & Berridge, 2001; Ranaldi et al., 2009). Further, blockade of the neurotransmitter dopamine (DA) (Dickinson et al., 2000; Lex & Hauber, 2008) or inactivation of DA-signaling neurons (Murschall & Hauber, 2006; Corbit et al., 2007) attenuates the ability of Pavlovian cues to potentiate instrumental responding.
The neural underpinnings of PIT are poorly understood, but have been shown to involve a host of limbic structures, such as the central and basolateral nuclei of the amgydala (Blundell et al., 2001; Hall et al., 2001; Holland & Gallagher, 2003) and dorsal regions of the striatum (Corbit & Janak, 2007; Homayoun & Moghaddam, 2009). Given the involvement of dopaminergic processes in modulating the transfer effect, it is not surprising that the nucleus accumbens (NAc) – a primary target of dopaminergic terminals arising from the ventral tegmental area – is also involved in supporting the PIT effect. Neurotoxic lesions of the NAc abolish PIT without affecting more general features of instrumental or Pavlovian conditioning separately (de Borchgrave et al., 2002), whereas delivery of amphetamine or corticotropin-releasing factor within the NAc enhances transfer (Wyvell & Berridge, 2000; Pecina et al., 2006). However, the specific roles that these accumbal regions contribute to the transfer effect remain controversial. For example, in one set of findings, lesions of the core but not the shell of the NAc selectively abolished PIT (Hall et al., 2001; Cardinal et al., 2002a), whereas the opposite finding demonstrating the selective involvement of the NAc shell in PIT has also been reported (Corbit et al., 2001). However, selective blockade of DA receptors at the time of transfer produced pronounced deficits in the PIT effect after infusion of the D1 antagonist SCH-23390 (and, to a lesser extent, the D2 antagonist raclopride) into either the core or shell (Lex & Hauber, 2008), suggesting that both regions may play an important role in this task.
This set of conflicting data argues that subregions of the NAc encode key features of learning that enable Pavlovian cues to modulate ongoing goal-directed instrumental behavior, and that this coding is critically dependent upon DA. However, the lack of temporal specificity inherent in the above techniques, such as permanent lesions or long-term blockade, may obscure the more subtle effects that these regions contribute to this task. To address this, we recorded from single neurons in the NAc core and shell during the performance of PIT. Further, we assessed how neural encoding was altered by cocaine, a drug that acts by blocking DA reuptake in the synapse of NAc neurons, by comparing neural firing in animals with a history of cocaine self-administration with naive and saline-infused controls.
The present data provide an important insight into the specific roles of NAc subregions during PIT. In all groups tested, there was a selective behavioral enhancement in lever pressing in the presence of the CS+ cue that was not seen in the presence of the CS− cue. However, rats with a history of cocaine self-administration showed transfer that was significantly more robust than either control group. At the neural level, evidence was found that both the core and shell contributed important facets of encoding critical to supporting successful transfer. In all groups, core neurons were reliably biased in encoding information about cues, rewards and operant task performance compared with the shell, and cue-related encoding in the core was correlated with the degree of behavioral transfer. In contrast, in naive rats, only shell neurons showed cue-modulated responses during lever press (PIT-modulated neurons) that were correlated with task performance. However, following chronic cocaine taking, shell but not core neurons showed enhanced encoding for all task-related events compared with controls, whereas both core and shell showed a dramatic increase in the percentage of PIT-modulated neural activity to the press.
In contrast, the analysis of foodcup entries and neural activity that encoded these responses highlights the specificity of the instrumental transfer feature of the PIT task. Although cocaine experience resulted in a significant potentiation of the PIT effect for lever pressing, it did not translate into more general behaviors in the task such as foodcup activity. These findings indicate that psychostimulant experience did not simply increase hyperactivity in the box, nor did it lead to a differential response conflict between the instrumental and Pavlovian responses during transfer. Instead, cocaine experience selectively enhanced the instrumental response in the presence of the CS+, a feature that was reflected in both the behavior and neural response.
In the present study, encoding information about Pavlovian cues in naive animals was largely a function of the NAc core, although a few shell neurons encoded this associative information. This pattern of encoding has been demonstrated reliably in previous studies, whether the cues predict natural rewards such as sucrose (Setlow et al., 2003; Day et al., 2006; Jones et al., 2008) or drugs of abuse such as cocaine (Hollander & Carelli, 2007). These neural representations encode not only the identity of these cues, but also the motivational significance and predictive value of the associated outcome. For example, studies from this laboratory have repeatedly demonstrated that NAc core neurons show little overlap between cues predictive of cocaine and cues predictive of natural reward (Carelli et al., 2000; Carelli & Wondolowski, 2003). Further, in a go/no-go task, NAc core neurons rapidly encoded new associations, and rapidly switched or lost this cue selectivity when response contingencies were reversed (Setlow et al., 2003). Studies employing neurotoxic lesion support these correlational findings; post-training core but not shell lesions impair performance on simple Pavlovian conditioning (Parkinson et al., 1999; Cardinal et al., 2002b), whereas lesions of the NAc centered on the core during a cued go/no-go task resulted in behavioral deficits suggestive that rats were insensitive to cued outcome value (Schoenbaum & Setlow, 2003). Further, reversible inactivation of the NAc core but not shell has been shown to selectively disrupt cue-induced reinstatement of cocaine self-administration (Fuchs et al., 2004). These data argue for a specific role for the NAc core for acquiring critical cue-related information for guiding behavior.
Interestingly, although much cue encoding was dependent on the core, only shell neurons in naive animals showed cue-modulated operant encoding that was correlated with the behavioral performance of PIT. Several studies have now suggested that the shell is critical for the transfer effect. For example, Corbit et al. (2001) showed that lesions of the NAc shell made prior to conditioning failed to impair either Pavlovian or instrumental conditioning, but selectively abolished cue-potentiated transfer, whereas NAc core lesions had no effect on transfer. Similarly, intrashell infusions of amphetamine (Wyvell & Berridge, 2000) or corticotropin-releasing factor (Pecina et al., 2006) results in potentiating the transfer effect, whereas lesions of the shell but not the core block this amphetamine potentiating effect (Parkinson et al., 1999).
These findings are somewhat at odds with other work that has shown specificity for the NAc core in PIT (Hall et al., 2001; de Borchgrave et al., 2002). In those studies, normal Pavlovian and instrumental conditioning were largely unaffected, but transfer was impaired. Importantly, in those studies, lesions of the core were made prior to any conditioning, whereas the above work by Parkinson et al. (1999) showing the importance of the shell was performed in experiments where the lesion was administered after first-order conditioning but prior to transfer (Parkinson et al., 1999). This suggests an important distinction between the acquisition of Pavlovian information vs. the potentiation of instrumental responding in the presence of learned cues.
In line with this finding, the enhancement of PIT following a period of prolonged drug-taking was accompanied by a concurrent increase in shell-specific neural encoding. These results mirror the findings from Parkinson et al. (1999) in which post-training shell lesions abolished the ability for amphetamine to potentiate already-learned responses. As in their study, this suggests that the shell acts to modulate previously learned Pavlovian and instrumental information, specifically those for which the drug inducing the alteration in behavior (either amphetamine or cocaine) was not the reinforcer being used to guide PIT. Similarly, amphetamine infusions into the NAc shell at the time of PIT significantly enhanced the transfer effect (Wyvell & Berridge, 2000). However, in both of these circumstances, the drug was present at the time of transfer, whereas in the present study and others (Ranaldi et al., 2009), animals were drug abstinent for 1 week prior to testing. Thus, the present findings suggest that repeated cocaine exposure may change the sensitivity of shell neurons to PIT-related stimuli, a mechanism that may be gated by prolonged exposure to phasic DA release. Intriguingly, previous studies have shown that DA release in the NAc following cocaine infusions is largely confined to the shell (Aragona et al., 2008). Cocaine self-administration may thus result in inducing a shell-specific DA-dependent process in which animals become exquisitely sensitive to task-related stimuli and rewards, and thus may be at greater risk for subsequent relapse.
Given these converging data, one model for these results that is in line with the present findings suggests a role of the NAc core neurons in learning the motivational significance of cues early in learning, whereas the core may become less important after the associations are fully learned. The naive animals reported here show such a pattern; core neurons reliably encoded cue-related information and, further, the degree to which this was learned predicted success on later transfer. However, these neural representations did not appear to modulate lever-pressing activity during PIT, suggesting a less essential role in expressing that behavior. Shell neurons showed a different pattern of activity in line with this model. Although not as involved with the encoding of cue-related information as the core, cells that were cue-modulated at the time of press were significantly correlated with performance on transfer. If this model is correct, we would predict that transient inactivation of the core, but not shell, during learning would impair subsequent transfer, whereas inactivation of the shell, but not core, at the time of transfer would have a similar transfer-inhibiting effect.
Previous work in this laboratory has also shown that, following cocaine abstinence, cue and task-related encoding are selectively potentiated in the core, but not the shell (Hollander & Carelli, 2005, 2007). However, in those studies, modulation was found for drug-related stimuli and responses, whereas in the present study, drug exposure altered encoding for non-drug (natural) reward during novel learning. Notably, in the earlier study, associative encoding for drug-related stimuli necessarily occurred while the cocaine was onboard, whereas in the present study, all animals had the opportunity to learn about Pavlovian and instrumental responses for natural reward while drug naive. Thus, in the earlier studies, these factors may strongly contribute to biasing rats towards core-specific encoding during learning, whereas in the present study, cocaine exposure may potentiate already-learned representations that may be more shell-dependent.
Thus, we predict that the role of repeated cocaine exposure would have differing effects from the present findings if presented prior to training. A series of work has now suggested that repeated cocaine exposure prior to learning can result in profound deficits in acquisition. For example, cocaine-treated rats have been shown to have impairments in acquiring normal Pavlovian (Schoenbaum & Setlow, 2005; Saddoris et al., 2010) and operant task (Schoenbaum et al., 2004; Calu et al., 2007; Roesch et al., 2007) performance. If animals are unable to learn about cue–outcome or response–outcome associations normally as a result of cocaine exposure (a putatively core-dependent process), then such cocaine exposure should result in impaired, not enhanced, PIT due to poor initial learning, but not because of poor transfer specifically.
Given that both the core and shell appear to coordinate activity to produce the PIT effect, it is not known how the core and shell subregions would coordinate activity in the course of learning to produce this phenomenon. Interestingly, many facets of NAc encoding presented here mirror results previously found in the amygdala. For example, similar to the core, lesions of the basolateral amygdala (BLA) disrupt behavior sensitive to Pavlovian cue encoding in similar tasks (Schoenbaum et al., 1998, 2003b; Balleine et al., 2003; Pickens et al., 2003), while also causing aberrant cue encoding in distally connected regions such as the prefrontal cortex (Schoenbaum et al., 2003a) and NAc (Ambroggi et al., 2008; Jones et al., 2010). In contrast, the central nucleus of the amygdala (CN) has been shown to be important for attention for learning (Gallagher et al., 1990; Hatfield et al., 1996; Parkinson et al., 2000b; Haney et al., 2010), but less important for detailed cue–outcome associative learning. Consequently, similar to differences between the core and shell in the NAc, BLA and CN show a similar dissociation in PIT. CN lesions abolish potentiating transfer effects, whereas BLA lesions only appear to abolish the behavioral selectivity (i.e. only pressing the CS+-associated lever) of the PIT (Blundell et al., 2001; Hall et al., 2001; Holland & Gallagher, 2003; Corbit & Balleine, 2005).
These core/BLA and shell/CN parallels suggest a larger system by which the amygdala and NAc coordinate activity to produce cue-modulated instrumental behavior. Indeed, BLA inputs to the NAc (Heimer et al., 1991; Brog et al., 1993) appear to be critical for supporting cue-related learning, as asymmetric lesions of the BLA and NAc block the ability for rats to use Pavlovian cues to support new learning (Setlow et al., 2002), whereas inactivation of the BLA selectively alters NAc core encoding during appetitive conditioning (Ambroggi et al., 2008; Jones et al., 2010). However, CN fibers do not terminate in the NAc shell; instead they presumably influence NAc activity via an indirect pathway through midbrain DA-expressing neurons. Consistent with this, inactivation of the ventral tegmental area abolished PIT (Corbit et al., 2007), whereas dopaminergic receptor blockade in the NAc attenuated transfer (Lex & Hauber, 2008). Conversely, amphetamine, which increases DA vesicular release, potentiates PIT after being selectively infused into the shell (Parkinson et al., 1999; Wyvell & Berridge, 2000). Thus, as the anatomical projections from the amygdala complex at the level of the ventral striatum (whether direct or indirect) are heavily intermixed, these functional parallels suggest that there is probably a necessary interplay between glutamatergic and dopaminergic processes that may differentially impact the ways in which motivational and detailed sensory information is coded within the NAc.
In conclusion, these results present an important basis for understanding the neural underpinnings of PIT in the NAc, and how this neural circuit is fundamentally altered following repeated exposure to cocaine and its resultant modulation of DA action in the NAc. Future work will need to investigate how this neural encoding acts within larger circuits of the limbic system such as the amygdala and dorsal striatum, and how such circuits are modulated by DA inputs.