The psychological and neurochemical mechanisms of drug memory reconsolidation: implications for the treatment of addiction

Authors


Dr. Amy L. Milton, as above.
E-mail: alm46@cam.ac.uk

Abstract

Memory reconsolidation is the process by which memories, destabilised at retrieval, require restabilisation to persist in the brain. It has been demonstrated that even old, well-established memories require reconsolidation following retrieval; therefore, memory reconsolidation could potentially be exploited to disrupt, or even erase, aberrant memories that underlie psychiatric disorders, thereby providing a novel therapeutic target. Drug addiction is one such disorder; it is both chronic and relapsing, and one prominent risk factor for a relapse episode is the presentation of environmental cues that have previously been associated with drugs of abuse. This ‘cue-induced relapse’ can be accounted for in psychological terms by reinforcing memories of the pavlovian association between the cue and the drug, which can thus influence behaviour through at least three psychologically and neurobiologically dissociable mechanisms: conditioned reinforcement, conditioned approach and conditioned motivation. As each of these psychological processes could contribute to the resumption of drug-seeking following abstinence, it is important to develop treatments that can reduce drug-seeking re-established via influences on each or all of these pavlovian processes, in order to minimise the risk of a subsequent relapse. Investigation of the memory reconsolidation mechanisms of the memories underlying conditioned reinforcement, conditioned approach and conditioned motivation indicate that they depend upon different neurochemical systems, including the glutamatergic and adrenergic systems within limbic corticostriatal circuitry. We also discuss here the subsequent translation to the clinic of this preclinical work.

Introduction

Memory reconsolidation (Lewis, 1979; Nader, 2003) is the process by which memories are destabilised at retrieval and require a process of restabilisation, or reconsolidation, in order to persist in the brain, and to be updated (Rodriguez-Ortiz et al., 2005; Winters et al., 2009), strengthened (Lee, 2008) or functionally integrated with older memories (see Lee, 2009, for review and Table 1 for definition of key terms relevant to reconsolidation studies). The neurochemical, intracellular and genetic mechanisms underlying reconsolidation are becoming increasingly understood, and it has been proposed that disruption of reconsolidation may provide a novel form of treatment for anxiety disorders, such as post-traumatic stress disorder, in humans (Debiec & LeDoux, 2006; Brunet et al., 2008; Kindt et al., 2009; Schiller et al., 2010), based upon preclinical work using aversive conditioning procedures in rodents (Nader et al., 2000; Debiec & LeDoux, 2004). We (Lee et al., 2005, 2006a; Milton et al., 2008a,b) and others (Miller & Marshall, 2005; Bernardi et al., 2006; Valjent et al., 2006) have hypothesised that, as for psychiatric disorders based upon maladaptive aversive memories, the mechanisms underlying memory reconsolidation could be exploited to provide a pro-abstinence, anti-relapse treatment for disorders based upon maladaptive appetitive memories, such as in drug addiction.

Table 1.   Definitions of terms relating to memory reconsolidation
TermDefinition
ReconsolidationThe hypothetical process by which a memory (the engram) requires restabilisation following its destabilisation at retrieval.
RetrievalThe returning of a memory to a state where the information within the memory can be utilised for behaviour and/or consciously experienced.
ReactivationThe procedure of inducing memory retrieval, so that it also induces the destabilisation of the memory (engram). The term ‘reactivation’ can also be used procedurally, to refer to the behavioural training session in which the memory is ‘reactivated’. Reactivation, unlike retrieval, has the connotation that the memory has entered the ‘active state’ from the ‘inactive state’, as defined by Lewis (1979).
DestabilisationThe molecular process underlying memory lability, induced by memory reactivation. This process probably depends upon protein degradation and activity of the ubiquitin proteasome system (Lee et al., 2008; Kaang et al., 2009).
RestabilisationThe molecular process underlying the stabilisation of memory, following its destabilisation at retrieval. This process probably involves protein synthesis, immediate–early gene activation and protein kinase activity(see Tronson & Taylor, 2007, for review).

Drug addiction is a chronic, relapsing disorder, synonymous with the Diagnostic and Statistical Manual of Mental Disorders (DSM-IVR) classification of ‘substance dependence’. The chronic, relapsing nature of the disorder (the propensity to relapse persists long after any acute signs of withdrawal have abated; Gawin & Kleber, 1992) is a major problematic aspect of effective treatment. Relapse, the resumption of drug-seeking and drug-taking behaviour following a period of abstinence, can be unconscious, automatic and habitual (Tiffany, 1990; Everitt & Robbins, 2005) and is markedly influenced by the presence of environmental stimuli and contexts that have been paired previously with drug use. These drug-associated conditioned stimuli (CSs), or cues, can induce craving and activate limbic corticostriatal circuitry in abstinent human addicts (Ehrman et al., 1992; Grant et al., 1996; Childress et al., 1999; Garavan et al., 2000; Litt et al., 2000; Sinha et al., 2000; Kilts et al., 2001; Bonson et al., 2002). Furthermore, drug CSs precipitate relapse behaviour in many different animal models of addiction (de Wit & Stewart, 1981; Crombag & Shaham, 2002) including cue-induced reinstatement following extinction or enforced abstinence (as used in studies of the ‘incubation’ phenomenon; Lu et al., 2005), conditioned place preference, and second-order schedules (see Markou et al., 1993; Everitt & Robbins, 2000; and Sanchis-Segura & Spanagel, 2008, for reviews of different preclinical models of drug addiction and relapse). Drug CSs also activate the same key corticostriatal regions in preclinical rodent models of addiction (Boujabit et al., 2003; Schiffer et al., 2009) as in studies of human addicts. These previously neutral environmental stimuli, which may be drug-specific paraphernalia such as syringes or more general stimuli such as particular individuals or locations, are associated with the addictive drug’s effects in a pavlovian manner, and subsequently these drug-associated CSs come to influence drug-seeking and relapse behaviour through the memories they evoke. Therefore, it is hypothesised that the disruption of these learned associations would reduce the subsequent risk of relapse.

As stated above, the disruption of old, well-established memories can potentially be achieved by exploiting memory reconsolidation mechanisms. Reconsolidation is usually studied in preclinical models by first training animals on a behavioural task involving pavlovian conditioning; subsequently, animals are briefly re-exposed to the conditioned stimuli established during the training session, often under the influence of a drug that, it is hypothesised, interferes with the restabilisation of the memory (an ‘amnestic agent’). Animals are subsequently tested on behavioural tasks dependent upon the reactivated memory, in the absence of the amnestic agent. Control groups are given the putative amnestic agent in the absence of re-exposure to the training stimuli. Therefore, for a deficit in memory reconsolidation to be demonstrated, the reduction in the subsequently tested behaviour, dependent on the original memory, will only be seen in those groups that received the amnestic agent in conjunction with memory retrieval. By understanding the neurochemical and intracellular mechanisms underlying memory reconsolidation, and the optimal way in which to induce memory reactivation in clinical populations, treatments could be developed to prevent the reconsolidation of maladaptive memories. However, developing reconsolidation-based, pro-abstinence, anti-relapse treatments for drug addiction is not merely an incremental step from the development of reconsolidation-based clinical treatments for anxiety disorders; the disorders are very different in their development and in their psychological and neurobiological basis. The development of a reconsolidation-based treatment for relapse prevention therefore requires an understanding of not only the neurochemical and intracellular mechanisms that underlie the reconsolidation process but also the different psychological processes and neurobiological substrates that influence drug-seeking behaviour, based on appropriate preclinical models of drug addiction.

Mechanisms of memory reconsolidation

Although originally described in the 1960s (Misanin et al., 1968; Schneider & Sherman, 1968), research into memory reconsolidation stalled until the early parts of the 21st century, when it was revived by studies primarily focusing on conditioned fear memories (Nader et al., 2000; Debiec et al., 2002; Lee et al., 2004; Suzuki et al., 2004). Subsequent research began to investigate the clinical utility of therapies based upon the disruption of memory reconsolidation: the first studies hypothesised that ‘reconsolidation-based’ therapies could be used to treat post-traumatic stress disorder (Debiec & LeDoux, 2006; Brunet et al., 2008; Kindt et al., 2009). Later work suggested that reconsolidation-based therapies might also provide a pro-abstinence, anti-relapse treatment for drug addiction (Lee et al., 2005; Miller & Marshall, 2005) as the reconsolidation of appetitive memories using addictive drug reinforcers, such as cocaine and morphine, was shown to be disrupted by gene knockdown and inhibition of the protein kinase MEK (also known as mitogen-activated protein kinase kinase; MAPKK) in specific neural sites.

These early studies of appetitive memory reconsolidation began to characterise the gene expression changes and intracellular signalling pathways necessary for memory restabilisation to occur following destabilisation at retrieval. Drug memory reconsolidation was shown to be dependent upon protein synthesis (Lee et al., 2005; Milekic et al., 2006; Robinson & Franklin, 2007b), the protein product of the plasticity-related immediate–early gene zif268 (Lee et al., 2005, 2006a), and the activity of protein kinases, including extracellular signal-related kinase (ERK or MAPK; Valjent et al., 2006) and the kinase of this protein (MEK or MAPKK; Miller & Marshall, 2005). Although gene knockdown in specific brain areas is not therapeutically feasible, targeting of intracellular protein kinase signalling pathways could be of potentially great therapeutic benefit, as protein kinases such as ERK/MAPK and MEK/MAPKK are activated by signalling through many different neurotransmitter receptors and they are thought ultimately to affect the transcription of genes required for memory reconsolidation, such as zif268 (see Tronson & Taylor, 2007, for review). Similarly, studies investigating the neurochemical basis of drug memory reconsolidation can be considered to be of translational relevance, as they identify tractable pharmacological targets at the level of the neuronal membrane and may allow modulation of immediate–early gene transcription and protein synthesis without the requirement for centrally-administered amnestic agents.

Studies of the neurochemical mechanisms of appetitive memory reconsolidation have focused mostly upon two neurotransmitter receptors: the NMDA subtype of glutamate receptor (NMDAR) and the β-adrenergic receptor (βAR), both of which had been identified previously as necessary for the reconsolidation of conditioned fear memories (Debiec & LeDoux, 2004; Lee et al., 2006b) following the earlier discovery of their involvement in emotional memory consolidation (Falls et al., 1992; Fanselow & Kim, 1994; McGaugh & Cahill, 1997; Ferry et al., 1999; Sara et al., 1999).

Glutamatergic mechanisms

Glutamatergic signalling at the NMDAR has been shown to be necessary not only for the reconsolidation of drug memories but also for the upregulation of Zif268 observed following to re-exposure to cocaine-associated CSs. NMDAR antagonists, given in conjunction with retrieval, disrupt the reconsolidation of place preferences conditioned to cocaine (Kelley et al., 2007; Brown et al., 2008; Itzhak, 2008), amphetamine (Sadler et al., 2007; Sakurai et al., 2007) and morphine (Zhai et al., 2008). Although conditioned place preference is difficult to interpret in terms of the psychological associations underlying the behaviour (White & Carr, 1985; Everitt & Robbins, 1992; Schechter & Calcagnetti, 1993), the reactivation-dependent amnesia caused by the administration of NMDAR antagonists in this procedure suggests that targeting and modulating NMDAR activity and NMDAR-mediated signalling may be of therapeutic value. Furthermore, NMDAR antagonists have been shown to disrupt the reconsolidation of drug memories in procedures that are more amenable to psychological analysis, such as cue-induced reinstatement of alcohol-seeking (von der Goltz et al., 2009) and the acquisition of a new instrumental response for a conditioned reinforcer previously associated with cocaine (Milton et al., 2008a). This disruption of reconsolidation by the administration of NMDAR antagonists has been linked directly to the expression of the immediate–early gene zif268, which is necessary for reconsolidation to occur (Bozon et al., 2003; Lee et al., 2004). Thus, administration of the NMDAR antagonist d-(2R)-amino-5-phosphonovaleric acid (d-APV) into the basolateral amygdala of rats previously trained to associate a CS with intravenous cocaine reinforcement markedly reduced the expression of Zif268 protein within the amygdala, which is normally increased following re-exposure to cocaine CSs (Thomas et al., 2003), but only if the NMDAR antagonist was given in conjunction with memory retrieval (Fig. 1; Milton et al., 2008a). To date, it has not been investigated whether Zif268 expression in response to cocaine CSs is linked to NMDARs, and thus also depends upon ERK activation, or whether it is dependent upon other protein kinases such as protein kinase A (PKA) and calcium–calmodulin dependent kinases, which can activate the serum-response element (SRE) or the cAMP-response element (CRE) of the zif268 gene. However, in the developing brain (Zhou et al., 2009) and in the adult hippocampus (Davis et al., 2000) the regulation of zif268 expression through NMDAR-mediated signalling does depend upon ERK, suggesting that this could be one mechanism through which the increase in Zif268 expression is achieved.

Figure 1.

 NMDAR antagonism, in conjunction with retrieval, reduced the expression of Zif268 in the basolateral amygdala following re-exposure to a cocaine-associated CS. d-APV-infused animals (black bars) showed reduced Zif268 expression relative to vehicle-infused controls (white bars) when the NMDAR antagonist was (a) given in conjunction with retrieval, but not (b) administered without retrieval. Representative Western blots are shown, quantifying Zif268 and β-actin expression relative to vehicle-infused controls. White squares represent vehicle-infused animals, black squares d-APV-infused animals. Modified, with permission, from Milton et al. (2008a,b.)

To date, all of the data concerning glutamatergic mechanisms of drug memory reconsolidation have been focused upon the NMDA subtype of glutamate receptor, with no published research concerning requirements for the necessity of activity at the AMPA subtype of glutamate receptor (AMPAR) for reconsolidation. This is despite the known importance of AMPAR-mediated transmission for memory retrieval (Bianchin et al., 1993; Bast et al., 2005; Winters & Bussey, 2005), which might suggest an involvement of AMPAR activity in the transition to the ‘active state’ of the memory (Lewis, 1979) and hence memory destabilisation. However, contrary to this hypothesis but consistent with research on CS–fear memory reconsolidation (Ben Mamou et al., 2006), AMPAR antagonism does not appear to be involved in the destabilisation of CS–drug memories (Fig. 2). Blocking the activity of receptors involved in the induction of memory lability should prevent the destabilisation of the memory, and therefore the memory should be protected from the amnestic effects of drugs that prevent the restabilisation process, such as protein synthesis inhibitors. However, the administration of the specific AMPAR antagonist LY293558 did not prevent the amnestic effects of a subsequent infusion of the protein synthesis inhibitor anisomycin, and therefore it can be concluded that AMPAR antagonism did not prevent the CS–drug memory from destabilising at retrieval.

Figure 2.

 AMPAR antagonism did not prevent the destabilisation of a CS–cocaine memory. (a) Experimental procedure. Rats were trained to associate self-administered intravenous cocaine with a light CS, and subsequently tested on the capacity of this CS to act as a conditioned reinforcer, supporting the acquisition of a new instrumental response. Prior to a memory reactivation session, the rats were microinfused with either vehicle or the AMPAR antagonist LY293558 directly into the basolateral amygdala. Immediately following the reactivation session, they were microinfused with either vehicle or the protein synthesis inhibitor anisomycin. (b) Any drug that prevents the destabilisation of the CS–drug memory should prevent the amnestic effect of anisomycin on subsequent responding for conditioned reinforcement; however, LY293558 did not prevent the anisomycin-induced memory deficit in pressing of the ‘active’ lever, which produced the previously cocaine-associated CS. All rats showed a preference for the active lever (F1,26 = 60.424, P < 0.001) and increased their responding with training (F2.8,73.5 = 3.157, P < 0.05). However, only animals that had received vehicle, and not anisomycin, following memory reactivation showed an increase in responding for the CS with extended training (Lever × Session × Anisomycin: F6,156 = 3.307, P < 0.01). n = 7 or 8 per group.

The elucidation of the neurochemical, intracellular and genetic mechanisms of memory destabilisation will be a critical area for future research; to date, the extent of the knowledge concerning destabilisation is that it is dependent upon intracellular protein degradation mechanisms (Artinian et al., 2008; Lee, 2008; Lee et al., 2008; Kaang et al., 2009) and that it is prevented by selective antagonism at the glutamate (NMDA) receptor subunit 2 (NR2B)-containing subtype of NMDAR prior to memory retrieval (Ben Mamou et al., 2006). The dependence of destabilisation upon NMDARs is counterintuitive considering the evidence outlined above that NMDAR activity is required for the restabilisation of memories following retrieval. Therefore, understanding the balance of activity between NR2B-containing and non-NR2B-containing NMDARs in mediating the destabilisation and restabilisation processes is likely to be of great importance in understanding the glutamatergic mechanisms of memory reconsolidation.

Adrenergic mechanisms

Antagonism at βARs has been investigated as a mechanism for disrupting drug memory reconsolidation, as previous data showed that fear memory reconsolidation was disrupted by the administration of beta-blockers in rodents (Debiec & LeDoux, 2004, 2006) and humans (Brunet et al., 2008; Kindt et al., 2009). Initial studies indicated that drug memories require activity at the βAR to reconsolidate; propranolol was shown to disrupt a place preference conditioned to morphine (Robinson & Franklin, 2007a) and cocaine (Bernardi et al., 2006, 2009; Fricks-Gleason & Marshall, 2008), and also to prevent a cocaine-associated CS from acting as a conditioned reinforcer in subsequent testing if propranolol was given in conjunction with memory retrieval (Milton et al., 2008b). Therefore, the first wave of studies suggested that propranolol or similar βAR blockers may be a promising treatment for the disruption of drug memory reconsolidation. However, more recent work has indicated limitations on propranolol as a form of treatment. Propranolol, given at reactivation, is not effective at reducing subsequent cue-induced reinstatement of cocaine-seeking following enforced abstinence (Fig. 3a; Milton & Everitt, 2009), even when combined with multiple memory reactivation-treatment sessions (Fig. 3b; A. L. Milton, unpublished PhD thesis), a procedure that was effective in disrupting cocaine conditioned place preference (Fricks-Gleason & Marshall, 2008).

Figure 3.

 The effects of propranolol, administered in conjunction with reactivation, on subsequent cue-induced reinstatement. CS-omission groups received saline at memory reactivation but no CS presentation during the relapse test. (a) Propranolol, administered in conjunction with a single memory reactivation session, did not subsequently reduce cue-induced reinstatement. All of the rats showed a preference for the ‘active’ lever, which had previously been associated with the CS and cocaine reinforcement (F1,15 = 288.661, P < 0.001) and although there was a main effect of Group (F2,15 = 29.033, P < 0.001) this was driven by differences between the CS-omission group and the other two groups (all P < 0.001), which did not differ from each other (P = 0.987); n = 6 per group. (b) Propranolol, administered in conjunction with two memory reactivation sessions, did not subsequently reduce cue-induced reinstatement. All rats showed a preference for the active lever (F1,13 = 73.388, P < 0.001) and although there was an interaction of Lever × Group (F2,13 = 4.613, P < 0.05) this was driven by a failure to discriminate between the active and inactive lever in the CS-omission group (P > 0.05) whilst the vehicle- and propranolol-treated groups did discriminate (all P < 0.001); n = 3–9 per group. Fig. 6a adapted from Milton & Everitt, 2009; Fig. 6b, A. L. Milton, unpublished PhD thesis.

These data concerning cue-induced reinstatement indicate that there is a difference in the way in which NMDAR antagonism and βAR antagonism are producing their amnestic effects. This could be due to different ways in which these drugs modulate the activity of intracellular signalling pathways, as there are at least two intracellular signalling pathways that could act in parallel to lead to increased expression of Zif268 (Davis et al., 2000). However, we have focused upon an alternative hypothesis: that the requirement for activity at NMDARs and βARs may differ according to the specific memory undergoing reconsolidation and the neural substrates upon which the reconsolidating memory depends. This has been most extensively investigated with respect to the different pavlovian memories that are formed during the learning of an appetitive CS–unconditioned stimulus (US) association, which we have hypothesised are relevant to relapse behaviour in drug addiction (Everitt et al., 2001).

More than a single CS–drug memory: psychological processes underlying relapse behaviour

Drug addiction, and therefore relapse behaviour, can be conceptualised as a disorder of aberrant memory formation (Everitt et al., 2001; Robbins & Everitt, 2002). During the experience of drug-seeking and drug-taking a number of associations are learned, including associations between the drug-seeking and drug-taking actions and the drug outcome (instrumental ‘action–outcome’ associations), environmental stimuli and the drug outcome (pavlovian ‘stimulus–outcome’ associations) and between the environmental stimuli and the drug-seeking and drug-taking actions (‘stimulus–response’ associations, leading to habits). With extended experience, we have argued that the instrumental drug-seeking and drug-taking behaviour of an individual gradually changes from being predominantly goal-directed in nature to a more automatic stimulus–response habit that ultimately becomes compulsive (Everitt & Robbins, 2005). However, to date neither goal-directed action–outcome nor habitual stimulus–response instrumental memories have been shown to reconsolidate (Hernandez & Kelley, 2004), and therefore reconsolidation-based treatments intended to disrupt the instrumental memories underlying drug-seeking behaviour have been little investigated. By contrast, pavlovian memories are known to undergo reconsolidation upon retrieval, and therefore these memories could potentially be targeted with a reconsolidation-based, and therefore anti-relapse, treatment.

Pavlovian conditioning is a highly adaptive mechanism by which an animal can predict future reinforcement, based upon its knowledge of the salience and value of environmental cues. The pavlovian memories formed by the association of previously motivationally neutral environmental CSs with a drug of abuse can influence drug-seeking and relapse behaviour in at least three psychologically and neurobiologically separable ways (Fig. 4; see Everitt et al., 2001, for review). Drug-associated stimuli can: (i) act as conditioned reinforcers and thereby support drug-seeking responses; (ii) support conditioned approach behaviour and thereby engage attention to likely drug locations; or (iii) induce conditioned motivation for the drug, thereby energising drug-seeking responses. All of these associations therefore have the potential to modulate the likelihood of instrumental drug-seeking and relapse behaviour when CSs are present in the environment.

Figure 4.

 The ‘three routes to relapse’ produced by the effects of pavlovian drug-associated conditioned stimuli over instrumental drug-seeking and relapse behaviour.

Conditioned reinforcement

Conditioned reinforcement refers to the phenomenon by which a pavlovian conditioned stimulus has the capacity to become reinforcing in its own right, because of its association with a reinforcing outcome. By definition, a conditioned reinforcer should be able to support the acquisition of a novel instrumental seeking response that has not previously been associated with primary reinforcement (Mackintosh, 1974). Conditioned reinforcement is found in both specific and general forms (Burke et al., 2007, 2008), with the specific form revealing the sensory-specific representation of the US by the associated CS and the general form revealing the motivational valence of the CS–US association. The specific and general conditioned reinforcing properties of CSs are critically dependent on limbic corticostriatal circuitry, including the basolateral amygdala (Everitt et al., 1989; Burns et al., 1993; Parkinson et al., 2001; Burke et al., 2007), the ventral striatum (Parkinson et al., 1999) and the orbitofrontal cortex (OFC; Pears et al., 2003), with recent work suggesting that the OFC is particularly important for the outcome-specific form of conditioned reinforcement (Burke et al., 2008).

The conditioned reinforcing properties of a CS influence instrumental behaviour by supporting the drug-seeking response over extended delays and instrumental chains, and under extinction conditions (Cardinal et al., 2003). Second-order reinforcement schedules (Arroyo et al., 1998; Alderson et al., 2000; Everitt & Robbins, 2000) reveal this impact of conditioned reinforcers on drug-seeking, as responses are reinforced by a drug-associated CS, before attainment of primary reinforcement, such as cocaine. Even under conditions of nonreinforcement, responding for conditioned reinforcement is remarkably persistent and resistant to extinction, with one study demonstrating that animals will continue to work vigorously for a conditioned reinforcer that was last associated with primary drug reinforcement 2 months earlier (Di Ciano & Everitt, 2004). Furthermore, animals will continue to respond for a conditioned reinforcer even when its associated primary reinforcer has been devalued (Davis & Smith, 1976; Parkinson et al., 2005). Therefore, conditioned reinforcing properties of drug-associated CSs are associated with a persistent risk of relapse, long after the last pairing of the CS with the primary drug reinforcer, and even if the primary drug reinforcer could be ‘devalued’ by treatment with a receptor antagonist (e.g. naloxone for opiates), a nausea-inducing drug (e.g. disulfiram for alcohol) or elimination of a drug’s effect (e.g. by cocaine vaccine). Conditioned reinforcement has been shown to support the maintenance of drug-seeking behaviour and the acquisition of new drug-seeking responses for a number of different drugs of abuse, including alcohol (Smith et al., 1977; Löf et al., 2007), nicotine (Olausson et al., 2004), opiates (Davis & Smith, 1976; Di Ciano & Everitt, 2004) and cocaine (Arroyo et al., 1998; Di Ciano & Everitt, 2004), indicating its importance for drug-seeking behaviour and relapse.

Conditioned approach

Conditioned approach behaviour, also known as pavlovian approach and usually measured in autoshaping tasks, is the phenomenon by which an animal will approach a CS that has been associated with the behaviourally noncontingent presentation of an appetitive reinforcer (Brown & Jenkins, 1968). This behaviour has an automatic, involuntary quality (Williams & Williams, 1969) and frequently animals will display to the CS the consummatory response appropriate to the primary reinforcer (Breland & Breland, 1961). Conditioned approach behaviour is likely to be of relevance to relapse to drug addiction and to influence drug-seeking behaviour by bringing an individual into proximity with the drug-associated cues and context, where successful drug-seeking and drug-taking responses have previously been made.

Conditioned approach, like conditioned reinforcement, is dependent upon limbic corticostriatal circuitry but it is dissociable from both conditioned reinforcement and conditioned motivation in its neurobiological basis. In contrast to conditioned reinforcement, which is known to depend upon the basolateral amygdala, conditioned approach is dependent upon the central nucleus of the amygdala (Parkinson et al., 2000a), requiring dopaminergic signalling within this area (Hitchcott & Phillips, 1998). It has been hypothesised that the central nucleus forms a functional circuit with the nucleus accumbens, anterior cingulate cortex (Parkinson et al., 2000b) and orbitofrontal cortex (Chudasama & Robbins, 2003). The electrophysiological responses of nucleus accumbens neurons have been shown to encode conditioned approach (Day et al., 2006) but although the nucleus accumbens is necessary for the expression of conditioned approach it does not seem to be necessary for the acquisition of the behaviour (Cardinal et al., 2002; Blaiss & Janak, 2009).

Conditioned approach has been demonstrated most clearly to CSs associated with ingestive reinforcers such as food and sweet liquids, and also to orally-administered alcohol, where there is a clear location of reinforcer delivery and a consummatory response (Krank, 2003; Cunningham & Patel, 2007; Krank et al., 2008). There have been attempts to observe conditioned approach to CSs associated with intravenous drug reinforcement (Kearns & Weiss, 2004; Uslaner et al., 2006) but their success has been limited, perhaps due to the absence of a spatially localised consummatory response (for example, Kearns & Weiss, 2004; found that rats were more likely to approach a cocaine-associated stimulus when they had been exposed to a seeking–taking schedule during training, in which the taking lever models the consummatory responding that is more readily measured in animals retrieving and ingesting food). Conditioned approach is likely to be of clinical relevance to relapse as, for example, in preclinical models the instrumental self-administration of alcohol is increased when the location of an alcohol-associated CS is consistent with the location of a lever that produces alcohol delivery, as compared to when the cue and the lever are spatially separate (Krank et al., 2008). Conditioned approach is probably of particular importance where the conditioned stimulus and the response manipulandum are the same (as in alcohol addiction, where cues such as glasses are linked to the instrumental response of drinking; Tomie, 1995, 1996; Tomie et al., 2008). It has even been argued (Tomie et al., 2002) that conditioned approach can be conceptualised as an animal model of ‘binge drinking’, as it can be used to induce high levels of alcohol consumption in a short period of time, though this has yet to be demonstrated for binge intake of other drugs of abuse.

Conditioned motivation

In addition to mediating conditioned approach and acting as conditioned reinforcers, pavlovian CSs are also capable of invigorating instrumental behaviour; this is termed ‘pavlovian-instrumental transfer’ (PIT) or conditioned motivation (Bindra, 1968). As for conditioned reinforcement, PIT occurs in both general and specific forms (Dickinson & Balleine, 2002), which depend upon at least partially dissociable neurobiological substrates. Specific PIT is dependent upon the basolateral amygdala, as is conditioned reinforcement (Corbit & Balleine, 2005) and, perhaps somewhat counterintuitively, the nucleus accumbens shell (Corbit et al., 2001), whilst general PIT, as for conditioned approach, is dependent upon the central nucleus of the amygdala (Hall et al., 2001; de Borchgrave et al., 2002; Holland & Gallagher, 2003; Corbit & Balleine, 2005) and the nucleus accumbens core (Hall et al., 2001). Conditioned motivation also depends upon dopaminergic signalling, as antagonism of D1 and D2 receptors in the nucleus accumbens also disrupts PIT (Dickinson et al., 2000; Lex & Hauber, 2008) and inactivation of the ventral tegmental area reduces PIT (Murschall & Hauber, 2006; Corbit et al., 2007).

Conditioned motivation may be an important factor in the precipitation of relapse, as drug-seeking can be increased by both drug-associated contexts and discrete CSs. Drug-associated contexts can act as occasion setters to increase drug intake (see Crombag et al., 2008, for review) and it has been demonstrated that alcohol-associated CSs increase alcohol-seeking behaviour through a general PIT effect (Glasner et al., 2005; Corbit & Janak, 2007).

Therefore, drug-associated conditioned stimuli can influence relapse behaviour through at least three different processes: conditioned reinforcement, conditioned approach and conditioned motivation (Fig. 4). Although these processes are psychologically and neurobiologically separable, and can be studied in isolation in a laboratory setting, in the real world of an addicted individual attempting to remain abstinent, all of these processes can be engaged by drug-associated stimuli and are therefore able simultaneously or sequentially to contribute to relapse; effectively, in pavlovian terms there are ‘three routes to relapse’. Thus, it can be hypothesised that all three processes need to be targeted in an anti-relapse, pro-abstinence treatment for addiction. Therapies based upon the extinction of pavlovian associations, such as ‘cue exposure therapy’, where addicted individuals are presented with drug paraphernalia in the absence of drug reinforcement, have had limited success (Conklin & Tiffany, 2002), both because some aspects of these pavlovian associations, such as conditioned reinforcement, are resistant to extinction (Di Ciano & Everitt, 2004) and also because of the context-specificity of extinction (Bouton, 2002; Conklin & Tiffany, 2002), such that any clinic-based therapy does not transfer effectively to the addict’s environment. Therefore, the alternative approach of memory disruption through interfering with the mechanisms underlying reconsolidation of drug memories at retrieval is potentially of great therapeutic value, as diminishing or even erasing these is more likely to reduce the risk of relapse in the long-term than creating a new, context-specific ‘CS–no drug’ memory, as is the case in extinction-based therapies.

Synthesis: the neurochemical mechanisms of addictive drug memory reconsolidation for specific representations

The procedure of cue-induced reinstatement following enforced abstinence, which is used to model relapse in previously self-administering animals, confounds each of the influences by which pavlovian CS–drug memories modulate instrumental relapse behaviour. The finding that NMDAR antagonism, given in conjunction with retrieval, reduces cue-induced reinstatement (Milton et al., 2008a) whilst βAR antagonism at retrieval does not (Fig. 3) has led us to investigate separately the neurochemical mechanisms underlying the reconsolidation of the memories underlying conditioned reinforcement, conditioned approach and conditioned motivation.

We have previously demonstrated that the reconsolidation of the memory underlying conditioned reinforcement is dependent upon both NMDAR-mediated transmission and βAR-mediated transmission (Milton et al., 2008a,b). We have also recently found that, as for the reconsolidation of memories for highly palatable sucrose elicited by re-exposure to associated CSs (Lee & Everitt, 2008), the reconsolidation of the pavlovian memories underlying conditioned approach and conditioned motivation elicited by presentation of alcohol-associated CSs are also dependent upon NMDAR-mediated signalling at reactivation (A.L. Milton, M.J.W. Schramm, J.R. Wawrzynski, F. Gore, F. Oikonomou-Mpegeti, N.Q. Wang, D. Samuel, D. Economidou, B. Everitt, unpublished observations; see Figs 5 and 6). The impairment of conditioned approach and conditioned motivation for alcohol-associated CSs and conditioned reinforcement for cocaine CSs, the ‘three routes to relapse’ (Fig. 4), by the administration of NMDAR antagonists at drug memory retrieval is consistent with the finding that NMDAR antagonism impairs the reconsolidation of the CS–drug memories that contribute to cue-induced reinstatement of both cocaine- and alcohol-seeking (Milton et al., 2008a; von der Goltz et al., 2009).

Figure 5.

 NMDAR antagonism, but not βAR antagonism, in conjunction with retrieval, disrupted the memory underlying PIT for an alcohol-associated CS. Animals were trained separately to associate a pavlovian CS with alcohol delivery, and to orally self-administer alcohol by pressing a lever. (a) Half of the rats underwent a memory reactivation session in conjunction with systemic administration of saline, an NMDAR antagonist or a βAR antagonist, and (b) half received the administration of saline, an NMDAR antagonist or a βAR antagonist without retrieval. Only the NMDAR antagonist given in conjunction with retrieval prevented the invigoration of instrumental responding by the presentation of the pavlovian CS at test; responding was no different from ‘chance’ levels, i.e. equal responding in the presence and absence of the CS, in this group. Adapted from Milton et al., submitted.

Figure 6.

 NMDAR antagonism, but not βAR antagonism, in conjunction with retrieval, disrupted the memory underlying conditioned approach towards an alcohol-associated CS. Rats were trained to associate a light CS+ in one location with the behaviourally noncontingent delivery of alcohol, and a light CS− with no outcome. (a) Half of the rats underwent a memory reactivation session in conjunction with systemic administration of saline, an NMDAR antagonist or a βAR antagonist, and (b) half received the administration of saline, an NMDAR antagonist or a βAR antagonist without retrieval. All of the rats showed discrimination between approach towards the CS+ (black dots) and the CS− (white dots) in a subsequent test conducted in extinction, except for the rats that received the NMDAR antagonist in conjunction with memory retrieval. Adapted from Milton et al., submitted.

Whilst the administration of propranolol in conjunction with memory retrieval impairs subsequent drug conditioned place preference (Bernardi et al., 2006, 2009; Robinson & Franklin, 2007a; Fricks-Gleason & Marshall, 2008) and the capacity of drug-associated CSs to act as conditioned reinforcers (Milton et al., 2008b), propranolol given in conjunction with drug memory reactivation is not effective at disrupting the reconsolidation of the pavlovian memories underlying conditioned motivation and conditioned approach. This has been demonstrated for CSs associated with sucrose reinforcement (Lee & Everitt, 2008) and, more recently, for CSs associated with ethanol reinforcement (Milton et al., submitted; Figs 5 and 6). These data indicate that propranolol is ineffective at disrupting reconsolidation of the memories underlying two out of the three psychological ‘routes to relapse’ by which pavlovian CSs influence instrumental drug-seeking behaviour. This could be due to differences in the intracellular signalling pathways initiated by activity at the NMDAR and βAR, though it is not clear why βAR antagonism would be sufficient to disrupt the reconsolidation of the association underlying conditioned reinforcement, but not conditioned motivation or conditioned approach. Alternatively, it is known that the circuits underlying pavlovian conditioned approach and conditioned motivation are at least partially dissociable from that underlying conditioned reinforcement (Everitt et al., 2001). Thus, it might be predicted that a requirement for βAR-mediated signalling in an area required for conditioned reinforcement, but not for conditioned approach and conditioned motivation, might account for the inability of propranolol, given at retrieval, to prevent reconsolidation of the associations underlying conditioned approach and motivation.

Therefore, although disruption of βAR-mediated signalling may be effective in disrupting the reconsolidation of some types of drug memory, the translation of propranolol-induced disruption of drug memory reconsolidation may be problematic, as propranolol appears not to disrupt all of the pavlovian representations influencing instrumental drug-seeking behaviour and does not prevent approach towards locations of past successful drug-seeking. It is unknown what the effects would be of disrupting only some of the ‘routes to relapse’ in the clinic; whether disrupting one out of the three ‘routes to relapse’ would lead to: (i) a less effective anti-relapse treatment via an increased propensity to relearn the disrupted CS–drug memory through the other, intact memories underlying the other processes maintaining drug-seeking behaviour; or (ii) even an increase in the capacity of the remaining, intact CS–drug memories to influence relapse behaviour. Further work is required to address this issue, and also to identify alternative neurochemical and intracellular mechanisms engaged at retrieval, disruption of which, like antagonism at NMDARs at retrieval, would fully disrupt CS–drug memories and thereby prevent all three of the routes to relapse.

Future directions: other neurotransmitter-mediated mechanisms of reconsolidation of specific representations

By comparison to glutamatergic and adrenergic mechanisms, there has been little research investigating the necessity of other neurotransmitter systems on drug memory reconsolidation. Of these, cholinergic signalling has been the most extensively investigated, with the demonstration that administration of the muscarinic acetylcholine receptor antagonist scopolamine at memory retrieval results in amnesia for a place preference conditioned to morphine (Zhai et al., 2008) or cocaine (Kelley et al., 2007). This is consistent with the hypothesis that cholinergic signalling has an important mnemonic function (Hasselmo, 2006). Similarly, endocannabinoids acting at the CB1 receptor, which may also have a mnemonic function (Mallet & Beninger, 1998), have been shown to be necessary for the reconsolidation of a place preference conditioned to methamphetamine (Yu et al., 2009). Nitric oxide, which, it has been hypothesised, is a retrograde signal in long-term potentiation (O’Dell et al., 1991), has also been shown to be necessary for drug memory reconsolidation. Mice with the nitric oxide synthase (NOS) gene knocked out can acquire a place preference to cocaine but the memory does not persist following retrieval, an effect that was attributed to a failure of reconsolidation mechanisms (Itzhak & Anderson, 2007). Similarly, wild-type mice with a place preference conditioned to cocaine show a reduction in subsequent place preference if a NOS inhibitor is given in conjunction with memory reactivation (Itzhak, 2008). The necessity of these neurotransmitters for the reconsolidation of specific psychological associations underlying pavlovian drug memories has yet to be investigated.

It is perhaps surprising that there is little or no evidence to suggest that dopamine is involved in reconsolidation, especially as the generation of a ‘prediction error’ has been thought to be an important factor in inducing the destabilisation process at retrieval. It has been hypothesised (Pedreira et al., 2004; Eisenhardt & Menzel, 2007) that the mechanisms underlying memory destabilisation, or the induction of memory lability, are activated when the organism detects a ‘mismatch’ between the training and reactivation conditions, namely when there is a ‘prediction error’. Dopaminergic signalling encodes reward prediction errors (Schultz et al., 1995, 1997; Waelti et al., 2001) and therefore an increase in dopamine transmission would be predicted to occur under the ‘mismatch’ conditions of the memory reactivation session. Dopaminergic receptors are also known to interact with NMDARs and influence NMDAR-mediated synaptic plasticity processes (Gurden et al., 2000; Smith-Roe & Kelley, 2000; Floresco et al., 2001; Lee et al., 2002; Dudman et al., 2003) and hence signalling mediated by dopamine receptors might be thought to influence memory reconsolidation mechanisms. To date, however, no published data support this hypothesis.

Conclusions: pro-abstinence, anti-relapse treatments for addiction based upon the disruption of drug memory reconsolidation

The disruption of drug memory reconsolidation could potentially provide a form of pro-abstinence, anti-relapse treatment for drug addiction, which would not be as sensitive to spontaneous recovery, reinstatement or renewal effects as are current therapies based upon cue-exposure or extinction (Conklin & Tiffany, 2002). Furthermore, treatments based upon the disruption of reconsolidation would be predicted to require few, and possibly even a single, treatment with a memory-disrupting drug in order to increase the likelihood of long-lasting abstinence from drugs of abuse. This would clearly be advantageous in avoiding the compliance and tolerance issues associated with more extended, prophylactic anti-relapse treatments. However, there are several challenges that remain in developing a reconsolidation-based treatment for drug addiction. The first of these is that the multitude of ways in which a pavlovian CS, by causing memory retrieval, can influence instrumental drug-seeking and relapse behaviour needs to be addressed experimentally; as discussed above, not all amnestic treatments are equally effective at disrupting the memories that underlie the three routes to relapse discussed herein: conditioned reinforcement, conditioned motivation and conditioned approach. Consequently, treatments such as propranolol, which do not impair the reconsolidation of all of the pavlovian associations underlying these effects, are less effective at reducing subsequent drug-seeking behaviour in a more translationally relevant behavioural model of relapse, cue-induced reinstatement following enforced abstinence. The psychological deconstruction of preclinical models of drug addiction, and studies of the isolated component processes, are an important part of developing reconsolidation-based treatments for the clinic, and understanding the limits on reconsolidation, the so-called ‘boundary conditions’ that prevent reconsolidation occurring (Dudai & Eisenberg, 2004; Nader & Hardt, 2009).

Secondly, treatments need to be developed that can target neurotransmitter systems involved in drug memory reconsolidation, such as NMDAR-mediated signalling, without producing unacceptable side-effects in human patients. The administration of NMDAR antagonists to humans may be of limited clinical use because these drugs have psychotomimetic side effects, but indirect methods of targeting NMDAR-mediated transmission could be of great use in the treatment of addiction. Identification of other neurochemical and intracellular mechanisms involved in the reconsolidation of the memories associated with the various routes to relapse may also lead to the development of new pharmacological treatments that could be applied clinically. Finally, the ‘boundary conditions’ that prevent memory reconsolidation require further investigation; understanding the factors that permit and prevent memory reconsolidation will allow, for instance, more effective behavioural procedures for memory reactivation to be developed. However, considering the comparatively short period in which drug memory reconsolidation has been researched, much progress has been made in understanding the intracellular and neurochemical mechanisms of the process, providing a basis for its future exploitation as a form of treatment for drug addiction in the clinic.

Acknowledgements

The preparation of this manuscript was supported by a UK Medical Research Council Programme Grant, no. G9536855, to B.J.E., and was conducted within the Behavioural and Clinical Neuroscience Institute, which is an initiative jointly funded by the MRC and the Wellcome Trust. A.L.M. was also supported by a Research Fellowship from Downing College, Cambridge.

Abbreviations
AMPAR

AMPA subtype of glutamate receptor

CRE

cAMP-response element

CS

conditioned stimulus

d-APV

d-(2R)-amino-5-phosphonovaleric acid

ERK (also known as MAPK)

extracellular signal-related kinase

MAPK

mitogen-activated protein kinase

MEK (also known as MAPKK)

MAPK/ERK kinase

NMDAR

NMDA subtype of glutamate receptor

PIT

pavlovian-instrumental transfer

SRE

serum-response element

βAR

β-adrenergic receptor

Ancillary