Neuroanatomy of the reinforcement system of the brain
Christian P. Müller SGDP-Center, Institute of Psychiatry, King‘s College London
Reinforcement research
...learn about mechanisms of euphoria, reward, and reinforcement. How to get happy on purpose?
drugs as seemingly „artificial paradises“
Reinforcement research
... De Quincey, Baudelaire, Freud....
drugs as seemingly „artificial paradises“
Reinforcement research
Research on mechanisms of addiction was and is closely linked with research on euphoria, hedonia, reward, and reinforcement
drugs as seemingly „artificial paradises“
Reinforcement circuits of the brain
Olds & Milner (1954): Intracranial Self-stimulation
Reinforcement circuits of the brain
Olds & Milner (1954): Intracranial Self-stimulation around
lateral hypothalamus
assumption: this brain region mediates the „pleasure“ of sensory stiumuli and drugs
James Olds: „pleasure center“ hypothesis
The dopamine hypothesis
Roy Wise: Dopamine-hypothesis (1980)
Dopamine-synapse codes for the hedonic/reward value of a stimulus in the brain The dopaminergic synapse is the place in the brain where the hedonic value of a stimulus is associated with its sensory properties.
Beyond ncl. accumbens dopamine
Neurons
Neurotransmitters
Projections
Brain areas
Which brain areas are involved in reinforcement?
Brain areas animal models of addiction
inactivation of a brain area • lesion: removal, cooling, use of neurotoxins • local anaesthetics
post mortem measurements • neurtoransmitters • autoradiography
Brain areas human addicts or occasional users - acute intoxication - withdrawal
Imageing e.g. fMRI, PET
humans and animals
Brain areas ventromed. thalamus prefrontal cortex
ventral pallidum
ncl. accumbens
hypothalamus
amygdala
ventral tegmental area
hippocampus
Projections How are these areas interconnected ?
Projections Neuroanatomy
anterograde tracer • injected in area of origin • transported along the axons to projection areas
retrograde tracer • injected in target area • uptaken by synapses, retrograde transport along the axons to soma
Projections
Berridge & Robinson, TINS (2003)
Projections
ventromed. thalamus ventral pallidum
prefrontal cortex
hypothalamus
ncl. accumbens amygdala
ventral tegmental area
hippocampus
Projections prefrontal cortex
premotor. cortex ventromed. thalamus
hippocampus
amygdala
ncl. accumbens
ventral pallidum
hypothalamus ventral tegmental area
Projections prefrontal cortex
• receives highly processed information from all sense modalities • convergence area • important a.o. for plannig of behavior and impulse control
Projections prefrontal cortex
premotor. cortex ventromed. thalamus
hippocampus
amygdala
ncl. accumbens
ventral pallidum
hypothalamus ventral tegmental area
Projections • essential for learning (not for retrieval) hippocampus
• especially for spacial information • important for spatial information (spatial cues) regarding to drug use
Projections prefrontal cortex
premotor. cortex ventromed. thalamus
hippocampus
amygdala
ncl. accumbens
ventral pallidum
hypothalamus ventral tegmental area
Projections
• important for emotion, especially anxiety
amygdala
• involved in discrete cue processing related to drug use • stores also stimulus-reward associations
Projections • hippocampus and amygdala input gate PFC input to Nac
prefrontal cortex
hippocampus
amygdala
ncl. accumbens
• bring Nac neurons in depolarized „sensitive“ state
O‘Donnell and Grace, J. Neurosci., 1995
Projections prefrontal cortex
premotor. cortex ventromed. thalamus
hippocampus
amygdala
ncl. accumbens
ventral pallidum
hypothalamus ventral tegmental area
Projections • several nuclei with distinct functions • processes information about internal milieu of the body (e.g. glucose, wasser, temperatur) • may be involved in processing of hedonic value of drugs hypothalamus
Projections prefrontal cortex
premotor. cortex ventromed. thalamus
hippocampus
amygdala
ncl. accumbens
ventral pallidum
hypothalamus ventral tegmental area
Projections •
neuronal activity increases when after a certain behaviour an unexpected „reward“ (e.g. drugreward) occurs
•
pre-ceeding stimulus is assigned an incentive salience („wanting“ rather than „liking“)
•
repeated occurrence may lead to incentive sensitization
ventral tegmental area
Projections When stimulus occurs: • increase likelyhood for rewarded behaviour • incentive salience „energizes“ addiction-related behaviour
ventral tegmental area
The nature of the dopaminergic input
The nature of the dopaminergic input • DA neurons fire upon free unpredicted rewards and during learning of stimulusreward associations • no DA activation when association is learned (familiar)
The nature of the dopaminergic input • during correct learning trials: DA neurons fire upon reward • during incorrect learning trials (error): drop in activity of DA neurons at times of expected reward
DA neuronal activity codes not for reward per se but for prediction error
Projections prefrontal cortex
premotor. cortex ventromed. thalamus
hippocampus
amygdala
ncl. accumbens
ventral pallidum
hypothalamus ventral tegmental area
Projections limbic-motor interface between structures processing sensory and interoceptive information and motor output structures ncl. accumbens
translates „motivation to action“ Mogenson et al., Prog Neurobiol. (1983)
Projections - processes changes in the predictive importance of sensory stimuli
ncl. accumbens
- projects to motor circuits to influence behavioural planning
Projections -
important in the pre-habitual stage of drug seeking and self-administration
ncl. accumbens
-
once these behaviours became habits – less important ( dorsal striatum)
Projections prefrontal cortex
premotor. cortex ventromed. thalamus
hippocampus
amygdala
ncl. accumbens
ventral pallidum
hypothalamus ventral tegmental area
Projections premotor. cortex ventromed. thalamus
ventral pallidum
motor nuclei involved in planing and execution of locomotor behaviour
Transmitter
How to identify other important neurotransmitters
Transmitter Addiction-related behavioural paradigms
Transmitter-inactivation • by synthesis blockers • reducing precursor levels (e.g. tyrosin-free diet) • transmitter-specific neurotoxines (e.g. 6OH-DA; 5,7-DHT) • receptor-antagonists or knock-out
systemic or local
Transmitter Addiction-related behavioural paradigms
Transmitter-stimulation • by blocking metabolizing enzymes (e.g. MAO blocker; AChE-blocker) • releaser or reuptake blocker • precursor (tyrosin-rich diet; L-DOPA) • receptor-agonists or overexpression
systemic or local
Transmitter Important transmitters: serotonin
+/-
GABA
-
Glutamat
+
Transmitter Glu +
premotor. cortex
prefrontal cortex
ventromed. thalamus GABA Glu +
hippocampus Glu +
amygdala
Ncl. accumbens
Glu + DA +/-
GABA -
ventral pallidum
hypothalamus ventral tegmental area
Transmitter
Glu +
DA +/Glu +
GABA -
Ncl. accumbens
Transmitter
Ncl. accumbens
McBride et al. Behav Brain Res. (1999)
Transmitter Furtyher modulating transmitters: • Noradrenaline • Acetylcholine • Histamine • CRF • other neuropeptides • ???
The refined model: Ikemoto & Panksepp (1999)
Distinction between mechanisms of approximation and consumption
The refined model: Ikemoto & Panksepp (1999) Required for approximation:
declarative perception • memory • stimulus-respons associations • nigrostriatal DA-system
incentive attribution • to stimulus
The refined model: Ikemoto & Panksepp (1999) Two different systems for approximation:
1.) Flexible system • During learning of new incentives (outcome dependent)
2.) Habit system • for well trained behavioral responses (stimulus dependent)
The refined model: Ikemoto & Panksepp (1999)
1.) Flexible approximation • Incentive detected (at low stimulus reward association )
• Ncl. accumbens DA activated • DA „energices“ flexible approximation response • ... and enhances incentive properties of stimulus representation
The refined model: Ikemoto & Panksepp (1999)
2.) Habituated approximation • well established stimulusresponse association • Independent from Nac and Nac DA • requires nigrostriatal DA system
The refined model: Ikemoto & Panksepp (1999)
Consumption • by brain stem mechanisms • Nac DA only role in learning/ incentive assignment for declarative memories • if consumed stimulus (UCS) pleasant and unexpected: incentive assignment to antecedent CS • until stimulus-response association formed
Anatomical evidence
• Combination of anterograde and retrograde tracing in Macaque monkeys
Importance for addiction
• trained Rhesus monkeys to self-administer cocaine • measured glucose utilization after acute (5d) and chronic exposure (100d) after last session with 14C-DG • reported initial decrease in ncl. accumbens • with increased administration: spreading to dorsal striatum
Importance for addiction
Importance for addiction • disconnection study • interuption of striato-nigral-striato circuitry • Cocaine selfadministration: second order schedule
DA antagonist lesion of Nac core
Belin and Everitt, Neuron, 2008
Importance for addiction • decreased drug seeking behaviour in trained rats in second order schedule of reinforcement
Belin and Everitt, Neuron, 2008
Literature Di Chiara G (2002) Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav Brain Res 137:75-114. Everitt BJ, Robbins TW (2005) Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature Neuroscience 8(11):1481-1489. Kalivas PW, Volkow ND (2005) The neural basis of addiction: A pathology of motivation and choice. American Journal of Psychiatry 162:1403-1413. Koob GF, Sanna PP, Bloom FE (1998) Neuroscience of addiction. Neuron 21:467-476. Koob GF, Le MM (2008) Addiction and the brain antireward system. Annu Rev Psychol 59:29-53. McBride WJ, Murphy JM, Ikemoto S (1999) Localization of brain reinforcement mechanisms: intracranial self-administration and intracranial place-conditioning studies. Behav Brain Res 101:129152. Robinson TE, Berridge KC (2003) Addiction. Annu Rev Psychol 54:25-53. Wise RA (2002) Brain reward circuitry: Insights from unsensed incentives. Neuron 36:229-240.
