Neuron, Vol. 40, 1251–1257, December 18, 2003, Copyright 2003 by Cell Press

Direct Activation of the Ventral Striatum in Anticipation of Aversive Stimuli Jimmy Jensen,1 Anthony R. McIntosh,2,3 Adrian P. Crawley,4 David J. Mikulis,4 Gary Remington,1,5 and Shitij Kapur1,5,* 1 PET Centre Centre for Addiction and Mental Health 2 Rotman Research Institute Baycrest Geriatric Centre Toronto, Ontario 3 Department of Psychology 4 Department of Medical Imaging 5 Department of Psychiatry University of Toronto Toronto, Ontario Canada

Summary The brain “reward” system, centered on the limbic ventral striatum, plays a critical role in the response to pleasure and pain. The ventral striatum is activated in animal and human studies during anticipation of appetitive/pleasurable events, but its role in aversive/ painful events is less clear. Here we present data from three human fMRI studies based on aversive conditioning using unpleasant cutaneous electrical stimulation and show that the ventral striatum is reliably activated. This activation is observed during anticipation and is not a consequence of relief after the aversive event. Further, the ventral striatum is activated in anticipation regardless of whether there is an opportunity to avoid the aversive stimulus or not. Our data suggest that the ventral striatum, a crucial element of the brain “reward” system, is directly activated in anticipation of aversive stimuli. Introduction The ability to anticipate pleasure and pain is a fundamental ability of animals and humans. The brain “reward” system centered on the limbic ventral striatum plays a critical role in this ability (Ikemoto and Panksepp, 1999; Schultz et al., 1993). It is suggested that neurons in the ventral striatum, i.e., nucleus accumbens, ventral caudate, and ventral putamen, have access to central representations of reward and thereby participate in the processing of information underlying the motivational control of goal-directed behavior (Schultz et al., 1992). The ventral striatum is appropriately connected to subserve such a function. It receives afferents from diverse limbic/paralimbic regions, i.e., insula, hippocampus, amygdala, and prefrontal cortex, which all are known to be important for motivational processes (Kalivas et al., 1993; Mesulam, 2000; Mogenson et al., 1993), and it projects mainly to the ventral globus pallidus (Mesulam, 2000; Mogenson et al., 1993), thereby being a gateway “from motivation to action” (Mogenson et al., 1993). *Correspondence: [email protected]

The ventral striatum is critically involved in processing appetitive/pleasurable events (Berns et al., 2001; Berridge and Robinson, 1998; Knutson et al., 2001). Animal studies have shown that in appetitive events, the anticipation of reward activates the ventral striatum more than the actual consumption, e.g., dopamine release occurs more robustly in this region during reward anticipation than during reward consumption (Berridge and Robinson, 1998; Ikemoto and Panksepp, 1999; Schultz, 1998). For example, recently Phillips and coworkers (Phillips et al., 2003) showed that dopamine increases in the nucleus accumbens in rats in response to cues that have a learned association with cocaine. In keeping with this, in humans the ventral striatum has been reported to be activated as measured using fMRI, in anticipation of monetary, gustatory, and olfactory rewards (Berns et al., 2001; Breiter et al., 2001; Elliott et al., 2000; Gottfried et al., 2002; Knutson et al., 2001; O’Doherty et al., 2002). The role of the ventral striatum in processing aversive events is currently controversial (Horvitz, 2000, 2002; Mirenowicz and Schultz, 1996; Redgrave et al., 1999; Spanagel and Weiss, 1999). Based on animal studies, some authors argue for a primary role of the ventral striatum in all motivated behaviors (Horvitz, 2002; Salamone, 1994); others argue that it is not involved in aversive events (Schultz, 1998) or propose that any seeming involvement of the reward system in aversive events could be secondary to the relief reward experienced at the offset of aversive stimuli (Horvitz, 2000). Some groups have failed to find an activation of the ventral striatum in anticipation of aversive stimuli in humans (Breiter et al., 2001; Gottfried et al., 2002; Knutson et al., 2001; O’Doherty et al., 2002), but these groups used abstract or mild aversive stimuli (e.g., monetary loss, taste of saline). In animals, mild aversive events like air puffs to the arm of monkeys (Mirenowicz and Schultz, 1996) or taste of hypertonic saline (Mirenowicz and Schultz, 1996) also failed to recruit the ventral striatum. However, stronger and more aversive stimuli like foot shock (Sorg and Kalivas, 1991) and anxiogenic drugs (McCullough and Salamone, 1992) do reliably activate it. Using thermal pain as an aversive stimulus, Becerra and colleagues (Becerra et al., 2001) reported that in addition to the pain circuit, the ventral striatum was activated in humans. Their study was not particularly designed to examine if this activation was observed during anticipation of the aversive stimulus since it used a blocked design with varying temperatures. They found however that the reward circuitry seemed to be activated earlier than regions more closely associated with somatosensory perception. In light of this background, the focus of the present study was to examine: (1) whether mere anticipation of an aversive event leads to activation of the ventral striatum; (2) whether this activation is direct, i.e., related to aversive anticipation or whether it is secondary to the offset of the aversive stimulus; and (3) whether the possibility of actively avoiding the aversive stimulus yields similar results as being passively subjected to it.

Neuron 1252

Figure 1. Statistical Parametric Maps from Experiment 1 Showing Activations in the Ventral Striatum and Anterior Insula The statistical parametric maps (SPM) are the results of a contrast with a greater neuronal response to the S⫹ stimuli relative to the S⫺ stimuli. An uncorrected p value of 0.001 was used as the threshold. The views of the brain selected are at z ⫽ ⫺2 for the axial and y ⫽ 6 for the coronal. The colors refer to t values.

To do so, we used individually titrated, “unpleasant but tolerable,” electrical stimulations as the aversive stimulus (AvS). This AvS was chosen since it is a concrete experience (as opposed to an abstract notion such as loss of points in a game), the intensity of the electrical stimulation for each subject can be adjusted to a given level of unpleasantness; and finally, foot shock has been one of the most commonly used AvS in animal experiments where the ventral striatum has been implicated. Results Experiment 1 To examine if the anticipation of AvS activates the ventral striatum, 11 subjects saw randomly sequenced colored circles appear on a screen, where one color (S⫹) was followed 5 s after onset by a brief (200 ms) AvS in one-third of the trials. The intensity of the AvS was titrated individually prior to the experiment to a level where the subjects experienced it as “unpleasant but tolerable” (mean ⫾ SD; 58V ⫾ 21V). The other colored circles (S⫺) had no consequences. We compared the blood-oxygen level-dependent (BOLD) responses for S⫹ versus S⫺ trials using a random effects analysis implemented in SPM99. Only the S⫹ trials not followed by shock were used in the contrast. The data showed differential activations in bilateral ventral striatum, bilateral anterior insula, and right anterior cingulate (Figures 1 and 2; Table 1). To ensure the reliability of this finding, we analyzed the data using another image analysis software, FSL3, where the time-series

analyses are done in native space rather than on spatially normalized data. The FSL3 analysis confirmed the SPM99 findings and yielded three large clusters with peak activations in the right ventral striatum, left ventral striatum, and left anterior insula respectively (Table 2). These clusters also contained local maxima in the region of right anterior insula and in the region of right anterior cingulate—thus replicating the observations of the SPM analysis. No other regions were activated. Thus, we observed reliable activation of the ventral striatum with mere anticipation of an aversive event. Experiment 2 To address the question of whether the activations in ventral striatum were associated with the onset of the S⫹ (as would be predicted by a “direct” activation idea) or to the offset of the S⫹ (as would be expected if the activation of the ventral striatum was related to the relief from termination of the aversive stimulus), a second experiment was conducted. The experimental paradigm in experiment 2 was similar to the first (n ⫽ 6; US ⫽ 52V ⫾ 25V), but a different imaging protocol was used. Based on an a priori hypothesis, we focused the imaging on the region containing the ventral striatum that is optimally done using coronal acquisition. By focusing on the brain region of interest it became possible to double the sampling rate (TR ⫽ 1.2 s versus TR ⫽ 2.3 s), and thus we collected four volumes of the region of interest during anticipation compared to two in experiment 1 allowing a much more precise estimation of the time course of activation. We modeled the hypothesis as an explicit contrast: (S⫹ onset ⫺ S⫹ offset) ⫺ (S⫺ onset ⫺ S⫺ offset). Activations were obtained in bilateral ventral striatum (left: coordinates ⫺12, 12, ⫺10; peak Z ⫽ 3.60;

Table 1. Activations Comparing Evoked Responses in Anticipation of an Aversive Stimulus and Anticipation of No Consequence, S⫹ versus S⫺, Using SPM99 Region Ventral striatum Left Right Anterior Insula Left Right Anterior Cingulate Right Figure 2. Average ␤ Values in the Ventral Striatum Peak Voxels in Experiment 1 Are Shown with Standard Error Bars

Coordinates

ml

Peak Z

⫺16, 4, ⫺4 24, ⫺4, 4

2.2 2.0

3.85 3.93

⫺36, 8, 0 32, 12, 0

4.8 1.3

4.19 3.43

4, 36, 32

1.1

3.93

Data are thresholded at p ⬍ 0.001 (uncorrected) and only clusters with ⬎15 voxels are reported.

Ventral Striatum Anticipating Aversive Stimuli 1253

Table 2. Activations Comparing Evoked Responses in Anticipation of an Aversive Stimulus and Anticipation of No Consequence, S⫹ versus S⫺, Using FSL3 Region Ventral striatum Left Right Anterior insula Left

Coordinates

ml

Peak Z

p Value (Extent)

⫺8, 10, ⫺2 12, 6, ⫺4

7.5 17.5

4.60 5.29

⬍0.01 ⬍0.001

⫺44, 14, 6

4.9

3.95

⬍0.05

Data are thresholded at Z ⫽ 2.33 and corrected for cluster size.

p ⬍ 0.01 corrected; right: coordinates 12, 10, ⫺2; peak Z ⫽ 4.37; p ⬍ 0.01 corrected; individual ␤ values are shown in Figure 3). Further, the BOLD responses in the peak voxels in the ventral striatum suggest that it is the S⫹ onset rather than offset that activate this region (Figure 4). We also found responses in bilateral anterior insula, but not in the cingulate or in the amygdala. Experiment 3 Organisms seldom respond passively to expected aversive stimuli—they try to escape from them. In experiment 3 we wanted to investigate whether an active avoidance task would yield results similar to the passive paradigms used in experiments 1 and 2. This experiment was based on conditioned avoidance where a motor response was required to avoid the AvS and a neutral stimulus. Seventeen subjects saw the same colored circles as in the previous experiments that denoted anticipation—but the circles now predicted that a target stimulus would appear soon and the subject would have to respond to the target as quickly as possible by pressing a mouse button. One of the colors (S⫹) told the subject that if they did not respond quickly enough during the target they would receive the brief AvS to the index finger. Another color (S⫺), which served as a motor/attentional control condition also required the subjects to respond as quickly as possible, but the consequence in the case of failure was a visual star presented to the screen. The “as quickly as possible” threshold for avoidance was adjusted such that subjects should be successful only in about 75% of the trials. The subjects were told beforehand that S⫹ would be followed by the AvS if not suc-

Figure 3. The Figure Shows Individual ␤ Values in Peak Voxels in the Ventral Striatum Obtained in Experiment 2 The dashed lines refer to mean values. VS, ventral striatum.

cessfully avoided while S⫺ would be followed by a visual star. We collected the subject’s reaction times (RT) for their avoidance response. Further, to assess autonomic arousal, galvanic skin responses (GSR) were also sampled. The titrated intensity level of the AvS was 62V ⫾ 24V. No differences were obtained in the frequency of successfully avoided S⫹ trials and S⫺ trials [77% ⫾ 4% versus 74% ⫾ 8%; t(16) ⫽ 1.80, p ⫽ n.s.]. In keeping with the aversive consequences of S⫹, the avoidance RT were however shorter for the S⫹ events compared to the S⫺ events [255 ⫾ 37 versus 278 ⫾ 48 ms; t(16) ⫽ 3.26, p ⬍ 0.01]. A higher frequency of GSR above 0.05 ␮S was obtained in the S⫹ trials compared to the S⫺ trials [69% ⫾ 36% versus 37% ⫾ 27%; t(8) ⫽ 3.34, p ⬍ 0.01]. The analysis of the fMRI data yielded seven clusters when contrasting S⫹ versus S⫺ (Table 3). We found clusters with peak voxels in regions similar to the passive experiments 1 and 2, i.e., bilateral ventral striatum, bilateral anterior insula, and anterior cingulate (see Figure 5). There were also two additional clusters in the right cerebellum and in the right supramarginal gyrus respectively. The plot of signal change versus time displayed similar responses for S⫹ as the passive paradigm in Experiment 2, while the control condition also showed an increased BOLD response (Figure 6). However, the response peaks in the ventral striatum for S⫹ were about twice as high as for S⫺. Discussion While a previous study has shown that experience of painful thermal stimulation activates the ventral striatum (Becerra et al., 2001), this is to our knowledge the first study to show that mere anticipation of an aversive stimulus activates the ventral striatum in humans. The results are reliable as they have been confirmed in three experiments over 34 subjects, with two different imaging sequences, with two different paradigms, and further confirmed in experiment 1 with two different methods of analysis. The onset/offset analysis confirms a direct activation by the stimulus associated with the AvS, rather than an effect secondary to some kind of relief. There seems to be a link between dopamine, reward, and brain activations in the ventral striatum. Cocaine administration, which leads to dopamine release, in rats has been reported to result in brain activations in the ventral striatum (Marota et al., 2000) which is part of the mesolimbic dopaminergic system that has been implicated in the acutely rewarding actions of cocaine (Breiter et al., 1997). Our findings are consistent with

Neuron 1254

Figure 4. The Percent Signal Change for Peak Voxels in Experiment 2 in the Left Ventral Striatum and Right Ventral Striatum Relative to Time Are Shown with Standard Error Bars The left ventral striatum coordinates: x ⫽ ⫺12, y ⫽ 12, z ⫽ ⫺10; right ventral striatum: x ⫽ 12, y ⫽ 10, z ⫽ ⫺2.

the hypothesis that the anticipation of rewarding (or aversive) events activates the ventral striatum (Berridge and Robinson, 1998; Ikemoto and Panksepp, 1999). These data are consistent with several studies in the animal literature which point to a role of the ventral striatum in modulating behavior to aversive and painful stimuli (for review see Salamone, 1994) suggesting a more general role for the ventral striatum in responding to salient stimuli (Horvitz, 2002). Our data are consistent with a broader bivalent role of the ventral striatum in humans. The failure of previous studies in humans (Breiter et al., 2001; Gottfried et al., 2002; Knutson et al., 2001; O’Doherty et al., 2002) to see activation in anticipation of aversive stimuli may have been due to the mild and perhaps abstract nature of those aversive stimuli The findings in experiment 3, which involved an aversive condition requiring action and a neutral condition also requiring action, support this more general “motivational salience” role for this system rather than being specifically involved in appetitive reward. Besides avoiding the aversive stimulus, the subjects were requested to try to avoid a neutral stimulus and the subjects complied well with the instructions since there are no differences in the number of avoided trials between event types. Further, the avoidance RT is only 9% longer for the S⫺ compared to S⫹ events and GSR above threshold were obtained in 37% of the S⫺ events. These data suggest that the subjects seem to experience some kind of autonomic arousal in the neutral trials although to a significantly lesser extent compared to aversive events. Both event types had reasonable size of BOLD signal change in peak voxels in the ventral striatum (Figure 6), but the aversive event was of larger magnitude compared to the more neutral event. Thus, the findings of the present study support the salience system hypothesis (Horvitz, 2002; Kapur, 2003). Although our paradigm is not conventional Pavlovian conditioning, i.e., we told our subjects beforehand about the association between S⫹ and AvS, it is similar. Activations of the anterior insula and anterior cingulate have been robust in studies with classical aversive conditioning paradigms (Buchel et al., 1998, 1999), but also in anticipation of reward-related outcome (Critchley et al., 2001). The anterior insula and anterior cingulate are considered paralimbic regions and have major inputs from

the amygdala (Mesulam, 2000). The anterior insula relays sensory information back to the amygdala and has been reported to be involved in tactile learning and reactions to pain (Mesulam, 2000) as well as in processing emotionally relevant contexts such as disgust (Phillips et al., 1997). The anterior cingulate is associated with heterogeneous behavioral functions including attention, memory, learning, motivation, pain perception, and visceral function (Casey et al., 1994; Devinsky et al., 1995; Mesulam, 2000). Thus, like the ventral striatum, these structures have extensive inputs from the amygdala, and the activation of the anterior insula and anterior cingulate in concert with the ventral striatum probably represents the negative psychological experience associated with aversive events. Somewhat surprisingly, we did not find direct activation of the amygdala, which has been reported by groups using aversive Pavlovian conditioning (Buchel et al., 1998, 1999; LaBar et al., 1998). A reason for the absence of amygdala activation is probably that the subjects did not have to learn the association between S⫹ and AvS during the experiment and we did not use an extinction phase. The amygdala seems to be activated mainly during learning phases in conditioning paradigms (Buchel et al., 1998, 1999; LaBar et al., 1998), i.e., during acquisition and extinction phases of the association. For example,

Table 3. Activations Comparing Evoked Responses in Anticipation of Avoiding an Aversive Stimulus and Anticipation of Avoiding a Neutral Stimulus, S⫹ versus S⫺, Using SPM99 Region Ventral striatum Left Right Anterior Insula Left Right Anterior Cingulate Left Cerebellum Right Supramarginal gyrus Right

Coordinates

ml

Peak Z

⫺12, 4, ⫺8 8, 4, ⫺4

1.2 2.4

4.73 3.90

⫺40, 20, ⫺4 56, 8, ⫺4 20, 36, ⫺4

6.6 1.5 2.7

4.89 4.29 3.97

⫺8, 4, 32

37.6

6.34

20, ⫺48, ⫺32

30.5

5.44

52, ⫺28, 24

5.2

5.04

Data are thresholded at p ⬍ 0.001 (uncorrected) and only clusters with ⬎15 voxels are reported.

Ventral Striatum Anticipating Aversive Stimuli 1255

Figure 5. Statistical Parametric Maps from Experiment 3 Showing Activations in the Ventral Striatum, Anterior Cingulate, and Anterior Insula The statistical parametric maps (SPM) are the results of a contrast with a greater neuronal response to the S⫹ stimuli relative to the S⫺ stimuli. An uncorrected p value of 0.001 was used as the threshold. The views of the brain selected are at z ⫽ ⫺4 for the axial and y ⫽ 6 for the coronal. The colors refer to t values.

the studies by Buchel and collaborators used a function of time as a regressor that resulted in a significant time x condition interaction for amygdala. We did not use any functions of time since we were interested in anticipation. Another reason might be that other studies using aversive conditioning sometimes exclude subjects who were determined not to show conditioned responses based on their galvanic skin response (e.g., Buchel et al., 1999). We did not exclude any subjects on the basis of any auxiliary measure. Finally, it might be that the amygdala is close to the regions sensitive to magnetic susceptibility (Patterson et al., 2002); we inspected our images in the region of ventral striatum for possible signal dropout but not in the region of amygdala. The additional activations in the cerebellum and right supramarginal gyrus in experiment 3 are consistent with earlier findings in studies investigating preparatory motor functions (Adam et al., 2003; Ramnani and Miall, 2003; Toni et al., 2001). Since both event types in our study demanded motor preparation, our findings suggest that these regions are involved in evaluating the potential motor significance of sensory stimuli similar to the results from Toni and collaborators (Toni et al., 2001). In summary then, our data are consistent with a bivalent role of the ventral striatum, i.e., being a gateway from motivation to action regardless of the valence of the motivation. These findings support the idea that the mesolimbic dopaminergic “reward” system is probably better conceptualized as a salience system (i.e., involved in all outcomes that are likely to motivate the subject’s behavior) (Horvitz, 2002; Kapur, 2003). Experimental Procedures Experiment 1 Subjects Eleven right-handed subjects (five females) aged 28 ⫾ 6 years gave written informed consent and participated in the study according to the guidelines of the local ethical review board.

In an initial session before the scanning, all subjects underwent a structured interview concerning their physical and psychiatric health history. Only healthy subjects were included. Experimental Protocol The paradigm was based on classical aversive conditioning using a 33% partial reinforcement schedule with cutaneous electrical stimulation to the left index finger as aversive stimulus (AvS). The intensity of the 200 ms AvS was titrated individually until it reached a level where the subject said it was “unpleasant but tolerable.” As cues (S⫹ and S⫺) we used yellow/blue circles with a duration of 5 s. The AvS immediately followed the offset of the S⫹ circle in 33% (S⫹paired) of the trials. The S⫺ circle had no consequences. The subjects were told before the experiment began that the S⫹ circles sometimes were followed by the AvS, but were not informed about the reinforcement schedule. A fixation cross was presented between trials and a fixed intertrial interval of 8.8 s was used. The experiment consisted of 75 randomized trials: 15 S⫹ presentations paired with the AvS (S⫹paired), 30 S⫹ trials without (S⫹), and 30 S⫺ trials. Before the experiment started, two additional S⫹paired trials were used to get the subjects familiar with the experimental setup. Apparatus The AvS was delivered by a stimulating bar electrode (30 mm electrode spacing; Chalgren Enterprises, Gilroy, CA) placed on the left index finger using a gel as electrolyte. The electrode was attached to a Grass Instruments SD-9 stimulator (Grass-Telefactor, West Warwick, RI) via well-isolated coaxial cable leads through a waveguide. The subjects used an adjustable mirror located above their eyes to view the back-projected S images on a screen placed at the foot of the scanner bed. The E-prime software (Psychology Software Tools, Inc., Pittsburg, PA) controlled the stimulus presentations, triggered the stimulator, and collected the RT in experiment 3. Image Acquisition MRI scans were acquired by a GE Signa 1.5 T scanner (General Electric, Waukesha, WI) equipped with a standard head coil. In a single session, 480 volumes (28 contiguous axial 4.4 mm thick slices) covering the whole brain were acquired using a T2*-sensitive spiral sequence (TR ⫽ 2300 ms; TE ⫽ 40 ms; flip angle 85⬚; matrix 64 ⫻ 64; FOV 200 ⫻ 200 mm). For localization purposes, IR-Prepped 3D FSPGR T1-weighted anatomical images (124 contiguous axial 1.5 mm thick slices) were acquired (TR ⫽ 12 ms; TE ⫽ 5.4 ms; flip angle 20⬚; matrix 256 ⫻ 256; FOV 200 ⫻ 200 mm). The images were visually inspected for signal dropout due to magnetic susceptibility in the region of ventral striatum. Volumes acquired during AvS were discarded for all subjects as the images showed artifacts in the slices obtained during the delivery of the AvS.

Figure 6. The Percent Signal Change for Peak Voxels in Experiment 3 in the Left Ventral Striatum and Right Ventral Striatum Relative to Time Are Shown with Standard Error Bars Left ventral striatum coordinates: x ⫽ ⫺12, y ⫽ 4, z ⫽ ⫺8 and right ventral striatum: x ⫽ 8, y ⫽ 4, z ⫽ ⫺4.

Neuron 1256

SPM99 Analysis All volumes were realigned to the first volume (Friston et al., 1995b) and the anatomical image was coregistered to a functional image to ensure that they were aligned. Finally, the images were spatially normalized (Friston et al., 1995a) to a standard EPI template (Evans et al., 1993), resampled at 4 ⫻ 4 ⫻ 4 mm and smoothed using a 10 mm FWHM isotropic kernel. The data were analyzed by modeling the event types as ␦ functions convolved with a synthetic hemodynamic response function (HRF). The contrast used in the present study tested for greater responses evoked by S⫹ relative to S⫺. The data were thresholded at p ⬍ 0.001 (uncorrected) and only clusters ⬎15 voxels are reported. FSL3 Analysis FEAT (FMRI Expert Analysis Tool) Version 5.00 was used with the following preprocessing steps: brain extraction using BET (Brain Extraction Tool) (Smith, 2002); motion correction (Jenkinson et al., 2002); resampled at 2 ⫻ 2 ⫻ 2 mm and spatial smoothed using a Gaussian kernel of FWHM 6 mm; time-series statistical analysis carried out using FILM (FMRIB’s Improved Linear Model); coregistration using FLIRT (Jenkinson et al., 2002; Jenkinson and Smith, 2001); and higher-level analysis by FLAME (FMRIB’s Local Analysis of Mixed Effects). Clusters were determined by a statistical threshold Z ⬎ 2.3 and a corrected cluster significance threshold of p ⫽ 0.05 (Friston et al., 1994; Worsley et al., 1992). Experiment 2 Six right-handed subjects (one female) aged 25 ⫾ 3 years participated. The procedure was the same as described in experiment 1. In a single session, 950 partial volumes (15 contiguous coronal 4.0 mm thick slices; 5 slices posterior to the anterior commissure and 10 anterior to it) were acquired using a T2*-sensitive spiral sequence (TR ⫽ 1200 ms; TE ⫽ 40 ms; flip angle 76⬚; matrix 64 ⫻ 64; FOV 200 ⫻ 200 mm). For localization purposes, whole-brain IRprepped 3D FSPGR T1-weighted anatomical images (124 contiguous coronal 1.5 mm thick slices) were acquired (TR ⫽ 12 ms; TE ⫽ 5.4 ms; flip angle 20⬚; matrix 256 ⫻ 256; FOV 220 ⫻ 220 mm). The preprocessing steps of data were the same as in the SPM99 analysis in experiment 1 with the exceptions that the normalized images were resampled at 2 ⫻ 2 ⫻ 2 mm and smoothed using a 6 mm FWHM isotropic kernel and a larger mask was constructed using a modification by Dr. K. Christoff (www-psych.stanford.edu/ kalina/). A larger mask was needed since three subjects showed some signal dropout in the region of the medial ventral striatum, probably due to magnetic susceptibility. The mask is constructed by using the subject’s normalized inplane anatomical image, segmenting out the gray and the white matter from it, then combining them, and then smoothing it with a large kernel (20 mm), in order to be lenient as to which voxels would be included. A fixed effects analysis was performed and small volume corrections (p ⬍ 0.05) were used. These volumes were defined by consulting brain atlases (Duvernoy, 1999; Talairach and Tournoux, 1988). The volumes of special interest in this study, the ventral striatum, were defined as spheres with an 8 mm radius with center coordinates x ⫽ ⫾18, y ⫽ 12, z ⫽ ⫺6. Experiment 3 This paradigm was based on conditioned avoidance. Seventeen right-handed subjects (twelve females) aged 31 ⫾ 9 years participated. The subjects saw a cue (a colored circle) that denoted a preparatory phase. A color of the circle (S⫹) told the subjects that in this trial an AvS would be delivered to the left index finger if not avoided while another color (S⫺) of the circle told indicated that a visual star would appear on the screen if not avoided. The cues (S⫹ and S⫺) prepared the subjects for a target stimulus (a red square) that was presented at the screen. In order to avoid the stimuli the subjects had to press a button on an MRI-compatible mouse during the target with their right index finger. The duration of the target was set to 320 ms in the first trial for both event types. The duration of the targets was subsequently adjusted depending on the subject’s performance by adding 40 ms to the next target’s duration when failing on the current one while deducting 10 ms from the next target’s duration if succeeding on the current one. However, the target durations for the two trial types were independent of each

other. In total, 40 S⫹ trials and 40 S⫺ trials were used. The subjects were told to try to avoid both types of consequences. The cue was presented for 3 s and there was a 1.5 s gap between cue offset and target onset. The AvS immediately followed the targets offset when not avoided. A fixation cross was presented between trials and each trial lasted for 13.8 s; i.e., the intertrial interval was about 9 s depending on the subject’s reaction time. The 200 ms AvS was titrated individually until it reached the intensity where the subject said it was “unpleasant but tolerable” as described in the previous experiments and delivered to the left index finger. The visual star was presented for 200 ms when failing to avoid it. GSR Recording and Analysis The GSR was continuously monitored by PowerLab 2/20 (AD Instruments, Castle Hill, Australia) via long well isolated cables through a wave guide. MRI compatible Ag/AgCl electrodes attached to the terminal phalynx on the left middle and ring finger respectively were used. The GSR was sampled at 10 Hz. Since we did not have access to MRI compatible electrodes in the beginning of the experiment, GSR data are available for nine subjects only. To correct for possible MRI-induced artifacts, the GSR signal was digitally low-pass filtered using a cut-off value of 2 Hz. To determine the GSR, the peak value within the 10 s following the cue onset was taken and subtracted by the mean value of the 500 ms before cue onset. The frequency of values higher than 0.05 ␮S was calculated for each of the four trials types modeled: successful avoidance of AvS, successful avoidance of visual star, failed avoidance of AvS, and failed avoidance of visual star where the two latter were of no interest. Image Acquisition and Analysis The image acquisition, SPM99 preprocessing, and SPM99 analysis was similar to experiment 1. However, 483 volumes were acquired and four regressors were modeled for in the analysis: successful avoidance of AvS (S⫹ trials), successful avoidance of visual star (S⫺ trials), failed avoidance of AvS, and failed avoidance of visual star where the two latter regressors were of no interest. Further, we used a fixed effects model in SPM since subject-specific regressors might introduce variance inhomogenities due to the unbalanced design obtained, which violates the assumptions for a random effects model within SPM (Holmes and Friston, 1998). The cue onsets were defined as event onsets in the model similarly to the other experiments. Acknowledgments We would like to thank Terry Bell, Peter Bloomfield, Garry Detzler, Ted Harris-Brandts, and Irina Vitcu for assisting with their technical expertise. This study was supported from a CRC Chair to S.K. Received: March 18, 2003 Revised: July 11, 2003 Accepted: October 23, 2003 Published: December 17, 2003 References Adam, J.J., Backes, W., Rijcken, J., Hofman, P., Kuipers, H., and Jolles, J. (2003). Rapid visuomotor preparation in the human brain: a functional MRI study. Brain Res. Cogn. Brain Res. 16, 1–10. Becerra, L., Breiter, H.C., Wise, R., Gonzalez, R.G., and Borsook, D. (2001). Reward circuitry activation by noxious thermal stimuli. Neuron 32, 927–946. Berns, G.S., McClure, S.M., Pagnoni, G., and Montague, P.R. (2001). Predictability modulates human brain response to reward. J. Neurosci. 21, 2793–2798. Berridge, K.C., and Robinson, T.E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369. Breiter, H.C., Aharon, I., Kahneman, D., Dale, A., and Shizgal, P. (2001). Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30, 619–639. Breiter, H.C., Gollub, R.L., Weisskoff, R.M., Kennedy, D.N., Makris, N., Berke, J.D., Goodman, J.M., Kantor, H.L., Gastfriend, D.R., Riorden, J.P., et al. (1997). Acute effects of cocaine on human brain activity and emotion. Neuron 19, 591–611.

Ventral Striatum Anticipating Aversive Stimuli 1257

Buchel, C., Morris, J., Dolan, R.J., and Friston, K.J. (1998). Brain systems mediating aversive conditioning: an event-related fMRI study. Neuron 20, 947–957. Buchel, C., Dolan, R.J., Armony, J.L., and Friston, K.J. (1999). Amygdala-hippocampal involvement in human aversive trace conditioning revealed through event-related functional magnetic resonance imaging. J. Neurosci. 19, 10869–10876. Casey, K.L., Minoshima, S., Berger, K.L., Koeppe, R.A., Morrow, T.J., and Frey, K.A. (1994). Positron emission tomographic analysis of cerebral structures activated specifically by repetitive noxious heat stimuli. J. Neurophysiol. 71, 802–807. Critchley, H.D., Mathias, C.J., and Dolan, R.J. (2001). Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 29, 537–545. Devinsky, O., Morrell, M.J., and Vogt, B.A. (1995). Contributions of anterior cingulate cortex to behaviour. Brain 118, 279–306. Duvernoy, H.M. (1999). The Human Brain: Surface, Blood Supply, and Three-Dimensional Sectional Anatomy (New York: Springer). Elliott, R., Friston, K.J., and Dolan, R.J. (2000). Dissociable neural responses in human reward systems. J. Neurosci. 20, 6159–6165. Evans, A.C., Collins, D.L., Mills, S.R., Brown, E.D., Kelly, R.L., and Peters, T.M. (1993). 3D statistical neuroanatomical models from 305 MRI volumes. Proceedings of the Institute of Electrical and Electronics Engineering - Nuclear Science Symposium and Medical Imaging 3, 1813–1817. Friston, K.J., Worsley, K.J., Frackowiak, R.S., Maziotta, J.C., and Evans, A.C. (1994). Assessing the significance of focal activations using their spatial extent. Hum. Brain Mapp. 1, 214–220. Friston, K.J., Ashburner, J., Frith, C.D., Poline, J.B., Heather, J., and Frackowiak, R.S. (1995a). Spatial registration and normalization of images. Hum. Brain Mapp. 2, 1–25. Friston, K.J., Holmes, A.P., Worsley, K.J., Poline, J.B., Frith, C.D., and Frackowiak, R.S. (1995b). Statistical parametric maps in functional imaging: a general linear approach. Hum. Brain Mapp. 2, 189–210. Gottfried, J.A., O’Doherty, J., and Dolan, R.J. (2002). Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J. Neurosci. 22, 10829– 10837. Holmes, A.P., and Friston, K.J. (1998). Generalizability, random effects, and population inference. Neuroimage 7, S754. Horvitz, J.C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96, 651–656. Horvitz, J.C. (2002). Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum. Behav. Brain Res. 137, 65–74. Ikemoto, S., and Panksepp, J. (1999). The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res. Brain Res. Rev. 31, 6–41. Jenkinson, M., and Smith, S. (2001). A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5, 143–156. Jenkinson, M., Bannister, P., Brady, M., and Smith, S. (2002). Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17, 825–841. Kalivas, P.W., Churchill, L., and Klitenick, M.A. (1993). The circuitry mediating the translation of motivational stimuli into adaptive motor responses. In Limbic Motor Circuits and Neuropsychiatry, P.W. Kalivas and C.D. Barnes, eds. (Boca Raton, FL: CRC Press), pp. 237–287. Kapur, S. (2003). Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am. J. Psychiatry 160, 13–23. Knutson, B., Adams, C.M., Fong, G.W., and Hommer, D. (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21, RC159. LaBar, K.S., Gatenby, J.C., Gore, J.C., LeDoux, J.E., and Phelps, E.A. (1998). Human amygdala activation during conditioned fear acquisition and extinction: a mixed-trial fMRI study. Neuron 20, 937–945.

Marota, J.J., Mandeville, J.B., Weisskoff, R.M., Moskowitz, M.A., Rosen, B.R., and Kosofsky, B.E. (2000). Cocaine activation discriminates dopaminergic projections by temporal response: an fMRI study in Rat. Neuroimage 11, 13–23. McCullough, L.D., and Salamone, J.D. (1992). Anxiogenic drugs beta-CCE and FG 7142 increase extracellular dopamine levels in nucleus accumbens. Psychopharmacology (Berl.) 109, 379–382. Mesulam, M.-M. (2000). Behavioral neuroanatomy: large-scale networks, association cortex, frontal syndromes, the limbic system, and hemispheric specializations. In Principles of Behavioral and Cognitive Neurology, M.-M. Mesulam, ed. (New York: Oxford University Press, Inc.), pp. 1–120. Mirenowicz, J., and Schultz, W. (1996). Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449–451. Mogenson, G.J., Brudzynski, S.M., Wu, M., Yang, C.R., and Yim, C.C.Y. (1993). From motivation to action: a review of dopaminergic regulation of limbic, nucleus accumbens, ventral pallidum, pedunculopontine nucleus circuittries involved in limbic-motor integration. In Limbic Motor Circuits and Neuropsychiatry, P.W. Kalivas and C. D. Barnes, eds. (Boca Raton, FL: CRS Press), pp. 193–236. O’Doherty, J.P., Deichmann, R., Critchley, H.D., and Dolan, R.J. (2002). Neural responses during anticipation of a primary taste reward. Neuron 33, 815–826. Patterson, J.C., II, Ungerleider, L.G., and Bandettini, P.A. (2002). Task-independent functional brain activity correlation with skin conductance changes: an fMRI study. Neuroimage 17, 1797–1806. Phillips, M.L., Young, A.W., Senior, C., Brammer, M., Andrew, C., Calder, A.J., Bullmore, E.T., Perrett, D.I., Rowland, D., Williams, S.C., et al. (1997). A specific neural substrate for perceiving facial expressions of disgust. Nature 389, 495–498. Phillips, P.E., Stuber, G.D., Heien, M.L., Wightman, R.M., and Carelli, R.M. (2003). Subsecond dopamine release promotes cocaine seeking. Nature 422, 614–618. Ramnani, N., and Miall, R.C. (2003). Instructed delay activity in the human prefrontal cortex is modulated by monetary reward expectation. Cereb. Cortex 13, 318–327. Redgrave, P., Prescott, T.J., and Gurney, K. (1999). Is the shortlatency dopamine response too short to signal reward error? Trends Neurosci. 22, 146–151. Salamone, J.D. (1994). The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav. Brain Res. 61, 117–133. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27. Schultz, W., Apicella, P., Scarnati, E., and Ljungberg, T. (1992). Neuronal activity in monkey ventral striatum related to the expectation of reward. J. Neurosci. 12, 4595–4610. Schultz, W., Apicella, P., and Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13, 900–913. Smith, S.M. (2002). Fast robust automated brain extraction. Hum. Brain Mapp. 17, 143–155. Sorg, B.A., and Kalivas, P.W. (1991). Effects of cocaine and footshock stress on extracellular dopamine levels in the ventral striatum. Brain Res. 559, 29–36. Spanagel, R., and Weiss, F. (1999). The dopamine hypothesis of reward: past and current status. Trends Neurosci. 22, 521–527. Talairach, J., and Tournoux, P. (1988). Co-Planar Stereotaxic Atlas of the Human Brain (New York: Thieme). Toni, I., Thoenissen, D., and Zilles, K. (2001). Movement preparation and motor intention. Neuroimage 14, S110–S117. Worsley, K.J., Evans, A.C., Marrett, S., and Neelin, P. (1992). A threedimensional statistical analysis for CBF activation studies in human brain. J. Cereb. Blood Flow Metab. 12, 900–918.