1460-9568/asset/EJN_left.gif?v=1&s=3d5223e01f3255a775e3d3fc8d54bc1c38677c6c)
You have free access to this content
Subjective neuronal coding of reward: temporal value discounting and risk
Article first published online: 24 MAY 2010
DOI: 10.1111/j.1460-9568.2010.07282.x
© The Author (2010). Journal Compilation © Federation of European Neuroscience Societies and Blackwell Publishing Ltd
Issue
European Journal of Neuroscience
Special Issue: On the occasion of the 7th FENS Forum, Amsterdam 2010
Volume 31, Issue 12, pages 2124–2135, June 2010
Additional Information
How to Cite
Schultz, W. (2010), Subjective neuronal coding of reward: temporal value discounting and risk. European Journal of Neuroscience, 31: 2124–2135. doi: 10.1111/j.1460-9568.2010.07282.x
Publication History
- Issue published online: 21 JUN 2010
- Article first published online: 24 MAY 2010
- Received 11 March 2010, revised 14 April 2010, accepted 15 April 2010
Abbreviations
- BOLD
blood oxygen level-dependent
- ITI
intertrial interval
- PEST
parameter estimation by sequential testing
- PIP
peak interval procedure
References
- (1974) Impulse control in pigeons. J. Exp. Anal. Behav., 21, 485–489.
- (1975) Specious rewards: a behavioral theory of impulsiveness and impulse control. Psychol. Bull., 82, 463–496.
- , & (2001) Size matters: impact of item size and quantity on array choice by chimpanzees (Pan troglodytes). J. Comp. Psychol., 115, 106–110.
- , & (1980) An empirical demonstration of risk-sensitive foraging preferences. Anim. Behav., 28, 820–830.
- , , , , & (1990) Risk-sensitivity: ambient temperaure effects foraging choice. Anim. Behav., 39, 338–345.
- , , , & (2001) Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science, 292, 2499–2501.
- & (1996) Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. J. Neurophysiol., 75, 1673–1686.
- , , , , , & (2007) Activation of prefrontal cortex by transcranial direct current stimulation reduces appetite for risk during ambiguous decision making. J. Neurosci., 27, 6212–6218.
- , & (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299, 1898–1902.
- , & (2008) The temporal precision of reward prediction in dopamine neurons. Nat. Neurosci., 11, 966–973.
- , & (2009) Short term temporal discounting of reward value in human ventral striatum. J. Neurophysiol., 101, 1507–1523.
- , , , , & (2006) Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. J. Neurosci., 26, 13213–13217.
- , , , & (1999) Theory and method in the quantitative analysis of ‘impulsive choice’ behaviour: implications for psychopharmacology. Psychopharmacology, 146, 362–372.
- (1980) CS-US interval as a determinant of the form of Pavlovian appetitive conditioned responses. J. Exp. Psychol. Anim. Behav. Process., 6, 155–174.
- , & (1998) Influence of reward expectation on behavior-related neuronal activity in primate striatum. J. Neurophysiol., 80, 947–963.
- & (2002) Within-subject comparison of real and hypothetical money rewards in delay discounting. J. Exp. Anal. Behav., 77, 129–146.
- & (2007) The neural correlates of subjective value during intertemporal choice. Nat. Neurosci., 10, 1625–1633.
- , , , , , , & (2004) Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology, 175, 206–214.
- , , & (2009) Valuation of uncertain and delayed rewards in primate prefrontal cortex. Neural Netw., 22, 294–304.
- & (1995) Modeling myopic decisions: evidence for hyperbolic delay-discounting within subjects and amounts. Organ. Behav. Hum. Decis. Process., 64, 22–30.
- , , , , , & (2006) Disruption of right prefrontal cortex by low-frequency repetitive transcranial magnetic stimulation induces risk-taking behavior. J. Neurosci., 26, 6469–6472.
- , , , & (2005) Distributed neural representation of expected value. J. Neurosci., 25, 4806–4812.
- & (2008) Influence of reward delays on responses of dopamine neurons. J. Neurosci., 28, 7837–7846.
- (1990) A Course in Microeconomic Theory. Pearson Education Ltd, Harlow.
- & (1979) Approximating expected utility by a function of mean and variance. Am. Econ. Rev., 69, 308–317.
- & (1992) Anomalies in intertemporal choice: evidence and an interpretation. Q. J. Econ., 107, 573–597.
- (2000) Utility of Gains and Losses: Measurement-Theoretical and Experimental Approaches. Lawrence Erlbaum, Mahwah, NJ.
- (1952) Portfolio selection. J. Finance, 7, 77–91.
- , , & (2004) Separate neural systems value immediate and delayed monetary rewards. Science, 306, 503–507.
- , , , & (2007) Time discounting for primary rewards. J. Neurosci., 27, 5796–5804.
- (2005) Neuropsychology of timing and time perception. Brain Cogn., 58, 1–8.
- (2004) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr. Opin. Neurobiol., 14, 769–776.
- (1927) Conditioned Reflexes. Oxford University Press, London.
- , & (2006) Neural differentiation of expected reward and risk in human subcortical structures. Neuron, 51, 381–390.
- , , & (2007) Planning for the future by western scrub-jays. Nature, 445, 919–921.
- (1967) Pavlovian conditioning and its proper control procedures. Psychol. Rev., 74, 71–80.
- , , & (1997) Determination of discount functions in rats with an adjusting-amount procedure. J. Exp. Anal. Behav., 67, 353–366.
- (1981) Isolation of an internal clock. J. Exp. Psychol. Anim. Behav. Process., 7, 242–268.
- & (1988) Adjusting delay to reinforcement: comparing choice in pigeons and humans. J. Exp. Psychol. Anim. Behav. Process., 14, 105–117.
- & (2005a) Neuronal activity dependent on anticipated and elapsed delay in macaque prefrontal cortex, frontal and supplementary eye fields, and premotor cortex. J. Neurophysiol., 94, 1469–1497.
- & (2005b) Neuronal activity in orbitofrontal cortex reflects the value of time. J. Neurophysiol., 94, 2457–2471.
- , & (2006) Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron, 51, 509–520.
- , & (2007) Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci., 10, 1615–1624.
- , , , & (2009) Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J. Neurosci., 29, 13365–13376.
- , , & (2005) Representation of action-specific reward values in the striatum. Science, 310, 1337–1340.
- (1937) Some aspects of the pure theory of capital. Q. J. Econ., 51, 469–496.
- & (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J. Neurophysiol., 63, 607–624.
- (1975) Pseudoconditioned jaw movements of the rabbit reflect associations conditioned to contextual background cues. J. Exp. Psychol. Anim. Behav. Process., 104, 245–260.
- & (1998) Reinforcement Learning. MIT Press, Cambridge, MA.
- , , , , & (2004) Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci., 7, 887–893.
- , & (2003) Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci., 23, 10402–10410.
- , & (2005) Adaptive coding of reward value by dopamine neurons. Science, 307, 1642–1645.
- , , & (2007a) Learning-related human brain activations reflecting individual finances. Neuron, 54, 167–175.
- , , & (2007b) Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neurophysiol., 97, 1621–1632.
- , , , & (2009) Risk-dependent reward value signal in human prefrontal cortex. Proc. Natl Acad. Sci. USA, 106, 7185–7190.
- , & (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature, 412, 43–48.
- (1994) Separating marginal utility and probabilistic risk aversion. Theor. Decis., 36, 1–44.
- (1996) Reward expectancy in primate prefrontal neurons. Nature, 382, 629–632.
- , , , & (2001) Behavioral reactions reflecting differential expectations of outcomes in monkeys. Exp. Brain Res., 140, 511–518.
- & (1997) Perceived risk attitudes: relating risk perception to risky choice. Manage. Sci., 43, 123–144.
- , & (2002) A domain-specific risk-attitude scale: measuring risk perceptions and risk behaviors. J. Behav. Decis. Mak., 15, 263–290.
- , & (2004) Predicting risk sensitivity in humans and lower animals: risk as variance or coefficient of variation. Psychol. Rev., 111, 430–445.
- , , & (2004) Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci., 24, 4718–4722.
- (2002) Brain reward circuitry: insights from unsensed incentives. Neuron, 36, 229–240.
- , & (2007) Time and decision making: differential contribution of the posterior insular cortex and the striatum during a delay discounting task. Exp. Brain Res., 179, 643–653.