1098-1063/asset/HIPO_centre.gif?v=1&s=de7b1567cd4ae1ef6fe4c3be01183f9dbdcd4d8b)
Research Article
You have full text access to this OnlineOpen article
Correlates of reward-predictive value in learning-related hippocampal neural activity
Article first published online: 2 JAN 2009
DOI: 10.1002/hipo.20535
Copyright © 2009 Wiley-Liss, Inc.
1098-1063/asset/cover.gif?v=1&s=006e5eacb07d69754fc016e21b7d8264e80128d3 "Hippocampus")
Additional Information
How to Cite
Okatan, M. (2009), Correlates of reward-predictive value in learning-related hippocampal neural activity. Hippocampus, 19: 487–506. doi: 10.1002/hipo.20535
Publication History
- Issue published online: 13 APR 2009
- Article first published online: 2 JAN 2009
- Manuscript Accepted: 28 OCT 2008
REFERENCES
- ,,,. 2007. Hippocampal CA1 place cells encode intended destination on a maze with multiple choice points. J Neurosci 27: 9769–9779.
- ,,,. 1992. Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J Neurophysiol 68: 945–960.
- ,. 2002. The neural basis of functional brain imaging signals. Trends Neurosci 25: 621–625.
- ,,. 2004. Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7: 404–410.
- ,. 1982. Simulation of anticipatory responses in classical conditioning by a neuron-like adaptive element. Behav Brain Res 4: 221–235.
- ,,. 1983. Neuronlike adaptive elements that can solve difficult learning problems. IEEE Trans Syst Man Cybern 13: 834–846.
- ,,,. 2008. Focal basal ganglia lesions are associated with impairments in reward-based reversal learning. Brain 131: 829–841.
- ,,,,. 1998. A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells. J Neurosci 18: 7411–7425.
- ,,,,. 2001. An analysis of neural receptive field plasticity by point process adaptive filtering. PNAS 98: 12261–12266.
- ,. 2002. Model Selection and Multimodel Inference. New York: Springer.
- . 2002. Theta oscillations in the hippocampus. Neuron 33: 325–340.
- ,,,,. 2008. Place cell firing correlates with memory deficits and amyloid plaque burden in Tg2576 Alzheimer mouse model. PNAS 105: 7863–7868.
- ,,,. 1993. Modification of the responses of hippocampal neurons in the monkey during the learning of a conditional spatial response task. Hippocampus 3: 29–42.Direct Link:
- ,,,,,,,,,,,. 1999. Impaired synaptic plasticity and learning in aged amyloid precursor protein transgenic mice. Nat Neurosci 2: 271–276.
- ,. 1995. Neuronal activity in the supplementary eye field during acquisition of conditional oculomotor associations. J Neurophysiol 73: 1101–1121.
- ,,. 1988. Maximum likelihood identification of neuronal point process systems. Biol Cybern 59: 265–275.
- ,. 2003. An Introduction to the Theory of Point Processes, 2nd ed. New York: Springer-Verlag.
- ,,. 1977. Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39: 1–38.
- ,. 1998. An Introduction to the Bootstrap. New York: Chapman & Hall/CRC.
- . 1999. The hippocampus and mechanisms of declarative memory. Behav Brain Res 103: 123–133.
- ,. 2005. Bridging the gap between brain and behavior: Cognitive and neural mechanisms of episodic memory. J Exp Anal Behav 84: 619–629.
- ,,. 1986. Normal olfactory discrimination learning set and facilitation of reversal learning after medial-temporal damage in rats: Implications for an account of preserved learning abilities in amnesia. J Neurosci 6: 1876–1884.
- ,,,,. 1999. The hippocampus, memory, and place cells: is it spatial memory or a memory space? Neuron 23: 209–226.
- ,. 2006. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440: 680–683.
- ,,. 1989. Spatial selectivity of rat hippocampal neurons: Dependence on preparedness for movement. Science 244: 1580–1582.
- ,,. 2002. Comparison of hippocampal, amygdala, and perirhinal projections to the nucleus accumbens: Combined anterograde and retrograde tracing study in the macaque brain. J Comp Neurol 450: 345–365.Direct Link:
- ,,. 1989. Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. J Neurophysiol 61: 331–349.
- ,,. 1997. Individually distinctive odours represent individual conspecifics in rats. Anim Behav 53: 935–944.
- . 1987. Circuitry of primate prefrontal cortex and regulation of behavior by representational memory. In: MountcastleVB, PlumF, GeigerSR, editors. Handbook of Physiology. The Nervous System, Vol. 5. Bethesda, MD: American Physiological Society. pp 373–417.
- ,,. 1984. Dual pathways connecting the dorsolateral prefrontal cortex with the hippocampal formation and parahippocampal cortex in the rhesus monkey. Neuroscience 12: 719–743.
- . 1996. Adaptive Filter Theory. Englewood Cliffs, NJ: Prentice-Hall.
- ,,. 1995. A model of how the basal ganglia generate and use reward signals that predict reinforcement. In: HoukJC,DavisJL,BeiserDG, editors. Models of Information Processing in the Basal Ganglia. Cambridge: MIT Press.
- ,,. 2003. Reward modulates neuronal activity in the hippocampus of the rat. Behav Brain Res 142: 181–191.
- ,. 2003. Organization of projections from the medial temporal cortical areas to the ventral striatum in macaque monkeys. Korean J Biol Sci 7: 237–248.
- ,,. 1996. Reinforcement learning: A survey. J Artif Intell Res 4: 237–285.
- ,. 2000. Dopamine bonuses. NIPS 13: 131–137.
- ,. 1998. Effects of nitric oxide synthase inhibitor NG-Nitro-L-Arginine Methyl Ester on spatial and cued leaning. Neuroscience 83: 837–841.
- ,,. 1996. Topographic organization of the ventral striatum afferent projection from amygdaloid complex and hippocampal formation. No To Shinkei 48: 534–542.
- ,,,,. 2006. Gradual translocation of spatial correlates of neuronal firing in the hippocampus toward prospective reward locations. Neuron 51: 639–650.
- ,,,,,. 2005. Independent codes for spatial and episodic memory in hippocampal neuronal ensembles. Science 309: 619–623.
- ,. 2000. Response differences in monkey TE and perirhinal cortex: Stimulus association related to reward schedules. J Neurophysiol 83: 1677–1692.
- ,,. 1992. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67: 145–163.
- ,,. 2003. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 339–346.
- ,,. 2004. The neural substrates of reward processing in humans: The modern role of fMRI. Neuroscientist 10: 260–268.
- ,. 1989. Generalized Linear Models, 2nd ed. New York: Chapman & Hall.
- ,,. 1997. Experience-dependent, asymmetric expansion of hippocampal place fields. PNAS 94: 8918–8921.
- ,,,. 1995. Bee foraging in uncertain environments using predictive hebbian learning. Nature 377: 725–728.
- ,,. 1996. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936–1947.
- ,,. 2004. Computational roles for dopamine in behavioral control. Nature 431: 760–767.
- ,,. 2006. Imaging valuation models in human choice. Annu Rev Neurosci 29: 417–448.
- ,,. 2005. Dopamine, uncertainty and TD learning. Behav Brain Funct 1: 6.
- ,,,,. 2003. Temporal difference models and reward-related learning in the human brain. Neuron 28: 329–337.
- ,,,. 2006. Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum. Neuron 49: 157–166.
- ,,. 2005. Analyzing functional connectivity using a network likelihood model of ensemble neural spiking activity. Neural Comput 17: 1927–1961.
- ,. 1971. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Res 34: 171–175.
- ,. 1978. The Hippocampus as a Cognitive Map. Oxford: Clarendon Press.
- ,,,. 2008. Internally generated cell assembly sequences in the rat hippocampus. Science 321: 1322–1327.
- . 1999. Spatial view cells and the representation of place in the primate hippocampus. Hippocampus 9: 467–480.Direct Link:
- ,. 2005. Reward-spatial view representations and learning in the primate hippocampus. J Neurosci 25: 6167–6174.
- ,. 1990. Dopamine neurons of the monkey midbrain: Contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63: 592–606.
- ,,,. 2005. Representation of action-specific reward values in the striatum. Science 310: 1337–1340.
- ,. 1994. Discrimination learning and concept formation in the Arabian horse. J Anim Sci 72: 3080–3087.
- ,. 1994. What are the memory systems of 1994? In: SchacterDL,TulvingE, Editors. Memory Systems. Cambridge: MIT Press. p 1–38.
- ,,. 1999. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci 19: 1876–1884.
- . 1998. Predictive reward signal of dopamine neurons. J Neurophysiol 80: 1–27.
- . 2000. Multiple reward signals in the brain. Nat Rev Neurosci 1: 199–207.
- . 2004. Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr Opin Neurobiol 14: 139–147.
- ,,. 1993. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13: 900–913.
- ,,. 1997. A neural substrate of prediction and reward. Science 275: 1593–1599.
- ,,,,,,,. 2004. Temporal difference models describe higher-order learning in humans. Nature 429: 664–667.
- ,. 1988. Nonparametric Statistics for the Behavioral Sciences. New York: McGraw-Hill.
- ,. 1997. Reinforcement learning for dynamic channel allocation in cellular telephone systems. In Advances in Neural Information Processing Systems: Proceedings of the 1996 Conference. Cambridge, MA: MIT Press. p 974–980.
- ,. 2003. State-space estimation from point process observations. Neural Comput 15: 965–991.
- ,,,,,,,,. 2004. Dynamic analysis of learning in behavioral experiments. J Neurosci 24: 447–461.
- ,. 1991. Random Point Processes in Time and Space 2nd ed. New York: Springer-Verlag.
- ,,. 1993. The structure and organization of memory. Annu Rev Psychol 44: 453–495.
- ,. 1999. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91: 871–890.
- ,. 2001. Temporal difference model reproduces anticipatory neural activity. Neural Comput 13: 841–862.
- . 1988. Learning to predict by the method of temporal difference. Machine Learn 3: 9–44.
- ,. 1981. Toward a modern theory of adaptive networks: Expectation and prediction. Psychol Rev 88: 135–170.
- ,. 1987. A temporal-difference model of classical conditioning. Proceedings of the Ninth Annual Conference of the Cognitive Science Society, Seattle, WA. pp 355–378.
- ,. 1998. Reinforcement Learning: An Introduction. Cambridge: MIT Press. p 261–290.
- ,,,,,. 2004. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7: 887–893.
- . 1994. TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Comput 6: 215–219.
- ,,. 2001. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412: 43–48.
- ,. 1993. Dynamics of the hippocampal ensemble code for space. Science 261: 1055–1058.
- ,,,,,. 2003. Single neurons in the monkey hippocampus and learning of new associations. Science 300: 1578–1581.
- ,. 2006. The role of the basal ganglia in habit formation. Nat Rev Neurosci 7: 464–476.