Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal

Overview

Journal Neuron

Publisher Cell Press

Specialty Neurology

Date 2005 Jul 6

PMID 15996553

Citations 556

Authors

Hannah M Bayer

Paul W Glimcher

Affiliations

Soon will be listed here.

Abstract

The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.

Citing Articles

Natural behaviour is learned through dopamine-mediated reinforcement.

Kasdin J, Duffy A, Nadler N, Raha A, Fairhall A, Stachenfeld K Nature. 2025; .

PMID: 40074908 DOI: 10.1038/s41586-025-08729-1.

Exploring Habits in Anorexia Nervosa: Promise, Pitfalls, and Progress.

Foerde K Curr Psychiatry Rep. 2025; .

PMID: 40016535 DOI: 10.1007/s11920-025-01588-7.

Altered neural signalling during reward anticipation in children and early adolescents with high psychotic-like experiences.

Sen P, Knolle F Neuroimage Clin. 2025; 45:103756.

PMID: 39983553 PMC: 11889563. DOI: 10.1016/j.nicl.2025.103756.

Devaluing memories of reward: a case for dopamine.

Fry B, Russell N, Fex V, Mo B, Pence N, Beatty J Commun Biol. 2025; 8(1):161.

PMID: 39900665 PMC: 11790953. DOI: 10.1038/s42003-024-07440-7.

Evidence for direct dopaminergic connections between substantia nigra pars compacta and thalamus in young healthy humans.

Cirillo G, Caiazzo G, Franza F, Cirillo M, Papa M, Esposito F Front Neural Circuits. 2025; 18():1522421.

PMID: 39850841 PMC: 11754968. DOI: 10.3389/fncir.2024.1522421.

References

Gallistel C, Gibbon J . Time, rate, and conditioning. Psychol Rev. 2000; 107(2):289-344. DOI: 10.1037/0033-295x.107.2.289. View

Frank M, Seeberger L, OReilly R . By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science. 2004; 306(5703):1940-3. DOI: 10.1126/science.1102941. View

Glimcher P, Ciaramitaro V, Platt M, Bayer H, Brown M, Handel A . Application of neurosonography to experimental physiology. J Neurosci Methods. 2001; 108(2):131-44. DOI: 10.1016/s0165-0270(01)00365-x. View

Cools R, Barker R, Sahakian B, Robbins T . Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cereb Cortex. 2001; 11(12):1136-43. DOI: 10.1093/cercor/11.12.1136. View

Marsh B, Kacelnik A . Framing effects and risky decisions in starlings. Proc Natl Acad Sci U S A. 2002; 99(5):3352-5. PMC: 122522. DOI: 10.1073/pnas.042491999. View

Packard M, Knowlton B . Learning and memory functions of the Basal Ganglia. Annu Rev Neurosci. 2002; 25:563-93. DOI: 10.1146/annurev.neuro.25.112701.142937. View

Daw N, Kakade S, Dayan P . Opponent interactions between serotonin and dopamine. Neural Netw. 2002; 15(4-6):603-16. DOI: 10.1016/s0893-6080(02)00052-7. View

Montague P, Berns G . Neural economics and the biological substrates of valuation. Neuron. 2002; 36(2):265-84. DOI: 10.1016/s0896-6273(02)00974-1. View

Rogers R, Tunbridge E, Bhagwagar Z, Drevets W, Sahakian B, Carter C . Tryptophan depletion alters the decision-making of healthy volunteers through altered processing of reward cues. Neuropsychopharmacology. 2002; 28(1):153-62. DOI: 10.1038/sj.npp.1300001. View

10.

Fiorillo C, Tobler P, Schultz W . Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003; 299(5614):1898-902. DOI: 10.1126/science.1077349. View

11.

Satoh T, Nakai S, Sato T, Kimura M . Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci. 2003; 23(30):9913-23. PMC: 6740875. View

12.

Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O . Dopamine neurons can represent context-dependent prediction error. Neuron. 2004; 41(2):269-80. DOI: 10.1016/s0896-6273(03)00869-9. View

13.

Kawagoe R, Takikawa Y, Hikosaka O . Reward-predicting activity of dopamine and caudate neurons--a possible mechanism of motivational control of saccadic eye movement. J Neurophysiol. 2003; 91(2):1013-24. DOI: 10.1152/jn.00721.2003. View

14.

Sutton R, Barto A . Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev. 1981; 88(2):135-70. View

15.

LJUNGBERG T, Apicella P, Schultz W . Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol. 1992; 67(1):145-63. DOI: 10.1152/jn.1992.67.1.145. View

16.

Schultz W, Apicella P, LJUNGBERG T . Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci. 1993; 13(3):900-13. PMC: 6576600. View

17.

Mirenowicz J, Schultz W . Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol. 1994; 72(2):1024-7. DOI: 10.1152/jn.1994.72.2.1024. View

18.

Knowlton B, Mangels J, Squire L . A neostriatal habit learning system in humans. Science. 1996; 273(5280):1399-402. DOI: 10.1126/science.273.5280.1399. View

19.

Schultz W, Dayan P, Montague P . A neural substrate of prediction and reward. Science. 1997; 275(5306):1593-9. DOI: 10.1126/science.275.5306.1593. View

20.

Handel A, Glimcher P . Response properties of saccade-related burst neurons in the central mesencephalic reticular formation. J Neurophysiol. 1997; 78(4):2164-75. DOI: 10.1152/jn.1997.78.4.2164. View