Dopaminergic Prediction Errors in the Ventral Tegmental Area Reflect a Multithreaded Predictive Model

Overview

Journal Nat Neurosci

Date 2023 Apr 20

PMID 37081296

Authors

Yuji K Takahashi

Thomas A Stalnaker

Lauren E Mueller

Sevan K Harootonian

Angela J Langdon

Geoffrey Schoenbaum

Affiliations

Soon will be listed here.

Abstract

Dopamine neuron activity is tied to the prediction error in temporal difference reinforcement learning models. These models make significant simplifying assumptions, particularly with regard to the structure of the predictions fed into the dopamine neurons, which consist of a single chain of timepoint states. Although this predictive structure can explain error signals observed in many studies, it cannot cope with settings where subjects might infer multiple independent events and outcomes. In the present study, we recorded dopamine neurons in the ventral tegmental area in such a setting to test the validity of the single-stream assumption. Rats were trained in an odor-based choice task, in which the timing and identity of one of several rewards delivered in each trial changed across trial blocks. This design revealed an error signaling pattern that requires the dopamine neurons to access and update multiple independent predictive streams reflecting the subject's belief about timing and potentially unique identities of expected rewards.

Citing Articles

The curious case of dopaminergic prediction errors and learning associative information beyond value.

Kahnt T, Schoenbaum G Nat Rev Neurosci. 2025; 26(3):169-178.

PMID: 39779974 DOI: 10.1038/s41583-024-00898-8.

Dopaminergic responses to identity prediction errors depend differently on the orbitofrontal cortex and hippocampus.

Takahashi Y, Zhang Z, Kahnt T, Schoenbaum G bioRxiv. 2025; .

PMID: 39763911 PMC: 11702580. DOI: 10.1101/2024.12.11.628003.

Generalized cue reactivity in rat dopamine neurons after opioids.

Lehmann C, Miller N, Nair V, Costa K, Schoenbaum G, Moussawi K Nat Commun. 2025; 16(1):321.

PMID: 39747036 PMC: 11697388. DOI: 10.1038/s41467-024-55504-3.

Reward Bases: A simple mechanism for adaptive acquisition of multiple reward types.

Millidge B, Song Y, Lak A, Walton M, Bogacz R PLoS Comput Biol. 2024; 20(11):e1012580.

PMID: 39561186 PMC: 11614280. DOI: 10.1371/journal.pcbi.1012580.

Zhang Z, Takahashi Y, Montesinos-Cartegena M, Kahnt T, Langdon A, Schoenbaum G Nat Commun. 2024; 15(1):8911.

PMID: 39414794 PMC: 11484966. DOI: 10.1038/s41467-024-53308-z.

References

Keiflin R, Pribut H, Shah N, Janak P . Ventral Tegmental Dopamine Neurons Participate in Reward Identity Predictions. Curr Biol. 2018; 29(1):93-103.e3. PMC: 6324975. DOI: 10.1016/j.cub.2018.11.050. View

Kobayashi S, Schultz W . Influence of reward delays on responses of dopamine neurons. J Neurosci. 2008; 28(31):7837-46. PMC: 3844811. DOI: 10.1523/JNEUROSCI.1600-08.2008. View

Montague P, Dayan P, Sejnowski T . A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci. 1996; 16(5):1936-47. PMC: 6578666. View

Lak A, Nomoto K, Keramati M, Sakagami M, Kepecs A . Midbrain Dopamine Neurons Signal Belief in Choice Accuracy during a Perceptual Decision. Curr Biol. 2017; 27(6):821-832. PMC: 5819757. DOI: 10.1016/j.cub.2017.02.026. View

Jo Y, Mizumori S . Prefrontal Regulation of Neuronal Activity in the Ventral Tegmental Area. Cereb Cortex. 2015; 26(10):4057-4068. PMC: 5028001. DOI: 10.1093/cercor/bhv215. View

Pan W, Schmidt R, Wickens J, Hyland B . Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci. 2005; 25(26):6235-42. PMC: 6725057. DOI: 10.1523/JNEUROSCI.1478-05.2005. View

Schultz W . Dopamine reward prediction-error signalling: a two-component response. Nat Rev Neurosci. 2016; 17(3):183-95. PMC: 5549862. DOI: 10.1038/nrn.2015.26. View

Tobler P, Dickinson A, Schultz W . Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci. 2003; 23(32):10402-10. PMC: 6741002. View

Langdon A, Sharpe M, Schoenbaum G, Niv Y . Model-based predictions for dopamine. Curr Opin Neurobiol. 2017; 49:1-7. PMC: 6034703. DOI: 10.1016/j.conb.2017.10.006. View

10.

Stalnaker T, Howard J, Takahashi Y, Gershman S, Kahnt T, Schoenbaum G . Dopamine neuron ensembles signal the content of sensory prediction errors. Elife. 2019; 8. PMC: 6839916. DOI: 10.7554/eLife.49315. View

11.

Watabe-Uchida M, Eshel N, Uchida N . Neural Circuitry of Reward Prediction Error. Annu Rev Neurosci. 2017; 40:373-394. PMC: 6721851. DOI: 10.1146/annurev-neuro-072116-031109. View

12.

Howard J, Kahnt T . Identity prediction errors in the human midbrain update reward-identity expectations in the orbitofrontal cortex. Nat Commun. 2018; 9(1):1611. PMC: 5913228. DOI: 10.1038/s41467-018-04055-5. View

13.

Wilson R, Takahashi Y, Schoenbaum G, Niv Y . Orbitofrontal cortex as a cognitive map of task space. Neuron. 2014; 81(2):267-279. PMC: 4001869. DOI: 10.1016/j.neuron.2013.11.005. View

14.

Suri R, Schultz W . A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience. 1999; 91(3):871-90. DOI: 10.1016/s0306-4522(98)00697-6. View

15.

Glimcher P . Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci U S A. 2011; 108 Suppl 3:15647-54. PMC: 3176615. DOI: 10.1073/pnas.1014269108. View

16.

Cohen J, Haesler S, Vong L, Lowell B, Uchida N . Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature. 2012; 482(7383):85-8. PMC: 3271183. DOI: 10.1038/nature10754. View

17.

Sharpe M, Chang C, Liu M, Batchelor H, Mueller L, Jones J . Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nat Neurosci. 2017; 20(5):735-742. PMC: 5413864. DOI: 10.1038/nn.4538. View

18.

Eshel N, Bukwich M, Rao V, Hemmelder V, Tian J, Uchida N . Arithmetic and local circuitry underlying dopamine prediction errors. Nature. 2015; 525(7568):243-6. PMC: 4567485. DOI: 10.1038/nature14855. View

19.

Takahashi Y, Roesch M, Wilson R, Toreson K, ODonnell P, Niv Y . Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat Neurosci. 2011; 14(12):1590-7. PMC: 3225718. DOI: 10.1038/nn.2957. View

20.

Mirenowicz J, Schultz W . Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol. 1994; 72(2):1024-7. DOI: 10.1152/jn.1994.72.2.1024. View