Causal Evidence Supporting the Proposal That Dopamine Transients Function As Temporal Difference Prediction Errors

Overview

Journal Nat Neurosci

Date 2020 Jan 22

PMID 31959935

Citations 36

Authors

Etienne J P Maes

Melissa J Sharpe

Alexandra A Usypchuk

Megan Lozzi

Chun Yun Chang

Matthew P H Gardner

Geoffrey Schoenbaum

Mihaela D Iordanova

Affiliations

Soon will be listed here.

Abstract

Reward-evoked dopamine transients are well established as prediction errors. However, the central tenet of temporal difference accounts-that similar transients evoked by reward-predictive cues also function as errors-remains untested. In the present communication we addressed this by showing that optogenetically shunting dopamine activity at the start of a reward-predicting cue prevents second-order conditioning without affecting blocking. These results indicate that cue-evoked transients function as temporal-difference prediction errors rather than reward predictions.

Citing Articles

Contextual cues facilitate dynamic value encoding in the mesolimbic dopamine system.

Fraser K, Collins V, Wolff A, Ottenheimer D, Bornhoft K, Pat F Curr Biol. 2025; 35(4):746-760.e5.

PMID: 39855205 PMC: 11859769. DOI: 10.1016/j.cub.2024.12.031.

Parabrachial Calca neurons mediate second-order conditioning.

Park S, Zhu A, Cao F, Palmiter R Nat Commun. 2024; 15(1):9721.

PMID: 39521770 PMC: 11550384. DOI: 10.1038/s41467-024-53977-w.

Individual differences in decision-making shape how mesolimbic dopamine regulates choice confidence and change-of-mind.

Kocharian A, Redish A, Rothwell P bioRxiv. 2024; .

PMID: 39345599 PMC: 11429702. DOI: 10.1101/2024.09.16.613237.

Chronic stress deficits in reward behaviour co-occur with low nucleus accumbens dopamine activity during reward anticipation specifically.

Zhang C, Dulinskas R, Ineichen C, Greter A, Sigrist H, Li Y Commun Biol. 2024; 7(1):966.

PMID: 39123076 PMC: 11316117. DOI: 10.1038/s42003-024-06658-9.

Dopamine Release in the Nucleus Accumbens Core Encodes the General Excitatory Components of Learning.

Taira M, Millard S, Verghese A, DiFazio L, Hoang I, Jia R J Neurosci. 2024; 44(35).

PMID: 38969504 PMC: 11358529. DOI: 10.1523/JNEUROSCI.0120-24.2024.

References

Glimcher P . Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci U S A. 2011; 108 Suppl 3:15647-54. PMC: 3176615. DOI: 10.1073/pnas.1014269108. View

Sharpe M, Chang C, Liu M, Batchelor H, Mueller L, Jones J . Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nat Neurosci. 2017; 20(5):735-742. PMC: 5413864. DOI: 10.1038/nn.4538. View

Keiflin R, Pribut H, Shah N, Janak P . Ventral Tegmental Dopamine Neurons Participate in Reward Identity Predictions. Curr Biol. 2018; 29(1):93-103.e3. PMC: 6324975. DOI: 10.1016/j.cub.2018.11.050. View

Mahmud A, Petrov P, Esber G, Iordanova M . The serial blocking effect: a testbed for the neural mechanisms of temporal-difference learning. Sci Rep. 2019; 9(1):5962. PMC: 6461709. DOI: 10.1038/s41598-019-42244-4. View

Rizley R, Rescorla R . Associations in second-order conditioning and sensory preconditioning. J Comp Physiol Psychol. 1972; 81(1):1-11. DOI: 10.1037/h0033333. View

Steinberg E, Keiflin R, Boivin J, Witten I, Deisseroth K, Janak P . A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci. 2013; 16(7):966-73. PMC: 3705924. DOI: 10.1038/nn.3413. View

Olejnik S, Algina J . Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychol Methods. 2003; 8(4):434-47. DOI: 10.1037/1082-989X.8.4.434. View

Sharpe M, Killcross S . The prelimbic cortex contributes to the down-regulation of attention toward redundant cues. Cereb Cortex. 2012; 24(4):1066-74. DOI: 10.1093/cercor/bhs393. View

Chang C, Gardner M, Conroy J, Whitaker L, Schoenbaum G . Brief, But Not Prolonged, Pauses in the Firing of Midbrain Dopamine Neurons Are Sufficient to Produce a Conditioned Inhibitor. J Neurosci. 2018; 38(41):8822-8830. PMC: 6181314. DOI: 10.1523/JNEUROSCI.0144-18.2018. View

10.

Kim H, Malik A, Mikhael J, Bech P, Tsutsui-Kimura I, Sun F . A Unified Framework for Dopamine Signals across Timescales. Cell. 2020; 183(6):1600-1616.e25. PMC: 7736562. DOI: 10.1016/j.cell.2020.11.013. View

11.

Chang C, Gardner M, Di Tillio M, Schoenbaum G . Optogenetic Blockade of Dopamine Transients Prevents Learning Induced by Changes in Reward Features. Curr Biol. 2017; 27(22):3480-3486.e3. PMC: 5698141. DOI: 10.1016/j.cub.2017.09.049. View

12.

Gardner M, Schoenbaum G, Gershman S . Rethinking dopamine as generalized prediction error. Proc Biol Sci. 2018; 285(1891). PMC: 6253385. DOI: 10.1098/rspb.2018.1645. View

13.

Schultz W, Dayan P, Montague P . A neural substrate of prediction and reward. Science. 1997; 275(5306):1593-9. DOI: 10.1126/science.275.5306.1593. View