Microstimulation of the Human Substantia Nigra Alters Reinforcement Learning

Overview

Journal J Neurosci

Specialty Neurology

Date 2014 May 16

PMID 24828643

Citations 18

Authors

Ashwin G Ramayya

Amrit Misra

Gordon H Baltuch

Michael J Kahana

Affiliations

Soon will be listed here.

Abstract

Animal studies have shown that substantia nigra (SN) dopaminergic (DA) neurons strengthen action-reward associations during reinforcement learning, but their role in human learning is not known. Here, we applied microstimulation in the SN of 11 patients undergoing deep brain stimulation surgery for the treatment of Parkinson's disease as they performed a two-alternative probability learning task in which rewards were contingent on stimuli, rather than actions. Subjects demonstrated decreased learning from reward trials that were accompanied by phasic SN microstimulation compared with reward trials without stimulation. Subjects who showed large decreases in learning also showed an increased bias toward repeating actions after stimulation trials; therefore, stimulation may have decreased learning by strengthening action-reward associations rather than stimulus-reward associations. Our findings build on previous studies implicating SN DA neurons in preferentially strengthening action-reward associations during reinforcement learning.

Citing Articles

Human Substantia Nigra Neurons Encode Reward Expectations.

Imtiaz Z, Kato A, Kopell B, Qasim S, Davis A, Nunez Martinez L bioRxiv. 2024; .

PMID: 38766086 PMC: 11100806. DOI: 10.1101/2024.05.10.593406.

From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement.

Fraser K, Pribut H, Janak P, Keiflin R J Neurosci. 2023; 43(21):3895-3908.

PMID: 37185097 PMC: 10217998. DOI: 10.1523/JNEUROSCI.0028-23.2023.

Motor context modulates substantia nigra pars reticulata spike activity in patients with Parkinson's disease.

Tekriwal A, Felsen G, Ojemann S, Abosch A, Thompson J J Neurol Neurosurg Psychiatry. 2022; 93(4):386-394.

PMID: 35193951 PMC: 10593310. DOI: 10.1136/jnnp-2021-326962.

Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges.

Dennison J, Sazhin D, Smith D Wiley Interdiscip Rev Cogn Sci. 2022; 13(3):e1589.

PMID: 35137549 PMC: 9124684. DOI: 10.1002/wcs.1589.

Neuromodulators and Long-Term Synaptic Plasticity in Learning and Memory: A Steered-Glutamatergic Perspective.

Bazzari A, Parri H Brain Sci. 2019; 9(11).

PMID: 31683595 PMC: 6896105. DOI: 10.3390/brainsci9110300.

References

Sugrue L, Corrado G, Newsome W . Choosing the greater of two goods: neural currencies for valuation and decision making. Nat Rev Neurosci. 2005; 6(5):363-75. DOI: 10.1038/nrn1666. View

Haber S, Fudge J, McFarland N . Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci. 2000; 20(6):2369-82. PMC: 6772499. View

Niv Y, Daw N, Joel D, Dayan P . Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl). 2006; 191(3):507-20. DOI: 10.1007/s00213-006-0502-4. View

Frank M, Seeberger L, OReilly R . By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science. 2004; 306(5703):1940-3. DOI: 10.1126/science.1102941. View

Zigmond M, Abercrombie E, Berger T, Grace A, Stricker E . Compensations after lesions of central dopaminergic neurons: some clinical and basic implications. Trends Neurosci. 1990; 13(7):290-6. DOI: 10.1016/0166-2236(90)90112-n. View

Schultz W, Dayan P, Montague P . A neural substrate of prediction and reward. Science. 1997; 275(5306):1593-9. DOI: 10.1126/science.275.5306.1593. View

Bayer H, Glimcher P . Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron. 2005; 47(1):129-41. PMC: 1564381. DOI: 10.1016/j.neuron.2005.05.020. View

Ma S, Rinne J, Collan Y, Roytta M, Rinne U . A quantitative morphometrical study of neuron degeneration in the substantia nigra in Parkinson's disease. J Neurol Sci. 1996; 140(1-2):40-5. DOI: 10.1016/0022-510x(96)00069-x. View

Pan W, Brown J, Dudman J . Neural signals of extinction in the inhibitory microcircuit of the ventral midbrain. Nat Neurosci. 2012; 16(1):71-8. PMC: 3563090. DOI: 10.1038/nn.3283. View

10.

Glimcher P . Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci U S A. 2011; 108 Suppl 3:15647-54. PMC: 3176615. DOI: 10.1073/pnas.1014269108. View

11.

Tsai H, Zhang F, Adamantidis A, Stuber G, Bonci A, De Lecea L . Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science. 2009; 324(5930):1080-4. PMC: 5262197. DOI: 10.1126/science.1168878. View

12.

Hollerman J, Grace A . The effects of dopamine-depleting brain lesions on the electrophysiological activity of rat substantia nigra dopamine neurons. Brain Res. 1990; 533(2):203-12. DOI: 10.1016/0006-8993(90)91341-d. View

13.

ODoherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan R . Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004; 304(5669):452-4. DOI: 10.1126/science.1094285. View

14.

Zaghloul K, Blanco J, Weidemann C, McGill K, Jaggi J, Baltuch G . Human substantia nigra neurons encode unexpected financial rewards. Science. 2009; 323(5920):1496-9. PMC: 2839450. DOI: 10.1126/science.1167342. View

15.

Lau B, Glimcher P . Value representations in the primate striatum during matching behavior. Neuron. 2008; 58(3):451-63. PMC: 2427158. DOI: 10.1016/j.neuron.2008.02.021. View

16.

Ungless M, Grace A . Are you or aren't you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci. 2012; 35(7):422-30. PMC: 3383926. DOI: 10.1016/j.tins.2012.02.003. View

17.

Henny P, Brown M, Northrop A, Faunes M, Ungless M, Magill P . Structural correlates of heterogeneous in vivo activity of midbrain dopaminergic neurons. Nat Neurosci. 2012; 15(4):613-9. PMC: 4242968. DOI: 10.1038/nn.3048. View

18.

Daw N, ODoherty J, Dayan P, Seymour B, Dolan R . Cortical substrates for exploratory decisions in humans. Nature. 2006; 441(7095):876-9. PMC: 2635947. DOI: 10.1038/nature04766. View

19.

Foerde K, Race E, Verfaellie M, Shohamy D . A role for the medial temporal lobe in feedback-driven learning: evidence from amnesia. J Neurosci. 2013; 33(13):5698-704. PMC: 3865542. DOI: 10.1523/JNEUROSCI.5217-12.2013. View

20.

Tepper J, Martin L, Anderson D . GABAA receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons. J Neurosci. 1995; 15(4):3092-103. PMC: 6577766. View