An Effect of Serotonergic Stimulation on Learning Rates for Rewards Apparent After Long Intertrial Intervals

Overview

Journal Nat Commun

Specialty Biology

Date 2018 Jun 28

PMID 29946069

Citations 39

Authors

Kiyohito Iigaya

Madalena S Fonseca

Masayoshi Murakami

Zachary F Mainen

Peter Dayan

Affiliations

Soon will be listed here.

Abstract

Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning.

Citing Articles

Acetylcholine modulates prefrontal outcome coding during threat learning under uncertainty.

Tu G, Wen P, Halawa A, Takehara-Nishiuchi K Elife. 2025; 13.

PMID: 40042523 PMC: 11882142. DOI: 10.7554/eLife.102986.

Computational and Neural Evidence for Altered Fast and Slow Learning from Losses in Problem Gambling.

Iigaya K, Larsen T, Fong T, ODoherty J J Neurosci. 2024; 45(1).

PMID: 39557579 PMC: 11694394. DOI: 10.1523/JNEUROSCI.0080-24.2024.

Investigating Transfer Learning in Noisy Environments: A Study of Predecessor and Successor Features in Spatial Learning Using a T-Maze.

Seo I, Lee H Sensors (Basel). 2024; 24(19).

PMID: 39409459 PMC: 11479366. DOI: 10.3390/s24196419.

Dopamine transients encode reward prediction errors independent of learning rates.

Mah A, Golden C, Constantinople C Cell Rep. 2024; 43(10):114840.

PMID: 39395170 PMC: 11571066. DOI: 10.1016/j.celrep.2024.114840.

Monoaminergic Modulation of Learning and Cognitive Function in the Prefrontal Cortex.

Boyle N, Betts S, Lu H Brain Sci. 2024; 14(9).

PMID: 39335398 PMC: 11429557. DOI: 10.3390/brainsci14090902.

References

Crockett M, Clark L, Robbins T . Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J Neurosci. 2009; 29(38):11993-9. PMC: 2775933. DOI: 10.1523/JNEUROSCI.2513-09.2009. View

Schultz W, Dayan P, Montague P . A neural substrate of prediction and reward. Science. 1997; 275(5306):1593-9. DOI: 10.1126/science.275.5306.1593. View

Daw N, Kakade S, Dayan P . Opponent interactions between serotonin and dopamine. Neural Netw. 2002; 15(4-6):603-16. DOI: 10.1016/s0893-6080(02)00052-7. View

Sugrue L, Corrado G, Newsome W . Matching behavior and the representation of value in the parietal cortex. Science. 2004; 304(5678):1782-7. DOI: 10.1126/science.1094765. View

Loewenstein Y, Seung H . Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity. Proc Natl Acad Sci U S A. 2006; 103(41):15224-9. PMC: 1622804. DOI: 10.1073/pnas.0505220103. View

Cohen J, Amoroso M, Uchida N . Serotonergic neurons signal reward and punishment on multiple timescales. Elife. 2015; 4. PMC: 4389268. DOI: 10.7554/eLife.06346. View

Boureau Y, Dayan P . Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology. 2010; 36(1):74-97. PMC: 3055522. DOI: 10.1038/npp.2010.151. View

Fusi S, Asaad W, Miller E, Wang X . A neural circuit model of flexible sensorimotor mapping: learning and forgetting on multiple timescales. Neuron. 2007; 54(2):319-33. PMC: 2833020. DOI: 10.1016/j.neuron.2007.03.017. View

Fletcher P, Korth K, Chambers J . Selective destruction of brain serotonin neurons by 5,7-dihydroxytryptamine increases responding for a conditioned reward. Psychopharmacology (Berl). 2000; 147(3):291-9. DOI: 10.1007/s002130051170. View

10.

Aston-Jones G, Cohen J . An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci. 2005; 28:403-50. DOI: 10.1146/annurev.neuro.28.061604.135709. View

11.

Hart A, Rutledge R, Glimcher P, Phillips P . Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J Neurosci. 2014; 34(3):698-704. PMC: 3891951. DOI: 10.1523/JNEUROSCI.2489-13.2014. View

12.

Gong S, Doughty M, Harbaugh C, Cummins A, Hatten M, Heintz N . Targeting Cre recombinase to specific neuron populations with bacterial artificial chromosome constructs. J Neurosci. 2007; 27(37):9817-23. PMC: 6672645. DOI: 10.1523/JNEUROSCI.2707-07.2007. View

13.

Lee S, Shimojo S, ODoherty J . Neural computations underlying arbitration between model-based and model-free learning. Neuron. 2014; 81(3):687-99. PMC: 3968946. DOI: 10.1016/j.neuron.2013.11.028. View

14.

Miyazaki K, Miyazaki K, Tanaka K, Yamanaka A, Takahashi A, Tabuchi S . Optogenetic activation of dorsal raphe serotonin neurons enhances patience for future rewards. Curr Biol. 2014; 24(17):2033-40. DOI: 10.1016/j.cub.2014.07.041. View

15.

Nassar M, Wilson R, Heasly B, Gold J . An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J Neurosci. 2010; 30(37):12366-78. PMC: 2945906. DOI: 10.1523/JNEUROSCI.0822-10.2010. View

16.

Barraclough D, Conroy M, Lee D . Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci. 2004; 7(4):404-10. DOI: 10.1038/nn1209. View

17.

Walker S, Robbins T, Roberts A . Differential contributions of dopamine and serotonin to orbitofrontal cortex function in the marmoset. Cereb Cortex. 2008; 19(4):889-98. PMC: 2651474. DOI: 10.1093/cercor/bhn136. View

18.

Kishida K, Sandberg S, Lohrenz T, Comair Y, Saez I, Phillips P . Sub-second dopamine detection in human striatum. PLoS One. 2011; 6(8):e23291. PMC: 3150430. DOI: 10.1371/journal.pone.0023291. View

19.

Hawkins R, Abrams T, Carew T, Kandel E . A cellular mechanism of classical conditioning in Aplysia: activity-dependent amplification of presynaptic facilitation. Science. 1983; 219(4583):400-5. DOI: 10.1126/science.6294833. View

20.

Correia P, Lottem E, Banerjee D, Machado A, Carey M, Mainen Z . Transient inhibition and long-term facilitation of locomotion by phasic optogenetic activation of serotonin neurons. Elife. 2017; 6. PMC: 5308893. DOI: 10.7554/eLife.20975. View