The Role of Efference Copy in Striatal Learning

Overview

Journal Curr Opin Neurobiol

Specialties Biology
Neurology

Date 2014 Feb 26

PMID 24566242

Citations 24

Authors

Michale S Fee

Affiliations

Soon will be listed here.

Abstract

Reinforcement learning requires the convergence of signals representing context, action, and reward. While models of basal ganglia function have well-founded hypotheses about the neural origin of signals representing context and reward, the function and origin of signals representing action are less clear. Recent findings suggest that exploratory or variable behaviors are initiated by a wide array of 'action-generating' circuits in the midbrain, brainstem, and cortex. Thus, in order to learn, the striatum must incorporate an efference copy of action decisions made in these action-generating circuits. Here we review several recent neural models of reinforcement learning that emphasize the role of efference copy signals. Also described are ideas about how these signals might be integrated with inputs signaling context and reward.

Citing Articles

The role of motor cortex in motor sequence execution depends on demands for flexibility.

Mizes K, Lindsey J, Escola G, Olveczky B Nat Neurosci. 2024; 27(12):2466-2475.

PMID: 39496797 DOI: 10.1038/s41593-024-01792-3.

Exploration biases forelimb reaching strategies.

Mosberger A, Sibener L, Chen T, Rodrigues H, Hormigo R, Ingram J Cell Rep. 2024; 43(4):113958.

PMID: 38520691 PMC: 11097405. DOI: 10.1016/j.celrep.2024.113958.

Dynamics of striatal action selection and reinforcement learning.

Lindsey J, Markowitz J, Gillis W, Datta S, Litwin-Kumar A bioRxiv. 2024; .

PMID: 38464083 PMC: 10925202. DOI: 10.1101/2024.02.14.580408.

Layer 5 Intratelencephalic Neurons in the Motor Cortex Stably Encode Skilled Movement.

Shinotsuka T, Tanaka Y, Terada S, Hatano N, Matsuzaki M J Neurosci. 2023; 43(43):7130-7148.

PMID: 37699714 PMC: 10601372. DOI: 10.1523/JNEUROSCI.0428-23.2023.

Generative models of birdsong learning link circadian fluctuations in song variability to changes in performance.

Brudner S, Pearson J, Mooney R PLoS Comput Biol. 2023; 19(5):e1011051.

PMID: 37126511 PMC: 10150982. DOI: 10.1371/journal.pcbi.1011051.

References

HOOVER J, Strick P . Multiple output channels in the basal ganglia. Science. 1993; 259(5096):819-21. DOI: 10.1126/science.7679223. View

Prather J, Peters S, Nowicki S, Mooney R . Precise auditory-vocal mirroring in neurons for learned vocal communication. Nature. 2008; 451(7176):305-10. DOI: 10.1038/nature06492. View

Schultz W . Predictive reward signal of dopamine neurons. J Neurophysiol. 1998; 80(1):1-27. DOI: 10.1152/jn.1998.80.1.1. View

Hikosaka O, Nakahara H, Rand M, Sakai K, Lu X, Nakamura K . Parallel neural networks for learning sequential procedures. Trends Neurosci. 1999; 22(10):464-71. DOI: 10.1016/s0166-2236(99)01439-3. View

Mysore S, Knudsen E . A shared inhibitory circuit for both exogenous and endogenous control of stimulus selection. Nat Neurosci. 2013; 16(4):473-8. PMC: 3609877. DOI: 10.1038/nn.3352. View

Kunzle H, Akert K . Efferent connections of cortical, area 8 (frontal eye field) in Macaca fascicularis. A reinvestigation using the autoradiographic technique. J Comp Neurol. 1977; 173(1):147-64. DOI: 10.1002/cne.901730108. View

Hollerman J, Schultz W . Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci. 1999; 1(4):304-9. DOI: 10.1038/1124. View

Shen W, Flajolet M, Greengard P, Surmeier D . Dichotomous dopaminergic control of striatal synaptic plasticity. Science. 2008; 321(5890):848-51. PMC: 2833421. DOI: 10.1126/science.1160575. View

Schultz W, Dayan P, Montague P . A neural substrate of prediction and reward. Science. 1997; 275(5306):1593-9. DOI: 10.1126/science.275.5306.1593. View

10.

Alexander G, DeLong M, Strick P . Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci. 1986; 9:357-81. DOI: 10.1146/annurev.ne.09.030186.002041. View

11.

Izhikevich E . Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex. 2007; 17(10):2443-52. DOI: 10.1093/cercor/bhl152. View

12.

Aronov D, Veit L, Goldberg J, Fee M . Two distinct modes of forebrain circuit dynamics underlie temporal patterning in the vocalizations of young songbirds. J Neurosci. 2011; 31(45):16353-68. PMC: 3241969. DOI: 10.1523/JNEUROSCI.3009-11.2011. View

13.

Frank M . Computational models of motivated action selection in corticostriatal circuits. Curr Opin Neurobiol. 2011; 21(3):381-6. DOI: 10.1016/j.conb.2011.02.013. View

14.

Daw N, Doya K . The computational neurobiology of learning and reward. Curr Opin Neurobiol. 2006; 16(2):199-204. DOI: 10.1016/j.conb.2006.03.006. View

15.

Frey U, Morris R . Synaptic tagging and long-term potentiation. Nature. 1997; 385(6616):533-6. DOI: 10.1038/385533a0. View

16.

Kao M, Doupe A, Brainard M . Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature. 2005; 433(7026):638-43. DOI: 10.1038/nature03127. View

17.

Kozhevnikov A, Fee M . Singing-related activity of identified HVC neurons in the zebra finch. J Neurophysiol. 2006; 97(6):4271-83. DOI: 10.1152/jn.00952.2006. View

18.

Pasupathy A, Miller E . Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature. 2005; 433(7028):873-6. DOI: 10.1038/nature03287. View

19.

Berns G, Sejnowski T . A computational model of how the basal ganglia produce sequences. J Cogn Neurosci. 1998; 10(1):108-21. DOI: 10.1162/089892998563815. View

20.

Sommer M, Wurtz R . Brain circuits for the internal monitoring of movements. Annu Rev Neurosci. 2008; 31:317-38. PMC: 2813694. DOI: 10.1146/annurev.neuro.31.060407.125627. View