A Distributed, Hierarchical and Recurrent Framework for Reward-based Choice

Overview

Journal Nat Rev Neurosci

Specialty Neurology

Date 2017 Feb 18

PMID 28209978

Citations 101

Authors

Laurence T Hunt

Benjamin Y Hayden

Affiliations

Soon will be listed here.

Abstract

Many accounts of reward-based choice argue for distinct component processes that are serial and functionally localized. In this Opinion article, we argue for an alternative viewpoint, in which choices emerge from repeated computations that are distributed across many brain regions. We emphasize how several features of neuroanatomy may support the implementation of choice, including mutual inhibition in recurrent neural networks and the hierarchical organization of timescales for information processing across the cortex. This account also suggests that certain correlates of value are emergent rather than represented explicitly in the brain.

Citing Articles

Structural analysis and treatment of acute liver injury of polysaccharides from the .

Ding Z, Ying Q, Lei J, Zhang N, Xia Y, Du Y Front Pharmacol. 2025; 16:1544302.

PMID: 40066337 PMC: 11891217. DOI: 10.3389/fphar.2025.1544302.

The Relationship Between Schizotypal Personality Traits and Temporal Discounting: The Role of the Date/Delay Effect.

Keidel K, Murawski C, Pantelis C, Ettinger U Schizophr Bull. 2025; 51(Supplement_2):S64-S73.

PMID: 40037816 PMC: 11879521. DOI: 10.1093/schbul/sbad110.

Common neural choice signals can emerge artefactually amid multiple distinct value signals.

Fromer R, Nassar M, Ehinger B, Shenhav A Nat Hum Behav. 2024; 8(11):2194-2208.

PMID: 39242928 PMC: 11576515. DOI: 10.1038/s41562-024-01971-z.

Neural interactions in the human frontal cortex dissociate reward and punishment learning.

Combrisson E, Basanisi R, Gueguen M, Rheims S, Kahane P, Bastin J Elife. 2024; 12.

PMID: 38941238 PMC: 11213568. DOI: 10.7554/eLife.92938.

Multiattribute Decision-making in Macaques Relies on Direct Attribute Comparisons.

Perkins A, Gillis Z, Rich E J Cogn Neurosci. 2024; 36(9):1879-1897.

PMID: 38940740 PMC: 11324248. DOI: 10.1162/jocn_a_02208.

References

Strait C, Blanchard T, Hayden B . Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron. 2014; 82(6):1357-66. PMC: 4086796. DOI: 10.1016/j.neuron.2014.04.032. View

Rustichini A, Padoa-Schioppa C . A neuro-computational model of economic decisions. J Neurophysiol. 2015; 114(3):1382-98. PMC: 4556855. DOI: 10.1152/jn.00184.2015. View

Pezzulo G, Cisek P . Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition. Trends Cogn Sci. 2016; 20(6):414-424. DOI: 10.1016/j.tics.2016.03.013. View

Badre D, DEsposito M . Is the rostro-caudal axis of the frontal lobe hierarchical?. Nat Rev Neurosci. 2009; 10(9):659-69. PMC: 3258028. DOI: 10.1038/nrn2667. View

Murray J, Bernacchia A, Freedman D, Romo R, Wallis J, Cai X . A hierarchy of intrinsic timescales across primate cortex. Nat Neurosci. 2014; 17(12):1661-3. PMC: 4241138. DOI: 10.1038/nn.3862. View

Dombrowski S, Hilgetag C, Barbas H . Quantitative architecture distinguishes prefrontal cortical systems in the rhesus monkey. Cereb Cortex. 2001; 11(10):975-88. DOI: 10.1093/cercor/11.10.975. View

Noonan M, Walton M, Behrens T, Sallet J, Buckley M, Rushworth M . Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc Natl Acad Sci U S A. 2010; 107(47):20547-52. PMC: 2996698. DOI: 10.1073/pnas.1012246107. View

Philiastides M, Biele G, Heekeren H . A mechanistic account of value computation in the human brain. Proc Natl Acad Sci U S A. 2010; 107(20):9430-5. PMC: 2889112. DOI: 10.1073/pnas.1001732107. View

Maunsell J . Neuronal representations of cognitive state: reward or attention?. Trends Cogn Sci. 2004; 8(6):261-5. DOI: 10.1016/j.tics.2004.04.003. View

10.

Peck C, Lau B, Salzman C . The primate amygdala combines information about space and value. Nat Neurosci. 2013; 16(3):340-8. PMC: 3596258. DOI: 10.1038/nn.3328. View

11.

Hunt L, Woolrich M, Rushworth M, Behrens T . Trial-type dependent frames of reference for value comparison. PLoS Comput Biol. 2013; 9(9):e1003225. PMC: 3772056. DOI: 10.1371/journal.pcbi.1003225. View

12.

Schacter D, Addis D, Hassabis D, Martin V, Nathan Spreng R, Szpunar K . The future of memory: remembering, imagining, and the brain. Neuron. 2012; 76(4):677-94. PMC: 3815616. DOI: 10.1016/j.neuron.2012.11.001. View

13.

Neubert F, Mars R, Sallet J, Rushworth M . Connectivity reveals relationship of brain areas for reward-guided learning and decision making in human and monkey frontal cortex. Proc Natl Acad Sci U S A. 2015; 112(20):E2695-704. PMC: 4443352. DOI: 10.1073/pnas.1410767112. View

14.

Cai X, Kim S, Lee D . Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron. 2011; 69(1):170-82. PMC: 3034314. DOI: 10.1016/j.neuron.2010.11.041. View

15.

Shadlen M, Shohamy D . Decision Making and Sequential Sampling from Memory. Neuron. 2016; 90(5):927-39. PMC: 4891701. DOI: 10.1016/j.neuron.2016.04.036. View

16.

Louie K, Grattan L, Glimcher P . Reward value-based gain control: divisive normalization in parietal cortex. J Neurosci. 2011; 31(29):10627-39. PMC: 3285508. DOI: 10.1523/JNEUROSCI.1237-11.2011. View

17.

Leathers M, Olson C . In monkeys making value-based decisions, LIP neurons encode cue salience and not action value. Science. 2012; 338(6103):132-5. PMC: 3705639. DOI: 10.1126/science.1226405. View

18.

Song H, Yang G, Wang X . Reward-based training of recurrent neural networks for cognitive and value-based tasks. Elife. 2017; 6. PMC: 5293493. DOI: 10.7554/eLife.21492. View

19.

Kouneiher F, Charron S, Koechlin E . Motivation and cognitive control in the human prefrontal cortex. Nat Neurosci. 2009; 12(7):939-45. DOI: 10.1038/nn.2321. View

20.

Plassmann H, ODoherty J, Rangel A . Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J Neurosci. 2007; 27(37):9984-8. PMC: 6672655. DOI: 10.1523/JNEUROSCI.2131-07.2007. View