Common Neural Code for Reward and Information Value

Overview

Journal Proc Natl Acad Sci U S A

Specialty Science

Date 2019 Jun 13

PMID 31186358

Citations 39

Authors

Kenji Kobayashi

Ming Hsu

Affiliations

Soon will be listed here.

Abstract

Adaptive information seeking is critical for goal-directed behavior. Growing evidence suggests the importance of intrinsic motives such as curiosity or need for novelty, mediated through dopaminergic valuation systems, in driving information-seeking behavior. However, valuing information for its own sake can be highly suboptimal when agents need to evaluate instrumental benefit of information in a forward-looking manner. Here we show that information-seeking behavior in humans is driven by subjective value that is shaped by both instrumental and noninstrumental motives, and that this subjective value of information (SVOI) shares a common neural code with more basic reward value. Specifically, using a task where subjects could purchase information to reduce uncertainty about outcomes of a monetary lottery, we found information purchase decisions could be captured by a computational model of SVOI incorporating utility of anticipation, a form of noninstrumental motive for information seeking, in addition to instrumental benefits. Neurally, trial-by-trial variation in SVOI was correlated with activity in striatum and ventromedial prefrontal cortex. Furthermore, cross-categorical decoding revealed that, within these regions, SVOI and expected utility of lotteries were represented using a common code. These findings provide support for the common currency hypothesis and shed insight on neurocognitive mechanisms underlying information-seeking behavior.

Citing Articles

Metacognition biases information seeking in assessing ambiguous news.

Guigon V, Villeval M, Dreher J Commun Psychol. 2024; 2(1):122.

PMID: 39702410 PMC: 11659316. DOI: 10.1038/s44271-024-00170-w.

Three diverse motives for information sharing.

Vellani V, Glickman M, Sharot T Commun Psychol. 2024; 2(1):107.

PMID: 39506099 PMC: 11541573. DOI: 10.1038/s44271-024-00144-y.

Representations of the intrinsic value of information in mouse orbitofrontal cortex.

Bussell J, Badman R, Marton C, Bromberg-Martin E, Abbott L, Rajan K bioRxiv. 2024; .

PMID: 39416043 PMC: 11482914. DOI: 10.1101/2023.10.13.562291.

To know or not to know? Curiosity and the value of prospective information in animals.

Ajuwon V, Monteiro T, Schnell A, Clayton N Learn Behav. 2024; 53(1):114-127.

PMID: 39414697 PMC: 11880187. DOI: 10.3758/s13420-024-00647-y.

A neural code supporting prospective probabilistic reasoning for instrumental information demand in humans.

Singletary N, Horga G, Gottlieb J Commun Biol. 2024; 7(1):1242.

PMID: 39358516 PMC: 11447085. DOI: 10.1038/s42003-024-06927-7.

References

Kakade S, Dayan P . Dopamine: generalization and bonuses. Neural Netw. 2002; 15(4-6):549-59. DOI: 10.1016/s0893-6080(02)00048-5. View

Ishii S, Yoshida W, Yoshimoto J . Control of exploitation-exploration meta-parameter in reinforcement learning. Neural Netw. 2002; 15(4-6):665-87. DOI: 10.1016/s0893-6080(02)00056-4. View

BERLYNE D . A theory of human curiosity. Br J Psychol. 1954; 45(3):180-91. DOI: 10.1111/j.2044-8295.1954.tb01243.x. View

Daw N, Niv Y, Dayan P . Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci. 2005; 8(12):1704-11. DOI: 10.1038/nn1560. View

Daw N, ODoherty J, Dayan P, Seymour B, Dolan R . Cortical substrates for exploratory decisions in humans. Nature. 2006; 441(7095):876-9. PMC: 2635947. DOI: 10.1038/nature04766. View

Bunzeck N, Duzel E . Absolute coding of stimulus novelty in the human substantia nigra/VTA. Neuron. 2006; 51(3):369-79. DOI: 10.1016/j.neuron.2006.06.021. View

Cohen J, McClure S, Yu A . Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos Trans R Soc Lond B Biol Sci. 2007; 362(1481):933-42. PMC: 2430007. DOI: 10.1098/rstb.2007.2098. View

Rangel A, Camerer C, Montague P . A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci. 2008; 9(7):545-56. PMC: 4332708. DOI: 10.1038/nrn2357. View

Wittmann B, Daw N, Seymour B, Dolan R . Striatal activity underlies novelty-based choice in humans. Neuron. 2008; 58(6):967-73. PMC: 2535823. DOI: 10.1016/j.neuron.2008.04.027. View

10.

Krebs R, Schott B, Schutze H, Duzel E . The novelty exploration bonus and its attentional modulation. Neuropsychologia. 2009; 47(11):2272-81. DOI: 10.1016/j.neuropsychologia.2009.01.015. View

11.

Bromberg-Martin E, Hikosaka O . Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron. 2009; 63(1):119-26. PMC: 2723053. DOI: 10.1016/j.neuron.2009.06.009. View

12.

Kang M, Hsu M, Krajbich I, Loewenstein G, McClure S, Wang J . The wick in the candle of learning: epistemic curiosity activates reward circuitry and enhances memory. Psychol Sci. 2009; 20(8):963-73. DOI: 10.1111/j.1467-9280.2009.02402.x. View

13.

Haber S, Knutson B . The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology. 2009; 35(1):4-26. PMC: 3055449. DOI: 10.1038/npp.2009.129. View

14.

Smith D, Hayden B, Truong T, Song A, Platt M, Huettel S . Distinct value signals in anterior and posterior ventromedial prefrontal cortex. J Neurosci. 2010; 30(7):2490-5. PMC: 2856318. DOI: 10.1523/JNEUROSCI.3319-09.2010. View

15.

Rangel A, Hare T . Neural computations associated with goal-directed choice. Curr Opin Neurobiol. 2010; 20(2):262-70. DOI: 10.1016/j.conb.2010.03.001. View

16.

Grabenhorst F, Rolls E . Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci. 2011; 15(2):56-67. DOI: 10.1016/j.tics.2010.12.004. View

17.

Bromberg-Martin E, Hikosaka O . Lateral habenula neurons signal errors in the prediction of reward information. Nat Neurosci. 2011; 14(9):1209-16. PMC: 3164948. DOI: 10.1038/nn.2902. View

18.

Jepma M, Verdonschot R, van Steenbergen H, Rombouts S, Nieuwenhuis S . Neural mechanisms underlying the induction and relief of perceptual curiosity. Front Behav Neurosci. 2012; 6:5. PMC: 3277937. DOI: 10.3389/fnbeh.2012.00005. View

19.

Levy D, Glimcher P . The root of all value: a neural common currency for choice. Curr Opin Neurobiol. 2012; 22(6):1027-38. PMC: 4093837. DOI: 10.1016/j.conb.2012.06.001. View

20.

McNamee D, Rangel A, ODoherty J . Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex. Nat Neurosci. 2013; 16(4):479-85. PMC: 3665508. DOI: 10.1038/nn.3337. View