Human Belief State-Based Exploration and Exploitation in an Information-Selective Symmetric Reversal Bandit Task

Overview

Journal Comput Brain Behav

Date 2021 Aug 9

PMID 34368622

Citations 1

Authors

Lilla Horvath

Stanley Colcombe

Michael Milham

Shruti Ray

Philipp Schwartenbeck

Dirk Ostwald

Affiliations

Soon will be listed here.

Abstract

Supplementary Information: The online version contains supplementary material available at 10.1007/s42113-021-00112-3.

Citing Articles

Active inference and the two-step task.

Gijsen S, Grundei M, Blankenburg F Sci Rep. 2022; 12(1):17682.

PMID: 36271279 PMC: 9586964. DOI: 10.1038/s41598-022-21766-4.

References

Gijsen S, Grundei M, Lange R, Ostwald D, Blankenburg F . Neural surprise in somatosensory Bayesian learning. PLoS Comput Biol. 2021; 17(2):e1008068. PMC: 7880500. DOI: 10.1371/journal.pcbi.1008068. View

Worthy D, Otto A, Maddox W . Working-memory load and temporal myopia in dynamic decision making. J Exp Psychol Learn Mem Cogn. 2012; 38(6):1640-58. PMC: 3419784. DOI: 10.1037/a0028146. View

Daw N, ODoherty J, Dayan P, Seymour B, Dolan R . Cortical substrates for exploratory decisions in humans. Nature. 2006; 441(7095):876-9. PMC: 2635947. DOI: 10.1038/nature04766. View

Chakroun K, Mathar D, Wiehler A, Ganzer F, Peters J . Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making. Elife. 2020; 9. PMC: 7266623. DOI: 10.7554/eLife.51260. View

Gershman S . Deconstructing the human algorithms for exploration. Cognition. 2018; 173:34-42. PMC: 5801139. DOI: 10.1016/j.cognition.2017.12.014. View

Speekenbrink M, Konstantinidis E . Uncertainty and exploration in a restless bandit problem. Top Cogn Sci. 2015; 7(2):351-67. DOI: 10.1111/tops.12145. View

Stephan K, Penny W, Daunizeau J, Moran R, Friston K . Bayesian model selection for group studies. Neuroimage. 2009; 46(4):1004-17. PMC: 2703732. DOI: 10.1016/j.neuroimage.2009.03.025. View

Bartolo R, Averbeck B . Prefrontal Cortex Predicts State Switches during Reversal Learning. Neuron. 2020; 106(6):1044-1054.e4. PMC: 7422923. DOI: 10.1016/j.neuron.2020.03.024. View

Hauser T, Iannaccone R, Stampfli P, Drechsler R, Brandeis D, Walitza S . The feedback-related negativity (FRN) revisited: new insights into the localization, meaning and network organization. Neuroimage. 2013; 84:159-68. DOI: 10.1016/j.neuroimage.2013.08.028. View

10.

Wilson R, Collins A . Ten simple rules for the computational modeling of behavioral data. Elife. 2019; 8. PMC: 6879303. DOI: 10.7554/eLife.49547. View

11.

Schwartenbeck P, Passecker J, Hauser T, FitzGerald T, Kronbichler M, Friston K . Computational mechanisms of curiosity and goal-directed exploration. Elife. 2019; 8. PMC: 6510535. DOI: 10.7554/eLife.41703. View

12.

Navarro D, Newell B, Schulze C . Learning and choosing in an uncertain world: An investigation of the explore-exploit dilemma in static and dynamic environments. Cogn Psychol. 2016; 85:43-77. DOI: 10.1016/j.cogpsych.2016.01.001. View

13.

Cohen J, McClure S, Yu A . Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos Trans R Soc Lond B Biol Sci. 2007; 362(1481):933-42. PMC: 2430007. DOI: 10.1098/rstb.2007.2098. View

14.

Glascher J, Hampton A, ODoherty J . Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex. 2008; 19(2):483-95. PMC: 2626172. DOI: 10.1093/cercor/bhn098. View

15.

Dayan P, Daw N . Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci. 2008; 8(4):429-53. DOI: 10.3758/CABN.8.4.429. View

16.

Wu C, Schulz E, Speekenbrink M, Nelson J, Meder B . Generalization guides human exploration in vast decision spaces. Nat Hum Behav. 2019; 2(12):915-924. DOI: 10.1038/s41562-018-0467-4. View

17.

Cogliati Dezza I, Yu A, Cleeremans A, Alexander W . Learning the value of information and reward over time when solving exploration-exploitation problems. Sci Rep. 2017; 7(1):16919. PMC: 5717252. DOI: 10.1038/s41598-017-17237-w. View

18.

Brand H, Woods P, Sakoda J . Anticipation of reward as a function of partial reinforcement. J Exp Psychol. 1956; 52(1):18-22. DOI: 10.1037/h0045510. View

19.

Ostwald D, Starke L, Hertwig R . A normative inference approach for optimal sample sizes in decisions from experience. Front Psychol. 2015; 6:1342. PMC: 4564691. DOI: 10.3389/fpsyg.2015.01342. View

20.

Hampton A, Bossaerts P, ODoherty J . The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci. 2006; 26(32):8360-7. PMC: 6673813. DOI: 10.1523/JNEUROSCI.1010-06.2006. View