Naturalistic Reinforcement Learning

Overview

Journal Trends Cogn Sci

Date 2023 Sep 30

PMID 37777463

Authors

Toby Wise

Kara Emery

Angela Radulescu

Affiliations

Soon will be listed here.

Abstract

Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.

Citing Articles

Approach-avoidance conflict recruits lateral frontoparietal and cinguloinsular networks in a predator-prey game setting.

Ni Y, Potter R, James T Cogn Affect Behav Neurosci. 2025; .

PMID: 40011405 DOI: 10.3758/s13415-025-01278-0.

When to stop social learning from a predecessor in an information-foraging task.

Suganuma H, Naito A, Katahira K, Kameda T Evol Hum Sci. 2025; 7:e2.

PMID: 39935447 PMC: 11810515. DOI: 10.1017/ehs.2024.29.

Schemas, reinforcement learning and the medial prefrontal cortex.

Bein O, Niv Y Nat Rev Neurosci. 2025; 26(3):141-157.

PMID: 39775183 DOI: 10.1038/s41583-024-00893-z.

Association between SOFA score and risk of acute kidney injury in patients with diabetic ketoacidosis: an analysis of the MIMIC-IV database.

Hua Y, Ding N, Jing H, Xie Y, Wu H, Wu Y Front Endocrinol (Lausanne). 2025; 15:1462330.

PMID: 39764255 PMC: 11700799. DOI: 10.3389/fendo.2024.1462330.

Medical Education: Considerations for a Successful Integration of Learning with and Learning about AI.

Domros-Zoungrana D, Rajaeean N, Boie S, Froling E, Lenz C J Med Educ Curric Dev. 2024; 11:23821205241284719.

PMID: 39654541 PMC: 11626671. DOI: 10.1177/23821205241284719.

References

Wimmer G, Poldrack R . Reward learning and working memory: Effects of massed versus spaced training and post-learning delay period. Mem Cognit. 2021; 50(2):312-324. PMC: 8821056. DOI: 10.3758/s13421-021-01233-7. View

Keren H, Zheng C, Jangraw D, Chang K, Vitale A, Rutledge R . The temporal representation of experience in subjective mood. Elife. 2021; 10. PMC: 8241441. DOI: 10.7554/eLife.62051. View

Li C, Aivar M, Tong M, Hayhoe M . Memory shapes visual search strategies in large-scale environments. Sci Rep. 2018; 8(1):4324. PMC: 5847537. DOI: 10.1038/s41598-018-22731-w. View

Wu C, Schulz E, Speekenbrink M, Nelson J, Meder B . Generalization guides human exploration in vast decision spaces. Nat Hum Behav. 2019; 2(12):915-924. DOI: 10.1038/s41562-018-0467-4. View

Eckstein M, Master S, Xia L, Dahl R, Wilbrecht L, Collins A . The interpretation of computational model parameters depends on the context. Elife. 2022; 11. PMC: 9635876. DOI: 10.7554/eLife.75474. View

Nicholas J, Daw N, Shohamy D . Uncertainty alters the balance between incremental learning and episodic memory. Elife. 2022; 11. PMC: 9810331. DOI: 10.7554/eLife.81679. View

Allen K, Smith K, Tenenbaum J . Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning. Proc Natl Acad Sci U S A. 2020; 117(47):29302-29310. PMC: 7703630. DOI: 10.1073/pnas.1912341117. View

Wise T, Dolan R . Associations between aversive learning processes and transdiagnostic psychiatric symptoms in a general population sample. Nat Commun. 2020; 11(1):4179. PMC: 7443146. DOI: 10.1038/s41467-020-17977-w. View

Park S, Miller D, Nili H, Ranganath C, Boorman E . Map Making: Constructing, Combining, and Inferring on Abstract Cognitive Maps. Neuron. 2020; 107(6):1226-1238.e8. PMC: 7529977. DOI: 10.1016/j.neuron.2020.06.030. View

10.

Molinaro G, Collins A . Intrinsic rewards explain context-sensitive valuation in reinforcement learning. PLoS Biol. 2023; 21(7):e3002201. PMC: 10374061. DOI: 10.1371/journal.pbio.3002201. View

11.

Keramati M, Smittenaar P, Dolan R, Dayan P . Adaptive integration of habits into depth-limited planning defines a habitual-goal-directed spectrum. Proc Natl Acad Sci U S A. 2016; 113(45):12868-12873. PMC: 5111694. DOI: 10.1073/pnas.1609094113. View

12.

Tavares R, Mendelsohn A, Grossman Y, Williams C, Shapiro M, Trope Y . A Map for Social Navigation in the Human Brain. Neuron. 2015; 87(1):231-43. PMC: 4662863. DOI: 10.1016/j.neuron.2015.06.011. View

13.

Lee S, Shimojo S, ODoherty J . Neural computations underlying arbitration between model-based and model-free learning. Neuron. 2014; 81(3):687-99. PMC: 3968946. DOI: 10.1016/j.neuron.2013.11.028. View

14.

Collins A, Frank M . How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. Eur J Neurosci. 2012; 35(7):1024-35. PMC: 3390186. DOI: 10.1111/j.1460-9568.2011.07980.x. View

15.

Garvert M, Dolan R, Behrens T . A map of abstract relational knowledge in the human hippocampal-entorhinal cortex. Elife. 2017; 6. PMC: 5407855. DOI: 10.7554/eLife.17086. View

16.

Iigaya K, Yi S, Wahle I, Tanwisuth S, Cross L, ODoherty J . Neural mechanisms underlying the hierarchical construction of perceived aesthetic value. Nat Commun. 2023; 14(1):127. PMC: 9873760. DOI: 10.1038/s41467-022-35654-y. View

17.

Tomov M, Schulz E, Gershman S . Multi-task reinforcement learning in humans. Nat Hum Behav. 2021; 5(6):764-773. DOI: 10.1038/s41562-020-01035-y. View

18.

Behrens T, Muller T, Whittington J, Mark S, Baram A, Stachenfeld K . What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior. Neuron. 2018; 100(2):490-509. DOI: 10.1016/j.neuron.2018.10.002. View

19.

Amir O, Tyomkin L, Hart Y . Adaptive search space pruning in complex strategic problems. PLoS Comput Biol. 2022; 18(8):e1010358. PMC: 9394844. DOI: 10.1371/journal.pcbi.1010358. View

20.

Eldar E, Roth C, Dayan P, Dolan R . Decodability of Reward Learning Signals Predicts Mood Fluctuations. Curr Biol. 2018; 28(9):1433-1439.e7. PMC: 5954908. DOI: 10.1016/j.cub.2018.03.038. View