Developmental Changes in Exploration Resemble Stochastic Optimization

Overview

Journal Nat Hum Behav

Specialties Psychology
Social Sciences

Date 2023 Aug 17

PMID 37591981

Authors

Anna P Giron

Simon Ciranka

Eric Schulz

Wouter van den Bos

Azzurra Ruggeri

Bjorn Meder

Charley M Wu

Affiliations

Soon will be listed here.

Abstract

Human development is often described as a 'cooling off' process, analogous to stochastic optimization algorithms that implement a gradual reduction in randomness over time. Yet there is ambiguity in how to interpret this analogy, due to a lack of concrete empirical comparisons. Using data from n = 281 participants ages 5 to 55, we show that cooling off does not only apply to the single dimension of randomness. Rather, human development resembles an optimization process of multiple learning parameters, for example, reward generalization, uncertainty-directed exploration and random temperature. Rapid changes in parameters occur during childhood, but these changes plateau and converge to efficient values in adulthood. We show that while the developmental trajectory of human parameters is strikingly similar to several stochastic optimization algorithms, there are important differences in convergence. None of the optimization algorithms tested were able to discover reliably better regions of the strategy space than adult participants on this task.

Citing Articles

Perceptual Novelty Drives Early Exploration in a Bottom-Up Manner.

Gao M, Sloutsky V Dev Sci. 2025; 28(3):e70002.

PMID: 40033792 PMC: 11876794. DOI: 10.1111/desc.70002.

The connecting brain in context: How adolescent plasticity supports learning and development.

Baker A, Galvan A, Fuligni A Dev Cogn Neurosci. 2024; 71():101486.

PMID: 39631105 PMC: 11653146. DOI: 10.1016/j.dcn.2024.101486.

Decrease in decision noise from adolescence into adulthood mediates an increase in more sophisticated choice behaviors and performance gain.

Scholz V, Waltmann M, Herzog N, Horstmann A, Deserno L PLoS Biol. 2024; 22(11):e3002877.

PMID: 39541313 PMC: 11563475. DOI: 10.1371/journal.pbio.3002877.

Humans flexibly integrate social information despite interindividual differences in reward.

Witt A, Toyokawa W, Lala K, Gaissmaier W, Wu C Proc Natl Acad Sci U S A. 2024; 121(39):e2404928121.

PMID: 39302964 PMC: 11441569. DOI: 10.1073/pnas.2404928121.

Sensitivity to the Instrumental Value of Choice Increases Across Development.

Nussenbaum K, Katzman P, Lu H, Zorowitz S, Hartley C Psychol Sci. 2024; 35(8):933-947.

PMID: 38900963 PMC: 11693699. DOI: 10.1177/09567976241256961.

References

Moran R, Symmonds M, Dolan R, Friston K . The brain ages optimally to model its environment: evidence from sensory learning over the adult lifespan. PLoS Comput Biol. 2014; 10(1):e1003422. PMC: 3900375. DOI: 10.1371/journal.pcbi.1003422. View

Nussenbaum K, Hartley C . Reinforcement learning across development: What insights can we draw from a decade of research?. Dev Cogn Neurosci. 2019; 40:100733. PMC: 6974916. DOI: 10.1016/j.dcn.2019.100733. View

Gopnik A, OGrady S, Lucas C, Griffiths T, Wente A, Bridgers S . Changes in cognitive flexibility and hypothesis search across human life history from childhood to adolescence to adulthood. Proc Natl Acad Sci U S A. 2017; 114(30):7892-7899. PMC: 5544286. DOI: 10.1073/pnas.1700811114. View

Walasek N, Frankenhuis W, Panchanathan K . Sensitive periods, but not critical periods, evolve in a fluctuating environment: a model of incremental development. Proc Biol Sci. 2022; 289(1969):20212623. PMC: 8848242. DOI: 10.1098/rspb.2021.2623. View

Kirkpatrick S, Gelatt Jr C, Vecchi M . Optimization by simulated annealing. Science. 1983; 220(4598):671-80. DOI: 10.1126/science.220.4598.671. View

Lucas C, Bridgers S, Griffiths T, Gopnik A . When children are better (or at least more open-minded) learners than adults: developmental differences in learning the forms of causal relationships. Cognition. 2014; 131(2):284-99. DOI: 10.1016/j.cognition.2013.12.010. View

Denison S, Bonawitz E, Gopnik A, Griffiths T . Rational variability in children's causal inferences: the Sampling Hypothesis. Cognition. 2012; 126(2):285-300. DOI: 10.1016/j.cognition.2012.10.010. View

Bonawitz E, Denison S, Gopnik A, Griffiths T . Win-Stay, Lose-Sample: a simple sequential algorithm for approximating Bayesian inference. Cogn Psychol. 2014; 74:35-65. DOI: 10.1016/j.cogpsych.2014.06.003. View

10.

Somerville L, Sasse S, Garrad M, Drysdale A, Abi Akar N, Insel C . Charting the expansion of strategic exploratory behavior during adolescence. J Exp Psychol Gen. 2016; 146(2):155-164. DOI: 10.1037/xge0000250. View

11.

Jepma M, Schaaf J, Visser I, Huizenga H . Uncertainty-driven regulation of learning and exploration in adolescents: A computational account. PLoS Comput Biol. 2020; 16(9):e1008276. PMC: 7549782. DOI: 10.1371/journal.pcbi.1008276. View

12.

Palminteri S, Kilford E, Coricelli G, Blakemore S . The Computational Development of Reinforcement Learning during Adolescence. PLoS Comput Biol. 2016; 12(6):e1004953. PMC: 4920542. DOI: 10.1371/journal.pcbi.1004953. View

13.

Rosenbaum G, Venkatraman V, Steinberg L, Chein J . The Influences of Described and Experienced Information on Adolescent Risky Decision Making. Dev Rev. 2018; 47:23-43. PMC: 5841249. DOI: 10.1016/j.dr.2017.09.003. View

14.

Baltes P, Staudinger U, Lindenberger U . Lifespan psychology: theory and application to intellectual functioning. Annu Rev Psychol. 2004; 50:471-507. DOI: 10.1146/annurev.psych.50.1.471. View

15.

Gopnik A . Scientific thinking in young children: theoretical advances, empirical research, and policy implications. Science. 2012; 337(6102):1623-7. DOI: 10.1126/science.1223416. View

16.

Schulz E, Wu C, Ruggeri A, Meder B . Searching for Rewards Like a Child Means Less Generalization and More Directed Exploration. Psychol Sci. 2019; 30(11):1561-1572. DOI: 10.1177/0956797619863663. View

17.

Dubois M, Bowler A, Moses-Payne M, Habicht J, Moran R, Steinbeis N . Exploration heuristics decrease during youth. Cogn Affect Behav Neurosci. 2022; 22(5):969-983. PMC: 9458685. DOI: 10.3758/s13415-022-01009-9. View

18.

Blanco N, Sloutsky V . Systematic exploration and uncertainty dominate young children's choices. Dev Sci. 2020; 24(2):e13026. PMC: 7867663. DOI: 10.1111/desc.13026. View

19.

van den Bos W, Cohen M, Kahnt T, Crone E . Striatum-medial prefrontal cortex connectivity predicts developmental changes in reinforcement learning. Cereb Cortex. 2011; 22(6):1247-55. PMC: 6283353. DOI: 10.1093/cercor/bhr198. View

20.

Blanco N, Love B, Ramscar M, Otto A, Smayda K, Maddox W . Exploratory decision-making as a function of lifelong experience, not cognitive decline. J Exp Psychol Gen. 2016; 145(3):284-297. PMC: 4755819. DOI: 10.1037/xge0000133. View

21.

Tenenbaum J, Kemp C, Griffiths T, Goodman N . How to grow a mind: statistics, structure, and abstraction. Science. 2011; 331(6022):1279-85. DOI: 10.1126/science.1192788. View