Evidence for Model-based Encoding of Pavlovian Contingencies in the Human Brain

Overview

Journal Nat Commun

Specialty Biology

Date 2019 Mar 9

PMID 30846685

Citations 16

Authors

Wolfgang M Pauli

Giovanni Gentile

Sven Collette

Julian M Tyszka

John P ODoherty

Affiliations

Soon will be listed here.

Abstract

Prominent accounts of Pavlovian conditioning successfully approximate the frequency and intensity of conditioned responses under the assumption that learning is exclusively model-free; that animals do not develop a cognitive map of events. However, these model-free approximations fall short of comprehensively capturing learning and behavior in Pavlovian conditioning. We therefore performed multivoxel pattern analysis of high-resolution functional MRI data in human participants to test for the encoding of stimulus-stimulus associations that could support model-based computations during Pavlovian conditioning. We found that dissociable sub-regions of the striatum encode predictions of stimulus-stimulus associations and predictive value, in a manner that is directly related to learning performance. Activity patterns in the orbitofrontal cortex were also found to be related to stimulus-stimulus as well as value encoding. These results suggest that the brain encodes model-based representations during Pavlovian conditioning, and that these representations are utilized in the service of behavior.

Citing Articles

Distributed midbrain responses signal the content of positive identity prediction errors.

Howard J, Edmonds D, Schoenbaum G, Kahnt T Curr Biol. 2024; 34(18):4240-4247.e4.

PMID: 39197457 PMC: 11421979. DOI: 10.1016/j.cub.2024.07.105.

Encoding of Predictive Associations in Human Prefrontal and Medial Temporal Neurons During Pavlovian Appetitive Conditioning.

Aquino T, Courellis H, Mamelak A, Rutishauser U, O Doherty J J Neurosci. 2024; 44(17).

PMID: 38423764 PMC: 11044193. DOI: 10.1523/JNEUROSCI.1628-23.2024.

Midbrain signaling of identity prediction errors depends on orbitofrontal cortex networks.

Liu Q, Zhao Y, Attanti S, Voss J, Schoenbaum G, Kahnt T Nat Commun. 2024; 15(1):1704.

PMID: 38402210 PMC: 10894191. DOI: 10.1038/s41467-024-45880-1.

Functional Connectivity Mechanisms Underlying Symptom Reduction Following Lisdexamfetamine Treatment in Binge-Eating Disorder: A Clinical Trial.

Griffiths K, Breukelaar I, Harvie G, Yang J, Foster S, Harris A Biol Psychiatry Glob Open Sci. 2024; 4(1):317-325.

PMID: 38298797 PMC: 10829641. DOI: 10.1016/j.bpsgos.2023.08.016.

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans.

Pool E, Pauli W, Cross L, ODoherty J Nat Commun. 2023; 14(1):8057.

PMID: 38052792 PMC: 10697955. DOI: 10.1038/s41467-023-43747-5.

References

Wilson R, Takahashi Y, Schoenbaum G, Niv Y . Orbitofrontal cortex as a cognitive map of task space. Neuron. 2014; 81(2):267-279. PMC: 4001869. DOI: 10.1016/j.neuron.2013.11.005. View

Bhandari A, Gagne C, Badre D . Just above Chance: Is It Harder to Decode Information from Prefrontal Cortex Hemodynamic Activity Patterns?. J Cogn Neurosci. 2018; 30(10):1473-1498. DOI: 10.1162/jocn_a_01291. View

Avants B, Duda J, Zhang H, Gee J . Multivariate normalization with symmetric diffeomorphisms for multivariate studies. Med Image Comput Comput Assist Interv. 2007; 10(Pt 1):359-66. DOI: 10.1007/978-3-540-75757-3_44. View

Samejima K, Ueda Y, Doya K, Kimura M . Representation of action-specific reward values in the striatum. Science. 2005; 310(5752):1337-40. DOI: 10.1126/science.1115270. View

Kriegeskorte N, Bandettini P . Analyzing for information, not activation, to exploit high-resolution fMRI. Neuroimage. 2007; 38(4):649-62. PMC: 2099257. DOI: 10.1016/j.neuroimage.2007.02.022. View

Howard J, Kahnt T . Identity-Specific Reward Representations in Orbitofrontal Cortex Are Modulated by Selective Devaluation. J Neurosci. 2017; 37(10):2627-2638. PMC: 5354319. DOI: 10.1523/JNEUROSCI.3473-16.2017. View

Schonberg T, Daw N, Joel D, ODoherty J . Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci. 2007; 27(47):12860-7. PMC: 6673291. DOI: 10.1523/JNEUROSCI.2496-07.2007. View

Parkinson J, Willoughby P, Robbins T, Everitt B . Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs Pavlovian approach behavior: further evidence for limbic cortical-ventral striatopallidal systems. Behav Neurosci. 2000; 114(1):42-63. View

Sadacca B, Wied H, Lopatina N, Saini G, Nemirovsky D, Schoenbaum G . Orbitofrontal neurons signal sensory associations underlying model-based inference in a sensory preconditioning task. Elife. 2018; 7. PMC: 5847331. DOI: 10.7554/eLife.30373. View

10.

Pauli W, Larsen T, Collette S, Tyszka J, Seymour B, ODoherty J . Distinct Contributions of Ventromedial and Dorsolateral Subregions of the Human Substantia Nigra to Appetitive and Aversive Learning. J Neurosci. 2015; 35(42):14220-33. PMC: 4683685. DOI: 10.1523/JNEUROSCI.2277-15.2015. View

11.

Seymour B, ODoherty J, Dayan P, Koltzenburg M, Jones A, Dolan R . Temporal difference models describe higher-order learning in humans. Nature. 2004; 429(6992):664-7. DOI: 10.1038/nature02581. View

12.

Yarkoni T, Poldrack R, Nichols T, Van Essen D, Wager T . Large-scale automated synthesis of human functional neuroimaging data. Nat Methods. 2011; 8(8):665-70. PMC: 3146590. DOI: 10.1038/nmeth.1635. View

13.

Pereira F, Mitchell T, Botvinick M . Machine learning classifiers and fMRI: a tutorial overview. Neuroimage. 2008; 45(1 Suppl):S199-209. PMC: 2892746. DOI: 10.1016/j.neuroimage.2008.11.007. View

14.

Yin H, Ostlund S, Knowlton B, Balleine B . The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci. 2005; 22(2):513-23. DOI: 10.1111/j.1460-9568.2005.04218.x. View

15.

Gottfried J, ODoherty J, Dolan R . Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science. 2003; 301(5636):1104-7. DOI: 10.1126/science.1087919. View

16.

Howard J, Kahnt T . Identity prediction errors in the human midbrain update reward-identity expectations in the orbitofrontal cortex. Nat Commun. 2018; 9(1):1611. PMC: 5913228. DOI: 10.1038/s41467-018-04055-5. View

17.

Pauli W, OReilly R, Yarkoni T, Wager T . Regional specialization within the human striatum for diverse psychological functions. Proc Natl Acad Sci U S A. 2016; 113(7):1907-12. PMC: 4763751. DOI: 10.1073/pnas.1507610113. View

18.

Prevost C, McNamee D, Jessup R, Bossaerts P, ODoherty J . Evidence for model-based computations in the human amygdala during Pavlovian conditioning. PLoS Comput Biol. 2013; 9(2):e1002918. PMC: 3578744. DOI: 10.1371/journal.pcbi.1002918. View

19.

Gorgolewski K, Burns C, Madison C, Clark D, Halchenko Y, Waskom M . Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in python. Front Neuroinform. 2011; 5:13. PMC: 3159964. DOI: 10.3389/fninf.2011.00013. View

20.

Bornstein A, Daw N . Multiplicity of control in the basal ganglia: computational roles of striatal subregions. Curr Opin Neurobiol. 2011; 21(3):374-80. PMC: 3269306. DOI: 10.1016/j.conb.2011.02.009. View