Exploring the Steps of Learning: Computational Modeling of Initiatory-actions Among Individuals with Attention-deficit/hyperactivity Disorder

Overview

Journal Transl Psychiatry

Specialties Psychiatry
Public Health

Date 2024 Jan 8

PMID 38191535

Authors

Gili Katabi

Nitzan Shahar

Affiliations

Soon will be listed here.

Abstract

Attention-deficit/hyperactivity disorder (ADHD) is characterized by difficulty in acting in a goal-directed manner. While most environments require a sequence of actions for goal attainment, ADHD was never studied in the context of value-based sequence learning. Here, we made use of current advancements in hierarchical reinforcement-learning algorithms to track the internal value and choice policy of individuals with ADHD performing a three-stage sequence learning task. Specifically, 54 participants (28 ADHD, 26 controls) completed a value-based reinforcement-learning task that allowed us to estimate internal action values for each trial and stage using computational modeling. We found attenuated sensitivity to action values in ADHD compared to controls, both in choice and reaction-time variability estimates. Remarkably, this was found only for first-stage actions (i.e., initiatory actions), while for actions performed just before outcome delivery the two groups were strikingly indistinguishable. These results suggest a difficulty in following value estimation for initiatory actions in ADHD.

References

Tsal Y, Shalev L, Mevorach C . The diversity of attention deficits in ADHD: the prevalence of four cognitive factors in ADHD versus controls. J Learn Disabil. 2005; 38(2):142-57. DOI: 10.1177/00222194050380020401. View

Button K, Ioannidis J, Mokrysz C, Nosek B, Flint J, Robinson E . Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013; 14(5):365-76. DOI: 10.1038/nrn3475. View

Pievsky M, McGrath R . The Neurocognitive Profile of Attention-Deficit/Hyperactivity Disorder: A Review of Meta-Analyses. Arch Clin Neuropsychol. 2017; 33(2):143-157. DOI: 10.1093/arclin/acx055. View

Addicott M, Pearson J, Schechter J, Sapyta J, Weiss M, Kollins S . Attention-deficit/hyperactivity disorder and the explore/exploit trade-off. Neuropsychopharmacology. 2020; 46(3):614-621. PMC: 8027173. DOI: 10.1038/s41386-020-00881-8. View

Sibley M, Graziano P, Ortiz M, Rodriguez L, Coxe S . Academic impairment among high school students with ADHD: The role of motivation and goal-directed executive functions. J Sch Psychol. 2019; 77:67-76. DOI: 10.1016/j.jsp.2019.10.005. View

Karalunas S, Geurts H, Konrad K, Bender S, Nigg J . Annual research review: Reaction time variability in ADHD and autism spectrum disorders: measurement and mechanisms of a proposed trans-diagnostic phenotype. J Child Psychol Psychiatry. 2014; 55(6):685-710. PMC: 4267725. DOI: 10.1111/jcpp.12217. View

Castellanos F, Sonuga-Barke E, Scheres A, Di Martino A, Hyde C, Walters J . Varieties of attention-deficit/hyperactivity disorder-related intra-individual variability. Biol Psychiatry. 2005; 57(11):1416-23. PMC: 1236991. DOI: 10.1016/j.biopsych.2004.12.005. View

Zohar A, Konfortes H . Diagnosing ADHD in Israeli adults: the psychometric properties of the adult ADHD Self Report Scale (ASRS) in Hebrew. Isr J Psychiatry Relat Sci. 2011; 47(4):308-15. View

Hauser T, Fiore V, Moutoussis M, Dolan R . Computational Psychiatry of ADHD: Neural Gain Impairments across Marrian Levels of Analysis. Trends Neurosci. 2016; 39(2):63-73. PMC: 4746317. DOI: 10.1016/j.tins.2015.12.009. View

10.

Dubois M, Bowler A, Moses-Payne M, Habicht J, Moran R, Steinbeis N . Exploration heuristics decrease during youth. Cogn Affect Behav Neurosci. 2022; 22(5):969-983. PMC: 9458685. DOI: 10.3758/s13415-022-01009-9. View

11.

Massat I, Slama H, Kavec M, Linotte S, Mary A, Baleriaux D . Working memory-related functional brain patterns in never medicated children with ADHD. PLoS One. 2012; 7(11):e49392. PMC: 3498108. DOI: 10.1371/journal.pone.0049392. View

12.

Schultz W, Dickinson A . Neuronal coding of prediction errors. Annu Rev Neurosci. 2000; 23:473-500. DOI: 10.1146/annurev.neuro.23.1.473. View

13.

Daw N, ODoherty J, Dayan P, Seymour B, Dolan R . Cortical substrates for exploratory decisions in humans. Nature. 2006; 441(7095):876-9. PMC: 2635947. DOI: 10.1038/nature04766. View

14.

Marx I, Hopcke C, Berger C, Wandschneider R, Herpertz S . The impact of financial reward contingencies on cognitive function profiles in adult ADHD. PLoS One. 2013; 8(6):e67002. PMC: 3688618. DOI: 10.1371/journal.pone.0067002. View

15.

Dezfouli A, Balleine B . Actions, action sequences and habits: evidence that goal-directed and habitual action control are hierarchically organized. PLoS Comput Biol. 2013; 9(12):e1003364. PMC: 3854489. DOI: 10.1371/journal.pcbi.1003364. View

16.

Demurie E, Roeyers H, Baeyens D, Sonuga-Barke E . Common alterations in sensitivity to type but not amount of reward in ADHD and autism spectrum disorders. J Child Psychol Psychiatry. 2011; 52(11):1164-73. DOI: 10.1111/j.1469-7610.2010.02374.x. View

17.

Epstein J, Langberg J, Rosen P, Graham A, Narad M, Antonini T . Evidence for higher reaction time variability for children with ADHD on a range of cognitive tasks including reward and event rate manipulations. Neuropsychology. 2011; 25(4):427-441. PMC: 3522094. DOI: 10.1037/a0022155. View

18.

Martinussen R, Hayden J, Hogg-Johnson S, Tannock R . A meta-analysis of working memory impairments in children with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry. 2005; 44(4):377-84. DOI: 10.1097/01.chi.0000153228.72591.73. View

19.

Faraone S, Banaschewski T, Coghill D, Zheng Y, Biederman J, Bellgrove M . The World Federation of ADHD International Consensus Statement: 208 Evidence-based conclusions about the disorder. Neurosci Biobehav Rev. 2021; 128:789-818. PMC: 8328933. DOI: 10.1016/j.neubiorev.2021.01.022. View

20.

Song P, Zha M, Yang Q, Zhang Y, Li X, Rudan I . The prevalence of adult attention-deficit hyperactivity disorder: A global systematic review and meta-analysis. J Glob Health. 2021; 11:04009. PMC: 7916320. DOI: 10.7189/jogh.11.04009. View