Goal-Directed Planning and Goal Understanding by Extended Active Inference: Evaluation Through Simulated and Physical Robot Experiments

Overview

Journal Entropy (Basel)

Publisher MDPI

Date 2022 Apr 23

PMID 35455132

Authors

Takazumi Matsumoto

Wataru Ohata

Fabien C Y Benureau

Jun Tani

Affiliations

Soon will be listed here.

Abstract

We show that goal-directed action planning and generation in a teleological framework can be formulated by extending the active inference framework. The proposed model, which is built on a variational recurrent neural network model, is characterized by three essential features. These are that (1) goals can be specified for both static sensory states, e.g., for goal images to be reached and dynamic processes, e.g., for moving around an object, (2) the model cannot only generate goal-directed action plans, but can also understand goals through sensory observation, and (3) the model generates future action plans for given goals based on the best estimate of the current state, inferred from past sensory observations. The proposed model is evaluated by conducting experiments on a simulated mobile agent as well as on a real humanoid robot performing object manipulation.

Citing Articles

Synergizing habits and goals with variational Bayes.

Han D, Doya K, Li D, Tani J Nat Commun. 2024; 15(1):4461.

PMID: 38796491 PMC: 11525633. DOI: 10.1038/s41467-024-48577-7.

Bio-Inspired Intelligent Systems: Negotiations between Minimum Manifest Task Entropy and Maximum Latent System Entropy in Changing Environments.

Fox S, Heikkila T, Halbach E, Soutukorva S Entropy (Basel). 2023; 25(11).

PMID: 37998233 PMC: 10670934. DOI: 10.3390/e25111541.

Incremental Learning of Goal-Directed Actions in a Dynamic Environment by a Robot Using Active Inference.

Matsumoto T, Ohata W, Tani J Entropy (Basel). 2023; 25(11).

PMID: 37998198 PMC: 10670890. DOI: 10.3390/e25111506.

References

Shimojo S . Postdiction: its implications on visual awareness, hindsight, and sense of agency. Front Psychol. 2014; 5:196. PMC: 3978293. DOI: 10.3389/fpsyg.2014.00196. View

Matsumoto T, Tani J . Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Network. Entropy (Basel). 2020; 22(5). PMC: 7517093. DOI: 10.3390/e22050564. View

Tani J . Model-based learning for mobile robot navigation from the dynamical systems perspective. IEEE Trans Syst Man Cybern B Cybern. 1996; 26(3):421-36. DOI: 10.1109/3477.499793. View

Ahmadi A, Tani J . How can a recurrent neurodynamic predictive coding model cope with fluctuation in temporal patterns? Robotic experiments on imitative interaction. Neural Netw. 2017; 92:3-16. DOI: 10.1016/j.neunet.2017.02.015. View

Friston K, Samothrakis S, Montague R . Active inference and agency: optimal control without cost functions. Biol Cybern. 2012; 106(8-9):523-41. DOI: 10.1007/s00422-012-0512-8. View

Parr T, Friston K . Generalised free energy and active inference. Biol Cybern. 2019; 113(5-6):495-513. PMC: 6848054. DOI: 10.1007/s00422-019-00805-w. View

Van de Maele T, Verbelen T, Catal O, De Boom C, Dhoedt B . Active Vision for Robot Manipulators Using the Free Energy Principle. Front Neurorobot. 2021; 15:642780. PMC: 7973267. DOI: 10.3389/fnbot.2021.642780. View

Friston K, Mattout J, Kilner J . Action understanding and active inference. Biol Cybern. 2011; 104(1-2):137-60. PMC: 3491875. DOI: 10.1007/s00422-011-0424-z. View

Tani J . Learning to generate articulated behavior through the bottom-up and the top-down interaction processes. Neural Netw. 2003; 16(1):11-23. DOI: 10.1016/s0893-6080(02)00214-9. View

10.

Kawato M . Internal models for motor control and trajectory planning. Curr Opin Neurobiol. 1999; 9(6):718-27. DOI: 10.1016/s0959-4388(99)00028-8. View

11.

Wolpert D, Miall R . Forward Models for Physiological Motor Control. Neural Netw. 1996; 9(8):1265-1279. DOI: 10.1016/s0893-6080(96)00035-4. View

12.

Friston K, Rigoli F, Ognibene D, Mathys C, Fitzgerald T, Pezzulo G . Active inference and epistemic value. Cogn Neurosci. 2015; 6(4):187-214. DOI: 10.1080/17588928.2015.1020053. View

13.

Ahmadi A, Tani J . A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition. Neural Comput. 2019; 31(11):2025-2074. DOI: 10.1162/neco_a_01228. View

14.

Yamashita Y, Tani J . Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Comput Biol. 2008; 4(11):e1000220. PMC: 2570613. DOI: 10.1371/journal.pcbi.1000220. View

15.

Catal O, Wauthier S, De Boom C, Verbelen T, Dhoedt B . Learning Generative State Space Models for Active Inference. Front Comput Neurosci. 2020; 14:574372. PMC: 7701292. DOI: 10.3389/fncom.2020.574372. View

16.

Kawato M, Maeda Y, Uno Y, Suzuki R . Trajectory formation of arm movement by cascade neural network model based on minimum torque-change criterion. Biol Cybern. 1990; 62(4):275-88. DOI: 10.1007/BF00201442. View

17.

Friston K . A theory of cortical responses. Philos Trans R Soc Lond B Biol Sci. 2005; 360(1456):815-36. PMC: 1569488. DOI: 10.1098/rstb.2005.1622. View

18.

Butz M, Bilkey D, Humaidan D, Knott A, Otte S . Learning, planning, and control in a monolithic neural event inference architecture. Neural Netw. 2019; 117:135-144. DOI: 10.1016/j.neunet.2019.05.001. View

19.

Ohata W, Tani J . Investigation of the Sense of Agency in Social Cognition, Based on Frameworks of Predictive Coding and Active Inference: A Simulation Study on Multimodal Imitative Interaction. Front Neurorobot. 2020; 14:61. PMC: 7509423. DOI: 10.3389/fnbot.2020.00061. View