Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

Overview

Journal Front Robot AI

Specialty Biomedical Engineering

Date 2021 Jan 27

PMID 33501307

Citations 19

Authors

Matteo Spezialetti

Giuseppe Placidi

Silvia Rossi

Affiliations

Soon will be listed here.

Abstract

A fascinating challenge in the field of human-robot interaction is the possibility to endow robots with emotional intelligence in order to make the interaction more intuitive, genuine, and natural. To achieve this, a critical point is the capability of the robot to infer and interpret human emotions. Emotion recognition has been widely explored in the broader fields of human-machine interaction and affective computing. Here, we report recent advances in emotion recognition, with particular regard to the human-robot interaction context. Our aim is to review the state of the art of currently adopted emotional models, interaction modalities, and classification strategies and offer our point of view on future developments and critical issues. We focus on facial expressions, body poses and kinematics, voice, brain activity, and peripheral physiological responses, also providing a list of available datasets containing data from these modalities.

Citing Articles

Gait-to-Gait Emotional Human-Robot Interaction Utilizing Trajectories-Aware and Skeleton-Graph-Aware Spatial-Temporal Transformer.

Li C, Seng K, Ang L Sensors (Basel). 2025; 25(3).

PMID: 39943373 PMC: 11820152. DOI: 10.3390/s25030734.

Preferred Distance in Human-Drone Interaction.

Wogerbauer E, von Castell C, Welsch R, Hecht H Vision (Basel). 2024; 8(4).

PMID: 39449392 PMC: 11503297. DOI: 10.3390/vision8040059.

Emotional responses of Korean and Chinese women to Hangul phonemes to the gender of an artificial intelligence voice.

Lee M, Lee G, Lee S, Lee J Front Psychol. 2024; 15:1357975.

PMID: 39135868 PMC: 11317464. DOI: 10.3389/fpsyg.2024.1357975.

Emerging Trends of Biomedical Signal Processing in Intelligent Emotion Recognition.

Goshvarpour A Brain Sci. 2024; 14(7).

PMID: 39061369 PMC: 11274954. DOI: 10.3390/brainsci14070628.

Designing, implementing and testing an intervention of affective intelligent agents in nursing virtual reality teaching simulations-a qualitative study.

Loizou M, Arnab S, Lameras P, Hartley T, Loizides F, Kumar P Front Digit Health. 2024; 6:1307817.

PMID: 38698890 PMC: 11063316. DOI: 10.3389/fdgth.2024.1307817.

References

Petrantonakis P, Hadjileontiadis L . Emotion recognition from EEG using higher order crossings. IEEE Trans Inf Technol Biomed. 2009; 14(2):186-97. DOI: 10.1109/TITB.2009.2034649. View

Ko B . A Brief Review of Facial Emotion Recognition Based on Visual Information. Sensors (Basel). 2018; 18(2). PMC: 5856145. DOI: 10.3390/s18020401. View

Goulart C, Valadao C, Delisle-Rodriguez D, Funayama D, Favarato A, Baldo G . Visual and Thermal Image Processing for Facial Specific Landmark Detection to Infer Emotions in a Child-Robot Interaction. Sensors (Basel). 2019; 19(13). PMC: 6650968. DOI: 10.3390/s19132844. View

Kragel P, LaBar K . Decoding the Nature of Emotion in the Brain. Trends Cogn Sci. 2016; 20(6):444-455. PMC: 4875847. DOI: 10.1016/j.tics.2016.03.011. View

Siegel E, Sands M, Van Den Noortgate W, Condon P, Chang Y, Dy J . Emotion fingerprints or emotion populations? A meta-analytic investigation of autonomic features of emotion categories. Psychol Bull. 2018; 144(4):343-393. PMC: 5876074. DOI: 10.1037/bul0000128. View

Akin M . Comparison of wavelet transform and FFT methods in the analysis of EEG signals. J Med Syst. 2002; 26(3):241-7. DOI: 10.1023/a:1015075101937. View

Goulart C, Valadao C, Delisle-Rodriguez D, Caldeira E, Bastos T . Emotion analysis in children through facial emissivity of infrared thermal imaging. PLoS One. 2019; 14(3):e0212928. PMC: 6426206. DOI: 10.1371/journal.pone.0212928. View

Lench H, Flores S, Bench S . Discrete emotions predict changes in cognition, judgment, experience, behavior, and physiology: a meta-analysis of experimental emotion elicitations. Psychol Bull. 2011; 137(5):834-55. DOI: 10.1037/a0024244. View

Ioannou S, Gallese V, Merla A . Thermal infrared imaging in psychophysiology: potentialities and limits. Psychophysiology. 2014; 51(10):951-63. PMC: 4286005. DOI: 10.1111/psyp.12243. View

10.

Wen G, Li H, Huang J, Li D, Xun E . Random Deep Belief Networks for Recognizing Emotions from Speech Signals. Comput Intell Neurosci. 2017; 2017:1945630. PMC: 5357547. DOI: 10.1155/2017/1945630. View

11.

Lindquist K, Siegel E, Quigley K, Barrett L . The hundred-year emotion war: are emotions natural kinds or psychological constructions? Comment on Lench, Flores, and Bench (2011). Psychol Bull. 2013; 139(1):255-263. PMC: 3556454. DOI: 10.1037/a0029038. View

12.

Palmiero M, Piccardi L . Frontal EEG Asymmetry of Mood: A Mini-Review. Front Behav Neurosci. 2017; 11:224. PMC: 5701669. DOI: 10.3389/fnbeh.2017.00224. View

13.

Saarimaki H, Ejtehadian L, Glerean E, Jaaskelainen I, Vuilleumier P, Sams M . Distributed affective space represents multiple emotion categories across the human brain. Soc Cogn Affect Neurosci. 2018; 13(5):471-482. PMC: 6007366. DOI: 10.1093/scan/nsy018. View

14.

Volkova E, de la Rosa S, Bulthoff H, Mohler B . The MPI emotional body expressions database for narrative scenarios. PLoS One. 2014; 9(12):e113647. PMC: 4252031. DOI: 10.1371/journal.pone.0113647. View

15.

Goodfellow I, Erhan D, Luc Carrier P, Courville A, Mirza M, Hamner B . Challenges in representation learning: a report on three machine learning contests. Neural Netw. 2015; 64:59-63. DOI: 10.1016/j.neunet.2014.09.005. View

16.

Krasne F, Fanselow M, Zelikowsky M . Design of a neurally plausible model of fear learning. Front Behav Neurosci. 2011; 5:41. PMC: 3145244. DOI: 10.3389/fnbeh.2011.00041. View

17.

Ghimire D, Lee J . Geometric feature-based facial expression recognition in image sequences using multi-class AdaBoost and support vector machines. Sensors (Basel). 2013; 13(6):7714-34. PMC: 3715259. DOI: 10.3390/s130607714. View

18.

Corneanu C, Simon M, Cohn J, Guerrero S . Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications. IEEE Trans Pattern Anal Mach Intell. 2016; 38(8):1548-68. PMC: 7426891. DOI: 10.1109/TPAMI.2016.2515606. View

19.

Trnka R, Lacev A, Balcar K, Kuska M, Tavel P . Modeling Semantic Emotion Space Using a 3D Hypercube-Projection: An Innovative Analytical Approach for the Psychology of Emotions. Front Psychol. 2016; 7:522. PMC: 4835669. DOI: 10.3389/fpsyg.2016.00522. View

20.

Kreibig S . Autonomic nervous system activity in emotion: a review. Biol Psychol. 2010; 84(3):394-421. DOI: 10.1016/j.biopsycho.2010.03.010. View