Self Supervised Learning Based Emotion Recognition Using Physiological Signals

Overview

Journal Front Hum Neurosci

Specialty Neurology

Date 2024 Apr 24

PMID 38655374

Authors

Min Zhang

Yanli Cui

Affiliations

Soon will be listed here.

Abstract

Introduction: The significant role of emotional recognition in the field of human-machine interaction has garnered the attention of many researchers. Emotion recognition based on physiological signals can objectively reflect the most authentic emotional states of humans. However, existing labeled Electroencephalogram (EEG) datasets are often of small scale.

Methods: In practical scenarios, a large number of unlabeled EEG signals are easier to obtain. Therefore, this paper adopts self-supervised learning methods to study emotion recognition based on EEG. Specifically, experiments employ three pre-defined tasks to define pseudo-labels and extract features from the inherent structure of the data.

Results And Discussion: Experimental results indicate that self-supervised learning methods have the capability to learn effective feature representations for downstream tasks without any manual labels.

References

Borra D, Bossi F, Rivolta D, Magosso E . Deep learning applied to EEG source-data reveals both ventral and dorsal visual stream involvement in holistic processing of social stimuli. Sci Rep. 2023; 13(1):7365. PMC: 10162973. DOI: 10.1038/s41598-023-34487-z. View

Zhao D, Tang F, Si B, Feng X . Learning joint space-time-frequency features for EEG decoding on small labeled data. Neural Netw. 2019; 114:67-77. DOI: 10.1016/j.neunet.2019.02.009. View

Borra D, Mondini V, Magosso E, Muller-Putz G . Decoding movement kinematics from EEG using an interpretable convolutional neural network. Comput Biol Med. 2023; 165:107323. DOI: 10.1016/j.compbiomed.2023.107323. View

Jia Z, Lin Y, Wang J, Ning X, He Y, Zhou R . Multi-View Spatial-Temporal Graph Convolutional Networks With Domain Generalization for Sleep Stage Classification. IEEE Trans Neural Syst Rehabil Eng. 2021; 29:1977-1986. PMC: 8556658. DOI: 10.1109/TNSRE.2021.3110665. View

Vahid A, Muckschel M, Stober S, Stock A, Beste C . Applying deep learning to single-trial EEG data provides evidence for complementary theories on action control. Commun Biol. 2020; 3(1):112. PMC: 7062698. DOI: 10.1038/s42003-020-0846-z. View

Borra D, Magosso E, Castelo-Branco M, Simoes M . A Bayesian-optimized design for an interpretable convolutional neural network to decode and analyze the P300 response in autism. J Neural Eng. 2022; 19(4). DOI: 10.1088/1741-2552/ac7908. View

Schirrmeister R, Springenberg J, Fiederer L, Glasstetter M, Eggensperger K, Tangermann M . Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp. 2017; 38(11):5391-5420. PMC: 5655781. DOI: 10.1002/hbm.23730. View

Farahat A, Reichert C, Sweeney-Reed C, Hinrichs H . Convolutional neural networks for decoding of covert attention focus and saliency maps for EEG feature visualization. J Neural Eng. 2019; 16(6):066010. DOI: 10.1088/1741-2552/ab3bb4. View

Borra D, Fantozzi S, Magosso E . A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision. Front Hum Neurosci. 2021; 15:655840. PMC: 8295472. DOI: 10.3389/fnhum.2021.655840. View

10.

Krishnan R, Rajpurkar P, Topol E . Self-supervised learning in medicine and healthcare. Nat Biomed Eng. 2022; 6(12):1346-1352. DOI: 10.1038/s41551-022-00914-1. View

11.

Zander T, Jatzev S . Context-aware brain-computer interfaces: exploring the information space of user, technical system and environment. J Neural Eng. 2011; 9(1):016003. DOI: 10.1088/1741-2560/9/1/016003. View

12.

Hochreiter S, Schmidhuber J . Long short-term memory. Neural Comput. 1997; 9(8):1735-80. DOI: 10.1162/neco.1997.9.8.1735. View

13.

Lawhern V, Solon A, Waytowich N, Gordon S, Hung C, Lance B . EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J Neural Eng. 2018; 15(5):056013. DOI: 10.1088/1741-2552/aace8c. View

14.

Borra D, Magosso E . Deep learning-based EEG analysis: investigating P3 ERP components. J Integr Neurosci. 2022; 20(4):791-811. DOI: 10.31083/j.jin2004083. View