EEG-based Emotion Recognition Using a Temporal-difference Minimizing Neural Network

Overview

Journal Cogn Neurodyn

Publisher Springer

Specialty Neurology

Date 2024 May 3

PMID 38699602

Authors

Xiangyu Ju

Ming Li

Wenli Tian

Dewen Hu

Affiliations

Soon will be listed here.

Abstract

Electroencephalogram (EEG) emotion recognition plays an important role in human-computer interaction. An increasing number of algorithms for emotion recognition have been proposed recently. However, it is still challenging to make efficient use of emotional activity knowledge. In this paper, based on prior knowledge that emotion varies slowly across time, we propose a temporal-difference minimizing neural network (TDMNN) for EEG emotion recognition. We use maximum mean discrepancy (MMD) technology to evaluate the difference in EEG features across time and minimize the difference by a multibranch convolutional recurrent network. State-of-the-art performances are achieved using the proposed method on the SEED, SEED-IV, DEAP and DREAMER datasets, demonstrating the effectiveness of including prior knowledge in EEG emotion recognition.

Citing Articles

Transformer-Driven Affective State Recognition from Wearable Physiological Data in Everyday Contexts.

Li F, Zhang D Sensors (Basel). 2025; 25(3).

PMID: 39943399 PMC: 11820912. DOI: 10.3390/s25030761.

Multi-source domain adaptation for EEG emotion recognition based on inter-domain sample hybridization.

Wu X, Ju X, Dai S, Li X, Li M Front Hum Neurosci. 2024; 18:1464431.

PMID: 39545146 PMC: 11560783. DOI: 10.3389/fnhum.2024.1464431.

HASTF: a hybrid attention spatio-temporal feature fusion network for EEG emotion recognition.

Hu F, Wang F, Bi J, An Z, Chen C, Qu G Front Neurosci. 2024; 18:1479570.

PMID: 39469033 PMC: 11513351. DOI: 10.3389/fnins.2024.1479570.

A novel signal channel attention network for multi-modal emotion recognition.

Du Z, Ye X, Zhao P Front Neurorobot. 2024; 18:1442080.

PMID: 39323931 PMC: 11422387. DOI: 10.3389/fnbot.2024.1442080.

CSA-SA-CRTNN: A Dual-Stream Adaptive Convolutional Cyclic Hybrid Network Combining Attention Mechanisms for EEG Emotion Recognition.

Qian R, Xiong X, Zhou J, Yu H, Sha K Brain Sci. 2024; 14(8).

PMID: 39199509 PMC: 11353053. DOI: 10.3390/brainsci14080817.

References

Lohani M, Payne B, Isaacowitz D . Emotional coherence in early and later adulthood during sadness reactivity and regulation. Emotion. 2017; 18(6):789-804. DOI: 10.1037/emo0000345. View

Levenson R . Blood, sweat, and fears: the autonomic architecture of emotion. Ann N Y Acad Sci. 2004; 1000:348-66. DOI: 10.1196/annals.1280.016. View

Kragel P, Hariri A, LaBar K . The Temporal Dynamics of Spontaneous Emotional Brain States and Their Implications for Mental Health. J Cogn Neurosci. 2021; 34(5):715-728. PMC: 9026845. DOI: 10.1162/jocn_a_01787. View

Zheng W, Liu W, Lu Y, Lu B, Cichocki A . EmotionMeter: A Multimodal Framework for Recognizing Human Emotions. IEEE Trans Cybern. 2018; 49(3):1110-1122. DOI: 10.1109/TCYB.2018.2797176. View

Dan-Glauser E, Gross J . Emotion regulation and emotion coherence: evidence for strategy-specific effects. Emotion. 2013; 13(5):832-42. PMC: 4106260. DOI: 10.1037/a0032672. View

Cimtay Y, Ekmekcioglu E . Investigating the Use of Pretrained Convolutional Neural Network on Cross-Subject and Cross-Dataset EEG Emotion Recognition. Sensors (Basel). 2020; 20(7). PMC: 7181114. DOI: 10.3390/s20072034. View

Torres P E, Torres E, Hernandez-Alvarez M, Yoo S . EEG-Based BCI Emotion Recognition: A Survey. Sensors (Basel). 2020; 20(18). PMC: 7570756. DOI: 10.3390/s20185083. View

Algarni M, Saeed F, Al-Hadhrami T, Ghabban F, Al-Sarem M . Deep Learning-Based Approach for Emotion Recognition Using Electroencephalography (EEG) Signals Using Bi-Directional Long Short-Term Memory (Bi-LSTM). Sensors (Basel). 2022; 22(8). PMC: 9033053. DOI: 10.3390/s22082976. View

Moon S, Chen C, Hsieh C, Wang J, Lee J . Emotional EEG classification using connectivity features and convolutional neural networks. Neural Netw. 2020; 132:96-107. DOI: 10.1016/j.neunet.2020.08.009. View

10.

Chen H, Jin M, Li Z, Fan C, Li J, He H . MS-MDA: Multisource Marginal Distribution Adaptation for Cross-Subject and Cross-Session EEG Emotion Recognition. Front Neurosci. 2021; 15:778488. PMC: 8688841. DOI: 10.3389/fnins.2021.778488. View

11.

Zhu M, Wang Q, Luo J . Emotion Recognition Based on Dynamic Energy Features Using a Bi-LSTM Network. Front Comput Neurosci. 2022; 15:741086. PMC: 8900638. DOI: 10.3389/fncom.2021.741086. View

12.

Shen F, Dai G, Lin G, Zhang J, Kong W, Zeng H . EEG-based emotion recognition using 4D convolutional recurrent neural network. Cogn Neurodyn. 2020; 14(6):815-828. PMC: 7568753. DOI: 10.1007/s11571-020-09634-1. View

13.

Shi L, Jiao Y, Lu B . Differential entropy feature for EEG-based vigilance estimation. Annu Int Conf IEEE Eng Med Biol Soc. 2013; 2013:6627-30. DOI: 10.1109/EMBC.2013.6611075. View

14.

Bao L, Qiu J, Tang H, Zheng W, Lu B . Investigating Sex Differences in Classification of Five Emotions from EEG and Eye Movement Signals. Annu Int Conf IEEE Eng Med Biol Soc. 2020; 2019:6746-6749. DOI: 10.1109/EMBC.2019.8857476. View

15.

Katsigiannis S, Ramzan N . DREAMER: A Database for Emotion Recognition Through EEG and ECG Signals From Wireless Low-cost Off-the-Shelf Devices. IEEE J Biomed Health Inform. 2017; 22(1):98-107. DOI: 10.1109/JBHI.2017.2688239. View

16.

Li X, Song D, Zhang P, Zhang Y, Hou Y, Hu B . Exploring EEG Features in Cross-Subject Emotion Recognition. Front Neurosci. 2018; 12:162. PMC: 5867345. DOI: 10.3389/fnins.2018.00162. View

17.

Feng L, Cheng C, Zhao M, Deng H, Zhang Y . EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM With Attention Mechanism. IEEE J Biomed Health Inform. 2022; 26(11):5406-5417. DOI: 10.1109/JBHI.2022.3198688. View

18.

Xiao G, Shi M, Ye M, Xu B, Chen Z, Ren Q . 4D attention-based neural network for EEG emotion recognition. Cogn Neurodyn. 2022; 16(4):805-818. PMC: 9279544. DOI: 10.1007/s11571-021-09751-5. View

19.

Goshvarpour A, Goshvarpour A . EEG spectral powers and source localization in depressing, sad, and fun music videos focusing on gender differences. Cogn Neurodyn. 2019; 13(2):161-173. PMC: 6426927. DOI: 10.1007/s11571-018-9516-y. View

20.

Ouyang D, Yuan Y, Li G, Guo Z . The Effect of Time Window Length on EEG-Based Emotion Recognition. Sensors (Basel). 2022; 22(13). PMC: 9269830. DOI: 10.3390/s22134939. View