3D Convolutional Neural Network Based on Spatial-spectral Feature Pictures Learning for Decoding Motor Imagery EEG Signal

Overview

Journal Front Neurorobot

Date 2024 Dec 25

PMID 39720668

Authors

Xiaoguang Li

Yaqi Chu

Xuejian Wu

Affiliations

Soon will be listed here.

Abstract

Non-invasive brain-computer interfaces (BCI) hold great promise in the field of neurorehabilitation. They are easy to use and do not require surgery, particularly in the area of motor imagery electroencephalography (EEG). However, motor imagery EEG signals often have a low signal-to-noise ratio and limited spatial and temporal resolution. Traditional deep neural networks typically only focus on the spatial and temporal features of EEG, resulting in relatively low decoding and accuracy rates for motor imagery tasks. To address these challenges, this paper proposes a 3D Convolutional Neural Network (P-3DCNN) decoding method that jointly learns spatial-frequency feature maps from the frequency and spatial domains of the EEG signals. First, the Welch method is used to calculate the frequency band power spectrum of the EEG, and a 2D matrix representing the spatial topology distribution of the electrodes is constructed. These spatial-frequency representations are then generated through cubic interpolation of the temporal EEG data. Next, the paper designs a 3DCNN network with 1D and 2D convolutional layers in series to optimize the convolutional kernel parameters and effectively learn the spatial-frequency features of the EEG. Batch normalization and dropout are also applied to improve the training speed and classification performance of the network. Finally, through experiments, the proposed method is compared to various classic machine learning and deep learning techniques. The results show an average decoding accuracy rate of 86.69%, surpassing other advanced networks. This demonstrates the effectiveness of our approach in decoding motor imagery EEG and offers valuable insights for the development of BCI.

References

Goldberger A, Amaral L, Glass L, Hausdorff J, Ivanov P, Mark R . PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation. 2000; 101(23):E215-20. DOI: 10.1161/01.cir.101.23.e215. View

Yin E, Zhou Z, Jiang J, Yu Y, Hu D . A Dynamically Optimized SSVEP Brain-Computer Interface (BCI) Speller. IEEE Trans Biomed Eng. 2014; 62(6):1447-56. DOI: 10.1109/TBME.2014.2320948. View

Yacine F, Salah H, Amar K, Ahmad K . A novel ANN adaptive Riemannian-based kernel classification for motor imagery. Biomed Phys Eng Express. 2022; 9(1). DOI: 10.1088/2057-1976/acaca2. View

Zhang D, Chen K, Jian D, Yao L . Motor Imagery Classification via Temporal Attention Cues of Graph Embedded EEG Signals. IEEE J Biomed Health Inform. 2020; 24(9):2570-2579. DOI: 10.1109/JBHI.2020.2967128. View

Li M, Li F, Pan J, Zhang D, Zhao S, Li J . The MindGomoku: An Online P300 BCI Game Based on Bayesian Deep Learning. Sensors (Basel). 2021; 21(5). PMC: 7956207. DOI: 10.3390/s21051613. View

Lawhern V, Solon A, Waytowich N, Gordon S, Hung C, Lance B . EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J Neural Eng. 2018; 15(5):056013. DOI: 10.1088/1741-2552/aace8c. View

Thenmozhi T, Helen R . Feature Selection Using Extreme Gradient Boosting Bayesian Optimization to upgrade the Classification Performance of Motor Imagery signals for BCI. J Neurosci Methods. 2021; 366:109425. DOI: 10.1016/j.jneumeth.2021.109425. View

Chen X, Zhao B, Wang Y, Gao X . Combination of high-frequency SSVEP-based BCI and computer vision for controlling a robotic arm. J Neural Eng. 2018; 16(2):026012. DOI: 10.1088/1741-2552/aaf594. View

Zhang R, Chen Y, Xu Z, Zhang L, Hu Y, Chen M . Recognition of single upper limb motor imagery tasks from EEG using multi-branch fusion convolutional neural network. Front Neurosci. 2023; 17:1129049. PMC: 9992961. DOI: 10.3389/fnins.2023.1129049. View

10.

Chen J, Xia Y, Zhou X, Vidal Rosas E, Thomas A, Loureiro R . fNIRS-EEG BCIs for Motor Rehabilitation: A Review. Bioengineering (Basel). 2023; 10(12). PMC: 10740927. DOI: 10.3390/bioengineering10121393. View

11.

Savic A, Lontis E, Mrachacz-Kersting N, Popovic M . Dynamics of movement-related cortical potentials and sensorimotor oscillations during palmar grasp movements. Eur J Neurosci. 2019; 51(9):1962-1970. DOI: 10.1111/ejn.14629. View

12.

Yin E, Zhou Z, Jiang J, Chen F, Liu Y, Hu D . A novel hybrid BCI speller based on the incorporation of SSVEP into the P300 paradigm. J Neural Eng. 2013; 10(2):026012. DOI: 10.1088/1741-2560/10/2/026012. View

13.

Wang W, Qi F, Wipf D, Cai C, Yu T, Li Y . Sparse Bayesian Learning for End-to-End EEG Decoding. IEEE Trans Pattern Anal Mach Intell. 2023; 45(12):15632-15649. DOI: 10.1109/TPAMI.2023.3299568. View

14.

Antony M, Sankaralingam B, Mahendran R, Gardezi A, Shafiq M, Choi J . Classification of EEG Using Adaptive SVM Classifier with CSP and Online Recursive Independent Component Analysis. Sensors (Basel). 2022; 22(19). PMC: 9573537. DOI: 10.3390/s22197596. View

15.

Yin E, Zhou Z, Jiang J, Chen F, Liu Y, Hu D . A speedy hybrid BCI spelling approach combining P300 and SSVEP. IEEE Trans Biomed Eng. 2013; 61(2):473-83. DOI: 10.1109/TBME.2013.2281976. View

16.

Li M, Ruan Z . A novel decoding method for motor imagery tasks with 4D data representation and 3D convolutional neural networks. J Neural Eng. 2021; 18(4). DOI: 10.1088/1741-2552/abf68b. View

17.

Pei Y, Luo Z, Zhao H, Xu D, Li W, Yan Y . A Tensor-Based Frequency Features Combination Method for Brain-Computer Interfaces. IEEE Trans Neural Syst Rehabil Eng. 2021; 30:465-475. DOI: 10.1109/TNSRE.2021.3125386. View

18.

Echtioui A, Zouch W, Ghorbel M, Mhiri C, Hamam H . Classification of BCI Multiclass Motor Imagery Task Based on Artificial Neural Network. Clin EEG Neurosci. 2023; 55(4):455-464. DOI: 10.1177/15500594221148285. View