FMRI Brain Decoding and Its Applications in Brain-Computer Interface: A Survey

Overview

Journal Brain Sci

Publisher MDPI

Date 2022 Feb 25

PMID 35203991

Authors

Bing Du

Xiaomu Cheng

Yiping Duan

Huansheng Ning

Affiliations

Soon will be listed here.

Abstract

Brain neural activity decoding is an important branch of neuroscience research and a key technology for the brain-computer interface (BCI). Researchers initially developed simple linear models and machine learning algorithms to classify and recognize brain activities. With the great success of deep learning on image recognition and generation, deep neural networks (DNN) have been engaged in reconstructing visual stimuli from human brain activity via functional magnetic resonance imaging (fMRI). In this paper, we reviewed the brain activity decoding models based on machine learning and deep learning algorithms. Specifically, we focused on current brain activity decoding models with high attention: variational auto-encoder (VAE), generative confrontation network (GAN), and the graph convolutional network (GCN). Furthermore, brain neural-activity-decoding-enabled fMRI-based BCI applications in mental and psychological disease treatment are presented to illustrate the positive correlation between brain decoding and BCI. Finally, existing challenges and future research directions are addressed.

Citing Articles

Advances in Neuroimaging and Deep Learning for Emotion Detection: A Systematic Review of Cognitive Neuroscience and Algorithmic Innovations.

Halkiopoulos C, Gkintoni E, Aroutzidis A, Antonopoulou H Diagnostics (Basel). 2025; 15(4).

PMID: 40002607 PMC: 11854508. DOI: 10.3390/diagnostics15040456.

Electroencephalography-Based Brain-Computer Interfaces in Rehabilitation: A Bibliometric Analysis (2013-2023).

Angulo Medina A, Aguilar Bonilla M, Rodriguez Giraldo I, Montenegro Palacios J, Caceres Gutierrez D, Liscano Y Sensors (Basel). 2024; 24(22).

PMID: 39598903 PMC: 11598414. DOI: 10.3390/s24227125.

Unique Cortical and Subcortical Activation Patterns for Different Conspecific Calls in Marmosets.

Jafari A, Dureux A, Zanini A, Menon R, Gilbert K, Everling S J Neurosci. 2024; 45(3.

PMID: 39516045 PMC: 11735661. DOI: 10.1523/JNEUROSCI.0670-24.2024.

Natural Image Reconstruction from fMRI Based on Node-Edge Interaction and Multi-Scale Constraint.

Kuang M, Zhan Z, Gao S Brain Sci. 2024; 14(3).

PMID: 38539622 PMC: 10968908. DOI: 10.3390/brainsci14030234.

Decoding Visual fMRI Stimuli from Human Brain Based on Graph Convolutional Neural Network.

Meng L, Ge K Brain Sci. 2022; 12(10).

PMID: 36291327 PMC: 9599823. DOI: 10.3390/brainsci12101394.

References

Zotev V, Krueger F, Phillips R, Alvarez R, Simmons W, Bellgowan P . Self-regulation of amygdala activation using real-time FMRI neurofeedback. PLoS One. 2011; 6(9):e24522. PMC: 3169601. DOI: 10.1371/journal.pone.0024522. View

Huth A, Nishimoto S, Vu A, Gallant J . A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron. 2012; 76(6):1210-24. PMC: 3556488. DOI: 10.1016/j.neuron.2012.10.014. View

Engel S, Glover G, Wandell B . Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cereb Cortex. 1997; 7(2):181-92. DOI: 10.1093/cercor/7.2.181. View

Birbaumer N, Veit R, Lotze M, Erb M, Hermann C, Grodd W . Deficient fear conditioning in psychopathy: a functional magnetic resonance imaging study. Arch Gen Psychiatry. 2005; 62(7):799-805. DOI: 10.1001/archpsyc.62.7.799. View

Mohanty R, Sinha A, Remsik A, Dodd K, Young B, Jacobson T . Machine Learning Classification to Identify the Stage of Brain-Computer Interface Therapy for Stroke Rehabilitation Using Functional Connectivity. Front Neurosci. 2018; 12:353. PMC: 5986965. DOI: 10.3389/fnins.2018.00353. View

Shen G, Horikawa T, Majima K, Kamitani Y . Deep image reconstruction from human brain activity. PLoS Comput Biol. 2019; 15(1):e1006633. PMC: 6347330. DOI: 10.1371/journal.pcbi.1006633. View

Cowen A, Chun M, Kuhl B . Neural portraits of perception: reconstructing face images from evoked brain activity. Neuroimage. 2014; 94:12-22. PMC: 4028096. DOI: 10.1016/j.neuroimage.2014.03.018. View

Caria A, Sitaram R, Veit R, Begliomini C, Birbaumer N . Volitional control of anterior insula activity modulates the response to aversive stimuli. A real-time functional magnetic resonance imaging study. Biol Psychiatry. 2010; 68(5):425-32. DOI: 10.1016/j.biopsych.2010.04.020. View

Han K, Wen H, Shi J, Lu K, Zhang Y, Fu D . Variational autoencoder: An unsupervised model for encoding and decoding fMRI activity in visual cortex. Neuroimage. 2019; 198:125-136. PMC: 6592726. DOI: 10.1016/j.neuroimage.2019.05.039. View

10.

Cox D, Savoy R . Functional magnetic resonance imaging (fMRI) "brain reading": detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage. 2003; 19(2 Pt 1):261-70. DOI: 10.1016/s1053-8119(03)00049-1. View

11.

Zhang Y, Tetrel L, Thirion B, Bellec P . Functional annotation of human cognitive states using deep graph convolution. Neuroimage. 2021; 231:117847. DOI: 10.1016/j.neuroimage.2021.117847. View

12.

Sereno M, Dale A, Reppas J, Kwong K, Belliveau J, Brady T . Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science. 1995; 268(5212):889-93. DOI: 10.1126/science.7754376. View

13.

Chen M, Han J, Hu X, Jiang X, Guo L, Liu T . Survey of encoding and decoding of visual stimulus via FMRI: an image analysis perspective. Brain Imaging Behav. 2013; 8(1):7-23. PMC: 4245050. DOI: 10.1007/s11682-013-9238-z. View

14.

Arns M, Batail J, Bioulac S, Congedo M, Daudet C, Drapier D . Neurofeedback: One of today's techniques in psychiatry?. Encephale. 2017; 43(2):135-145. DOI: 10.1016/j.encep.2016.11.003. View

15.

Linden D, Habes I, Johnston S, Linden S, Tatineni R, Subramanian L . Real-time self-regulation of emotion networks in patients with depression. PLoS One. 2012; 7(6):e38115. PMC: 3366978. DOI: 10.1371/journal.pone.0038115. View

16.

Pichiorri F, Morone G, Petti M, Toppi J, Pisotta I, Molinari M . Brain-computer interface boosts motor imagery practice during stroke recovery. Ann Neurol. 2015; 77(5):851-65. DOI: 10.1002/ana.24390. View

17.

Schoenmakers S, Guclu U, van Gerven M, Heskes T . Gaussian mixture models and semantic gating improve reconstructions from human brain activity. Front Comput Neurosci. 2015; 8:173. PMC: 4311641. DOI: 10.3389/fncom.2014.00173. View

18.

Logothetis N . The underpinnings of the BOLD functional magnetic resonance imaging signal. J Neurosci. 2003; 23(10):3963-71. PMC: 6741096. View

19.

Tajbakhsh N, Shin J, Gurudu S, Hurst R, Kendall C, Gotway M . Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?. IEEE Trans Med Imaging. 2016; 35(5):1299-1312. DOI: 10.1109/TMI.2016.2535302. View

20.

Walther D, Caddigan E, Fei-Fei L, Beck D . Natural scene categories revealed in distributed patterns of activity in the human brain. J Neurosci. 2009; 29(34):10573-81. PMC: 2774133. DOI: 10.1523/JNEUROSCI.0559-09.2009. View