Decoding Vowels and Consonants in Spoken and Imagined Words Using Electrocorticographic Signals in Humans

Overview

Journal J Neural Eng

Specialties Biomedical Engineering
Neurology

Date 2011 Jul 14

PMID 21750369

Citations 80

Authors

Xiaomei Pei

Dennis L Barbour

Eric C Leuthardt

Gerwin Schalk

Affiliations

Soon will be listed here.

Abstract

Several stories in the popular media have speculated that it may be possible to infer from the brain which word a person is speaking or even thinking. While recent studies have demonstrated that brain signals can give detailed information about actual and imagined actions, such as different types of limb movements or spoken words, concrete experimental evidence for the possibility to 'read the mind', i.e. to interpret internally-generated speech, has been scarce. In this study, we found that it is possible to use signals recorded from the surface of the brain (electrocorticography) to discriminate the vowels and consonants embedded in spoken and in imagined words, and we defined the cortical areas that held the most information about discrimination of vowels and consonants. The results shed light on the distinct mechanisms associated with production of vowels and consonants, and could provide the basis for brain-based communication using imagined speech.

Citing Articles

Generative language reconstruction from brain recordings.

Ye Z, Ai Q, Liu Y, de Rijke M, Zhang M, Lioma C Commun Biol. 2025; 8(1):346.

PMID: 40025160 PMC: 11873242. DOI: 10.1038/s42003-025-07731-7.

Imagined speech event detection from electrocorticography and its transfer between speech modes and subjects.

de Borman A, Wittevrongel B, Dauwe I, Carrette E, Meurs A, Van Roost D Commun Biol. 2024; 7(1):818.

PMID: 38969758 PMC: 11226700. DOI: 10.1038/s42003-024-06518-6.

Feasibility of decoding covert speech in ECoG with a Transformer trained on overt speech.

Komeiji S, Mitsuhashi T, Iimura Y, Suzuki H, Sugano H, Shinoda K Sci Rep. 2024; 14(1):11491.

PMID: 38769115 PMC: 11106343. DOI: 10.1038/s41598-024-62230-9.

The speech neuroprosthesis.

Silva A, Littlejohn K, Liu J, Moses D, Chang E Nat Rev Neurosci. 2024; 25(7):473-492.

PMID: 38745103 PMC: 11540306. DOI: 10.1038/s41583-024-00819-9.

Representation of internal speech by single neurons in human supramarginal gyrus.

Wandelt S, Bjanes D, Pejsa K, Lee B, Liu C, Andersen R Nat Hum Behav. 2024; 8(6):1136-1149.

PMID: 38740984 PMC: 11199147. DOI: 10.1038/s41562-024-01867-y.

References

Crone N, Miglioretti D, Gordon B, Lesser R . Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. Brain. 1999; 121 ( Pt 12):2301-15. DOI: 10.1093/brain/121.12.2301. View

Sharp D, Scott S, Cutler A, Wise R . Lexical retrieval constrained by sound structure: the role of the left inferior frontal gyrus. Brain Lang. 2005; 92(3):309-19. DOI: 10.1016/j.bandl.2004.07.002. View

Mellinger J, Schalk G, Braun C, Preissl H, Rosenstiel W, Birbaumer N . An MEG-based brain-computer interface (BCI). Neuroimage. 2007; 36(3):581-93. PMC: 2017111. DOI: 10.1016/j.neuroimage.2007.03.019. View

Pulvermuller F, Fadiga L . Active perception: sensorimotor circuits as a cortical basis for language. Nat Rev Neurosci. 2010; 11(5):351-60. DOI: 10.1038/nrn2811. View

Hochberg L, Serruya M, Friehs G, Mukand J, Saleh M, Caplan A . Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature. 2006; 442(7099):164-71. DOI: 10.1038/nature04970. View

DaSalla C, Kambara H, Sato M, Koike Y . Single-trial classification of vowel speech imagery using common spatial patterns. Neural Netw. 2009; 22(9):1334-9. DOI: 10.1016/j.neunet.2009.05.008. View

Leuthardt E, Schalk G, Wolpaw J, Ojemann J, Moran D . A brain-computer interface using electrocorticographic signals in humans. J Neural Eng. 2005; 1(2):63-71. DOI: 10.1088/1741-2560/1/2/001. View

Caramazza A, Chialant D, Capasso R, Miceli G . Separable processing of consonants and vowels. Nature. 2000; 403(6768):428-30. DOI: 10.1038/35000206. View

Aoki F, Fetz E, Shupe L, Lettich E, Ojemann G . Changes in power and coherence of brain activity in human sensorimotor cortex during performance of visuomotor tasks. Biosystems. 2001; 63(1-3):89-99. DOI: 10.1016/s0303-2647(01)00149-6. View

10.

Wolpaw J . Brain-computer interfaces as new brain output pathways. J Physiol. 2007; 579(Pt 3):613-9. PMC: 2151370. DOI: 10.1113/jphysiol.2006.125948. View

11.

Miller K, Schalk G, Fetz E, den Nijs M, Ojemann J, Rao R . Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc Natl Acad Sci U S A. 2010; 107(9):4430-5. PMC: 2840149. DOI: 10.1073/pnas.0913697107. View

12.

Suppes P, Han B . Brain-wave representation of words by superposition of a few sine waves. Proc Natl Acad Sci U S A. 2000; 97(15):8738-43. PMC: 27018. DOI: 10.1073/pnas.140228397. View

13.

Carreiras M, Price C . Brain activation for consonants and vowels. Cereb Cortex. 2008; 18(7):1727-35. DOI: 10.1093/cercor/bhm202. View

14.

Pei X, Leuthardt E, Gaona C, Brunner P, Wolpaw J, Schalk G . Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition. Neuroimage. 2010; 54(4):2960-72. PMC: 3020260. DOI: 10.1016/j.neuroimage.2010.10.029. View

15.

Gunduz A, Sanchez J, Carney P, Principe J . Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans Motor control features. Neural Netw. 2009; 22(9):1257-70. DOI: 10.1016/j.neunet.2009.06.036. View

16.

Formisano E, De Martino F, Bonte M, Goebel R . "Who" is saying "what"? Brain-based decoding of human voice and speech. Science. 2008; 322(5903):970-3. DOI: 10.1126/science.1164318. View

17.

Leuthardt E, Miller K, Anderson N, Schalk G, Dowling J, Miller J . Electrocorticographic frequency alteration mapping: a clinical technique for mapping the motor cortex. Neurosurgery. 2007; 60(4 Suppl 2):260-70. DOI: 10.1227/01.NEU.0000255413.70807.6E. View

18.

Crone N, Boatman D, Gordon B, Hao L . Induced electrocorticographic gamma activity during auditory perception. Brazier Award-winning article, 2001. Clin Neurophysiol. 2001; 112(4):565-82. DOI: 10.1016/s1388-2457(00)00545-9. View

19.

Sinai A, Bowers C, Crainiceanu C, Boatman D, Gordon B, Lesser R . Electrocorticographic high gamma activity versus electrical cortical stimulation mapping of naming. Brain. 2005; 128(Pt 7):1556-70. DOI: 10.1093/brain/awh491. View

20.

Miller K, Shenoy P, den Nijs M, Sorensen L, Rao R, Ojemann J . Beyond the gamma band: the role of high-frequency features in movement classification. IEEE Trans Biomed Eng. 2008; 55(5):1634-7. DOI: 10.1109/TBME.2008.918569. View