From Speech to Thought: the Neuronal Basis of Cognitive Units in Non-experimental, Real-life Communication Investigated Using ECoG
Overview
Affiliations
Exchange of thoughts by means of expressive speech is fundamental to human communication. However, the neuronal basis of real-life communication in general, and of verbal exchange of ideas in particular, has rarely been studied until now. Here, our aim was to establish an approach for exploring the neuronal processes related to cognitive "idea" units (IUs) in conditions of non-experimental speech production. We investigated whether such units corresponding to single, coherent chunks of speech with syntactically-defined borders, are useful to unravel the neuronal mechanisms underlying real-world human cognition. To this aim, we employed simultaneous electrocorticography (ECoG) and video recordings obtained in pre-neurosurgical diagnostics of epilepsy patients. We transcribed non-experimental, daily hospital conversations, identified IUs in transcriptions of the patients' speech, classified the obtained IUs according to a previously-proposed taxonomy focusing on memory content, and investigated the underlying neuronal activity. In each of our three subjects, we were able to collect a large number of IUs which could be assigned to different functional IU subclasses with a high inter-rater agreement. Robust IU-onset-related changes in spectral magnitude could be observed in high gamma frequencies (70-150 Hz) on the inferior lateral convexity and in the superior temporal cortex regardless of the IU content. A comparison of the topography of these responses with mouth motor and speech areas identified by electrocortical stimulation showed that IUs might be of use for extraoperative mapping of eloquent cortex (average sensitivity: 44.4%, average specificity: 91.1%). High gamma responses specific to memory-related IU subclasses were observed in the inferior parietal and prefrontal regions. IU-based analysis of ECoG recordings during non-experimental communication thus elicits topographically- and functionally-specific effects. We conclude that segmentation of spontaneous real-world speech in linguistically-motivated units is a promising strategy for elucidating the neuronal basis of mental processing during non-experimental communication.
Feasibility of decoding covert speech in ECoG with a Transformer trained on overt speech.
Komeiji S, Mitsuhashi T, Iimura Y, Suzuki H, Sugano H, Shinoda K Sci Rep. 2024; 14(1):11491.
PMID: 38769115 PMC: 11106343. DOI: 10.1038/s41598-024-62230-9.
Open multimodal iEEG-fMRI dataset from naturalistic stimulation with a short audiovisual film.
Berezutskaya J, Vansteensel M, Aarnoutse E, Freudenburg Z, Piantoni G, Branco M Sci Data. 2022; 9(1):91.
PMID: 35314718 PMC: 8938409. DOI: 10.1038/s41597-022-01173-0.
Glanz O, Hader M, Schulze-Bonhage A, Auer P, Ball T Front Hum Neurosci. 2022; 15:711886.
PMID: 35185491 PMC: 8854223. DOI: 10.3389/fnhum.2021.711886.
Kern M, Bert S, Glanz O, Schulze-Bonhage A, Ball T Commun Biol. 2019; 2:118.
PMID: 30937400 PMC: 6435746. DOI: 10.1038/s42003-019-0360-3.
Neurolinguistics Research Advancing Development of a Direct-Speech Brain-Computer Interface.
Cooney C, Folli R, Coyle D iScience. 2018; 8:103-125.
PMID: 30296666 PMC: 6174918. DOI: 10.1016/j.isci.2018.09.016.