Gestures Speed Up Responses to Questions

Overview

Journal Lang Cogn Neurosci

Specialty Neurology

Date 2024 May 30

PMID 38812611

Authors

Marlijn Ter Bekke

Linda Drijvers

Judith Holler

Affiliations

Soon will be listed here.

Abstract

Most language use occurs in face-to-face conversation, which involves rapid turn-taking. Seeing communicative bodily signals in addition to hearing speech may facilitate such fast responding. We tested whether this holds for co-speech hand gestures by investigating whether these gestures speed up button press responses to questions. Sixty native speakers of Dutch viewed videos in which an actress asked yes/no-questions, either with or without a corresponding iconic hand gesture. Participants answered the questions as quickly and accurately as possible via button press. Gestures did not impact response accuracy, but crucially, gestures sped up responses, suggesting that response planning may be finished earlier when gestures are seen. How much gestures sped up responses was not related to their timing in the question or their timing with respect to the corresponding information in speech. Overall, these results are in line with the idea that multimodality may facilitate fast responding during face-to-face conversation.

References

He Y, Gebhardt H, Steines M, Sammer G, Kircher T, Nagels A . The EEG and fMRI signatures of neural integration: An investigation of meaningful gestures and corresponding speech. Neuropsychologia. 2015; 72:27-42. DOI: 10.1016/j.neuropsychologia.2015.04.018. View

Levinson S . Turn-taking in Human Communication--Origins and Implications for Language Processing. Trends Cogn Sci. 2015; 20(1):6-14. DOI: 10.1016/j.tics.2015.10.010. View

Stivers T, Enfield N, Brown P, Englert C, Hayashi M, Heinemann T . Universals and cultural variation in turn-taking in conversation. Proc Natl Acad Sci U S A. 2009; 106(26):10587-92. PMC: 2705608. DOI: 10.1073/pnas.0903616106. View

Drijvers L, Holler J . The multimodal facilitation effect in human communication. Psychon Bull Rev. 2022; 30(2):792-801. PMC: 10104796. DOI: 10.3758/s13423-022-02178-x. View

Kelly S, Ozyurek A, Maris E . Two sides of the same coin: speech and gesture mutually interact to enhance comprehension. Psychol Sci. 2010; 21(2):260-7. DOI: 10.1177/0956797609357327. View

Nagels A, Kircher T, Steines M, Straube B . Feeling addressed! The role of body orientation and co-speech gesture in social communication. Hum Brain Mapp. 2015; 36(5):1925-36. PMC: 6869376. DOI: 10.1002/hbm.22746. View

Ter Bekke M, Drijvers L, Holler J . Hand Gestures Have Predictive Potential During Conversation: An Investigation of the Timing of Gestures in Relation to Speech. Cogn Sci. 2024; 48(1):e13407. DOI: 10.1111/cogs.13407. View

Roberts S, Torreira F, Levinson S . The effects of processing and sequence organization on the timing of turn taking: a corpus study. Front Psychol. 2015; 6:509. PMC: 4429583. DOI: 10.3389/fpsyg.2015.00509. View

Dargue N, Sweller N, Jones M . When our hands help us understand: A meta-analysis into the effects of gesture on comprehension. Psychol Bull. 2019; 145(8):765-784. DOI: 10.1037/bul0000202. View

10.

Miller J . Timecourse of coactivation in bimodal divided attention. Percept Psychophys. 1986; 40(5):331-43. DOI: 10.3758/bf03203025. View

11.

Wu Y, Coulson S . Iconic Gestures Facilitate Discourse Comprehension in Individuals With Superior Immediate Memory for Body Configurations. Psychol Sci. 2015; 26(11):1717-27. DOI: 10.1177/0956797615597671. View

12.

Levinson S, Torreira F . Timing in turn-taking and its implications for processing models of language. Front Psychol. 2015; 6:731. PMC: 4464110. DOI: 10.3389/fpsyg.2015.00731. View

13.

Landis J, Koch G . The measurement of observer agreement for categorical data. Biometrics. 1977; 33(1):159-74. View

14.

Bogels S, Magyari L, Levinson S . Neural signatures of response planning occur midway through an incoming question in conversation. Sci Rep. 2015; 5:12881. PMC: 4525376. DOI: 10.1038/srep12881. View

15.

Drijvers L, Vaitonyte J, Ozyurek A . Degree of Language Experience Modulates Visual Attention to Visible Speech and Iconic Gestures During Clear and Degraded Speech Comprehension. Cogn Sci. 2019; 43(10):e12789. PMC: 6790953. DOI: 10.1111/cogs.12789. View

16.

Kendrick K, Holler J, Levinson S . Turn-taking in human face-to-face interaction is multimodal: gaze direction and manual gestures aid the coordination of turn transitions. Philos Trans R Soc Lond B Biol Sci. 2023; 378(1875):20210473. PMC: 9985971. DOI: 10.1098/rstb.2021.0473. View

17.

Holle H, Rein R . EasyDIAg: A tool for easy determination of interrater agreement. Behav Res Methods. 2014; 47(3):837-47. DOI: 10.3758/s13428-014-0506-7. View

18.

Zhang Y, Frassinelli D, Tuomainen J, Skipper J, Vigliocco G . More than words: word predictability, prosody, gesture and mouth movements in natural language comprehension. Proc Biol Sci. 2021; 288(1955):20210500. PMC: 8292779. DOI: 10.1098/rspb.2021.0500. View

19.

Holler J, Kendrick K, Levinson S . Processing language in face-to-face conversation: Questions with gestures get faster responses. Psychon Bull Rev. 2017; 25(5):1900-1908. DOI: 10.3758/s13423-017-1363-z. View

20.

Drijvers L, Ozyurek A, Jensen O . Hearing and seeing meaning in noise: Alpha, beta, and gamma oscillations predict gestural enhancement of degraded speech comprehension. Hum Brain Mapp. 2018; 39(5):2075-2087. PMC: 5947738. DOI: 10.1002/hbm.23987. View