Preverbal Infants Utilize Cross-modal Semantic Congruency in Artificial Grammar Acquisition

Overview

Journal Sci Rep

Specialty Science

Date 2018 Aug 25

PMID 30139964

Citations 1

Authors

Chia-Huei Tseng

Hiu Mei Chow

Yuen Ki Ma

Jie Ding

Affiliations

Soon will be listed here.

Abstract

Learning in a multisensory world is challenging as the information from different sensory dimensions may be inconsistent and confusing. By adulthood, learners optimally integrate bimodal (e.g. audio-visual, AV) stimulation by both low-level (e.g. temporal synchrony) and high-level (e.g. semantic congruency) properties of the stimuli to boost learning outcomes. However, it is unclear how this capacity emerges and develops. To approach this question, we examined whether preverbal infants were capable of utilizing high-level properties with grammar-like rule acquisition. In three experiments, we habituated pre-linguistic infants with an audio-visual (AV) temporal sequence that resembled a grammar-like rule (A-A-B). We varied the cross-modal semantic congruence of the AV stimuli (Exp 1: congruent syllables/faces; Exp 2: incongruent syllables/shapes; Exp 3: incongruent beeps/faces) while all the other low-level properties (e.g. temporal synchrony, sensory energy) were constant. Eight- to ten-month-old infants only learned the grammar-like rule from AV congruent stimuli pairs (Exp 1), not from incongruent AV pairs (Exp 2, 3). Our results show that similar to adults, preverbal infants' learning is influenced by a high-level multisensory integration gating system, pointing to a perceptual origin of bimodal learning advantage that was not previously acknowledged.

Citing Articles

Visual-auditory interactions on explicit and implicit information processing.

Lo L, Lai C Cogn Process. 2022; 23(2):179-189.

PMID: 35142948 DOI: 10.1007/s10339-022-01077-2.

References

Bahrick L, Lickliter R . Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. Dev Psychol. 2000; 36(2):190-201. PMC: 2704001. DOI: 10.1037//0012-1649.36.2.190. View

Laurienti P, Kraft R, Maldjian J, Burdette J, Wallace M . Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res. 2004; 158(4):405-14. DOI: 10.1007/s00221-004-1913-2. View

Kim R, Seitz A, Shams L . Benefits of stimulus congruency for multisensory facilitation of visual learning. PLoS One. 2008; 3(1):e1532. PMC: 2211398. DOI: 10.1371/journal.pone.0001532. View

Koo T, Li M . A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016; 15(2):155-63. PMC: 4913118. DOI: 10.1016/j.jcm.2016.02.012. View

Starkey P, Spelke E, Gelman R . Detection of intermodal numerical correspondences by human infants. Science. 1983; 222(4620):179-81. DOI: 10.1126/science.6623069. View

Grassi M, Casco C . Audiovisual bounce-inducing effect: when sound congruence affects grouping in vision. Atten Percept Psychophys. 2010; 72(2):378-86. DOI: 10.3758/APP.72.2.378. View

Kim R, Peters M, Shams L . 0 + 1 > 1: How adding noninformative sound improves performance on a visual task. Psychol Sci. 2011; 23(1):6-12. DOI: 10.1177/0956797611420662. View

Endress A, Nespor M, Mehler J . Perceptual and memory constraints on language acquisition. Trends Cogn Sci. 2009; 13(8):348-53. DOI: 10.1016/j.tics.2009.05.005. View

Brainard D . The Psychophysics Toolbox. Spat Vis. 1997; 10(4):433-6. View

10.

Chen Y, Spence C . When hearing the bark helps to identify the dog: semantically-congruent sounds modulate the identification of masked pictures. Cognition. 2009; 114(3):389-404. DOI: 10.1016/j.cognition.2009.10.012. View

11.

Gogate L, Bahrick L, Watson J . A study of multimodal motherese: the role of temporal synchrony between verbal labels and gestures. Child Dev. 2000; 71(4):878-94. DOI: 10.1111/1467-8624.00197. View

12.

Kanayama N, Kimura K, Hiraki K . Cortical EEG components that reflect inverse effectiveness during visuotactile integration processing. Brain Res. 2014; 1598:18-30. DOI: 10.1016/j.brainres.2014.12.017. View

13.

Beebe B, Messinger D, Bahrick L, Margolis A, Buck K, Chen H . A systems view of mother-infant face-to-face communication. Dev Psychol. 2016; 52(4):556-71. PMC: 4808406. DOI: 10.1037/a0040085. View

14.

Cunillera T, Camara E, Laine M, Rodriguez-Fornells A . Words as anchors: known words facilitate statistical learning. Exp Psychol. 2010; 57(2):134-41. DOI: 10.1027/1618-3169/a000017. View

15.

Iordanescu L, Grabowecky M, Franconeri S, Theeuwes J, Suzuki S . Characteristic sounds make you look at target objects more quickly. Atten Percept Psychophys. 2010; 72(7):1736-41. PMC: 3261720. DOI: 10.3758/APP.72.7.1736. View

16.

Iordanescu L, Grabowecky M, Suzuki S . Object-based auditory facilitation of visual search for pictures and words with frequent and rare targets. Acta Psychol (Amst). 2010; 137(2):252-9. PMC: 3010345. DOI: 10.1016/j.actpsy.2010.07.017. View

17.

Meredith M, Nemitz J, Stein B . Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci. 1987; 7(10):3215-29. PMC: 6569162. View

18.

Srinivasan M, Carey S . The long and the short of it: on the nature and origin of functional overlap between representations of space and time. Cognition. 2010; 116(2):217-41. PMC: 2900540. DOI: 10.1016/j.cognition.2010.05.005. View

19.

Lewkowicz D, Minar N . Infants are not sensitive to synesthetic cross-modality correspondences: a comment on Walker et al. (2010). Psychol Sci. 2014; 25(3):832-4. PMC: 3954909. DOI: 10.1177/0956797613516011. View

20.

Meyerhoff H, Huff M . Semantic congruency but not temporal synchrony enhances long-term memory performance for audio-visual scenes. Mem Cognit. 2015; 44(3):390-402. DOI: 10.3758/s13421-015-0575-6. View