A Cross-Modal and Cross-lingual Study of Iconicity in Language: Insights From Deep Learning

Overview

Journal Cogn Sci

Specialty Psychology

Date 2022 Jun 6

PMID 35665953

Authors

Andrea Gregor de Varda

Carlo Strapparava

Affiliations

Soon will be listed here.

Abstract

The present paper addresses the study of non-arbitrariness in language within a deep learning framework. We present a set of experiments aimed at assessing the pervasiveness of different forms of non-arbitrary phonological patterns across a set of typologically distant languages. Different sequence-processing neural networks are trained in a set of languages to associate the phonetic vectorization of a set of words to their sensory (Experiment 1), semantic (Experiment 2), and word-class representations (Experiment 3). The models are then tested, without further training, in a set of novel instances in a language belonging to a different language family, and their performance is compared with a randomized baseline. We show that the three cross-domain mappings can be successfully transferred across languages and language families, suggesting that the phonological structure of the lexicon is pervaded with language-invariant cues about the words' meaning and their syntactic classes.

Citing Articles

The cognitive science of language diversity: achievements and challenges.

Benitez-Burraco A Cogn Process. 2025; .

PMID: 39998596 DOI: 10.1007/s10339-025-01262-z.

Cracking arbitrariness: A data-driven study of auditory iconicity in spoken English.

de Varda A, Marelli M Psychon Bull Rev. 2025; .

PMID: 39779657 DOI: 10.3758/s13423-024-02630-0.

THINGSplus: New norms and metadata for the THINGS database of 1854 object concepts and 26,107 natural object images.

Stoinski L, Perkuhn J, Hebart M Behav Res Methods. 2023; 56(3):1583-1603.

PMID: 37095326 PMC: 10991023. DOI: 10.3758/s13428-023-02110-8.

References

Graven T, Desebrock C . Bouba or kiki with and without vision: Shape-audio regularities and mental images. Acta Psychol (Amst). 2018; 188:200-212. DOI: 10.1016/j.actpsy.2018.05.011. View

Lupyan G, Winter B . Language is more abstract than you think, or, why aren't languages more iconic?. Philos Trans R Soc Lond B Biol Sci. 2018; 373(1752). PMC: 6015821. DOI: 10.1098/rstb.2017.0137. View

Dingemanse M, Blasi D, Lupyan G, Christiansen M, Monaghan P . Arbitrariness, Iconicity, and Systematicity in Language. Trends Cogn Sci. 2015; 19(10):603-615. DOI: 10.1016/j.tics.2015.07.013. View

Hebart M, Dickter A, Kidder A, Kwok W, Corriveau A, Van Wicklin C . THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images. PLoS One. 2019; 14(10):e0223792. PMC: 6793944. DOI: 10.1371/journal.pone.0223792. View

Borghi A, Binkofski F, Castelfranchi C, Cimatti F, Scorolli C, Tummolini L . The challenge of abstract concepts. Psychol Bull. 2017; 143(3):263-292. DOI: 10.1037/bul0000089. View

Lockwood G, Dingemanse M . Iconicity in the lab: a review of behavioral, developmental, and neuroimaging research into sound-symbolism. Front Psychol. 2015; 6:1246. PMC: 4547014. DOI: 10.3389/fpsyg.2015.01246. View

Reilly J, Hung J, Westbury C . Non-Arbitrariness in Mapping Word Form to Meaning: Cross-Linguistic Formal Markers of Word Concreteness. Cogn Sci. 2016; 41(4):1071-1089. DOI: 10.1111/cogs.12361. View

Cwiek A, Fuchs S, Draxler C, Asu E, Dediu D, Hiovain K . The effect is robust across cultures and writing systems. Philos Trans R Soc Lond B Biol Sci. 2021; 377(1841):20200390. PMC: 8591387. DOI: 10.1098/rstb.2020.0390. View

Crutch S, Warrington E . Abstract and concrete concepts have structurally different representational frameworks. Brain. 2004; 128(Pt 3):615-27. DOI: 10.1093/brain/awh349. View

10.

Ozturk O, Krehm M, Vouloumanos A . Sound symbolism in infancy: evidence for sound-shape cross-modal correspondences in 4-month-olds. J Exp Child Psychol. 2012; 114(2):173-86. DOI: 10.1016/j.jecp.2012.05.004. View

11.

Maurer D, Pathman T, Mondloch C . The shape of boubas: sound-shape correspondences in toddlers and adults. Dev Sci. 2006; 9(3):316-22. DOI: 10.1111/j.1467-7687.2006.00495.x. View

12.

Perniss P, Thompson R, Vigliocco G . Iconicity as a general property of language: evidence from spoken and signed languages. Front Psychol. 2011; 1:227. PMC: 3153832. DOI: 10.3389/fpsyg.2010.00227. View

13.

Levelt W, Roelofs A, Meyer A . A theory of lexical access in speech production. Behav Brain Sci. 2001; 22(1):1-38; discussion 38-75. DOI: 10.1017/s0140525x99001776. View

14.

Yamins D, DiCarlo J . Using goal-driven deep learning models to understand sensory cortex. Nat Neurosci. 2016; 19(3):356-65. DOI: 10.1038/nn.4244. View

15.

Parise C, Pavani F . Evidence of sound symbolism in simple vocalizations. Exp Brain Res. 2011; 214(3):373-80. DOI: 10.1007/s00221-011-2836-3. View

16.

Gunther F, Marelli M, Tureski S, Petilli M . ViSpa (Vision Spaces): A computer-vision-based representation system for individual images and concept prototypes, with large-scale evaluation. Psychol Rev. 2022; 130(4):896-934. DOI: 10.1037/rev0000392. View

17.

Hirata S, Ukita J, Kita S . Implicit phonetic symbolism in voicing of consonants and visual lightness using Garner's speeded classification task. Percept Mot Skills. 2012; 113(3):929-40. DOI: 10.2466/15.21.28.PMS.113.6.929-940. View

18.

LeCun Y, Bengio Y, Hinton G . Deep learning. Nature. 2015; 521(7553):436-44. DOI: 10.1038/nature14539. View

19.

Fryer L, Freeman J, Pring L . Touching words is not enough: how visual experience influences haptic-auditory associations in the "Bouba-Kiki" effect. Cognition. 2014; 132(2):164-73. DOI: 10.1016/j.cognition.2014.03.015. View

20.

Bremner A, Caparos S, Davidoff J, de Fockert J, Linnell K, Spence C . "Bouba" and "Kiki" in Namibia? A remote culture make similar shape-sound matches, but different shape-taste matches to Westerners. Cognition. 2012; 126(2):165-72. DOI: 10.1016/j.cognition.2012.09.007. View