The Grammar of Emoji? Constraints on Communicative Pictorial Sequencing

Overview

Journal Cogn Res Princ Implic

Specialty Psychology

Date 2019 Sep 1

PMID 31471857

Citations 5

Authors

Neil Cohn

Jan Engelen

Joost Schilperoord

Affiliations

Soon will be listed here.

Abstract

Emoji have become a prominent part of interactive digital communication. Here, we ask the questions: does a grammatical system govern the way people use emoji; and how do emoji interact with the grammar of written text? We conducted two experiments that asked participants to have a digital conversation with each other using only emoji (Experiment 1) or to substitute at least one emoji for a word in the sentences (Experiment 2). First, we found that the emoji-only utterances of participants remained at simplistic levels of patterning, primarily appearing as one-unit utterances (as formulaic expressions or responsive emotions) or as linear sequencing (for example, repeating the same emoji or providing an unordered list of semantically related emoji). Emoji playing grammatical roles (i.e., 'parts-of-speech') were minimal, and showed little consistency in 'word order'. Second, emoji were substituted more for nouns and adjectives than verbs, while also typically conveying nonredundant information to the sentences. These findings suggest that, while emoji may follow tendencies in their interactions with grammatical structure in multimodal text-emoji productions, they lack grammatical structure on their own.

Citing Articles

Help-Seeking Situations Related to Visual Interactions on Mobile Platforms and Recommended Designs for Blind and Visually Impaired Users.

Xie I, Choi W, Wang S, Lee H, Hong B, Wang N J Imaging. 2024; 10(8).

PMID: 39194994 PMC: 11355365. DOI: 10.3390/jimaging10080205.

Semantic and syntactic processing of emojis in sentential intermediate positions.

Tang M, Chen B, Zhao X, Zhao L Cogn Neurodyn. 2024; 18(4):1743-1752.

PMID: 39104667 PMC: 11297853. DOI: 10.1007/s11571-023-10037-1.

Grammatical structures of emoji in Japanese-language text conversations.

Sekine K, Ikuta M Cogn Res Princ Implic. 2024; 9(1):49.

PMID: 39073677 PMC: 11286883. DOI: 10.1186/s41235-024-00571-9.

Experimental evidence for a semantic typology of emoji: Inferences of co-, pro-, and post-text emoji.

Tieu L, Qiu J, Puvipalan V, Pasternak R Q J Exp Psychol (Hove). 2024; :17470218241255786.

PMID: 38752511 PMC: 11905322. DOI: 10.1177/17470218241255786.

Digital Visual Communication for Public Health: Design Proposal for a Vaccinated Emoji.

Boender T, Louis-Ferdinand N, Duschek G J Med Internet Res. 2022; 24(4):e35786.

PMID: 35389363 PMC: 8993141. DOI: 10.2196/35786.

References

Federmeier K, Kutas M . Meaning and modality: influences of context, semantic memory organization, and perceptual predictability on picture processing. J Exp Psychol Learn Mem Cogn. 2001; 27(1):202-24. View

Gershkoff-Stowe L . Is there a natural order for expressing semantic relations?. Cogn Psychol. 2002; 45(3):375-412. DOI: 10.1016/s0010-0285(02)00502-9. View

Madden C, Therriault D . Verb aspect and perceptual simulations. Q J Exp Psychol (Hove). 2009; 62(7):1294-302. DOI: 10.1080/17470210802696088. View

Langus A, Nespor M . Cognitive systems struggling for word order. Cogn Psychol. 2010; 60(4):291-318. DOI: 10.1016/j.cogpsych.2010.01.004. View

Cohn N, Paczynski M, Jackendoff R, Holcomb P, Kuperberg G . (Pea)nuts and bolts of visual narrative: structure and meaning in sequential image comprehension. Cogn Psychol. 2012; 65(1):1-38. PMC: 3331971. DOI: 10.1016/j.cogpsych.2012.01.003. View

Hafri A, Papafragou A, Trueswell J . Getting the gist of events: recognition of two-participant actions from brief displays. J Exp Psychol Gen. 2012; 142(3):880-905. PMC: 3657301. DOI: 10.1037/a0030045. View

Cohn N . Visual narrative structure. Cogn Sci. 2012; 37(3):413-52. DOI: 10.1111/cogs.12016. View

Cohn N, Paczynski M . Prediction, events, and the advantage of agents: the processing of semantic roles in visual narrative. Cogn Psychol. 2013; 67(3):73-97. PMC: 3895484. DOI: 10.1016/j.cogpsych.2013.07.002. View

Nigam A, Hoffman J, Simons R . N400 to semantically anomalous pictures and words. J Cogn Neurosci. 2013; 4(1):15-22. DOI: 10.1162/jocn.1992.4.1.15. View

10.

Ganis G, Kutas M, Sereno M . The search for "common sense": an electrophysiological study of the comprehension of words and pictures in reading. J Cogn Neurosci. 2013; 8(2):89-106. DOI: 10.1162/jocn.1996.8.2.89. View

11.

Cohn N . You're a good structure, Charlie Brown: the distribution of narrative categories in comic strips. Cogn Sci. 2014; 38(7):1317-59. PMC: 4159446. DOI: 10.1111/cogs.12116. View

12.

Cohn N, Jackendoff R, Holcomb P, Kuperberg G . The grammar of visual narrative: Neural evidence for constituent structure in sequential image comprehension. Neuropsychologia. 2014; 64:63-70. PMC: 4364919. DOI: 10.1016/j.neuropsychologia.2014.09.018. View

13.

Marno H, Langus A, Omidbeigi M, Asaadi S, Seyed-Allaei S, Nespor M . A new perspective on word order preferences: the availability of a lexicon triggers the use of SVO word order. Front Psychol. 2015; 6:1183. PMC: 4534792. DOI: 10.3389/fpsyg.2015.01183. View

14.

Dehaene S, Meyniel F, Wacongne C, Wang L, Pallier C . The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees. Neuron. 2015; 88(1):2-19. DOI: 10.1016/j.neuron.2015.09.019. View

15.

Cohn N . A multimodal parallel architecture: A cognitive framework for multimodal interactions. Cognition. 2015; 146:304-23. DOI: 10.1016/j.cognition.2015.10.007. View

16.

Strickland B . Language Reflects "Core" Cognition: A New Theory About the Origin of Cross-Linguistic Regularities. Cogn Sci. 2016; 41(1):70-101. DOI: 10.1111/cogs.12332. View

17.

Jackendoff R, Wittenberg E . Linear grammar as a possible stepping-stone in the evolution of language. Psychon Bull Rev. 2016; 24(1):219-224. DOI: 10.3758/s13423-016-1073-y. View

18.

Kaye L, Malone S, Wall H . Emojis: Insights, Affordances, and Possibilities for Psychological Science. Trends Cogn Sci. 2017; 21(2):66-68. DOI: 10.1016/j.tics.2016.10.007. View

19.

Manfredi M, Cohn N, Kutas M . When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative. Brain Lang. 2017; 169:28-38. PMC: 5465314. DOI: 10.1016/j.bandl.2017.02.001. View

20.

Cohn N, Paczynski M, Kutas M . Not so secret agents: Event-related potentials to semantic roles in visual event comprehension. Brain Cogn. 2017; 119:1-9. PMC: 5759974. DOI: 10.1016/j.bandc.2017.09.001. View