Psycholinguistic Mechanisms of Classifier Processing in Sign Language

Overview

Journal J Exp Psychol Learn Mem Cogn

Publisher American Psychological Association

Specialty Psychology

Date 2020 Nov 19

PMID 33211523

Citations 1

Authors

Julia Krebs

Evie Malaia

Ronnie B Wilbur

Dietmar Roehm

Affiliations

Soon will be listed here.

Abstract

Nonsigners viewing sign language are sometimes able to guess the meaning of signs by relying on the overt connection between form and meaning, or iconicity (cf. Ortega, Özyürek, & Peeters, 2020; Strickland et al., 2015). One word class in sign languages that appears to be highly iconic is classifiers: verb-like signs that can refer to location change or handling. Classifier use and meaning are governed by linguistic rules, yet in comparison with lexical verb signs, classifiers are highly variable in their morpho-phonology (variety of potential handshapes and motion direction within the sign). These open-class linguistic items in sign languages prompt a question about the mechanisms of their processing: Are they part of a gestural-semiotic system (processed like the gestures of nonsigners), or are they processed as linguistic verbs? To examine the psychological mechanisms of classifier comprehension, we recorded the electroencephalogram (EEG) activity of signers who watched videos of signed sentences with classifiers. We manipulated the sentence word order of the stimuli (subject-object-verb [SOV] vs. object-subject-verb [OSV]), contrasting the two conditions, which, according to different processing hypotheses, should incur increased processing costs for OSV orders. As previously reported for lexical signs, we observed an N400 effect for OSV compared with SOV, reflecting increased cognitive load for linguistic processing. These findings support the hypothesis that classifiers are a linguistic part of speech in sign language, extending the current understanding of processing mechanisms at the interface of linguistic form and meaning. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Citing Articles

Low-Frequency Entrainment to Visual Motion Underlies Sign Language Comprehension.

Malaia E, Borneman S, Krebs J, Wilbur R IEEE Trans Neural Syst Rehabil Eng. 2021; 29:2456-2463.

PMID: 34762589 PMC: 8720261. DOI: 10.1109/TNSRE.2021.3127724.

References

Krebs J, Wilbur R, Alday P, Roehm D . The Impact of Transitional Movements and Non-Manual Markings on the Disambiguation of Locally Ambiguous Argument Structures in Austrian Sign Language (ÖGS). Lang Speech. 2018; 62(4):652-680. DOI: 10.1177/0023830918801399. View

Emmorey K, Grabowski T, McCullough S, Ponto L, Hichwa R, Damasio H . The neural correlates of spatial language in English and American Sign Language: a PET study with hearing bilinguals. Neuroimage. 2005; 24(3):832-40. DOI: 10.1016/j.neuroimage.2004.10.008. View

Strickland B, Geraci C, Chemla E, Schlenker P, Kelepir M, Pfau R . Event representations constrain the structure of language: Sign language as a window into universally accessible linguistic biases. Proc Natl Acad Sci U S A. 2015; 112(19):5968-73. PMC: 4434776. DOI: 10.1073/pnas.1423080112. View

Roehm D, Bornkessel-Schlesewsky I, Rosler F, Schlesewsky M . To predict or not to predict: influences of task and strategy on the processing of semantic relations. J Cogn Neurosci. 2007; 19(8):1259-74. DOI: 10.1162/jocn.2007.19.8.1259. View

Hickok G, Wilson M, Clark K, Klima E, Kritchevsky M, Bellugi U . Discourse deficits following right hemisphere damage in deaf signers. Brain Lang. 1999; 66(2):233-48. DOI: 10.1006/brln.1998.1995. View

Malaia E, Borneman J, Wilbur R . Information Transfer Capacity of Articulators in American Sign Language. Lang Speech. 2017; 61(1):97-112. DOI: 10.1177/0023830917708461. View

Friedman D, Cycowicz Y, Gaeta H . The novelty P3: an event-related brain potential (ERP) sign of the brain's evaluation of novelty. Neurosci Biobehav Rev. 2001; 25(4):355-73. DOI: 10.1016/s0149-7634(01)00019-7. View

Malaia E, Wilbur R, Weber-Fox C . ERP evidence for telicity effects on syntactic processing in garden-path sentences. Brain Lang. 2008; 108(3):145-58. PMC: 2720995. DOI: 10.1016/j.bandl.2008.09.003. View

Alday P, Schlesewsky M, Bornkessel-Schlesewsky I . Towards a computational model of actor-based language comprehension. Neuroinformatics. 2013; 12(1):143-79. DOI: 10.1007/s12021-013-9198-x. View

10.

Emmorey K, McCullough S, Mehta S, Ponto L, Grabowski T . The biology of linguistic expression impacts neural correlates for spatial language. J Cogn Neurosci. 2012; 25(4):517-33. PMC: 3715382. DOI: 10.1162/jocn_a_00339. View

11.

Osterhout L, Holcomb P, Swinney D . Brain potentials elicited by garden-path sentences: evidence of the application of verb information during parsing. J Exp Psychol Learn Mem Cogn. 1994; 20(4):786-803. DOI: 10.1037//0278-7393.20.4.786. View

12.

Emmorey K, Damasio H, McCullough S, Grabowski T, Ponto L, Hichwa R . Neural systems underlying spatial language in American Sign Language. Neuroimage. 2002; 17(2):812-24. View

13.

Hall M, Mayberry R, Ferreira V . Cognitive constraints on constituent order: evidence from elicited pantomime. Cognition. 2013; 129(1):1-17. PMC: 4224279. DOI: 10.1016/j.cognition.2013.05.004. View

14.

Kallioinen P, Olofsson J, Nakeva von Mentzer C, Lindgren M, Ors M, Sahlen B . Semantic Processing in Deaf and Hard-of-Hearing Children: Large N400 Mismatch Effects in Brain Responses, Despite Poor Semantic Ability. Front Psychol. 2016; 7:1146. PMC: 4978721. DOI: 10.3389/fpsyg.2016.01146. View

15.

Picton T . The P300 wave of the human event-related potential. J Clin Neurophysiol. 1992; 9(4):456-79. DOI: 10.1097/00004691-199210000-00002. View

16.

Van Petten C, Luka B . Prediction during language comprehension: benefits, costs, and ERP components. Int J Psychophysiol. 2011; 83(2):176-90. DOI: 10.1016/j.ijpsycho.2011.09.015. View

17.

Atkinson J, Marshall J, Woll B, Thacker A . Testing comprehension abilities in users of British Sign Language following CVA. Brain Lang. 2005; 94(2):233-48. DOI: 10.1016/j.bandl.2004.12.008. View

18.

Bornkessel I, Schlesewsky M . The extended argument dependency model: a neurocognitive approach to sentence comprehension across languages. Psychol Rev. 2006; 113(4):787-821. DOI: 10.1037/0033-295X.113.4.787. View

19.

Atkinson J, Campbell R, Marshall J, Thacker A, Woll B . Understanding 'not': neuropsychological dissociations between hand and head markers of negation in BSL. Neuropsychologia. 2003; 42(2):214-29. DOI: 10.1016/s0028-3932(03)00186-6. View

20.

Philipp M, Bornkessel-Schlesewsky I, Bisang W, Schlesewsky M . The role of animacy in the real time comprehension of Mandarin Chinese: Evidence from auditory event-related brain potentials. Brain Lang. 2007; 105(2):112-33. DOI: 10.1016/j.bandl.2007.09.005. View