N200 and Late Components Reveal Text-emoji Congruency Effect in Affective Theory of Mind

Emojis are thought to be important for online communication, affecting not only our emotional state, but also our ability to infer the sender's emotional state, i.e., the affective theory of mind (aToM). However, it is unclear the role of text-emoji valence congruency in aToM judgements. Participants were presented with positive, negative, or neutral instant messages followed by positive or negative emoji and were required to infer the sender's emotional state as making valence and arousal ratings. Participants rated that senders felt more positive when they displayed positive emojis as opposed to negative emojis, and the senders were more aroused when valence between emoji and sentence was congruent. Event-related potentials were time-locked to emojis and analyzed by robust mass-univariate statistics, finding larger N200 for positive emojis relative to negative emojis in the negative sentence but not in the positive and neutral sentences, possibly reflecting conflict detection. Furthermore, the N400 effect was found between emotional and neutral sentences, but not between congruent and incongruent conditions, which may reflect a rapid bypassing of deeper semantic analysis. Critically, larger later positivity and negativity (600-900 ms) were found for incongruent combinations relative to congruent combinations in emotional sentences, which was more pronounced for positive sentence, reflecting the cognitive efforts needed for reevaluating the emotional meaning of emotional state attribution under incongruent combinations. These results suggest that emoji valence exerts different effects on positive and negative aToM judgments, and affective processing of sentence-emoji combinations precedes semantic processing, highlighting the importance of emojis in aToM.