Representations of Modality-specific Affective Processing for Visual and Auditory Stimuli Derived from Functional Magnetic Resonance Imaging Data

There is converging evidence that people rapidly and automatically encode affective dimensions of objects, events, and environments that they encounter in the normal course of their daily routines. An important research question is whether affective representations differ with sensory modality. This research examined the nature of the dependency of affect and sensory modality at a whole-brain level of analysis in an incidental affective processing paradigm. Participants were presented with picture and sound stimuli that differed in positive or negative valence in an event-related functional magnetic resonance imaging experiment. Global statistical tests, applied at a level of the individual, demonstrated significant sensitivity to valence within modality, but not valence across modalities. Modality-general and modality-specific valence hypotheses predict distinctly different multidimensional patterns of the stimulus conditions. Examination of lower dimensional representation of the data demonstrated separable dimensions for valence processing within each modality. These results provide support for modality-specific valence processing in an incidental affective processing paradigm at a whole-brain level of analysis. Future research should further investigate how stimulus-specific emotional decoding may be mediated by the physical properties of the stimuli.