McGurk Effect in Non-English Listeners: Few Visual Effects for Japanese Subjects Hearing Japanese Syllables of High Auditory Intelligibility

The McGurk effect is a phenomenon that demonstrates a perceptual fusion between auditory and visual (lip-read) information in speech perception under the condition of audio-visual discrepancy, created by dubbed video tapes. This paper investigated whether or not the McGurk effect could be extended to Japanese subjects listening to Japanese syllables of different auditory intelligibility. The audio and video signal of a female talker's speech for ten Japanese syllables (/ba/, /pa/, /ma/, /wa/, /da/, /ta/, /na/, /ra/, /ga/, /ka/) was combined on videotapes, giving 100 audio-visual stimuli. These stimuli were presented to ten Japanese subjects who were required to identify the stimuli as heard speech in both noise-added and noise-free conditions. For both conditions, the intelligibility of the auditory stimuli was measured, by presenting the audio-alone stimuli. The results showed that, in the noise-free condition, the McGurk effect was small and almost limited to auditory stimuli of which the intelligibility was less than 100%. In the noise-added condition, the McGurk effect was very strong and widespread. These results indicate that the "Japanese McGurk effect" is less easily induced than the English one, and that it depends on the auditory intelligibility of the speech signal.