When Knowing Can Replace Seeing in Audiovisual Integration of Actions

The ability to predict the effects of actions is necessary to behave properly in our physical and social world. Here, we describe how the ability to predict the consequence of complex gestures can change the way we integrate sight and sound when relevant visual information is missing. Six drummers and six novices were asked to judge audiovisual synchrony for drumming point-light displays where the visual information was manipulated to eliminate or include the drumstick-drumhead impact point. In the condition with only the arm information novices were unable to detect asynchrony whereas drummers were able to. Additionally, in the conditions that included the impact point drummers perceived the best alignment when the sight preceded the sound, while in the arm only condition they perceived the best alignment when the sound occurred together with or preceded the sight, as it would be expected if they were predicting the beat occurrence. Taken together these findings suggest that humans can acquire, through practice, internal models of action which can be used to replace missing information when integrating multisensory signals from the environment.