Spatial orienting in complex audiovisual environments
Article first published online: 24 APR 2013
Copyright © 2013 Wiley Periodicals, Inc.
Human Brain Mapping
Volume 35, Issue 4, pages 1597–1614, April 2014
How to Cite
Nardo, D., Santangelo, V. and Macaluso, E. (2014), Spatial orienting in complex audiovisual environments. Hum. Brain Mapp., 35: 1597–1614. doi: 10.1002/hbm.22276
- Issue published online: 20 MAR 2014
- Article first published online: 24 APR 2013
- Manuscript Accepted: 7 FEB 2013
- Manuscript Revised: 22 JAN 2013
- Manuscript Received: 9 NOV 2012
- The European Research Council under the European Union's Seventh Framework Program (FP7/2007-2013)/ERC grant agreement 242809
- The Italian Ministry of Health.
- eye movements;
- posterior parietal cortex
Previous studies on crossmodal spatial orienting typically used simple and stereotyped stimuli in the absence of any meaningful context. This study combined computational models, behavioural measures and functional magnetic resonance imaging to investigate audiovisual spatial interactions in naturalistic settings. We created short videos portraying everyday life situations that included a lateralised visual event and a co-occurring sound, either on the same or on the opposite side of space. Subjects viewed the videos with or without eye-movements allowed (overt or covert orienting). For each video, visual and auditory saliency maps were used to index the strength of stimulus-driven signals, and eye-movements were used as a measure of the efficacy of the audiovisual events for spatial orienting. Results showed that visual salience modulated activity in higher-order visual areas, whereas auditory salience modulated activity in the superior temporal cortex. Auditory salience modulated activity also in the posterior parietal cortex, but only when audiovisual stimuli occurred on the same side of space (multisensory spatial congruence). Orienting efficacy affected activity in the visual cortex, within the same regions modulated by visual salience. These patterns of activation were comparable in overt and covert orienting conditions. Our results demonstrate that, during viewing of complex multisensory stimuli, activity in sensory areas reflects both stimulus-driven signals and their efficacy for spatial orienting; and that the posterior parietal cortex combines spatial information about the visual and the auditory modality. Hum Brain Mapp 35:1597–1614, 2014. © 2013 Wiley Periodicals, Inc.