From symbols to sounds: Visual symbolic information activates sound representations

Authors

  • Andreas Widmann,

    1. Institut für Allgemeine Psychologie, University of Leipzig, Leipzig, Germany
    Search for more papers by this author
  • Teija Kujala,

    1. Helsinki Collegium for Advanced Studies, University of Helsinki, Helsinki, Finland
    2. Cognitive Brain Research Unit, Department of Psychology, University of Helsinki, Helsinki, Finland
    3. Helsinki Brain Research Center, Helsinki, Finland
    Search for more papers by this author
  • Mari Tervaniemi,

    1. Institut für Allgemeine Psychologie, University of Leipzig, Leipzig, Germany
    2. Cognitive Brain Research Unit, Department of Psychology, University of Helsinki, Helsinki, Finland
    3. Helsinki Brain Research Center, Helsinki, Finland
    Search for more papers by this author
  • Anu Kujala,

    1. Cognitive Brain Research Unit, Department of Psychology, University of Helsinki, Helsinki, Finland
    Search for more papers by this author
  • Erich Schröger

    1. Institut für Allgemeine Psychologie, University of Leipzig, Leipzig, Germany
    Search for more papers by this author

  • We are grateful to Anett Pfeiffer and Aileen Oeberst for their help in conducting the experiment. Also we are grateful to István Winkler, Martin Eimer, and the reviewers for their comments that helped us improve the article. The research was supported by grants from the Deutscher Akademischer Austauschdienst (DAAD; German Academic Exchange Service) and the European Union (Marie-Curie, QLK6-CT-2000-51227).

Address reprint requests to: Andreas Widmann, University of Leipzig, Institut für Allgemeine Psychologie, Seeburgstr. 14-20, D-04103 Leipzig, Germany. E-mail: widmann@uni-leipzig.de.

Abstract

Expectations on forthcoming sounds can speed up responding to environmental changes and can, thus, be a basis for successful adaptation. The present study investigated event-related brain potential (ERP) effects in situations where particular sounds were predicted on the basis of preceding visual information. Subjects had to map scorelike visual symbols to corresponding sounds. The sounds could be either congruent or occasionally incongruent with the corresponding symbol. When the auditory stimulation was incongruent with the visual information, a brain response was elicited starting as early as about 100 ms from the onset of the auditory stimulus. It had a bilateral frontal distribution and a polarity inversion at the mastoids compatible with the assumption of sources in auditory cortex. These results suggest that the auditory system can establish a representation of an expected stimulus on the basis of visual symbolic information.

Ancillary