Sounds Are Perceived as Louder When Accompanied by Visual Movement

In: Multisensory Research
View More View Less
  • 1 University of California, Riverside, CA, USA
  • | 2 Department of General Psychology, University of Padova, Italy
  • | 3 School of Psychology, University of Sussex, Falmer, Brighton, BN1 9QH, UK
  • | 4 Sackler Centre for Consciousness Science, University of Sussex, Brighton, UK

Purchase instant access (PDF download and unlimited online access):

€29.95$34.95

In this study, we present three experiments investigating the influence of visual movement on auditory judgements. In Experiments 1 and 2, two bursts of noise were presented and participants were required to judge which was louder using a forced-choice task. One of the two bursts was accompanied by a moving disc. The other burst either was accompanied by no visual stimulus (Experiment 1) or by a static disc (Experiment 2). When the two sounds were of identical intensity participants judged the sound accompanied by the moving disc as louder. The effect was greater when auditory stimuli were of the same intensity but it was still present for mid-to-high intensities. In a third, control, experiment participants judged the pitch (and not the loudness) of a pair of tones. Here the pattern was different: there was no effect of visual motion for sounds of the same pitch, with a reversed effect for mid-to-high pitch differences (the effect of motion lowered the pitch). This showed no shift of response towards the interval accompanied by the moving disc. In contrast, the effect on pitch was reversed in comparison to what observed for loudness, with mid-to-high frequency sound accompanied by motion rated as lower in pitch respect to the static intervals.

The natural tendency for moving objects to elicit sounds may lead to an automatic perceptual influence of vision over sound particularly when the latter is ambiguous. This is the first account of this novel audio-visual interaction.

  • Alais D., Burr D. (2004). No direction-specific bimodal facilitation for audiovisual motion detection, Cogn. Brain Res. 19, 185194.

  • Baumann O., Greenlee M. W. (2007). Neural correlates of coherent audiovisual motion perception, Cereb. Cortex 17, 14331443.

  • Ben-Artzi E., Marks L. E. (1995). Visual-auditory interaction in speeded classification: role of stimulus difference, Percept. Psychophys. 57, 11511162.

    • Search Google Scholar
    • Export Citation
  • Bolognini N., Frassinetti F., Serino A., Làdavas E. (2005). “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs, Exp. Brain Res. 160, 273282.

    • Search Google Scholar
    • Export Citation
  • Brainard D. H. (1997). The psychophysics toolbox, Spat. Vis. 10, 433436.

  • Bremmer F., Schlack A., Shah N. J., Zafiris O., Kubischik M., Hoffmann K. P., Zilles K., Fink G. R. (2001). Polymodal motion processing in posterior parietal and promotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys, Neuron 29, 287296.

    • Search Google Scholar
    • Export Citation
  • Frassinetti F., Bolognini N., Làdavas E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction, Exp. Brain Res. 147, 332343.

    • Search Google Scholar
    • Export Citation
  • Freeman E., Driver J. (2008). Direction of visual apparent motion driven solely by timing of a static sound, Curr. Biol. 18(16), 12621266.

    • Search Google Scholar
    • Export Citation
  • Grassi M., Casco C. (2012). Revealing the origin of the audiovisual bounce-inducing effect, See. Perceiv. 25, 223233.

  • Grassi M., Pavan A. (2012). The subjective duration of audiovisual looming and receding stimuli, Attent. Percept. Psychophys. 74, 13211333.

    • Search Google Scholar
    • Export Citation
  • Grassi M., Pastore M., Lemaitre G. (2013). Looking at the world with your ears: how do we get the size of an object from its sound? Acta Psychol. 143, 96104.

    • Search Google Scholar
    • Export Citation
  • Green D. M., Swets J. A. (1966). Signal Detection Theory and Psychophysics. Wiley, New York, NY, USA.

  • Jaekl P. M., Soto-Faraco S. (2010). Audiovisual contrast enhancement is articulated primarily via the M-pathway, Brain Res. 1366, 8592.

    • Search Google Scholar
    • Export Citation
  • Lewkowicz D. J., Minar N. J. (2014). Infants are not sensitive to synesthetic cross-modality correspondences. A comment to Walker et al. (2010), Psychol. Sci. 25, 832834.

    • Search Google Scholar
    • Export Citation
  • Lippert M., Logothetis N. K., Kayser C. (2007). Improvement of visual contrast detection by a simultaneous sound, Brain Res. 1173, 102109.

    • Search Google Scholar
    • Export Citation
  • Maniglia M., Grassi M., Casco C., Campana G. (2012). The origin of the audiovisual bounce inducing effect: a TMS study, Neuropsychologia 50, 14781482.

    • Search Google Scholar
    • Export Citation
  • McGurk H., MacDonald J. (1976). Hearing lips and seeing voices, Nature 264(5588), 746748.

  • Monaghan P., Mattock K., Walker P. (2012). The role of sound symbolism in language learning, J. Exp. Psycho. Learn. Mem. Cogn. 38, 11521164.

    • Search Google Scholar
    • Export Citation
  • Nava E., Grassi M., Turati C. (2016). Audio-visual, visuo-tactile and audio-tactile correspondences in preschoolers, Multisens. Res. 29, 93111.

    • Search Google Scholar
    • Export Citation
  • Neuhoff J. G. (2001). An adaptive bias in the perception of looming auditory motion, Ecol. Psychol. 13, 87110.

  • Odgaard E. C., Arieh Y., Marks L. E. (2003). Cross-modal enhancement of perceived brightness; sensory interaction versus response bias, Percept. Psychophys. 65, 123132.

    • Search Google Scholar
    • Export Citation
  • Odgaard E. C., Arieh Y., Marks L. E. (2004). Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation, Cogn. Affect. Behav. Neurosci. 4, 127132.

    • Search Google Scholar
    • Export Citation
  • Parise C. V., Spence C. (2009). ‘When birds of a feather flock together’: synesthetic correspondences modulate audiovisual integration in non-synesthetes, PLoS One 4(5). DOI:10.1371/journal.pone.0005664.

    • Search Google Scholar
    • Export Citation
  • Pelli D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies, Spat. Vis. 10, 437442.

  • Phillips-Silver J., Trainor L. J. (2007). Hearing what the body feels: auditory encoding of rhythmic movement, Cognition 105, 533546. DOI:10.1016/j.cognition.2006.11.006.

    • Search Google Scholar
    • Export Citation
  • Sadaghiani S., Maier J. X., Noppeney U. (2009). Natural, metaphoric and linguistic auditory direction signals have distinct influences on visual motion processing, J. Neurosci. 29, 64906499.

    • Search Google Scholar
    • Export Citation
  • Saenz M., Koch C. (2008). The sound of change: visually-induced auditory synesthesia, Curr. Biol. 18, R650R651.

  • Seitz A. R., Kim R., Shams L. (2006). Sound facilitates visual learning, Curr. Biol. 16, 14221427. DOI:10.1016/j.cub.2006.05.048.

  • Sekuler R., Sekuler A. B., Lau R. (1997). Sound alters visual motion perception, Nature 385(6614), 308.

  • Shams L., Kamitani Y., Shimojo S. (2000). Illusions: what you see is what you hear, Nature 408(6814), 788.

  • Soranzo A., Grassi M. (2014). PSYCHOACOUSTICS: a comprehensive MATLAB toolbox for auditory testing, Front. Psychol. 5, 712. DOI:10.3389/fpsyg.2014.00712.

    • Search Google Scholar
    • Export Citation
  • Soto-Faraco S., Kingstone A., Spence C. (2003). Multisensory contributions to the perception of motion, Neuropsychologia 41, 18471862.

  • Stein B. E., London N., Wilkinson L. K., Price D. D. (1996). Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis, J. Cogn. Neurosci. 8, 497506.

    • Search Google Scholar
    • Export Citation
  • Tanner T. A. J., Haller R. W., Atkinson R. C. (1967). Signal recognition as influenced by presentation schedules, Percept. Psychophys. 2, 349358.

    • Search Google Scholar
    • Export Citation
  • Teramoto W., Hidaka S., Sugita Y., Sakamoto S., Gyoba J., Iwaya Y., Suzuki Y. (2012). Sounds can alter the perceived direction of a moving visual object, J. Vis. 12(3). DOI:10.1167/12.3.11.

    • Search Google Scholar
    • Export Citation
  • Walker P., Bremner J. G., Mason U., Spring J., Mattock K., Slater A., Johnson S. P. (2010a). Preverbal infants’ sensitivity to synaesthetic cross-modality correspondences, Psychol. Sci. 21, 2125.

    • Search Google Scholar
    • Export Citation
  • Walker P., Francis B. J., Walker L. (2010b). The brightness-weight illusion darker objects look heavier but feel lighter, Exp. Psychol. 57, 462469.

    • Search Google Scholar
    • Export Citation
  • Yarrow K., Haggard P., Rothwell J. C. (2008). Vibrotactile-auditory interactions are post-perceptual, Perception 37, 11141130. DOI:10.1068/p5824.

    • Search Google Scholar
    • Export Citation

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 307 55 5
Full Text Views 222 8 2
PDF Views & Downloads 22 11 2