Sounds Are Perceived as Louder When Accompanied by Visual Movement
In: Multisensory ResearchSearch for other papers by Marcello Maniglia in
Current site
Google Scholar
Search for other papers by Massimo Grassi in
Current site
Google Scholar
4Sackler Centre for Consciousness Science, University of Sussex, Brighton, UK
Search for other papers by Jamie Ward in
Current site
Google Scholar
In this study, we present three experiments investigating the influence of visual movement on auditory judgements. In Experiments 1 and 2, two bursts of noise were presented and participants were required to judge which was louder using a forced-choice task. One of the two bursts was accompanied by a moving disc. The other burst either was accompanied by no visual stimulus (Experiment 1) or by a static disc (Experiment 2). When the two sounds were of identical intensity participants judged the sound accompanied by the moving disc as louder. The effect was greater when auditory stimuli were of the same intensity but it was still present for mid-to-high intensities. In a third, control, experiment participants judged the pitch (and not the loudness) of a pair of tones. Here the pattern was different: there was no effect of visual motion for sounds of the same pitch, with a reversed effect for mid-to-high pitch differences (the effect of motion lowered the pitch). This showed no shift of response towards the interval accompanied by the moving disc. In contrast, the effect on pitch was reversed in comparison to what observed for loudness, with mid-to-high frequency sound accompanied by motion rated as lower in pitch respect to the static intervals.
The natural tendency for moving objects to elicit sounds may lead to an automatic perceptual influence of vision over sound particularly when the latter is ambiguous. This is the first account of this novel audio-visual interaction.
Purchase
Buy instant access (PDF download and unlimited online access):
Institutional Login
Log in with Open Athens, Shibboleth, or your institutional credentials
Personal login
Log in with your brill.com account
-
Alais D., Burr D. (2004). No direction-specific bimodal facilitation for audiovisual motion detection, Cogn. Brain Res. 19, 185–194.
-
Baumann O., Greenlee M. W. (2007). Neural correlates of coherent audiovisual motion perception, Cereb. Cortex 17, 1433–1443.
-
Ben-Artzi E., Marks L. E. (1995). Visual-auditory interaction in speeded classification: role of stimulus difference, Percept. Psychophys. 57, 1151–1162.
-
Bolognini N., Frassinetti F., Serino A., Làdavas E. (2005). “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs, Exp. Brain Res. 160, 273–282.
-
Brainard D. H. (1997). The psychophysics toolbox, Spat. Vis. 10, 433–436.
-
Bremmer F., Schlack A., Shah N. J., Zafiris O., Kubischik M., Hoffmann K. P., Zilles K., Fink G. R. (2001). Polymodal motion processing in posterior parietal and promotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys, Neuron 29, 287–296.
-
Frassinetti F., Bolognini N., Làdavas E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction, Exp. Brain Res. 147, 332–343.
-
Freeman E., Driver J. (2008). Direction of visual apparent motion driven solely by timing of a static sound, Curr. Biol. 18(16), 1262–1266.
-
Grassi M., Casco C. (2012). Revealing the origin of the audiovisual bounce-inducing effect, See. Perceiv. 25, 223–233.
-
Grassi M., Pavan A. (2012). The subjective duration of audiovisual looming and receding stimuli, Attent. Percept. Psychophys. 74, 1321–1333.
-
Grassi M., Pastore M., Lemaitre G. (2013). Looking at the world with your ears: how do we get the size of an object from its sound? Acta Psychol. 143, 96–104.
-
Green D. M., Swets J. A. (1966). Signal Detection Theory and Psychophysics. Wiley, New York, NY, USA.
-
Jaekl P. M., Soto-Faraco S. (2010). Audiovisual contrast enhancement is articulated primarily via the M-pathway, Brain Res. 1366, 85–92.
-
Lewkowicz D. J., Minar N. J. (2014). Infants are not sensitive to synesthetic cross-modality correspondences. A comment to Walker et al. (2010), Psychol. Sci. 25, 832–834.
-
Lippert M., Logothetis N. K., Kayser C. (2007). Improvement of visual contrast detection by a simultaneous sound, Brain Res. 1173, 102–109.
-
Maniglia M., Grassi M., Casco C., Campana G. (2012). The origin of the audiovisual bounce inducing effect: a TMS study, Neuropsychologia 50, 1478–1482.
-
McGurk H., MacDonald J. (1976). Hearing lips and seeing voices, Nature 264(5588), 746–748.
-
Monaghan P., Mattock K., Walker P. (2012). The role of sound symbolism in language learning, J. Exp. Psycho. Learn. Mem. Cogn. 38, 1152–1164.
-
Nava E., Grassi M., Turati C. (2016). Audio-visual, visuo-tactile and audio-tactile correspondences in preschoolers, Multisens. Res. 29, 93–111.
-
Neuhoff J. G. (2001). An adaptive bias in the perception of looming auditory motion, Ecol. Psychol. 13, 87–110.
-
Odgaard E. C., Arieh Y., Marks L. E. (2003). Cross-modal enhancement of perceived brightness; sensory interaction versus response bias, Percept. Psychophys. 65, 123–132.
-
Odgaard E. C., Arieh Y., Marks L. E. (2004). Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation, Cogn. Affect. Behav. Neurosci. 4, 127–132.
-
Parise C. V., Spence C. (2009). ‘When birds of a feather flock together’: synesthetic correspondences modulate audiovisual integration in non-synesthetes, PLoS One 4(5). DOI:10.1371/journal.pone.0005664.
-
Pelli D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies, Spat. Vis. 10, 437–442.
-
Phillips-Silver J., Trainor L. J. (2007). Hearing what the body feels: auditory encoding of rhythmic movement, Cognition 105, 533–546. DOI:10.1016/j.cognition.2006.11.006.
-
Sadaghiani S., Maier J. X., Noppeney U. (2009). Natural, metaphoric and linguistic auditory direction signals have distinct influences on visual motion processing, J. Neurosci. 29, 6490–6499.
-
Saenz M., Koch C. (2008). The sound of change: visually-induced auditory synesthesia, Curr. Biol. 18, R650–R651.
-
Seitz A. R., Kim R., Shams L. (2006). Sound facilitates visual learning, Curr. Biol. 16, 1422–1427. DOI:10.1016/j.cub.2006.05.048.
-
Sekuler R., Sekuler A. B., Lau R. (1997). Sound alters visual motion perception, Nature 385(6614), 308.
-
Shams L., Kamitani Y., Shimojo S. (2000). Illusions: what you see is what you hear, Nature 408(6814), 788.
-
Soranzo A., Grassi M. (2014). PSYCHOACOUSTICS: a comprehensive MATLAB toolbox for auditory testing, Front. Psychol. 5, 712. DOI:10.3389/fpsyg.2014.00712.
-
Soto-Faraco S., Kingstone A., Spence C. (2003). Multisensory contributions to the perception of motion, Neuropsychologia 41, 1847–1862.
-
Stein B. E., London N., Wilkinson L. K., Price D. D. (1996). Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis, J. Cogn. Neurosci. 8, 497–506.
-
Tanner T. A. J., Haller R. W., Atkinson R. C. (1967). Signal recognition as influenced by presentation schedules, Percept. Psychophys. 2, 349–358.
-
Teramoto W., Hidaka S., Sugita Y., Sakamoto S., Gyoba J., Iwaya Y., Suzuki Y. (2012). Sounds can alter the perceived direction of a moving visual object, J. Vis. 12(3). DOI:10.1167/12.3.11.
-
Walker P., Bremner J. G., Mason U., Spring J., Mattock K., Slater A., Johnson S. P. (2010a). Preverbal infants’ sensitivity to synaesthetic cross-modality correspondences, Psychol. Sci. 21, 21–25.
-
Walker P., Francis B. J., Walker L. (2010b). The brightness-weight illusion darker objects look heavier but feel lighter, Exp. Psychol. 57, 462–469.
-
Yarrow K., Haggard P., Rothwell J. C. (2008). Vibrotactile-auditory interactions are post-perceptual, Perception 37, 1114–1130. DOI:10.1068/p5824.
| All Time | Past 365 days | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 706 | 92 | 9 |
| Full Text Views | 247 | 5 | 2 |
| PDF Views & Downloads | 67 | 10 | 4 |
Sounds Are Perceived as Louder When Accompanied by Visual Movement
In: Multisensory ResearchSearch for other papers by Marcello Maniglia in
Current site
Google Scholar
PubMed
Search for other papers by Massimo Grassi in
Current site
Google Scholar
PubMed
4Sackler Centre for Consciousness Science, University of Sussex, Brighton, UK
Search for other papers by Jamie Ward in
Current site
Google Scholar
PubMed
In this study, we present three experiments investigating the influence of visual movement on auditory judgements. In Experiments 1 and 2, two bursts of noise were presented and participants were required to judge which was louder using a forced-choice task. One of the two bursts was accompanied by a moving disc. The other burst either was accompanied by no visual stimulus (Experiment 1) or by a static disc (Experiment 2). When the two sounds were of identical intensity participants judged the sound accompanied by the moving disc as louder. The effect was greater when auditory stimuli were of the same intensity but it was still present for mid-to-high intensities. In a third, control, experiment participants judged the pitch (and not the loudness) of a pair of tones. Here the pattern was different: there was no effect of visual motion for sounds of the same pitch, with a reversed effect for mid-to-high pitch differences (the effect of motion lowered the pitch). This showed no shift of response towards the interval accompanied by the moving disc. In contrast, the effect on pitch was reversed in comparison to what observed for loudness, with mid-to-high frequency sound accompanied by motion rated as lower in pitch respect to the static intervals.
The natural tendency for moving objects to elicit sounds may lead to an automatic perceptual influence of vision over sound particularly when the latter is ambiguous. This is the first account of this novel audio-visual interaction.
| All Time | Past 365 days | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 706 | 92 | 9 |
| Full Text Views | 247 | 5 | 2 |
| PDF Views & Downloads | 67 | 10 | 4 |
