Sounds Are Perceived as Louder When Accompanied by Visual Movement

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?



Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.



Help

Have Institutional Access?



Access content through your institution. Any other coaching guidance?



Connect

In this study, we present three experiments investigating the influence of visual movement on auditory judgements. In Experiments 1 and 2, two bursts of noise were presented and participants were required to judge which was louder using a forced-choice task. One of the two bursts was accompanied by a moving disc. The other burst either was accompanied by no visual stimulus (Experiment 1) or by a static disc (Experiment 2). When the two sounds were of identical intensity participants judged the sound accompanied by the moving disc as louder. The effect was greater when auditory stimuli were of the same intensity but it was still present for mid-to-high intensities. In a third, control, experiment participants judged the pitch (and not the loudness) of a pair of tones. Here the pattern was different: there was no effect of visual motion for sounds of the same pitch, with a reversed effect for mid-to-high pitch differences (the effect of motion lowered the pitch). This showed no shift of response towards the interval accompanied by the moving disc. In contrast, the effect on pitch was reversed in comparison to what observed for loudness, with mid-to-high frequency sound accompanied by motion rated as lower in pitch respect to the static intervals.

The natural tendency for moving objects to elicit sounds may lead to an automatic perceptual influence of vision over sound particularly when the latter is ambiguous. This is the first account of this novel audio-visual interaction.

Multisensory Research

A Journal of Scientific Research on All Aspects of Multisensory Processing

Sections

References

AlaisD.BurrD. (2004). No direction-specific bimodal facilitation for audiovisual motion detection, Cogn. Brain Res. 19, 185194.

BaumannO.GreenleeM. W. (2007). Neural correlates of coherent audiovisual motion perception, Cereb. Cortex 17, 14331443.

Ben-ArtziE.MarksL. E. (1995). Visual-auditory interaction in speeded classification: role of stimulus difference, Percept. Psychophys. 57, 11511162.

BologniniN.FrassinettiF.SerinoA.LàdavasE. (2005). “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs, Exp. Brain Res. 160, 273282.

BrainardD. H. (1997). The psychophysics toolbox, Spat. Vis. 10, 433436.

BremmerF.SchlackA.ShahN. J.ZafirisO.KubischikM.HoffmannK. P.ZillesK.FinkG. R. (2001). Polymodal motion processing in posterior parietal and promotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys, Neuron 29, 287296.

FrassinettiF.BologniniN.LàdavasE. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction, Exp. Brain Res. 147, 332343.

FreemanE.DriverJ. (2008). Direction of visual apparent motion driven solely by timing of a static sound, Curr. Biol. 18(16), 12621266.

GrassiM.CascoC. (2012). Revealing the origin of the audiovisual bounce-inducing effect, See. Perceiv. 25, 223233.

GrassiM.PavanA. (2012). The subjective duration of audiovisual looming and receding stimuli, Attent. Percept. Psychophys. 74, 13211333.

GrassiM.PastoreM.LemaitreG. (2013). Looking at the world with your ears: how do we get the size of an object from its sound? Acta Psychol. 143, 96104.

GreenD. M.SwetsJ. A. (1966). Signal Detection Theory and Psychophysics. Wiley, New York, NY, USA.

JaeklP. M.Soto-FaracoS. (2010). Audiovisual contrast enhancement is articulated primarily via the M-pathway, Brain Res. 1366, 8592.

LewkowiczD. J.MinarN. J. (2014). Infants are not sensitive to synesthetic cross-modality correspondences. A comment to Walker et al. (2010), Psychol. Sci. 25, 832834.

LippertM.LogothetisN. K.KayserC. (2007). Improvement of visual contrast detection by a simultaneous sound, Brain Res. 1173, 102109.

ManigliaM.GrassiM.CascoC.CampanaG. (2012). The origin of the audiovisual bounce inducing effect: a TMS study, Neuropsychologia 50, 14781482.

McGurkH.MacDonaldJ. (1976). Hearing lips and seeing voices, Nature 264(5588), 746748.

MonaghanP.MattockK.WalkerP. (2012). The role of sound symbolism in language learning, J. Exp. Psycho. Learn. Mem. Cogn. 38, 11521164.

NavaE.GrassiM.TuratiC. (2016). Audio-visual, visuo-tactile and audio-tactile correspondences in preschoolers, Multisens. Res. 29, 93111.

NeuhoffJ. G. (2001). An adaptive bias in the perception of looming auditory motion, Ecol. Psychol. 13, 87110.

OdgaardE. C.AriehY.MarksL. E. (2003). Cross-modal enhancement of perceived brightness; sensory interaction versus response bias, Percept. Psychophys. 65, 123132.

OdgaardE. C.AriehY.MarksL. E. (2004). Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation, Cogn. Affect. Behav. Neurosci. 4, 127132.

PariseC. V.SpenceC. (2009). ‘When birds of a feather flock together’: synesthetic correspondences modulate audiovisual integration in non-synesthetes, PLoS One 4(5). DOI:10.1371/journal.pone.0005664.

PelliD. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies, Spat. Vis. 10, 437442.

Phillips-SilverJ.TrainorL. J. (2007). Hearing what the body feels: auditory encoding of rhythmic movement, Cognition 105, 533546. DOI:10.1016/j.cognition.2006.11.006.

SadaghianiS.MaierJ. X.NoppeneyU. (2009). Natural, metaphoric and linguistic auditory direction signals have distinct influences on visual motion processing, J. Neurosci. 29, 64906499.

SaenzM.KochC. (2008). The sound of change: visually-induced auditory synesthesia, Curr. Biol. 18, R650R651.

SeitzA. R.KimR.ShamsL. (2006). Sound facilitates visual learning, Curr. Biol. 16, 14221427. DOI:10.1016/j.cub.2006.05.048.

SekulerR.SekulerA. B.LauR. (1997). Sound alters visual motion perception, Nature 385(6614), 308.

ShamsL.KamitaniY.ShimojoS. (2000). Illusions: what you see is what you hear, Nature 408(6814), 788.

SoranzoA.GrassiM. (2014). PSYCHOACOUSTICS: a comprehensive MATLAB toolbox for auditory testing, Front. Psychol. 5, 712. DOI:10.3389/fpsyg.2014.00712.

Soto-FaracoS.KingstoneA.SpenceC. (2003). Multisensory contributions to the perception of motion, Neuropsychologia 41, 18471862.

SteinB. E.LondonN.WilkinsonL. K.PriceD. D. (1996). Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis, J. Cogn. Neurosci. 8, 497506.

TannerT. A. J.HallerR. W.AtkinsonR. C. (1967). Signal recognition as influenced by presentation schedules, Percept. Psychophys. 2, 349358.

TeramotoW.HidakaS.SugitaY.SakamotoS.GyobaJ.IwayaY.SuzukiY. (2012). Sounds can alter the perceived direction of a moving visual object, J. Vis. 12(3). DOI:10.1167/12.3.11.

WalkerP.BremnerJ. G.MasonU.SpringJ.MattockK.SlaterA.JohnsonS. P. (2010a). Preverbal infants’ sensitivity to synaesthetic cross-modality correspondences, Psychol. Sci. 21, 2125.

WalkerP.FrancisB. J.WalkerL. (2010b). The brightness-weight illusion darker objects look heavier but feel lighter, Exp. Psychol. 57, 462469.

YarrowK.HaggardP.RothwellJ. C. (2008). Vibrotactile-auditory interactions are post-perceptual, Perception 37, 11141130. DOI:10.1068/p5824.

Figures

  • Experiment 1. Proportion of louder responses for intervals presented with the moving stimulus. On the left, responses to trials where the intensity of the two sounds intervals was identical, on the right responses to trials where the intensity of two sounds was different. In each boxplot, the central mark is the median. The edges of the box are the 25th and 75th percentiles. The whiskers are the interquartile range (i.e., Q3–Q1) augmented by 50% and symbols are outliers. The black circles represent the mean and the vertical bars are +/− SEM.

    View in gallery
  • Proportion of louder responses as a function of stimulus intensity for Experiment 1. Dotted and dashed lines (maximum likelihood fit) represent fit on aggregated data for No movement and Movement intervals, respectively.

    View in gallery
  • Experiment 2. Proportion of louder responses for intervals presented with the moving stimulus. On the left, responses to trials where the intensity of the two sounds intervals was identical, on the right responses to trials where the intensity of two sounds was different. In each boxplot, the central mark is the median. The edges of the box are the 25th and 75th percentiles. The whiskers are the interquartile range (i.e., Q3–Q1) augmented by 50% and symbols are outliers. The black circles represent the mean and the vertical bars are +/− SEM.

    View in gallery
  • Proportion of louder responses as a function of stimulus intensity for Experiment 2. Dotted and dashed lines (maximum likelihood fit) represent fit on aggregated data for No movement and Movement intervals, respectively.

    View in gallery
  • Experiment 3. Proportion of ‘higher in pitch’ responses for intervals presented with the moving stimulus. On the left, responses to trials where the frequency of the two sounds intervals was identical, on the right responses to trials where the frequency of two sounds was different. In each boxplot, the central mark is the median. The edges of the box are the 25th and 75th percentiles. The whiskers are the interquartile range (i.e., Q3–Q1) augmented by 50% and symbols are outliers. The black circles represent the mean and the vertical bars are +/− SEM.

    View in gallery
  • Proportion of louder responses as a function of stimulus frequency for Experiment 3. Dotted and dashed lines (maximum likelihood fit) represent fit on aggregated data for No movement and Movement intervals, respectively.

    View in gallery

Information

Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 22 22 13
Full Text Views 5 5 5
PDF Downloads 0 0 0
EPUB Downloads 0 0 0