Transfer of Audio-Visual Temporal Training to Temporal and Spatial Audio-Visual Tasks

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


Temporal and spatial characteristics of sensory inputs are fundamental to multisensory integration because they provide probabilistic information as to whether or not multiple sensory inputs belong to the same event. The multisensory temporal binding window defines the time range within which two stimuli of different sensory modalities are merged into one percept and has been shown to depend on training. The aim of the present study was to evaluate the role of the training procedure for improving multisensory temporal discrimination and to test for a possible transfer of training to other multisensory tasks. Participants were trained over five sessions in a two-alternative forced-choice simultaneity judgment task. The task difficulty of each trial was either at each participant’s threshold (adaptive group) or randomly chosen (control group). A possible transfer of improved multisensory temporal discrimination on multisensory binding was tested with a redundant signal paradigm in which the temporal alignment of auditory and visual stimuli was systematically varied. Moreover, the size of the spatial audio-visual ventriloquist effect was assessed. Adaptive training resulted in faster improvements compared to the control condition. Transfer effects were found for both tasks: The processing speed of auditory inputs and the size of the ventriloquist effect increased in the adaptive group following the training. We suggest that the relative precision of the temporal and spatial features of a cross-modal stimulus is weighted during multisensory integration. Thus, changes in the precision of temporal processing are expected to enhance the likelihood of multisensory integration for temporally aligned cross-modal stimuli.

Transfer of Audio-Visual Temporal Training to Temporal and Spatial Audio-Visual Tasks

in Multisensory Research



AlaisD.BurrD. (2004). The ventriloquist effect results from near-optimal bimodal integrationCurr. Biol. 14257262.

BologniniN.FrassinettiF.SerinoA.LàdavasE. (2005). “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputsExp. Brain Res. 160273282.

CharbonneauG.VéronneauM.Boudrias-FournierC.LeporeF.CollignonO. (2013). The ventriloquist in periphery: impact of eccentricity-related reliability on audio-visual localizationJ. Vis. 1320. DOI:10.1167/13.12.20.

De NiearM. A.KooB.WallaceM. T. (2016). Multisensory perceptual learning is dependent upon task difficultyExp. Brain Res. 23432693277.

DiederichA.ColoniusH. (1987). Intersensory facilitation in the motor component? Psychol. Res. 492329.

DiederichA.ColoniusH. (2004). Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction timePercept. Psychophys. 6613881404.

DoehrmannO.NaumerM. J. (2008). Semantics and the multisensory brain: how meaning modulates processes of audio-visual integrationBrain Res. 1242136150.

DunningD. L.HolmesJ. (2014). Does working memory training promote the use of strategies on untrained working memory tasks? Mem. Cogn. 42854862.

DunningD. L.HolmesJ.GathercoleS. E. (2013). Does working memory training lead to generalized improvements in children with low working memory ? A randomized controlled trialDev. Sci. 6915925.

ErnstM. O.BanksM. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashionNature 415(6870) 429433.

FrassinettiF.BologniniN.LàdavasE. (2002). Enhancement of visual perception by cross-modal visuo-auditory interactionExp. Brain Res. 147332343.

GingrasG.RowlandB. A.SteinB. E. (2009). The differing impact of multisensory and unisensory integration on behaviorJ. Neurosci. 2948974902.

HabetsB.BrunsP.RöderB. (2017). Experience with cross-modal statistics reduces the sensitivity for audio-visual temporal asynchronySci. Rep. 71486. DOI:10.1038/s41598-017-01252-y.

HairstonW. D.WallaceT.VaughanJ. W.SteinB. E.NorrisJ. L.SchirilloJ. A. (2003). Visual localization ability influences cross-modal biasJ. Cogn. Neurosci. 152029.

HairstonW. D.BurdetteJ. H.FlowersD. L.WoodF. B.WallaceM. T. (2005). Altered temporal profile of visual-auditory multisensory interactions in dyslexiaExp. Brain Res. 166474480.

HershensonM. (1962). Reaction time as a measure of intersensory facilitationJ. Exp. Psychol. 63289293.

HolmesJ.GathercoleS. E.DunningD. L. (2009). Adaptive training leads to sustained enhancement of poor working memory in childrenDev. Sci. 12F9F15.

HughesH. C.Reuter-LorenzP. A.NozawaG.FendrichR. (1994). Visual-auditory interactions in sensorimotor processing: saccades versus manual responsesJ. Exp. Psychol. Hum. Percept. Perform. 20131153.

KayserC.ShamsL. (2015). Multisensory causal inference in the brainPLOS Biol. 13e1002075. DOI:10.1371/journal.pbio.1002075.

LeekM. R. (2001). Adaptive procedures in psychophysical researchPercept. Psychophys. 6312791292.

LovelaceC. T.SteinB. E.WallaceM. T. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detectionCogn. Brain Res. 17447453.

MaierJ. X.DiLucaM.NoppeneyU. (2011). Audiovisual asynchrony detection in human speechJ. Exp. Psychol. Hum. Percept. Perform. 37245256.

McGovernD. P.RoudaiaE.NewellF. N.RoachN. W. (2016a). Perceptual learning shapes multisensory causal inference via two distinct mechanismsSci. Rep. 624673. DOI:10.1038/srep24673.

McGovernD. P.AstleA. T.ClavinS. L.NewellF. N. (2016b). Task-specific transfer of perceptual learning across sensory modalitiesCurr. Biol. 26R20R21.

Metzler-BaddeleyC.BaddeleyR. J. (2009). Does adaptive training work? Appl. Cogn. Psychol. 266254266.

MillerJ. (1982). Divided attention: evidence for coactivation with redundant signalsCogn. Psychol. 14247279.

Morein-ZamirS.Soto-FaracoS.KingstoneA. (2003). Auditory capture of vision: examining temporal ventriloquismCogn. Brain Res. 17154163.

NavarraJ.Hartcher-O’BrienJ.PiazzaE.SpenceC. (2009). Adaptation to audiovisual asynchrony modulates the speeded detection of soundProc. Natl Acad. Sci. U.S.A. 10691699173.

NelsonW.HettingerL. J.CunninghamJ. A.BrickmanB. J.HaasM.McKinleyR. (1998). Effects of localized auditory information on visual target detection performance using a helmet-mounted displayHum. Fact. 40452460.

PlatF. M.PraamstraP.HorstinkM. W. I. M. (2000). Redundant-signals effects on reaction time, response force and movement-related potentials in Parkinson’s diseaseExp. Brain Res. 130533539.

PowersA.HillockA. R.WallaceT. (2009). Perceptual training narrows the temporal window of multisensory bindingJ. Neurosci. 291226512274.

PowersA.HeveyM.WallaceM. T. (2012). Neural correlates of multisensory perceptual learningJ. Neurosci. 3262636274.

PowersA. R.DunnA. H.WallaceM. T. (2016). Generalization of multisensory perceptual learningSci. Rep. 623374. DOI:10.1038/srep23374.

RadeauM.BertelsonP. (1987). Auditory–visual interaction and the timing of inputsPsychol. Res. 491722.

RoachN. W.HeronJ.WhitakerD.McGrawP. V. (2011). Asynchrony adaptation reveals neural population code for audio-visual timingProc. Biol. Sci. 27813141322.

SeitzA. R.NanezJ. E.HollowayS. R.WatanabeT. (2006). Perceptual learning of motion leads to faster flicker perceptionPloS One 1e28. DOI:10.1371/journal.pone.0000028.

SettiA.StapletonJ.LeahyD.WalshC.KennyR. A.NewellF. N. (2014). Improving the efficiency of multisensory integration in older adults: audio-visual temporal discrimination training reduces susceptibility to the sound-induced flash illusionNeuropsychologia 61259268.

ShamsL.KamitaniY.ShimojoS. (2000). Illusions. What you see is what you hearNature 408(6814) 788.

ShibataK.SasakiY.BangJ. W.WalshE. G.MachizawaM. G.TamakiM.ChangL.-H.WatanabeT. (2017). Overlearning hyperstabilizes a skill by rapidly making neurochemical processing inhibitory-dominantNat. Neurosci. 20470475.

SlutskyD.RecanzoneG. H. (2001). Temporal and spatial dependency of the ventriloquism effectNeuroreport 12710.

StevensonR. A.WallaceM. T. (2013). Multisensory temporal integration: task and stimulus dependenciesExp. Brain Res. 227249261.

StevensonR.WilsonM. M.PowersA. R.WallaceM. T. (2013). The effects of visual training on multisensory temporal processingExp. Brain Res. 225479489.

TreutweinB. (1995). Adaptive psychophysical proceduresVis. Res. 3525032522.

VatakisA.SpenceC. (2006). Audiovisual synchrony perception for music, speech and object actionsBrain Res. 1111134142.

WallaceM. T.RobersonG. E.HairstonW. D.SteinB. E.VaughanJ. W.SchirilloJ. A. (2004). Unifying multisensory signals across time and spaceExp. Brain Res. 158252258.

WatsonA. B.PelliD. G. (1983). QUEST: a Bayesian adaptive psychometric methodPercept. Psychophys. 33113120.

WelchR. B.WarrenD. H. (1980). Immediate perceptual response to intersensory discrepancyPsychol. Bull. 88638667.

ZhouY.HuangC.XuP.TaoL.QiuZ.LiX.LuZ.-L. (2006). Perceptual learning improves contrast sensitivity and visual acuity in adults with anisometropic amblyopiaVis. Res. 46739750.


  • View in gallery

    Schematic drawing of the training device used for the simultaneity judgment training. The device was placed 100 cm in front of the participant and had a visual angle of 41.6° (diameter: 76 cm). Sixteen loudspeakers with red LEDs (diameter: 5 mm; visual angle: 0.29°) attached were arranged at equal distances (22.5°). Tetris was played on the 17″ computer screen in the center of the device. The eyes of the participant were at the same height as the center of the computer screen.

  • View in gallery

    Experimental setup of the auditory localization task. Thirteen loudspeakers and red LEDs (diameter: 5 mm) were positioned at a distance of 100 cm in steps of 10° (total radius: 120°). Target positions were 0°, ±30° and ±60°, whereas the irrelevant visual stimuli were presented shifted by 0°, ±10°, ±20° or ±30° with respect to the target position.

  • View in gallery

    Thresholds in the simultaneity judgment training for the control and adaptive training group and each session (1–5). Error bars denote standard error of the mean (p<0.05, ∗∗p<0.01).

  • View in gallery

    Reaction times in the redundant target task for (A) the control group, (B) the adaptive group for each stimulus onset asynchrony (SOA) and (C) the mean reaction times to unimodal auditory and auditory-leading cross-modal stimuli in the pretraining (green) and posttraining (red) session. Negative values indicate auditory-leading cross-modal stimuli and positive values indicate visual-leading stimuli. Error bars denote standard error of the mean (p<0.05).

  • View in gallery

    Ventriloquist effect (VE) in the unimodal conditions of localization task: (A) shows the VE for the unimodal auditory and (B) for the unimodal visual condition in the pretest and the posttest of the control and the adaptive group. (C) Mean VE across both groups and time points.

  • View in gallery

    Ventriloquist effect (VE) in the auditory localization task: (A) and (B) show the VE of the control and the adaptive group, respectively, separated according to the irrelevant stimulus positions ±10°, ±20° and ±30°. (C) VE averaged over irrelevant stimulus positions ±10°, ±20° and ±30° separately for the pretest and posttest and for the control (left panel) and adaptive (right panel) group. Error bars denote the standard error of the mean (p<0.05).

  • View in gallery

    Number of responses per delta (in bins of 1°) when localizing the auditory target in the bimodal auditory localization task. Number of responses is shown at each irrelevant stimulus position of ±10°, ±20° and ±30° as a function of group (control and adaptive). The responses were calculated by averaging across the main positions ±30° and 0°. A positive delta means a shift of the indicated target location into the direction of the irrelevant visual stimulus and a negative delta means a shift into the opposite direction with respect to the irrelevant visual stimulus (regardless of relative left and right irrelevant visual stimulus position). Delta 0° indicates that there was no difference between the target location and the indicated location of the target. Dashed vertical lines show the discrepancy with respect to the loudspeaker at which the irrelevant visual stimulus was presented.

Index Card

Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 18 18 18
Full Text Views 14 14 14
PDF Downloads 3 3 3
EPUB Downloads 0 0 0