1 The Functional Electrical Neuroimaging Laboratory, Neuropsychology and Neurorehabilitation Service, Department of Clinical Neurosciences, Centre Hospitalier Universitaire Vaudois and University of Lausanne, Switzerland
2 Electroencephalography Brain Mapping Core, Center for Biomedical Imaging (CIBM), Lausanne, Switzerland
3 Radiology Department, Centre Hospitalier Universitaire Vaudois and University of Lausanne, Switzerland
4 Department of Hearing and Speech Sciences, Vanderbilt University, Nashville, TN, USA
This review article summarizes evidence that multisensory experiences at one point in time have long-lasting effects on subsequent unisensory visual and auditory object recognition. The efficacy of single-trial exposure to task-irrelevant multisensory events is its ability to modulate memory performance and brain activity to unisensory components of these events presented later in time. Object recognition (either visual or auditory) is enhanced if the initial multisensory experience had been semantically congruent and can be impaired if this multisensory pairing was either semantically incongruent or entailed meaningless information in the task-irrelevant modality, when compared to objects encountered exclusively in a unisensory context. Processes active during encoding cannot straightforwardly explain these effects; performance on all initial presentations was indistinguishable despite leading to opposing effects with stimulus repetitions. Brain responses to unisensory stimulus repetitions differ during early processing stages (∼100 ms post-stimulus onset) according to whether or not they had been initially paired in a multisensory context. Plus, the network exhibiting differential responses varies according to whether or not memory performance is enhanced or impaired. The collective findings we review indicate that multisensory associations formed via single-trial learning exert influences on later unisensory processing to promote distinct object representations that manifest as differentiable brain networks whose activity is correlated with memory performance. These influences occur incidentally, despite many intervening stimuli, and are distinguishable from the encoding/learning processes during the formation of the multisensory associations. The consequences of multisensory interactions thus persist over time to impact memory retrieval and object discrimination.
BeerA. L.PlankT.MeyerG.GreenleeM. W. (2013).
Combined diffusion-weighted and functional magnetic resonance imaging reveals a temporal-occipital network involved in auditory–visual object processingFront. Integr. Neurosci.75.
MurrayM. M.CappeC.RomeiV.MartuzziR.ThutG. (2012).
Auditory–visual multisensory interactions in humans: a synthesis of findings from behavior, ERPs, fMRI, and TMS in:
The New Handbook of Multisensory ProcessesSteinB. E. (Ed.) pp.
223–238. MIT PressCambridge, MA, USA.
MurrayM. M.SperdinH. F. (2010).
Single-trial multisensory learning and memory retrieval in:
Multisensory Object Perception in the Primate BrainKaiserJ.NaumerM. J. (Eds) pp.
191–208. SpringerHeidelberg, Germany.
ThelenA.MurrayM. M. (2013). Predicting individual differences in the impact of multisensory single-trial exposure upon subsequent object recognition. Neuroscience 2013 Abstracts Program No. 765.15483. Society for Neuroscience San Diego California USA.
ThelenA.TalsmaD.MurrayM. M. (submitted).
The efficacy of single-trial multisensory memories for visual and auditory object recognitionJ. Cognitive Neurosci.