Above the Mean: Examining Variability in Behavioral and Neural Responses to Multisensory Stimuli

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


Even when experimental conditions are kept constant, a robust and consistent finding in both behavioral and neural experiments designed to examine multisensory processing is striking variability. Although this variability has often been considered uninteresting noise (a term that is laden with strong connotations), emerging work suggests that differences in variability may be an important aspect in describing differences in performance between individuals and groups. In the current review, derived from a symposium at the 2015 International Multisensory Research Forum in Pisa, Italy, we focus on several aspects of variability as it relates to multisensory function. This effort seeks to expand our understanding of variability at levels of coding and analysis ranging from the single neuron through large networks and on to behavioral processes, and encompasses a number of the multimodal approaches that are used to evaluate and characterize multisensory processing including single-unit neurophysiology, electroencephalography (EEG), functional magnetic resonance imaging (fMRI), and electrocorticography (ECoG).

Multisensory Research

A Journal of Scientific Research on All Aspects of Multisensory Processing



AdamoN.HuoL.AdelsbergS.PetkovaE.CastellanosF. X.Di MartinoA. (2014). Response time intra-subject variability: commonalities between children with autism spectrum disorders and children with ADHD, Eur. Child Adolesc. Psychiatry 23, 6979.

AlaisD.NewellF. N.MamassianP. (2010). Multisensory processing in review: from physiology to behaviour, Seeing Perceiving 23, 338.

BaumS. H.BeauchampM. S. (2014). Greater BOLD variability in older compared with younger adults during audiovisual speech perception, PLoS ONE 9, e111121. DOI:10.1371/journal.pone.0111121.

BaumS. H.StevensonR. A.WallaceM. T. (2015). Behavioral, perceptual, and neural alterations in sensory and multisensory function in autism spectrum disorder, Progr. Neurobiol. 134, 140160.

BeckJ. M.MaW. J.PitkowX.LathamP. E.PougetA. (2012). Not noisy, just wrong: the role of suboptimal inference in behavioral variability, Neuron 74, 3039.

BielakA. A.HultschD. F.StraussE. H.MacdonaldS. W.HunterM. A. (2010a). Intraindividual variability is related to cognitive change in older adults: evidence for within-person coupling, Psychol. Aging 25, 575586.

BielakA. A.HultschD. F.StraussE. H.MacdonaldS. W.HunterM. A. (2010b). Intraindividual variability in reaction time predicts cognitive outcomes 5 years later, Neuropsychology 24, 731741.

CalvertG. A.SpenceC.SteinB. E. (2004). Handbook of Multisensory Processes. MIT Press, Cambridge, MS, USA.

CappeC.ThutG.RomeiV.MurrayM. M. (2009). Selective integration of auditory–visual looming cues by humans, Neuropsychologia 47, 10451052.

CappeC.ThutG.RomeiV.MurrayM. M. (2010). Auditory–visual multisensory interactions in humans: timing, topography, directionality, and sources, J. Neurosci. 30, 1257212580.

CappeC.ThelenA.RomeiV.ThutG.MurrayM. M. (2012). Looming signals reveal synergistic principles of multisensory integration, J. Neurosci. 32, 11711182.

CerellaJ.HaleS. (1994). The rise and fall in information-processing rates over the life span, Acta Psychol. 86, 109197.

ChristenM.KohnA.OttT.StoopR. (2006). Measuring spike pattern reliability with the Lempel–Ziv-distance, J. Neurosci. Methods 156, 342350.

ChristensenK.OlamiZ.BakP. (1992). Deterministic 1f noise in nonconserative models of self-organized criticality, Phys. Rev. Lett. 68, 24172420.

ColoniusH.DiederichA. (2006). The race model inequality: interpreting a geometric measure of the amount of violation, Psychol. Rev. 113, 148154.

ColoniusH.DiederichA. (2015). A new measure of multisensory integration in a single neuron based on dependent probability summation. Available at arXiv:1507.08505v1.

CoxR. W. (1996). AFNI: software for analysis and visualization of functional magnetic resonance neuroimages, Comput. Biomed. Res. 29, 162173.

Di MartinoA.GhaffariM.CurchackJ.ReissP.HydeC.VannucciM.PetkovaE.KleinD. F.CastellanosF. X. (2008). Decomposing intra-subject variability in children with attention-deficit/hyperactivity disorder, Biol. Psychiatry 64, 607614.

DuttaP.HornP. M. (1981). Low-frequency fluctuations in solids: 1f noise, Rev. Modern Phys. 53, 497499.

EngelA. K.FriesP.SingerW. (2001). Dynamic predictions: oscillations and synchrony in top–down processing, Nat. Rev. Neurosci. 2, 704716.

FriesP. (2015). Rhythms for cognition: communication through coherence, Neuron 88, 220235.

GarrettD. D.KovacevicN.McintoshA. R.GradyC. L. (2011). The importance of being variable, J. Neurosci. 31, 44964503.

GarrettD. D.MacdonaldS. W.CraikF. I. (2012). Intraindividual reaction time variability is malleable: feedback- and education-related reductions in variability with age, Front. Hum. Neurosci. 6, 101. DOI:10.3389/fnhum.2012.00101.

HeddenT.GabrieliJ. D. (2004). Insights into the ageing mind: a view from cognitive neuroscience, Nat. Rev. Neurosci. 5, 8796.

HultschD. F.MacdonaldS. W.DixonR. A. (2002). Variability in reaction time performance of younger and older adults, J. Gerontol. B, Psychol. Sci. Soc. Sci. 57, P101P115.

JenkinsL.MyersonJ.JoerdingJ. A.HaleS. (2000). Converging evidence that visuospatial cognition is more age-sensitive than verbal cognition, Psychol. Aging 15, 157175.

KriegeskorteN.MurM.BandettiniP. (2008). Representational similarity analysis — connecting the branches of systems neuroscience, Front. Syst. Neurosci. 2, 4. DOI:10.3389/neuro.06.004.2008.

KubanekJ.BrunnerP.GunduzA.PoeppelD.SchalkG. (2013). The tracking of speech envelope in the human cortex, PLoS ONE 8, e53398. DOI:10.1371/journal.pone.0053398.

LovdenM.LiS. C.ShingY. L.LindenbergerU. (2007). Within-person trial-to-trial variability precedes and predicts cognitive decline in old and very old age: longitudinal data from the Berlin Aging Study, Neuropsychologia 45, 28272838.

LovdenM.SchmiedekF.KennedyK. M.RodrigueK. M.LindenbergerU.RazN. (2013). Does variability in cognitive performance correlate with frontal brain volume? NeuroImage 64, 209215.

MacDonaldS. W.KarlssonS.RieckmannA.NybergL.BackmanL. (2012). Aging-related increases in behavioral variability: relations to losses of dopamine D1 receptors, J. Neurosci. 32, 81868191.

MercierM. R.MolholmS.FiebelkornI. C.ButlerJ. S.SchwartzT. H.FoxeJ. J. (2015). Neuro-oscillatory phase alignment drives speeded multisensory response times: an electro-corticographic investigation, J. Neurosci. 35, 85468557.

MorrellL. K.MorrellF. (1966). Evoked potentials and reaction times: a study of intra-individual variability, Electroencephalogr. Clin. Neurophysiol. 20, 567575.

MurphyK. J.WestR.ArmilioM. L.CraikF. I.StussD. T. (2007). Word-list-learning performance in younger and older adults: intra-individual performance variability and false memory, Neuropsychol., Dev. Cogn. B Aging Neuropsychol. Cogn. 14, 7094.

MurrayM. M.FoxeJ. J.WylieG. R. (2005). The brain uses single-trial multisensory memories to discriminate without awareness, NeuroImage 27, 473478.

MurrayM. M.WallaceM. T. (2012). The Neural Bases of Multisensory Processes. CRC Press, Boca Raton, FL, USA.

NathA. R.BeauchampM. S. (2011). Dynamic changes in superior temporal sulcus connectivity during perception of noisy audiovisual speech, J. Neurosci. 31, 17041714.

OhtsukiH.HauertC.LiebermanE.NowakM. A. (2006). A simple rule for the evolution of cooperation on graphs and social networks, Nature 441, 502505.

OostenveldR.FriesP.MarisE.SchoffelenJ. M. (2011). FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data, Comput. Intell. Neurosci. 2011, 156869. DOI:10.1155/2011/156869.

PeichM. C.HusainM.BaysP. M. (2013). Age-related decline of precision and binding in visual working memory, Psychol. Aging 28, 729743.

RomeiV.MurrayM. M.CappeC.ThutG. (2013). The contributions of sensory dominance and attentional bias to cross-modal enhancement of visual cortex excitability, J. Cogn. Neurosci. 25, 11221135.

RossL. A.Saint-AmourD.LeavittV. M.JavittD. C.FoxeJ. J. (2007). Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments, Cereb. Cortex 17, 11471153.

RossS. M. (2006). Simulation. Academic Press, Burlington, MA, USA.

SarkoD. K.NidifferA. R.PowersI. A.GhoseD.Hillock-DunnA.FisterM. C.KruegerJ.WallaceM. T. (2012). Spatial and temporal features of multisensory processes: bridging animal and human studies, in: The Neural Bases of Multisensory Processes, MurrayM. M.WallaceM. T. (Eds), pp.  192216. CRC Press, Boca Raton, FL, USA.

SperdinH. F.CappeC.FoxeJ. J.MurrayM. M. (2009). Early, low-level auditory-somatosensory multisensory interactions impact reaction time speed, Front. Integr. Neurosci. 3, 2. DOI:10.3389/neuro.07.002.2009.

StanfordT. R.QuessyS.SteinB. E. (2005). Evaluating the operations underlying multisensory integration in the cat superior colliculus, J. Neurosci. 25, 64996508.

SteinB. E.MeredithM. A. (1993). The Merging of the Senses. MIT Press, Cambridge, MA, USA.

StevensonR. A.WallaceM. T. (2013). Multisensory temporal integration: task and stimulus dependencies, Exp. Brain Res. 227, 249261.

StevensonR. A.GhoseD.FisterJ. K.SarkoD. K.AltieriN. A.NidifferA. R.KurelaL. R.SiemannJ. K.JamesT. W.WallaceM. T. (2014). Identifying and quantifying multisensory integration: a tutorial review, Brain Topogr. 27, 707730.

ThelenA.MatuszP. J.MurrayM. M. (2014). Multisensory context portends object memory, Curr. Biol. 24, R734R735.

VossR. F. (1992). Evolution of long-range fractal correlations and 1f noise in DNA base sequences, Phys. Rev. Lett. 68, 38053808.

WallaceM. T.RobersonG. E.HairstonW. D.SteinB. E.VaughanJ. W.SchirilloJ. A. (2004). Unifying multisensory signals across time and space, Exp. Brain Res. 158, 252258.

WestR.MurphyK. J.ArmilioM. L.CraikF. I.StussD. T. (2002). Lapses of intention and performance variability reveal age-related increases in fluctuations of executive control, Brain Cogn. 49, 402419.

YarkoniT.BarchD. M.GrayJ. R.ConturoT. E.BraverT. S. (2009). BOLD correlates of trial-by-trial reaction time variability in gray matter and white matter: a multi-study fMRI analysis, PLoS ONE 4, e4257. DOI:10.1371/journal.pone.0004257.


  • Comparison of the traditional and a proposed new index of multisensory integration. The traditional index defines multisensory integration (MSI) as the difference between the mean of the actual cross modal response (CM) and the maximum mean unisensory response (UM). The UM is defined as the maximum of the expected values (means) from the individual visual (E[V]) and auditory response distributions (E[A]). The proposed new index (MSI) first orders the individual trial by trial responses of the visual and auditory distributions in ascending and descending order, respectively. The maximum of each these pairings from the unisensory responses (largest visual response and smallest auditory response, secondly largest visual response and second smallest auditory response, etc.) is then taken, and the UM is defined as the mean of these maxima, representing the largest mean response that could occur using unisensory inputs alone. Integration with the MSI index is defined as the difference between UM and the actual crossmodal mean response. Both measures are amenable to statistical testing.

    View in gallery


Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 14 14 5
Full Text Views 3 3 3
PDF Downloads 1 1 1
EPUB Downloads 0 0 0