On the Temporal Precision of Thought: Individual Differences in the Multisensory Temporal Binding Window Predict Performance on Verbal and Nonverbal Problem Solving Tasks

in Multisensory Research
No Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


Although psychology is greatly preoccupied by the tight link between the way that individuals perceive the world and their intelligent, creative behavior, there is little experimental work on the relationship between individual differences in perception and cognitive ability in healthy populations. Here, individual differences in problem solving ability were examined in relation to multisensory perception as measured by tolerance for temporal asynchrony between auditory and visual inputs, i.e., the multisensory temporal binding window. The results demonstrated that enhanced performance in both verbal and nonverbal problem solving tasks (the Remote Associates Test and Raven’s Advanced Progressive Matrices Task) is predicted by a narrower audio-visual temporal binding window, which reflects greater sensitivity to subtle discrepancies in sensory inputs. This suggests that the precision of individuals’ temporal window of multisensory integration might mirror their capacities for complex reasoning and thus the precision of their thoughts.

On the Temporal Precision of Thought: Individual Differences in the Multisensory Temporal Binding Window Predict Performance on Verbal and Nonverbal Problem Solving Tasks

in Multisensory Research



Akbari ChermahiniS.HommelB. (2010). The (b) link between creativity and dopamine: spontaneous eye blink rates predict and dissociate divergent and convergent thinkingCognition 115458465.

Akbari ChermahiniS. A.HickendorffM.HommelB. (2012). Development and validity of a Dutch version of the remote associates task: an item-response theory approachThink. Skills Creat. 7177186.

AnsburgP. I.HillK. (2003). Creative and analytic thinkers differ in their use of attentional resourcesPers. Indiv. Dif. 3411411152.

BahrickL. E. (2010). Intermodal perception and selective attention to intersensory redundancy: implications for typical social development and autism in: Blackwell Handbook of Infant Development2nd edn.BremnerG.WachsT. D. (Eds) pp.  120166. Blackwell PublishingOxford, UK.

BahrickL. E.LickliterR. (2000). Intersensory redundancy guides attentional selectivity and perceptual learning in infancyDev. Psychol. 36190201.

BahrickL. E.LickliterR. (2012). The role of intersensory redundancy in early perceptual, cognitive, and social development in: Multisensory DevelopmentBremnerA. J.LewkowiczD. J.SpenceC. (Eds) pp.  183206. Oxford University PressOxford, UK.

BahrickL. E.LickliterR. (2014). Learning to attend selectively the dual role of intersensory redundancyCurr. Dir. Psychol. Sci. 23414420.

Bastien-ToniazzoM.StroumzaA.CavéC. (2009). Audio-visual perception and integration in developmental dyslexia: an exploratory study using the McGurk effectCurr. Psychol. Lett. 25214.

BebkoJ. M.WeissJ. A.DemarkJ. L.GomezP. (2006). Discrimination of temporal synchrony in intermodal events by children with autism and children with developmental disabilities without autismJ. Child Psychol. Psychiatry 478898.

BebkoJ. M.SchroederJ. H.WeissJ. A. (2014). The McGurk effect in children with autism and Asperger syndromeAutism Res. 75059.

BeemanM. J.BowdenE. M. (2000). The right hemisphere maintains solution-related activation for yet-to-be-solved problemsMem. Cogn. 2812311241.

BergC. A.StroughJ.CalderoneK. S.SansoneC.WeirC. (1998). The role of problem definitions in understanding age and context effects on strategies for solving everyday problemsPsychol. Aging 132944.

BishopC. W.MillerL. M. (2009). A multisensory cortical network for understanding speech in noiseJ. Cogn. Neurosci. 2117901804.

BoenkeL. T.DelianoM.OhlF. W. (2009). Stimulus duration influences perceived simultaneity in audiovisual temporal-order judgmentExp. Brain Res. 198233244.

BowdenE. M.Jung-BeemanM. (2003). Normative data for 144 compound remote associate problemsBehav. Res. Methods Instrum. Comput. 35634639.

CalvertG. A.ThesenT. (2004). Multisensory integration: methodological approaches and emerging principles in the human brainJ. Physiol. Paris 98191205.

CascioC. J.Foss-FeigJ. H.BurnetteC. P.HeacockJ. L.CosbyA. A. (2012). The rubber hand illusion in children with autism spectrum disorders: delayed influence of combined tactile and visual input on proprioceptionAutism 16406419.

ConreyB.PisoniD. B. (2006). Auditory–visual speech perception and synchrony detection for speech and nonspeech signalsJ. Acoust. Soc. Am. 11940654073.

De Boer-SchellekensL.EussenM.VroomenJ. (2013). Diminished sensitivity of audiovisual temporal order in autism spectrum disorderFront. Integr. Neurosci. 78. DOI:10.3389/fnint.2013.00008.

De GelderB.BertelsonP. (2003). Multisensory integration, perception and ecological validityTrends Cogn. Sci. 7460467.

De GelderB.VroomenJ.Van der HeideL. (1991). Face recognition and lip-reading in autismEur. J. Cogn. Psychol. 36986.

De GelderB.VroomenJ.AnnenL.MasthofE.HodiamontP. (2003). Audio-visual integration in schizophreniaSchizophr. Res. 59211218.

De GelderB.VroomenJ.De JongS. J.MasthoffE. D.TrompenaarsF. J.HodiamontP. (2005). Multisensory integration of emotional faces and voices in schizophrenicsSchizophr. Res. 72195203.

De JongJ. J.HodiamontP. P. G.Van den StockJ.De GelderB. (2009). Audiovisual emotion recognition in schizophrenia: reduced integration of facial and vocal affectSchizophr. Res. 107286293.

DeShonR. P.ChanD.WeissbeinD. A. (1995). Verbal overshadowing effects on raven’s advanced progressive matrices: evidence for multidimensional performance determinantsIntelligence 21135155.

DiederichA.ColoniusH. (2009). Crossmodal interaction in speeded responses: time window of integration modelProgr. Brain Res. 174119135.

DiederichA.ColoniusH. (2015). The time window of multisensory integration: relating reaction times and judgments of temporal orderPsychol. Rev. 122232241.

DiederichA.ColoniusH.SchomburgA. (2008). Assessing age-related multisensory enhancement with the time-window-of-integration modelNeuropsychologia 4625562562.

DillonR. F.PohlmannJ. T.LohmanD. F. (1981). A factor analysis of Raven’s advanced progressive matrices freed of difficulty factorsEduc. Psychol. Meas. 4112951302.

DixonN. F.SpitzL. (1980). The detection of auditory visual desynchronyPerception 9719721.

DonohueS. E.WoldorffM. G.MitroffS. R. (2010). Video game players show more precise multisensory temporal processing abilitiesAtten. Percept. Psychophys. 7211201129.

DonohueS. E.DarlingE. F.MitroffS. R. (2012). Links between multisensory processing and autismExp. Brain Res. 222377387.

DorfmanJ.ShamesV. A.KihlstromJ. F. (1996). Intuition, incubation, and insight: implicit cognition in problem solving in: Implicit CognitionUnderwoodG. (Ed.) pp.  257296. Oxford University PressOxford, UK.

EpelboimJ.SuppesP. (1997). Eye movements during geometrical problem solvingPerception 26(1 Suppl.) 138.

Foss-FeigJ. H.KwakyeL. D.CascioC. J.BurnetteC. P.KadivarH.StoneW. L.WallaceM. T. (2010). An extended multisensory temporal binding window in autism spectrum disordersExp. Brain Res. 203381389.

Foss-FeigJ. H.HeacockJ. L.CascioC. J. (2012). Tactile responsiveness patterns and their association with core features in autism spectrum disordersRes. Autism Spectr. Disord. 6337344.

FoucherJ. R.LacambreM.PhamB. T.GierschA.ElliottM. A. (2007). Low time resolution in schizophrenia: lengthened windows of simultaneity for visual, auditory and bimodal stimuliSchizophr. Res. 97118127.

FujisakiW.NishidaS. Y. (2009). Audio-tactile superiority over visuo-tactile and audio-visual combinations in the temporal resolution of synchrony perceptionExp. Brain Res. 198245259.

FujisakiW.ShimojoS.KashinoM.NishidaS. Y. (2004). Recalibration of audiovisual simultaneityNat. Neurosci. 7773778.

GirinL.SchwartzJ. L.FengG. (2001). Audio-visual enhancement of speech in noiseJ. Acoust. Soc. Am. 10930073020.

GondanM.NiederhausB.RöslerF.RöderB. (2005). Multisensory processing in the redundant-target effect: a behavioral and event-related potential studyPercept. Psychophys. 67713726.

GrantE. R.SpiveyM. J. (2003). Eye movements and problem solving guiding attention guides thoughtPsychol. Sci. 14462466.

GrantK. W.WaldenB. E.SeitzP. F. (1998). Auditory–visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditory–visual integrationJ. Acoust. Soc. Am. 10326772690.

HairstonW. D.LaurientiP. J.MishraG.BurdetteJ. H.WallaceM. T. (2003). Multisensory enhancement of localization under conditions of induced myopiaExp. Brain Res. 152404408.

HairstonW. D.BurdetteJ. H.FlowersD. L.WoodF. B.WallaceM. T. (2005). Altered temporal profile of visual–auditory multisensory interactions in dyslexiaExp. Brain Res. 166474480.

HershensonM. (1962). Reaction time as a measure of intersensory facilitationJ. Exp. Psychol. 63289293.

HillockA. R.PowersA. R.WallaceM. T. (2011). Binding of sights and sounds: age-related changes in multisensory temporal processingNeuropsychologia 49461467.

Hillock-DunnA.WallaceM. T. (2012). Developmental changes in the multisensory temporal binding window persist into adolescenceDev. Sci. 15688696.

HusserlE. (1991). On the Phenomenology of the Consciousness of Internal Time (1893–1917)Vol. 4. Springer Science & Business MediaDordrecht, The Netherlands.

IrwinJ. R.TornatoreL. A.BrancazioL.WhalenD. H. (2011). Can children with autism spectrum disorders “hear” a speaking face? Child Dev. 8213971403.

JustM. A.CarpenterP. A. (1985). Cognitive coordinate systems: accounts of mental rotation and individual differences in spatial abilityPsychol. Rev. 92137172.

KeetelsM.VroomenJ. (2007). No effect of auditory–visual spatial disparity on temporal recalibrationExp. Brain Res. 182559565.

KeetelsM.VroomenJ. (2008). Temporal recalibration to tactile–visual asynchronous stimuliNeurosci. Lett. 430130134.

KingA. J. (2005). Multisensory integration: strategies for synchronizationCurr. Biol. 15R339R341.

KingA. J.CalvertG. A. (2001). Multisensory integration: perceptual grouping by eye and earCurr. Biol. 11R322R325.

KnoblichG.OhlssonS.RaneyG. E. (2001). An eye movement study of insight problem solvingMem. Cogn. 2910001009.

KujalaT.LepistöT.NäätänenR. (2013). The neural basis of aberrant speech and audition in autism spectrum disordersNeurosci. Biobehav. Rev. 37697704.

KwakyeL. D.Foss-FeigJ. H.CascioC. J.StoneW. L.WallaceM. T. (2011). Altered auditory and multisensory temporal processing in autism spectrum disordersFront. Integr. Neurosci. 4129. DOI:10.3389/fnint.2010.00129.

LewkowiczD. J. (1996). Perception of auditory–visual temporal synchrony in human infantsJ. Exp. Psychol. Hum. Percept. Perform. 2210941106.

LewkowiczD. J. (2010). Infant perception of audio-visual speech synchronyDev. Psychol. 466677.

LewkowiczD. J.FlomR. (2014). The audiovisual temporal binding window narrows in early childhoodChild Dev. 85685694.

LovelaceC. T.SteinB. E.WallaceM. T. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detectionCognitive Brain Tesearch 17447453.

MassaroD. W.CohenM. M.SmeeleP. M. (1996). Perception of asynchronous and conflicting visual and auditory speechJ. Acoust. Soc. Am. 10017771786.

MednickS. (1962). The associative basis of the creative processPsychol. Rev. 69220232.

MégevandP.MolholmS.NayakA.FoxeJ. J. (2013). Recalibration of the multisensory temporal window of integration results from changing task demandsPLoS ONE 8e71608. DOI:10.1371/journal.pone.0071608.

MolholmS.RitterW.MurrayM. M.JavittD. C.SchroederC. E.FoxeJ. J. (2002). Multisensory auditory–visual interactions during early sensory processing in humans: a high-density electrical mapping studyCogn. Brain Res. 14115128.

MongilloE. A.IrwinJ. R.WhalenD. H.KlaimanC.CarterA. S.SchultzR. T. (2008). Audiovisual processing in children with and without autism spectrum disordersJ. Autism Dev. Disord. 3813491358.

NavarraJ.VatakisA.ZampiniM.Soto-FaracoS.HumphreysW.SpenceC. (2005). Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integrationCogn. Brain Res. 25499507.

NoelJ. P.WallaceM. T.Orchard-MillsE.AlaisD.Van der BurgE. (2015). True and perceived synchrony are preferentially associated with particular sensory pairingsSci. Rep. 517467. DOI:10.1038/srep17467.

OhlssonS. (2012). The problems with problem solving: reflections on the rise, current status, and possible future of a cognitive research paradigmJ. Problem Solv. 5101128.

PearlD.Yodashkin-PoratD.KatzN.ValevskiA.AizenbergD.SiglerM.WeizmanA.KikinzonL. (2009). Differences in audiovisual integration, as measured by McGurk phenomenon, among adult and adolescent patients with schizophrenia and age-matched healthy control groupsCompr. Psychiatry 50186192.

PowersA. R.HillockA. R.WallaceM. T. (2009). Perceptual training narrows the temporal window of multisensory bindingJ. Neurosci. 291226512274.

PowersA. R.HeveyM. A.WallaceM. T. (2012). Neural correlates of multisensory perceptual learningJ. Neurosci. 3262636274.

RavenJ. C. (1965). Advanced Progressive Matrices: Sets I and II. LewisLondon, UK.

RossL. A.Saint-AmourD.LeavittV. M.MolholmS.JavittD. C.FoxeJ. J. (2007). Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditionsSchizophr. Res. 97173183.

RussoN.FoxeJ. J.BrandweinA. B.AltschulerT.GomesH.MolholmS. (2010). Multisensory processing in children with autism: high-density electrical mapping of auditory–somatosensory integrationAutism Res. 3253267.

SchallJ. D.HanesD. P. (1993). Neural basis of saccade target selection in frontal eye field during visual searchNature 366467469.

SchoolerJ. W.MelcherJ. (1995). The ineffability of insight in: The Creative Cognition ApproachSmithS. M.WardT. B.FinkeR. A. (Eds) pp.  97133. MIT PressCambridge, MA, USA.

SekulerR.SekulerA. B.LauR. (1997). Sound alters visual motion perceptionNature 385308.

ShamsL. (2002). Integration in the brain — the subconscious alteration of visual perception by cross-modal integrationSci. Consc. Rev. 114.

SpearmanC.Wynn-JonesL. (1951). Human Ability. MacmillanLondon, UK.

SpenceC.BaddeleyR.ZampiniM.JamesR.ShoreD. I. (2003). Multisensory temporal order judgments: when two locations are better than onePercept. Psychophys. 65318328.

SpenceC.SquireS. (2003). Multisensory integration: maintaining the perception of synchronyCurr. Biol. 13R519R521.

SteinB. E.StanfordT. R. (2008). Multisensory integration: current issues from the perspective of the single neuronNat. Rev. Neurosci. 9255266.

StevensonR. A.JamesT. W. (2009). Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognitionNeuroimage 4412101223.

StevensonR. A.WallaceM. T. (2013). Multisensory temporal integration: task and stimulus dependenciesExp. Brain Res. 227249261.

StevensonR. A.ZemtsovR. K.WallaceM. T. (2011). Multisensory illusions and the temporal binding windowIperception 2903.

StevensonR. A.ZemtsovR. K.WallaceM. T. (2012). Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusionsJ. Exp. Psychol. Hum. Percept. Perform. 3815171529.

StevensonR. A.WilsonM. M.PowersA. R.WallaceM. T. (2013). The effects of visual training on multisensory temporal processingExp. Brain Res. 225479489.

StevensonR. A.SiemannJ. K.WoynaroskiT. G.SchneiderB. C.EberlyH. E.CamarataS. M.WallaceM. T. (2014a). Brief report: arrested development of audiovisual speech perception in autism spectrum disordersJ. Autism Dev. Disord 4414701477.

StevensonR. A.SegersM.FerberS.BarenseM. D.WallaceM. T. (2014b). The impact of multisensory integration deficits on speech perception in children with autism spectrum disordersFront. Psychol. 5379. DOI:10.3389/fpsyg.2014.00379.

StevensonR. A.SiemannJ. K.SchneiderB. C.EberlyH. E.WoynaroskiT. G.CamarataS. M.WallaceM. T. (2014c). Multisensory temporal integration in autism spectrum disordersJ. Neurosci. 34691697.

StevensonR. A.SegersM.FerberS.BarenseM. D.CamarataS.WallaceM. T. (in press). Keeping time in the brain: autism spectrum disorder and audiovisual temporal processingAutism Res. DOI:10.1002/aur.1566.

StoneJ. V.HunkinN. M.PorrillJ.WoodR.KeelerV.BeanlandM.PortM.PorterN. R. (2001). When is now? Perception of simultaneityProc. R. Soc. Lond. B Biol. Sci. 2683138.

SumbyW. H.PollackI. (1954). Visual contribution to speech intelligibility in noiseJ. Acoust. Soci. Am. 26212215.

SzycikG. R.MünteT. F.DilloW.MohammadiB.SamiiA.EmrichH. M.DietrichD. E. (2009). Audiovisual integration of speech is disturbed in schizophrenia: an fMRI studySchizophr. Res. 110111118.

ThomasL. E.LlerasA. (2007). Moving eyes and moving thought: on the spatial compatibility between eye movements and cognitionPsychonom. Bull. Rev. 14663668.

Vaillant-MolinaM.BahrickL. E. (2012). The role of intersensory redundancy in the emergence of social referencing in 51/2-month-old infantsDev. Psychol. 4819.

Van der SmagtM. J.van EngelandH.KemnerC. (2007). Brief report: can you see what is not there? Low-level auditory–visual integration in autism spectrum disorderJ. Autism Dev. Disord. 3720142019.

Van EijkR. L.KohlrauschA.JuolaJ. F.Van de ParS. (2008). Audiovisual synchrony and temporal order judgments: effects of experimental method and stimulus typePercept. Psychophys. 70955968.

Van WassenhoveV.GrantK. W.PoeppelD. (2007). Temporal window of integration in auditory–visual speech perceptionNeuropsychologia 45598607.

VatakisA.BakouA. E. (2015). Distorted multisensory experiences of order and simultaneity in: Time Distortions in Mind: Temporal Processing in Clinical PopulationsVatakisA.AllmanM. (Eds) pp.  136. BrillLeiden, The Netherlands.

VatakisA.SpenceC. (2006). Audiovisual synchrony perception for music, speech, and object actionsBrain Res. 1111134142.

VatakisA.SpenceC. (2010). Audiovisual temporal integration for complex speech, object-action, animal call, and musical stimuli in: Multisensory Object Perception in the Primate BrainNaumerM. J.KaiserJ. (Eds) pp.  95121. Springer-VerlagBerlin & Heidelberg, Germany.

VatakisA.NavarraJ.Soto-FaracoS.SpenceC. (2008). Audiovisual temporal adaptation of speech: temporal order versus simultaneity judgmentsExp. Brain Res. 185521529.

VisserE.ZwiersM. P.KanC. C.HoekstraL.Van OpstalA. J.BuitelaarJ. K. (2013). Atypical vertical sound localization and sound-onset sensitivity in people with autism spectrum disordersJ. Psychiatry Neurosci. 38398406.

VroomenJ.KeetelsM. (2010). Perception of intersensory synchrony: a tutorial reviewAtten. Percept. Psychophys. 72871884.

VroomenJ.KeetelsM.De GelderB.BertelsonP. (2004). Recalibration of temporal order perception by exposure to audio-visual asynchronyCogn. Brain Res. 223235.

WallaceM. T.StevensonR. A. (2014). The construct of the multisensory temporal binding window and its dysregulation in developmental disabilitiesNeuropsychologia 64105123.

WoynaroskiT. G.KwakyeL. D.Foss-FeigJ. H.StevensonR. A.StoneW. L.WallaceM. T. (2013). Multisensory speech perception in children with autism spectrum disordersJ. Autism Dev. Disord. 4328912902.

ZabelinaD. L.BeemanM. (2013). Short-term attentional perseveration associated with real-life creative achievementFront. Psychol. 4191. DOI:10.3389/fpsyg.2013.00191.

ZabelinaD. L.O’LearyD.PornpattananangkulN.NusslockR.BeemanM. (2015a). Creativity and sensory gating indexed by the P50: selective versus leaky sensory gating in divergent thinkers and creative achieversNeuropsychologia 697784.

ZabelinaD.SaportaA.BeemanM. (2015b). Flexible or leaky attention in creative people? Distinct patterns of attention for different types of creative thinkingMem. Cogn. 1111.

ZampiniM.ShoreD. I.SpenceC. (2003). Audiovisual temporal order judgmentsExp. Brain Res. 152198210.

ZampiniM.GuestS.ShoreD. I.SpenceC. (2005). Audio-visual simultaneity judgmentsPercept. Psychophys. 67531544.

ZmigrodS.HommelB. (2011). The relationship between feature binding and consciousness: evidence from asynchronous multi-modal stimuliConsc. Cogn. 20586593.

ZmigrodS.ZmigrodL. (2015). Zapping the gap: reducing the multisensory temporal binding window by means of transcranial direct current stimulation (tDCS)Consc. Cogn. 35143149.

ZmigrodS.De SonnevilleL. M. J.ColzatoL. S.SwaabH.HommelB. (2013). Cognitive control of feature bindings: evidence from children with autistic spectrum disorderPsychol. Res. 77147154.

ZmigrodS.ZmigrodL.HommelB. (2015). Zooming into creativity: individual differences in attentional global-local biases are linked to creative thinkingFront. Psychol. 61647. DOI:10.3389/fpsyg.2015.01647.


  • View in gallery

    Percentage of synchrony judgment with error bars as a function of stimuli onset asynchronies (SOAs). p<0.001.

  • View in gallery

    Scatter plot depicting the correlations between the synchrony judgment at SOA of 350 ms and the scores from both the verbal problem solving task (RAT) and the nonverbal problem solving task (Raven’s APM).

  • View in gallery

    Scatter plot depicting the correlations between the TBW in milliseconds and the scores from both the verbal problem solving task (RAT) and the nonverbal problem solving task (Raven’s APM).

  • View in gallery

    Percentage of synchrony judgment at SOA = 350 ms (with error bars) as a function of high scores and low scores for verbal (RAT) and nonverbal (Raven’s APM) complex problems. p<0.05.

  • View in gallery

    Percentage of synchrony judgment as a function of stimulus onset asynchronies (SOAs) and calculated temporal binding window (TBW) of males and females with error bars. p<0.05.


Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 29 29 9
Full Text Views 147 147 62
PDF Downloads 4 4 0
EPUB Downloads 0 0 0