Multisensory Integration in Self Motion Perception

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


Self motion perception involves the integration of visual, vestibular, somatosensory and motor signals. This article reviews the findings from single unit electrophysiology, functional and structural magnetic resonance imaging and psychophysics to present an update on how the human and non-human primate brain integrates multisensory information to estimate one’s position and motion in space. The results indicate that there is a network of regions in the non-human primate and human brain that processes self motion cues from the different sense modalities.

Multisensory Integration in Self Motion Perception

in Multisensory Research



AndersenR. A.SnyderL. H.BradleyD. C.XingJ. (1997). Multimodal representation of space in the posterior parietal cortex and its use in planning movementsAnnu. Rev. Neurosci. 20303330.

AngelakiD. E.GuY.DeAngelisG. C. (2011). Visual and vestibular cue integration for heading perception in extrastriate visual cortexJ. Physiol. 589825833.

AvillacM.DeneveS.OlivierE.PougetA.DuhamelJ. R. (2005). Reference frames for representing visual and tactile locations in parietal cortexNat. Neurosci. 8941949.

AvillacM.Ben HamedS.DuhamelJ. R. (2007). Multisensory integration in the ventral intraparietal area of the macaque monkeyJ. Neurosci. 2719221932.

BaranyR. (1907). Physiologie und Pathologie des Bogengangapparates beim Menschen. Franz Deuticke VerlagLeipzig, Germany.

BarlowH. B. (1990). A theory about the functional role and synaptic mechanism of visual after-effects in: Vision: Coding and EfficiencyBlakemoreC. B. (Ed.) pp.  363375. Cambridge University PressCambridge, UK.

BarlowH. B.HillR. M. (1963). Selective sensitivity to direction of movement in ganglion cells of the rabbit retinaScience 139(3553) 412414.

BeerA. L.WatanabeT.NiR.SasakiY.AndersenG. J. (2009). 3D surface perception from motion involves a temporal-parietal networkEur. J. Neurosci. 30703713.

BeintemaJ. A.Van den BergA. V. (1998). Heading detection using motion templates and eye velocity gain fieldsVis. Res. 3821552179.

BeintemaJ. A.Van den BergA. V.LappeM. (2004). Circular receptive field structures for flow analysis and heading detection in: The Structure of Receptive Fields for Flow Analysis and Heading DetectionVainaL. M.BeardsleyS. A.RushtonS. (Eds) pp.  223248. Kluwer Academic PublishersNorwell, MA, USA.

Ben HamedS.DuhamelJ. R.BremmerF.GrafW. (2002). Visual receptive field modulation in the lateral intraparietal area during attentive fixation and free gazeCereb. Cortex 12234245.

BensonA. J.KassJ. R.VogelH. (1986). European vestibular experiments on the Spacelab-1 mission: 4. Thresholds of perception of whole-body linear oscillationExp. Brain Res. 64264271.

BiagiL.CrespiS. A.TosettiM.MorroneM. C. (2015). BOLD response selective to flow-motion in very young infantsPLoS Biol. 13e1002260. DOI:10.1371/journal.pbio.1002260.

BillingtonJ.SmithA. T. (2015). Neural mechanisms for discounting head-roll-induced retinal motionJ. Neurosci. 3548514856.

BottiniG.PaulesuE.GandolaM.LoffredoS.ScarpaP.SterziR.SantilliI.DefantiC.ScialfaG.FazioF. (2005). Left caloric vestibular stimulation ameliorates right hemianesthesiaNeurology 6512781283.

BottiniG.GandolaM.SeddaA.FerrèE. R. (2013). Caloric vestibular stimulation: interaction between somatosensory system and vestibular apparatusFront. Integr. Neurosci. 766. DOI:10.3389/fnint.2013.00066.

BradleyD. C.MaxwellM.AndersenR. A.BanksM. S.ShenoyK. V. (1996). Mechanisms of heading perception in primate visual cortexScience 27315441547.

BrandtT.DichgansJ.BuchleW. (1974). Motion habituation: inverted self-motion perception and optokinetic after-nystagmusExp. Brain Res. 21337352.

BrandtT.BartensteinP.JanekA.DieterichM. (1998). Reciprocal inhibitory visual–vestibular interaction. Visual motion stimulation deactivates the parieto-insular vestibular cortexBrain 12117491758.

BremmerF. (2011). Multisensory space: from eye-movements to self-motionJ. Physiol. 589815823.

BremmerF.KubischikM.PekelM.LappeM.HoffmannK. P. (1999). Linear vestibular self-motion signals in monkey medial superior temporal areaAnn. N. Y. Acad. Sci. 871272281.

BremmerF.SchlackA.ShahN. J.ZafirisO.KubischikM.HoffmannK.ZillesK.FinkG. R. (2001). Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeysNeuron 29287296.

BremmerF.DuhamelJ.-R.Ben HamedS.GrafW. (2002a). Heading encoding in the macaque ventral intraparietal area (VIP)Eur. J. Neurosci. 1615541568.

BremmerF.KlamF.DuhamelJ.-R.Ben HamedS.GrafW. (2002b). Visual–vestibular interactive responses in the macaque ventral intraparietal area (VIP)Eur. J. Neurosci. 1615691586.

BremmerF.KubischikM.HoffmannK. P.KrekelbergB. (2009). Neural dynamics of saccadic suppressionJ. Neurosci. 291237412383.

BremmerF.KubischikM.PekelM.HoffmannK. P.LappeM. (2010). Visual selectivity for heading in monkey area MSTExp. Brain Res. 2005160.

BrittenK. H. (2008). Mechanisms of self-motion perceptionAnnu. Rev. Neurosci. 31389410.

ButlerJ. S.SmithS. T.CamposJ. L.BülthoffH. H. (2010). Bayesian integration of visual and vestibular signals for headingJ. Vis. 1023. DOI:10.1167/10.11.23.

CardinV.SmithA. T. (2010). Sensitivity of human visual and vestibular cortical regions to egomotion-compatible visual stimulationCereb. Cortex 2019641973.

ChenA.DeAngelisG. C.AngelakiD. E. (2011). Representation of vestibular and visual cues to self-motion in ventral intraparietal cortexJ. Neurosci. 311203612052.

ChenA.DeangelisG. C.AngelakiD. E. (2013). Functional specializations of the ventral intraparietal area for multisensory heading discriminationJ. Neurosci. 3335673581.

ChenX.DeAngelisG. C.AngelakiD. E. (2014). Eye-centered visual receptive fields in the ventral intraparietal areaJ. Neurophysiol. 112353361.

ClaeysK. G.LindseyD. SchutterE.OrbanG. A. (2003). A higher order motion region in human inferior parietal lobule: evidence from fMRINeuron 40631642.

CohenB.HennV.RaphanT.DennettD. (1981). Velocity storage, nystagmus, and visual–vestibular interactions in humansAnn. N. Y. Acad. Sci. 374421433.

ConiglioA. J.CraneB. T. (2014). Human yaw rotation aftereffects with brief duration rotations are inconsistent with velocity storageJ. Assoc. Res. Otolaryngol. 15305317.

CraneB. T. (2012). Fore–aft translation aftereffectsExp. Brain Res. 219477487.

CraneB. T. (2013). Limited interaction between translation and visual motion aftereffects in humansExp. Brain Res. 224165178.

CuturiL. F.MacNeilageP. R. (2013). Systematic biases in human heading estimationPLoS ONE 8e56862. DOI:10.1371/journal.pone.0056862.

CuturiL. F.MacNeilageP. R. (2014). Optic flow induces nonvisual self-motion aftereffectsCurr. Biol. 2428172821.

De WinkelK. N.KatliarM.BulthoffH. H. (2015). Forced fusion in multisensory heading estimationPLoS ONE 10e0127104. DOI:10.1371/journal.pone.0127104.

DeutschländerA.BenseS.StephanT.SchwaigerM.BrandtT.DieterichM. (2002). Sensory system interactions during simultaneous vestibular and visual stimulation in PETHum. Brain Mapp. 1692103.

DieterichM.BrandtT. (2008). Functional brain imaging of peripheral and central vestibular disordersBrain 13125382552.

DieterichM.BrandtT. (2015). The bilateral central vestibular system: its pathways, functions, and disordersAnn. N. Y. Acad. Sci. 13431026.

DieterichM.BucherS. F.SeelosK. C.BrandtT. (1998). Horizontal or vertical optokinetic stimulation activates visual motion-sensitive, ocular motor and vestibular cortex areas with right hemispheric dominance. An fMRI studyBrain 12114791495.

DieterichM.BenseS.LutzS.DrzezgaA.StephanT.BartensteinP.BrandtT. (2003). Dominance for vestibular cortical function in the non-dominant hemisphereCereb. Cortex 139941007.

DuffyC. J. (1998). MST neurons respond to optic flow and translational movementJ. Neurophysiol. 8018161827.

DuffyC. J. (2000). Optic flow analysis for self-movement perceptionInt. Rev. Neurobiol. 44199218.

DuffyC. J.WurtzR. H. (1991a). Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuliJ. Neurophysiol. 6513291345.

DuffyC. J.WurtzR. H. (1991b). Sensitivity of MST neurons to optic flow stimuli. II. Mechanisms of response selectivity revealed by small-field stimuliJ. Neurophysiol. 6513461359.

DuhamelJ. R.ColbyC. L.GoldbergM. E. (1998). Ventral intraparietal area of the macaque: congruent visual and somatic response propertiesJ. Neurophysiol. 79126136.

DukelowS. P.DeSouzaJ. F.CulhamJ. C.Van den BergA. V.MenonR. S.VilisT. (2001). Distinguishing subregions of the human MT+ complex using visual fields and pursuit eye movementsJ. Neurophysiol. 8619912000.

EickhoffS. B.WeissP. H.AmuntsK.FinkG. R.ZillesK. (2006). Identifying human parieto-insular vestibular cortex using fMRI and cytoarchitectonic mappingHum. Brain Mapp. 27611621.

EricksonR. G.ThierP. (1991). A neuronal correlate of spatial stability during periods of self-induced visual motionExp. Brain Res. 86608616.

FasoldO.von BrevernM.KuhbergM.PlonerC. J.VillringerA.LempertT.WenzelR. (2002). Human vestibular cortex as identified with caloric stimulation in functional magnetic resonance imagingNeuroImage 1713841393.

FattoriP.PitzalisS.GallettiC. (2009). The cortical visual area V6 in macaque and human brainsJ. Physiol. Paris 1038897.

FerrèE. R.SeddaA.GandolaM.BottiniG. (2011). How the vestibular system modulates tactile perception in normal subjects: a behavioural and physiological studyExp. Brain Res. 2082938.

FerrèE. R.DayB. L.BottiniG.HaggardP. (2013a). How the vestibular system interacts with somatosensory perception: a sham-controlled study with galvanic vestibular stimulationNeurosci. Lett. 5503540.

FerrèE. R.BottiniG.IannettiG. D.HaggardP. (2013b). The balance of feelings: vestibular modulation of bodily sensationsCortex 49748758.

FerrèE. R.KaliuzhnaM.HerbelinB.HaggardP.BlankeO. (2014). Vestibular-somatosensory interactions: effects of passive whole-body rotation on somatosensory detectionPLoS ONE 9e86379. DOI:10.1371/journal.pone.0086379.

FetschC. R.TurnerA. H.DeAngelisG. C.AngelakiD. E. (2009). Dynamic reweighting of visual and vestibular cues during self-motion perceptionJ. Neurosci. 291560115612.

FetschC. R.DeAngelisG. C.AngelakiD. E. (2013). Bridging the gap between theories of sensory cue integration and the physiology of multisensory neuronsNat. Rev. Neurosci. 14429442.

FrankS. M.GreenleeM. W. (2014). An MRI-compatible caloric stimulation device for the investigation of human vestibular cortexJ. Neurosci. Meth. 235208218.

FrankS. M.BaumannO.MattingleyJ. B.GreenleeM. W. (2014). Vestibular and visual responses in human posterior insular cortexJ. Neurophysiol. 11224812491.

FrankS. M.WirthA. M.GreenleeM. W. (subm.). Visual–vestibular processing in the human Sylvian fissure.

GallettiC.FattoriP. (2003). Neuronal mechanisms for detection of motion in the field of viewNeuropsychologia 4117171727.

GallettiC.SquatritoS.BattagliniP. P.Grazia MaioliM. (1984). ‘Real-motion’ cells in the primary visual cortex of macaque monkeysBrain Res. 30195110.

GallettiC.BattagliniP. P.AicardiG. (1988). ‘Real-motion’ cells in visual area V2 of behaving macaque monkeysExp. Brain Res. 69279288.

GallettiC.BattagliniP. P.FattoriP. (1990). ‘Real-motion’ cells in area V3A of macaque visual cortexExp. Brain Res. 826776.

GamberiniM.FattoriP.GallettiC. (2015). The medial parietal occipital areas in the macaque monkeyVis. Neurosci. 32E013. DOI:10.1017/S0952523815000103.

GibsonJ. J. (1950). The Perception of the Visual World. Houghton MifflinBoston, MA, USA.

GrabherrL.NicoucarK.MastF. W.MerfeldD. M. (2008). Vestibular thresholds for yaw rotation about an Earth-vertical axis as a function of frequencyExp. Brain Res. 186677681.

GranthamD. W.WightmanF. L. (1979). Auditory motion aftereffectsPercept. Psychophys. 26403408.

GreenleeM. W. (2000). Human cortical areas underlying the perception of optic flow: brain imaging studiesInt. Rev. Neurobiol. 44269292.

GuY.WatkinsP. V.AngelakiD. E.DeAngelisG. C. (2006). Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal areaJ. Neurosci. 267385.

GuY.DeAngelisG. C.AngelakiD. E. (2007). A functional link between area MSTd and heading perception based on vestibular signalsNat. Neurosci. 1010381047.

GuY.AngelakiD. E.DeAngelisG. C. (2008). Neural correlates of multisensory cue integration in macaque MSTdNat. Neurosci. 1112011210.

GuY.FetschC. R.AdeyemoB.DeAngelisG. C.AngelakiD. E. (2010). Decoding of MSTd population activity accounts for variations in the precision of heading perceptionNeuron 66596609.

GuY.DeAngelisG. C.AngelakiD. E. (2012). Causal links between dorsal medial superior temporal area neurons and multisensory heading perceptionJ. Neurosci. 3222992313.

GuldinW. O.GrüsserO. J. (1998). Is there a vestibular cortex? Trends Neurosci. 21254259.

HitierM.BesnardS.SmithP. F. (2014). Vestibular pathways involved in cognitionFront. Integr. Neurosci. 859. DOI:10.3389/fnint.2014.00059.

HoltenV.Van der SmagtM. J.DonkerS. F.VerstratenF. A. (2014). Illusory motion of the motion aftereffect induces postural swayPsychol. Sci. 2518311834.

HuangR.-S.ChenC.-F.SerenoM. I. (2015). Neural substrates underlying the passive observation and active control of translational egomotionJ. Neurosci. 3542584267.

HukA. C.DoughertyR. F.HeegerD. J. (2002). Retinotopy and functional subdivision of human areas MT and MSTJ. Neurosci. 2271957205.

IontaS.HeydrichL.LenggenhagerB.MouthonM.FornariE.ChapuisD.GassertR.BlankeO. (2011). Multisensory mechanisms in temporo-parietal cortex support self-location and first-person perspectiveNeuron 70363374.

KaliuzhnaM.PrsaM.GaleS.LeeS. J.BlankeO. (2015). Learning to integrate contradictory multisensory self-motion cue pairingsJ. Vis. 1515.1.10. DOI:10.1167/15.1.10.

KaminiarzA.SchlackA.HoffmannK. P.LappeM.BremmerF. (2014). Visual selectivity for heading in the macaque ventral intraparietal areaJ. Neurophysiol. 11224702480.

KitagawaN.IchiharaS. (2002). Hearing visual motion in depthNature 416(6877) 172174.

KleinschmidtA.ThiloK. V.BüchelC.GrestyM. A.BronsteinA. M.FrackowiakR. S. J. (2002). Neural correlates of visual-motion perception as object- or self-motionNeuroImage 16873882.

KoenderinkJ. J. (1986). Optic flowVis. Res. 26161179.

KommerellG.ThieleH. (1970). Der optokinetische Kurzreiznystagmus [Optokinetic short-stimulation nystagmus]Graefes Arch. Klin. Exp. Ophthalmol. 179(3) 220234.

KonkleT.MooreC. I. (2009). What can crossmodal aftereffects reveal about neural representation and dynamics? Commun. Integr. Biol. 2479481.

KonkleT.WangQ.HaywardV.MooreC. I. (2009). Motion aftereffects transfer between touch and visionCurr. Biol. 19745750.

KontsevichL. L.TylerC. W. (1999). Bayesian adaptive estimation of psychometric slope and thresholdVis. Res. 3927292737.

KördingK. P.BeierholmU.MaW. J.QuartzS.TenenbaumJ. B.ShamsL. (2007). Causal inference in multisensory perceptionPLoS ONE 2e943. DOI:10.1371/journal.pone.0000943.

LacknerJ. R.DiZioP. (2005). Vestibular, proprioceptive, and haptic contributions to spatial orientationAnnu. Rev. Psychol. 56115147.

LappeM.RauscheckerJ. P. (1993). A neural network for the processing of optic flow from ego-motion in man and higher mammalsNeural Comp. 5374391.

LappeM.RauscheckerJ. P. (1994). Heading detection from optic flowNature 369(6483) 712713.

LappeM.BremmerF.PekelM.ThieleA.HoffmannK. P. (1996). Optic flow processing in monkey STS: a theoretical and experimental approachJ. Neurosci. 1662656285.

LappeM.PekelM.HoffmannK.-P. (1998). Optokinetic eye movements elicited by radial optic flow in the macaque monkeyJ. Neurophysiol. 7914611480.

LappeM.BremmerF.Van den BergA. V. (1999). Perception of self-motion from visual flowTrends Cogn. Sci. 3329336.

LobelE.KleineJ.Le BihanD.Leroy-WilligA.BerthozA. (1998). Functional MRI of galvanic vestibular stimulationJ. Neurophysiol. 8026992709.

LopezC.BlankeO. (2011). The thalamocortical vestibular system in animals and humansBrain Res. Rev. 67119146.

LopezC.BlankeO.MastF. W. (2012). The human vestibular cortex revealed by coordinate-based activation likelihood estimation meta-analysisNeuroscience 212159179.

MacNeilageP. R.BanksM. S.BergerD. R.BülthoffH. H. (2007). A Bayesian model of the disambiguation of gravitoinertial force by visual cuesExp. Brain Res. 179263290.

MacNeilageP. R.BanksM. S.DeAngelisG. C.AngelakiD. E. (2010). Vestibular heading discrimination and sensitivity to linear acceleration in head and world coordinatesJ. Neurosci. 3090849094.

MatherG.PavanA.CampanaG.CascoC. (2008). The motion aftereffect reloadedTrends Cogn. Sci. 12481487.

MergnerT.RosemeierT. (1998). Interaction of vestibular, somatosensory and visual signals for postural control and motion perception under terrestrial and microgravity conditions — a conceptual modelBrain Res. Rev. 28118135.

MergnerT.NardiG.BeckerW.DeeckeL. (1983). The role of canal-neck interaction for the perception of horizontal trunk and head rotationExp. Brain Res. 49198208.

MorganM. L.DeAngelisG. C.AngelakiD. E. (2008). Multisensory integration in macaque visual cortex depends on cue reliabilityNeuron 59662673.

MorrisA. P.KubischikM.HoffmannK.-P.KrekelbergB.BremmerF. (2012). Dynamics of eye-position signals in the dorsal visual systemCurr. Biol. 22173179.

MorroneM. C.TosettiM.MontanaroD.FiorentiniA.CioniG.BurrD. C. (2000). A cortical area that responds specifically to optic flow, revealed by fMRINat. Neurosci. 313221328.

NiJ.TatalovicM.StraumannD.OlasagastiI. (2013). Gaze direction affects linear self-motion heading discrimination in humansEur. J. Neurosci. 3832483260.

OrbanG. A.FizeD.PeuskensH.DenysK.NelissenK.SunaertS.ToddJ.VanduffelW. (2003). Similarities and differences in motion processing between the human and macaque brain: evidence from fMRINeuropsychologia 4117571768.

PageW. K.DuffyC. J. (2003). Heading representation in MST: sensory interactions and population encodingJ. Neurophysiol. 8919942013.

PerroneJ. A.StoneL. S. (1994). A model of self-motion estimation within primate extrastriate visual cortexVis. Res. 3429172938.

PfeifferC.LopezC.SchmutzV.DuenasJ. A.MartuzziR.BlankeO. (2013). Multisensory origin of the subjective first-person perspective: visual, tactile, and vestibular mechanismsPLoS ONE 8e61751. DOI:10.1371/journal.pone.0061751.

PfeifferC.SchmutzV.BlankeO. (2014). Visuospatial viewpoint manipulation during full-body illusion modulates subjective first-person perspectiveExp. Brain Res. 23240214033.

PfeifferC.van ElkM.BernasconiF.BlankeO. (2016). Distinct vestibular effects on early and late somatosensory cortical processing in humansNeuroImage 125208219.

PitzalisS.SerenoM. I.CommitteriG.FattoriP.GalatiG.TosoniA.GallettiC. (2013). The human homologue of macaque area V6ANeuroImage 82517530.

PitzalisS.FattoriP.GallettiC. (2015). The human cortical areas V6 and V6AVis. Neurosci. 32E007. DOI:10.1017/S0952523815000048.

PriesolA. J.ValkoY.MerfeldD. M.LewisR. F. (2014). Motion perception in patients with idiopathic bilateral vestibular hypofunctionOtolaryngol. Head Neck Surg. 15010401042.

ProbstT.StraubeA.BlesW. (1985). Differential effects of ambivalent visual–vestibular–somatosensory stimulation on the perception of self-motionBehav. Brain Res. 167179.

PrsaM.GaleS.BlankeO. (2012). Self-motion leads to mandatory cue fusion across sensory modalitiesJ. Neurophysiol. 10822822291.

RieckeB. E.JordanJ. D. (2015). Comparing the effectiveness of different displays in enhancing illusions of self-movement (vection)Front. Psychol. 6713. DOI:10.3389/fpsyg.2015.00713.

RoachN. W.HeronJ.McGrawP. V. (2006). Resolving multisensory conflict: a strategy for balancing the costs and benefits of audio-visual integrationProc. R. Soc. B Biol. Sci. 27321592168.

SchlackA.HoffmannK. P.BremmerF. (2002). Interaction of linear vestibular and visual stimulation in the macaque ventral intraparietal area (VIP)Eur. J. Neurosci. 1618771886.

SeemungalB. M. (2014). The cognitive neurology of the vestibular systemCurr. Opin. Neurol. 27125132.

SenoT.ItoH.SunagaS. (2010). Vection aftereffects from expanding/contracting stimuliSeeing Perceiving 23273294.

SerenoM. I.HuangR. S. (2006). A human parietal face area contains aligned head-centered visual and tactile mapsNat. Neurosci. 913371343.

SerenoM. I.HuangR. S. (2014). Multisensory maps in parietal cortexCurr. Opin. Neurobiol. 243946.

ShenoyK. V.BradleyD. C.AndersenR. A. (1999). Influence of gaze rotation on the visual response of primate MSTd neuronsJ. Neurophysiol. 8127642786.

ShuZ. J.SwindaleN. V.CynaderM. S. (1993). Spectral motion produces an auditory after-effectNature 364(6439) 721723.

SmithA. T.WallM. B.ThiloK. V. (2012). Vestibular inputs to human motion-sensitive visual cortexCereb. Cortex 2210681077.

SommerM. A.WurtzR. H. (2008). Brain circuits for the internal monitoring of movementsAnnu. Rev. Neurosci. 31317338.

SperryR. W. (1950). Neural basis of the spontaneous optokinetic response produced by visual inversionJ. Comp. Physiol. Psychol. 43482489.

StephanT.DeutschländerA.NolteA.SchneiderE.WiesmannM.BrandtT.DieterichM. (2005). Functional MRI of galvanic vestibular stimulation with alternating currents at different frequenciesNeuroImage 26721732.

SunaertS.Van HeckeP.MarchalG.OrbanG. A. (1999). Motion-responsive regions of the human brainExp. Brain Res. 127355370.

SutherlandN. S. (1961). Figural aftereffects and apparent sizeQ. J. Exp. Psychol. 13222228.

TakahashiK.GuY.MayP. J.NewlandsS. D.DeAngelisG. C.AngelakiD. E. (2007). Multimodal coding of three-dimensional rotation and translation in area MSTd: comparison of visual and vestibular selectivityJ. Neurosci. 2797429768.

UesakiM.AshidaH. (2015). Optic-flow selective cortical sensory regions associated with self-reported states of vectionFront. Psychol. 6775. DOI:10.3389/fpsyg.2015.00775.

UpadhyayU. D.PageW. K.DuffyC. J. (2000). MST responses to pursuit across optic flow with motion parallaxJ. Neurophysiol. 84818826.

ValkoY.LewisR. F.PriesolA. J.MerfeldD. M. (2012). Vestibular labyrinth contributions to human whole-body motion discriminationJ. Neurosci. 321353713542.

VallarG.SterziR.BottiniG.CappaS.RusconiM. L. (1990). Temporary remission of left hemianesthesia after vestibular stimulation. A sensory neglect phenomenonCortex 26123131.

Van den BergA. V. (1993). Perception of headingNature 365(6446) 497498.

von HolstE.MittelstaedtH. (1950). Das ReafferenzprinzipNaturwissenschaften 37464476.

WallM. B.SmithA. T. (2008). The representation of egomotion in the human brainCurr. Biol. 18191194.

WallaceM. T.RobersonG.HairstonW. D.SteinB. E.VaughanJ. W.SchirilloJ. A. (2004). Unifying multisensory signals across time and spaceExp. Brain Res. 158252258.

WarrenW. H.HannonD. J. (1988). Direction of self-motion is perceived from optical flowNature 336(6195) 162163.

WarrenW. H.HannonD. J. (1990). Eye movements and optical flowJ. Opt. Soc. Am. A 7160169.

WatanabeJ.HayashiS.KajimotoH.TachiS.NishidaS. (2007). Tactile motion aftereffects produced by appropriate presentation for mechanoreceptorsExp. Brain Res. 180577582.

WexlerM.PaneraiF.LamouretI.DroulezJ. (2001). Self-motion and the perception of stationary objectsNature 409(6816) 8588.

YuC. P.PageW. K.GaborskiR.DuffyC. J. (2010). Receptive field dynamics underlying MST neuronal optic flow selectivityJ. Neurophysiol. 10327942807.

ZhangT.BrittenK. H. (2011). Parietal area VIP causally influences heading perception during pursuit eye movementsJ. Neurosci. 3125692575.

ZhangT.HeuerH. W.BrittenK. H. (2004). Parietal area VIP neuronal responses to heading stimuli are encoded in head-centered coordinatesNeuron 429931001.

zu EulenburgP.BaumgärtnerU.TreedeR.-D.DieterichM. (2013). Interoceptive and multimodal functions of the operculo-insular cortex: tactile, nociceptive and vestibular representationsNeuroImage 837586.


  • View in gallery

    Retinal flow fields and neuronal responses. The panels in the left column depict exemplary retinal flow fields seen by an observer moving across a ground plane. Here, heading as indicated by the arrow was always to the left, but the simulated eye movements differed, leading to markedly different retinal flow across the three eye movement conditions. Monkeys had to fixate a central target () in all cases. The right columns show the time resolved responses of a neuron from area VIP for all three different headings (as indicated by the post-stimulus time histograms, PSTHs, color traces) and the three different eye movement conditions. This neuron responded strongly for self motion to the left (indicated by the green response curves), irrespective of the underlying simulated eye movement. Medium responses were observed for movement straight ahead (blue response curves), while movement to the right (red response curves) induced in all three eye movement conditions inhibition of the ongoing activity with respect to baseline (Mann–Whitney rank test, p<0.001). Responses for a given self motion direction did not differ across eye movement conditions (ANOVA on ranks, 2 df, p>0.05 for each of the three heading directions).

  • View in gallery

    Population responses. Panels A and B depict the distribution of rank-order triplets for areas MST (light grey, online version: orange) and VIP (medium grey, online version: green) as raw numbers (A) and in percent (B). Data from 64 MST neurons and 48 VIP neurons resulted in a set of n=64×3+48×3=336 rank-order triplets. Peak average discharges for identical heading directions (i.e., triplet ⟨1-1-1⟩) were observed in 60/336 = 17.9% of the cases. Weakest discharges for identical headings (⟨3-3-3⟩) occurred in 62/336 = 18.4% of the cases and medium discharges for identical headings (⟨2-2-2⟩) occurred in 44/336 = 13.1% of the cases. Each of these rank-order triplets indicates the invariance of a neuron’s heading response with respect to simulated eye movements. Considering the occurrence of these three triplets together, eye movement invariances were found in 168/336 = 49.4% of the cases. This proportion was significantly larger than would have been expected, if responses for a given heading direction across the different eye movement conditions had been independent (χ2=117.5, 1 df, p<0.001). Importantly, we found this overrepresentation of eye movement invariance in each of the two areas individually: in 82/192 = 42.7% of the cases in area MST (χ2=49.4, 1 df, p<0.001), and in 84/144 = 58.3% of the cases in area VIP (χ2=72.6, 1 df, p<0.001). Panels C and D depict the time courses of the establishment of eye movement invariance (⟨1-1-1⟩) and of a random response scheme (⟨1-2-3⟩). Panel E depicts the distribution of the rank-order quadruplets. In area MST, we found a coincidence of heading preferences for simulated and real eye movements (rank-order quadruplet ⟨1-1-1-1⟩) in 20 out of 147 cases (13.6%). The weakest response for a given heading direction (⟨3-3-3-3⟩) was observed in 19/147 = 12.9% of the cases. A medium response for a given heading (⟨2-2-2-2⟩) was found in 13/147 = 8.8% of the cases. Each observed number of cases of the three eye movement invariances occurred significantly more often than would have been expected if tunings had been distributed uniformly (smallest χ2 value=8.5, 1 df, p<0.005). A similar result was obtained from the population of VIP neurons. Here, response peaks for a given heading direction were observed in 16/111 = 14.4% of the cases. Medium and weakest responses for a given heading direction were found in 18/111 = 16.2% and 10/111 = 9% of the cases, respectively. Again, each of these proportions differed significantly from a uniform distribution (smallest χ2 value=5.64, 1 df, p<0.03). Panel F shows the distribution of the response modulation for simulated (abscissa) and real (ordinate) eye movements. For details see main text. This figure is published in colour in the online version.

  • View in gallery

    Crossmodal self motion aftereffects. (A) Illustration of the experimental protocol. During the adaptation phase (left), subjects were presented with an optic flow simulating either forward or backward self motion at constant velocity (3 m/s). In the test phase (right), subjects experienced a 2-s passive linear fore–aft translation with a Gaussian velocity profile (vestibular-only test), and subsequently indicated the perceived movement direction. In a control condition, a visual-only test stimulus (far right) composed of expanding or contracting optic flow was used instead, and subjects indicated the direction of optic flow. Different adaption conditions were run in separate blocks of 50 trials each: baseline (no adaptation), forward (15 s), backward (15 s), as well as three forward blocks with shorter adapter durations of 7.5, 3.75, or 1.5 s. (B) Mean aftereffect (PSE) across subjects (n=20) following forward and backward adaptation with 15 s duration compared with the no-adapter baseline. (C) Mean aftereffect (expressed as forward minus baseline PSE) as a function of adapter duration (n=17). Subjects additionally rated their vection on a scale of 1 to 7, with 1 representing perception of object motion only and 7 representing perception of self motion only. Gray dots indicate mean subjective ratings. (D) Crossmodal and visual-only aftereffects (expressed as forward or backward minus baseline PSE) are uncorrelated (r=0.003; p=0.98) (n=15). To allow for comparison, backward aftereffects are multiplied by −1. All error bars show SE.

  • View in gallery

    Experimental setup and results. Participants are seated in the motion platform with a screen in front of them. Upper panel: Experiment 1. Lower panel: Experiment 2. Participants show optimal visuo-vestibular integration, their bimodal threshold (grey square, online version: red) being lower than their unimodal visual (open square, online version: blue) and vestibular (black) thresholds, and not different from the predictions of a Bayesian optimal integration model (open circle, online: red circle). This figure is published in colour in the online version.

  • View in gallery

    (A) Experimental setup testing for the effects of natural vestibular stimulation on tactile detection. A congruent trial is depicted: rotation direction corresponds to the side of tactile stimulation. (B) Results. Independently of congruency tactile sensitivity was improved during rotation in comparison to a no-rotation baseline. This figure is published in colour in the online version.

  • View in gallery

    (A) Cortical regions of interest studied. The figure shows an inflated representation of the right cerebral hemisphere of one individual with the locations and extents of hMT, hMST, hVIP, V6 and CSv indicated as grey-scaled (online version: coloured) overlays. In each case, BOLD activity elicited by the visual localizer that was used to define the region is shown in a slice through the brain of the same participant. A corresponding set of visual areas is present in the left hemisphere (not shown). Modified from Smith et al. (2012). (B) A ‘cut-out’ section of the flattened grey matter representation from each hemisphere of one participant, centered on the MT complex (dashed white line is the superior temporal sulcus). Vestibular activity (grey-scaled, online version: orange/yellow) is superimposed, together with the outlines of hMT (light grey, online version: green) and hMST (dark grey, online version: magenta) as defined with a visual localizer. Vestibular activity is apparent in hMST but not hMT and is confined to the anterior portion of hMST. Modified from Smith et al. (2012). (C) Visual stimulus used by Billington and Smith (2015). A circular patch of white dots appears to rotate during GVS. Physical rotation of the patch on the screen was used to null this illusory motion. (D) MVPA results for classifying the temporal phase of sinusoidal visual and vestibular rotations in the roll plane for five cortical visual areas. Chance performance is shown, along with the 95th percentile obtained from permutation testing as an indicator of statistical significance. Modified from Billington and Smith et al. (2015). This figure is published in colour in the online version.

  • View in gallery

    Organization of human vestibular cortex in lateral sulcus (also called sylvian fissure). Shown are activations in a sample participant (left hemisphere) during stimulation with visual motion and caloric vestibular cues (p<0.001, uncorrected). (A) Visual motion stimulation shows significant activations in the posterior insular cortex area (PIC) in the posterior end of the lateral sulcus. In addition to PIC, other motion-sensitive regions in visual and parietal cortex respond well to visual motion stimuli. (B) Caloric stimulation elicits activations in the vestibular network in lateral sulcus, including the putative center of cortical vestibular processing, the parieto-insular vestibular cortex area (PIVC). Activations during caloric stimulation are also evident in area PIC, suggesting that PIC is part of both, the vestibular and the visual motion processing networks After Frank et al., 2014 (with permission of the publisher) and Frank et al., submitted for publication. This figure is published in colour in the online version.

Index Card

Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 14 14 4
Full Text Views 18 18 9
PDF Downloads 7 7 4
EPUB Downloads 3 3 0