A Potential Role of Auditory Induced Modulations in Primary Visual Cortex

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


A biologically relevant event is normally the source of multiple, typically correlated, sensory inputs. To optimize perception of the outer world, our brain combines the independent sensory measurements into a coherent estimate. However, if sensory information is not readily available for every pertinent sense, the brain tries to acquire additional information via covert/overt orienting behaviors or uses internal knowledge to modulate sensory sensitivity based on prior expectations. Cross-modal functional modulation of low-level auditory areas due to visual input has been often described; however, less is known about auditory modulations of primary visual cortex. Here, based on some recent evidence, we propose that an unexpected auditory signal could trigger a reflexive overt orienting response towards its source and concomitantly increase the primary visual cortex sensitivity at the locations where the object is expected to enter the visual field. To this end, we propose that three major functionally specific pathways are employed in parallel. A stream orchestrated by the superior colliculus is responsible for the overt orienting behavior, while direct and indirect (via higher-level areas) projections from A1 to V1 respectively enhance spatiotemporal sensitivity and facilitate object detectability.

Multisensory Research

A Journal of Scientific Research on All Aspects of Multisensory Processing



AdamsD. L.HortonJ. C. (2003). A precise retinotopic map of primate striate cortex generated from the representation of angioscotomas, J. Neurosci. 23, 37713789.

BaizerJ. S.LockT. M.YouakimM. (1997). Projections from the claustrum to the prelunate gyrus in the monkey, Exp. Brain Res. 113, 564568.

BaroneP.BatardiereA.KnoblauchK.KennedyH. (2000). Laminar distribution of neurons in extrastriate areas projecting to visual areas V1 and V4 correlates with the hierarchical rank and indicates the operation of a distance rule, J. Neurosci. 20, 32633281.

BeneventoL. A.RezakM. (1976). The cortical projections of the inferior pulvinar and adjacent lateral pulvinar in the rhesus monkey (Macaca mulatta): an autoradiographic study, Brain Res. 108, 124.

BlankH.AnwanderA.von KriegsteinK. (2011). Direct structural connections between voice- and face-recognition areas, J. Neurosci. 31, 1290612915.

BologniniN.SennaI.MaravitaA.Pascual-LeoneA.MerabetL. B. (2010). Auditory enhancement of visual phosphene perception: the effect of temporal and spatial factors and of stimulus intensity, Neurosci. Lett. 477, 109114.

CallawayE. M. (1998). Local circuits in primary visual cortex of the macaque monkey, Annu. Rev. Neurosci. 21, 4774.

CalvertG. A. (2001). Crossmodal processing in the human brain: insights from functional neuroimaging studies, Cereb. Cortex 11, 11101123.

CalvertG. A.BrammerM. J.BullmoreE. T.CampbellR.IversenS. D.DavidA. S. (1999). Response amplification in sensory-specific cortices during crossmodal binding, NeuroReport 10, 26192623.

CarlisleN. B.AritaJ. T.PardoD.WoodmanG. F. (2011). Attentional templates in visual working memory, J. Neurosci. 31, 93159322.

ChennuS.NoreikaV.GueorguievD.BlenkmannA.KochenS.IbáñezA.OwenA. M.BekinschteinT. A. (2013). Expectation and attention in hierarchical auditory prediction, J. Neurosci. 33, 1119411205.

ChklovskiiD. B.KoulakovA. A. (2004). Maps in the brain: what can we learn from them? Annu. Rev. Neurosci. 27, 369392.

ClavagnierS.FalchierA.KennedyH. (2004). Long-distance feedback projections to area V1: implications for multisensory integration, spatial awareness, and visual consciousness, Cogn. Affect. Behav. Neurosci. 4, 117126.

CorneilB. D.MunozD. P. (2014). Overt responses during covert orienting, Neuron 82, 12301243.

ErnstM. O.BanksM. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion, Nature 415(6870), 429433.

FalchierA.ClavagnierS.BaroneP.KennedyH. (2002). Anatomical evidence of multimodal integration in primate striate cortex, J. Neurosci. 22, 57495759.

FellemanD. J.Van EssenD. C. (1991). Distributed hierarchical processing in the primate cerebral cortex, Cereb. Cortex 1, 147.

FetschC. R.DeAngelisG. C.AngelakiD. E. (2013). Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons, Nat. Rev. Neurosci. 14, 429442.

FreeseJ. L.AmaralD. G. (2005). The organization of projections from the amygdala to visual cortical areas TE and V1 in the macaque monkey, J. Comp. Neurol. 486, 295317.

FristonK. (2005). A theory of cortical responses, Philos. Trans. R. Soc. Lond. B Biol. Sci. 360(1456), 815836.

GhazanfarA. A.SchroederC. E. (2006). Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278285.

GhazanfarA. A.MaierJ. X.HoffmanK. L.LogothetisN. K. (2005). Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex, J. Neurosci. 25, 50045012.

GhazanfarA. A.ChandrasekaranC.LogothetisN. K. (2008). Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys, J. Neurosci. 28, 44574469.

HensonD. B. (1993). Visual Fields, 1st edn. Oxford Medical Publications, Oxford, UK.

HubelD. H.WieselT. N. (1968). Receptive fields and functional architecture of monkey striate cortex, J. Physiol. 195, 215243.

IssaN. P.TrepelC.StrykerM. P. (2000). Spatial frequency maps in cat visual cortex, J. Neurosci. 20, 85048514.

JayM. F.SparksD. L. (1984). Auditory receptive fields in primate superior colliculus shift with changes in eye position, Nature 309(5966), 345347.

KaasJ. H.HackettT. A. (2000). Subdivisions of auditory cortex and processing streams in primates, Proc. Natl Acad. Sci. USA 97, 1179311799.

KanwisherN. (2010). Functional specificity in the human brain: a window into the functional architecture of the mind, Proc. Natl Acad. Sci. USA 107, 1116311170.

KaraP.BoydJ. D. (2009). A micro-architecture for binocular disparity and ocular dominance in visual cortex, Nature 458(7238), 627631.

KayserC.PetkovC. I.AugathM.LogothetisN. K. (2007). Functional imaging reveals visual modulation of specific fields in auditory cortex, J. Neurosci. 27, 18241835.

KayserC.PetkovC. I.LogothetisN. K. (2008). Visual modulation of neurons in auditory cortex, Cereb. Cortex 18, 15601574.

KlierE. M.WangH.CrawfordJ. D. (2003). Three-dimensional eye-head coordination is implemented downstream from the superior colliculus, J. Neurophysiol. 89, 28392853.

KnillD. C.PougetA. (2004). The Bayesian brain: the role of uncertainty in neural coding and computation, Trends Neurosci. 27, 712719.

KokP.JeheeJ. F. M.De LangeF. P. (2012). Less is more: expectation sharpens representations in the primary visual cortex, Neuron 75, 265270.

KokP.BrouwerG. J.Van GervenM. A.De LangeF. P. (2013). Prior expectations bias sensory representations in visual cortex, J. Neurosci. 33, 1627516284.

KokP.FailingM. F.De LangeF. P. (2014). Prior expectations evoke stimulus templates in the primary visual cortex, J. Cogn. Neurosci. 26, 15461554.

KrauzlisR. J.ListonD.CarelloC. D. (2004). Target selection and the superior colliculus: goals, choices and hypotheses, Vis. Res. 44, 14451451.

KustovA. A.RobinsonD. L. (1996). Shared neural control of attentional shifts and eye movements, Nature 384(6604), 7477.

LakatosP.ChenC. M.O’ConnellM. N.MillsA.SchroederC. E. (2007). Neuronal oscillations and multisensory interaction in primary auditory cortex, Neuron 53, 279292.

LakatosP.KarmosG.MehtaA. D.UlbertI.SchroederC. E. (2008). Entrainment of neuronal oscillations as a mechanism of attentional selection, Science 320(5872), 110113.

LakatosP.O’ConnellM. N.BarczakA.MillsA.JavittD. C.SchroederC. E. (2009). The leading sense: supramodal control of neurophysiological context by attention, Neuron 64, 419430.

LandismanC. E.Ts’oD. Y. (2002). Color processing in macaque striate cortex: relationships to ocular dominance, cytochrome oxidase, and orientation, J. Neurophysiol. 87, 31263137.

LeeJ.GrohJ. M. (2012). Auditory signals evolve from hybrid- to eye-centered coordinates in the primate superior colliculus, J. Neurophysiol. 108, 227242.

LeungJ.AlaisD.CarlileS. (2008). Compression of auditory space during rapid head turns, Proc. Natl Acad. Sci. USA 105, 64926497.

LuH. D.RoeA. W. (2008). Functional organization of color domains in V1 and V2 of macaque monkey revealed by optical imaging, Cereb. Cortex 18, 516533.

LupyanG.WardE. J. (2013). Language can boost otherwise unseen objects into visual awareness, Proc. Natl Acad. Sci. USA 110, 1419614201.

MacalusoE.DriverJ. (2005). Multisensory spatial interactions: a window onto functional integration in the human brain, Trends Neurosci. 28, 264271.

McDonaldJ. J.Teder-SälejärviW. A.HillyardS. A. (2000). Involuntary orienting to sound improves visual perception, Nature 407(6806), 906908.

McManusJ. N. J.LiW.GilbertC. D. (2011). Adaptive shape processing in primary visual cortex, Proc. Natl Acad. Sci. USA 108, 97399746.

McPeekR. M.KellerE. L. (2004). Deficits in saccade target selection after inactivation of superior colliculus, Nat. Neurosci. 7, 757763.

MeredithM. A.SteinB. E. (1986). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration, J. Neurophysiol. 56, 640662.

MeredithM. A.ClemoH. R.SteinB. E. (1991). Somatotopic component of the multisensory map in the deep laminae of the cat superior colliculus, J. Comp. Neurol. 312, 353370.

MiddlebrooksJ. C.KnudsenE. I. (1984). A neural code for auditory space in the cat’s superior colliculus, J. Neurosci. 4, 26212634.

MorrellF. (1972). Visual system’s view of acoustic space, Nature 238(5358), 4446.

OhkiK.ChungS.Ch’ngY. H.KaraP.ReidR. C. (2005). Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex, Nature 433(7026), 597603.

OliverD. L.BeckiusG. E.BishopD. C.LoftusW. C.BatraR. (2003). Topography of interaural temporal disparity coding in projections of medial superior olive to inferior colliculus, J. Neurosci. 23, 74387449.

PosnerM. I.NissenM. J.OgdenW. C. (1978). Attended and unattended processing modes: the role of set for spatial location, in: Modes of Perceiving and Processing Information, PickH.SaltzmannE. (Eds), pp.  137157. John Wiley and Sons Inc., Hillsdale, NJ, USA.

RaoR. P.BallardD. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects, Nat. Neurosci. 2, 7987.

ReserD. H. (2000). Binaural interactions in primary auditory cortex of the awake macaque, Cereb. Cortex 10, 574584.

RocklandK. S.OjimaH. (2003). Multisensory convergence in calcarine visual areas in macaque monkey, Int. J. Psychophysiol. 50, 1926.

RolkeB.HofmannP. (2007). Temporal uncertainty degrades perceptual processing, Psychonom. Bull. Rev. 14, 522526.

RomeiV.MurrayM. M.MerabetL. B.ThutG. (2007). Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: implications for multisensory interactions, J. Neurosci. 27, 1146511472.

RomeiV.MurrayM. M.CappeC.ThutG. (2009). Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds, Curr. Biol. 19, 17991805.

RomeiV.GrossJ.ThutG. (2012). Sounds reset rhythms of visual cortex and corresponding human visual perception, Curr. Biol. 22, 807813.

SaenzM.LangersD. R. M. (2014). Tonotopic mapping of human auditory cortex, Hearing Res. 307, 4252.

SchroederC. E.FoxeJ. (2005). Multisensory contributions to low-level, ‘unisensory’ processing, Curr. Opin. Neurobiol. 15, 454458.

SchwarzD. A.LebedevM. A.HansonT. L.DimitrovD. F.LehewG.MeloyJ.RajangamS.SubramanianV.IfftP. J.LiZ.RamakrishnanA.TateA.ZhuangK. Z.NicolelisM. A. (2014). Chronic, wireless recordings of large-scale brain activity in freely moving rhesus monkeys, Nat. Meth. 11, 670676.

ShamsL.KimR. (2010). Crossmodal influences on visual perception, Phys. Life Rev. 7, 269284.

ShamsL.KamitaniY.ShimojoS. (2000). Illusions. What you see is what you hear, Nature 408(6814), 788.

SharmaJ.SugiharaH.KatzY.SchummersJ.TenenbaumJ.SurM. (in press). Spatial attention and temporal expectation under timed uncertainty predictably modulate neuronal responses in monkey V1, Cereb. Cortex.

SparksD. L. (1999). Conceptual issues related to the role of the superior colliculus in the control of gaze, Curr. Opin. Neurobiol. 9, 698707.

SpragueJ. M. (1996). Neural mechanisms of visual orienting responses, Prog. Brain Res. 112, 115.

SteinB. E.MeredithM. A. (1993). The Merging of the Senses. MIT Press, Cambridge, MA, USA.

SteinB. E.MeredithM. A.WallaceM. T. (1993). The visually responsive neuron and beyond: multisensory integration in cat and monkey, Prog. Brain Res. 95, 7990.

SteinB. E.StanfordT. R.RowlandB. A. (2009). The neural basis of multisensory integration in the midbrain: its organization and maturation, Hearing Res. 258, 415.

SteinB. E.StanfordT. R.RowlandB. A. (2014). Development of multisensory integration from the perspective of the individual neuron, Nat. Rev. Neurosci. 15, 520535.

SummerfieldC.EgnerT. (2009). Expectation (and attention) in visual cognition, Trends Cogn. Sci. 13, 403409.

TalsmaD.SenkowskiD.Soto-FaracoS.WoldorffM. G. (2010). The multifaceted interplay between attention and multisensory integration, Trends Cogn. Sci. 14, 400410.

Van der BurgE.TalsmaD.OliversC. N.HickeyC.TheeuwesJ. (2011). Early multisensory interactions affect the competition among multiple visual objects, NeuroImage 55, 12081218.

VanduffelW.ZhuQ.OrbanG. A. (2014). Monkey cortex through fMRI glasses, Neuron 83, 533550.

VaneyD. I.SivyerB.TaylorW. R. (2012). Direction selectivity in the retina: symmetry and asymmetry in structure and function, Nat. Rev. Neurosci. 13, 194208.

VroomenJ.De GelderB. (2000). Sound enhances visual perception: cross-modal effects of auditory organization on vision, J. Exp. Psychol. Hum. Percept. Perform. 26, 15831590.

WangY.CelebriniS.TrotterY.BaroneP. (2008). Visuo-auditory interactions in the primary visual cortex of the behaving monkey: electrophysiological evidence, BMC Neurosci. 9, 79.

WatkinsS.ShamsL.TanakaS.HaynesJ. D.ReesG. (2006). Sound alters activity in human V1 in association with illusory visual perception, NeuroImage 31, 12471256.

WatkinsS.ShamsL.JosephsO.ReesG. (2007). Activity in human V1 follows multisensory perception, NeuroImage 37, 572578.

WeiT.LiangX.HeY.ZangY.HanZ.CaramazzaA.BiY. (2012). Predicting conceptual processing capacity from spontaneous neuronal activity of the left middle temporal gyrus, J. Neurosci. 32, 481489.

WernerS.NoppeneyU. (2010). Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization, J. Neurosci. 30, 26622675.

WhiteB. J.MunozD. P. (2011). The superior colliculus, in: The Oxford Handbook of Eye Movements, LiversedgeS. P.GilchristI. D.EverlingS. (Eds), pp.  195214. Oxford University Press, Oxford, UK.

WittenI. B.KnudsenE. I. (2005). Why seeing is believing: merging auditory and visual worlds, Neuron 48, 489496.

WurtzR. H.AlbanoJ. E. (1980). Visual-motor function of the primate superior colliculus, Annu. Rev. Neurosci. 3, 189226.


  • Schematic representation of the field of hearing (FoH, light gray) and of the field of view (FoV, dark grey) of humans. While the normal FoH has a global scope, encompassing every angle around the head, the FoH is focal, covering a wedge of approximately 95° away from the nose and 60° towards the nose in the azimuthal plane and 75° downward and 60° upward for each eye. Outside the visual wedge, the two sensory fields do not overlap.

    View in gallery
  • Simplified plots for the main suggested anatomical pathways involved in our proposal. An auditory spatial cue conveying object identity would be able to trigger three functional streams of processing to facilitate the overt orienting response, spatial sensitivity and object selectivity. (a) After the cue reaches the cochlea, the auditory information is sent to the cochlear nuclei (CN) in the medulla, subsequently projected to the olivary nuclei (ON), where spatial features such azimuthal distance are calculated and the outputs are forwarded to the inferior colliculus (IC). At this point, one stream is sent to the superior colliculus (SC), where the overlapping cross-modal sensory and motor receptive fields promote sensory reflexive orientation to the expected location of the event. In parallel to that, auditory information flows through the medial geniculate nucleus (MGN) to the auditory cortex (AC). (b) In the primary auditory cortices, the partially processed auditory information would be once more split to play distinct but complementary functional roles: bottom-up enhancement of visual spatial sensitivity and top-down facilitation of visual detectability of an expected object, eventually converging to low-level visual areas, such as the primary visual cortex (V1). Monosynaptic A1–V1 projections would be responsible for the former function, while top-down processing pathways involving the inferior temporal cortex (IT) would associate the identity of the object conveyed by the sound with the visual memory of the same object, creating a visual expectation for it. The object-specific expectation would then be able to modulate the receptive fields of the visual cortices including V1, optimizing its visual identification. Cs: central sulcus; ips: intraparietal sulcus.

    View in gallery


Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 19 19 15
Full Text Views 10 10 10
PDF Downloads 3 3 3
EPUB Downloads 0 0 0