The Skin as a Medium for Sensory Substitution

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


The last 50 years or so has seen great optimism concerning the potential of sensory substitution and augmentation devices to enhance the lives of those with (or even those without) some form of sensory loss (in practice, this has typically meant those who are blind or suffering from low vision). One commonly discussed solution for those individuals who are blind has been to use one of a range of tactile–visual sensory substitution systems that represent objects captured by a camera as outline images on the skin surface in real-time (what Loomis, Klatzky and Giudice, 2012, term general-purpose sensory substitution devices). However, despite the fact that touch, like vision, initially codes information spatiotopically, I would like to argue that a number of fundamental perceptual, attentional, and cognitive limitations constraining the processing of tactile information mean that the skin surface is unlikely ever to provide such general-purpose sensory substitution capabilities. At present, there is little evidence to suggest that the extensive cortical plasticity that has been demonstrated in those who have lost (or never had) a sense can do much to overcome the limitations associated with trying to perceive high rates of spatiotemporally varying information presented via the skin surface (no matter whether that surface be the back, stomach, forehead, or tongue). Instead, the use of the skin will likely be restricted to various special-purpose devices that enable specific activities such as navigation, the control of locomotion, pattern perception, etc.

The Skin as a Medium for Sensory Substitution

in Multisensory Research



AbramsM.WintersD. (2003). Can you see with your tongue? The brain is so adaptable some researchers now think that any of the five senses can be rewired Discover 6th Jan. available from (accessed 27 June 2014).

AitkinL. M. (1976). Tonotopic organisation at higher levels of the auditory pathway in: International Reviews of Physiology Neurophysiology IIIVol. 10PorterR. (Ed.) pp.  249279. University Park PressBaltimore, MD, USA.

AlliusiE. A. (1960). On optimizing cutaneous communication: a respectful supplement to some adventures in tactile literacy in: Symposium on Cutaneous Sensitivity G. R. Hawkes (Ed.) pp. 39–45. U.S. Army Med. Res. Lab. Rep. No. 424.

AlsmithA. J. T.LongoM. R. (2014). Where exactly am I? Self-location judgements distribute between head and torsoConscious. Cogn. 247074.

ArnoldG.SpenceC.AuvrayM. (submitted). Which perspective is adopted when interpreting tactile stimuli? A review of the tactile symbol recognition paradigmPsychol. Bull.

AuvrayM.GallaceA.Hartcher-O’BrienJ.TanH. Z.SpenceC. (2008). Tactile and visual distractors induce change blindness for tactile stimuli presented on the fingertipsBrain Res. 1213111119.

Bach-y-RitaP. (1972). Brain Mechanisms in Sensory Substitution. Academic PressNew York, NY, USA.

Bach-y-RitaP. (2002). Sensory substitution and qualia in: Vision and MindNoëA.ThompsonE. (Eds) pp.  497514. MIT PressCambridge, MA, USA.

Bach-y-RitaP.CollinsC. C.SaundersF. A.WhiteB.ScaddenL. (1969). Vision substitution by tactile image projectionNature 221963964.

Bach-y-RitaP.KercelS. W. (2003). Sensory substitution and the human machine interfaceTrends Cogn. Sci. 7541546.

BertossaF.BesaM.FerrariR.FerriF. (2008). Point zero: a phenomenological inquiry into the seat of consciousnessPercept. Mot. Skills 107323335.

BieverC. (2005). The touchy-feely side of telecoms New Scientist 27th Feb. available from (accessed 27 June 2014).

CholewiakR. W.CraigJ. C. (1984). Vibrotactile pattern recognition and discrimination at several body sitesPercept. Psychophys. 35503514.

CohenL. G.CelnikP.Pascual-LeoneA.CorwellB.FaizL.DambrosiaJ.HondaM.SadatoN.GerloffC.CatalaM. D.HallettM. (1997). Functional relevance of cross-modal plasticity in blind humansNature 389180183.

CollinsC. C. (1970). Tactile television-mechanical and electrical image projectionIEEE Trans. Hum. Mach. Syst. MMS-116571.

CollinsC. C. (1985). On mobility aids for the blind in: Electronic Spatial Sensing for the BlindWarrenD. H.StelowE. R. (Eds) pp.  3564. Martinus NijhofBoston, MA, USA.

CorcoranD. W. J. (1977). The phenomena of the disembodied eye or is it a matter of personal geography? Perception 6247253.

CraigJ. C. (1974). Pictorial and abstract cutaneous displays in: Conference on Cutaneous Communication Systems and Devices F. A. Geldard (Ed.) pp. 78–83. Psychonomic Society Austin TX USA.

CraigJ. C. (1977). Vibrotactile pattern perception: extraordinary observersScience 196450452.

CraigJ. C.EvansP. M. (1987). Vibrotactile masking and the persistence of tactual featuresPercept. Psychophys. 42309317.

CraigJ. C.SherrickC. E. (1982). Dynamic tactile displays in: Tactile Perception: A SourcebookSchiffW.FoulkeE. (Eds) pp.  209233. Cambridge University PressCambridge, UK.

DiazA.BarrientosA.JacobsD. M.TraviesoD. (2012). Action contingent vibrotactile flow facilitates the detection of ground level obstacles with a partly virtual sensory substitution deviceHum. Mov. Sci. 3115711584.

FellemanD. J.Van EssenD. C. (1991). Distributed hierarchical processing in primate cerebral cortexCereb. Cortex 1147.

FringsC.AmmendtA.SpenceC. (2011). When seeing doesn’t matter: tactile negative priming is equally strong in the blind and the sightedJ. Exp. Psychol. Hum. Percept. Perform. 3711741181.

GalinskyA. D.MageeJ. C.InesiM. E.GruenfeldD. H. (2006). Power and perspectives not takenPsychol. Sci. 1710681074.

GallaceA.NgoM. K.SulaitisJ.SpenceC. (2012). Multisensory presence in virtual reality: possibilities and limitations in: Multiple Sensorial Media Advances and Applications: New Developments in MulSeMediaGhineaG.AndresF.GulliverS. (Eds) pp.  140. IGI GlobalHershey, PA, USA.

GallaceA.SpenceC. (2011). To what extent do Gestalt grouping principles influence tactile perception? Psychol. Bull. 137538561.

GallaceA.SpenceC. (2014). In Touch with the Future: The Sense of Touch from Cognitive Neuroscience to Virtual Reality. Oxford University PressOxford, UK.

GallaceA.TanH. Z.SpenceC. (2006a). The failure to detect tactile change: a tactile analog of visual change blindnessPsychonom. Bull. Rev. 13300303.

GallaceA.TanH. Z.SpenceC. (2006b). Numerosity judgments for tactile stimuli distributed over the body surfacePerception 35247266.

GallaceA.TanH. Z.SpenceC. (2007). The body surface as a communication system: the state of the art after 50 yearsPresence (Camb.) 16655676.

GallaceA.TanH. Z.SpenceC. (2008). Can tactile stimuli be subitised? An unresolved controversy within the literature on numerosity judgmentsPerception 37782800.

GallaceA.ZeedenS.RöderB.SpenceC. (2010). Lost in the move? Secondary task performance impairs the detection of tactile change on the body surfaceConscious. Cogn. 19215229.

GeldardF. A. (1960). Some neglected possibilities of communicationScience 13115831588.

GeldardF. A. (Ed.) (1974). Cutaneous Communication Systems and Devices. Psychonomic SocietyAustin, TX, USA.

GouldW. R.VierckC. J.LuckM. M. (1979). Cues supporting recognition of the orientation or direction of movement of tactile stimuli in: Sensory Functions of the Skin of HumansKenshaloD. R. (Ed.) pp.  6378. Plenum PressNew York, NY, USA.

GreggL. W. (1960). Some coding problems in the design of a cutaneous communication channelU.S. Army Medical Research Reports 42485102.

GuarnieroG. (1974). Experience of tactile visionPerception 3101104.

GuarnieroG. (1977). Tactile vision: a personal viewJournal of Visual Impairment and Blindness 71125130.

HawkesG. R. (Ed.) (1960). Symposium on Cutaneous Sensitivity Rep. U.S. Army Med. Res. Lab. 424 1–165.

HennessyJ. R. (1966). Cutaneous sensitivity communicationsHum. Fact. 8463469.

HillJ. W. (1971). Processing of tactual and visual point stimuli sequentially presented at high ratesJ. Exp. Psychol. 88340348.

HillJ. W.BlissJ. C. (1968). Perception of sequentially presented tactile point stimuliPercept. Psychophys. 4289295.

HoC.SpenceC. (2008). The Multisensory Driver: Implications for Ergonomic Car Interface Design. Ashgate PublishingAldershot, UK.

HonigZ. (2013). Triposo Travel Belt vibrates to help you locate attractions (video) available from (accessed 13 September 2013).

HooverR. E. (1950). The cane as a travel aid in: BlindnessZahlP. A. (Ed.) pp.  353365. Princeton University PressPrinceton, MA, USA.

HöttingK.RöderB. (2004). Hearing cheats touch, but less in congenitally blind than in sighted individualsPsychol. Sci. 156064.

HumphriesC.LiebenthalE.BinderJ. R. (2010). Tonotopic organization of human auditory cortexNeuroImage 5012021211.

JuravleG.DeubelH.SpenceC. (2011). Attention and suppression affect tactile perception in reach-to-grasp movementsActa Psychol. 138302310.

JuravleG.DeubelH.TanH. Z.SpenceC. (2010). Changes in tactile sensitivity over the timecourse of a goal-directed movementBehav. Brain Res. 208391401.

JuravleG.SpenceC. (2011). Juggling reveals a decisional component to tactile suppressionExp. Brain Res. 2138799.

KaczmarekK. A.Bach-y-RitaP. (1995). Tactile displays in: Virtual Environments and Advanced Interface DesignBarfieldW.FurnessT. A. (Eds) pp.  349414. Oxford University PressNew York, NY, USA.

KajimotoH.KannoY.TachiS. (2006). Forehead electro-tactile display for vision substitution Proc. Eurohaptics 1–6.

KotovskyK.BlissJ. C. (1963). Tactual representation of visual informationIEEE Trans Mil. Electron. MIL-7108113.

KrechD.CrutchfieldR. S. (1958). Elements of Psychology. A. A. KnopfNew York, NY, USA.

KupersR.PtitoM. (2004). Seeing’ through the tongue: cross-modal plasticity in the congenitally blindInt. Congr. Ser. 12707984.

LakatosS.ShepardR. N. (1997). Time-distance relations in shifting attention between locations on one’s bodyPercept. Psychophys. 59557566.

LavieN. (2005). Distracted and confused?: selective attention under loadTrends Cogn. Sci. 97582.

LeeJ.-H.SpenceC. (2008). Feeling what you hear: task-irrelevant sounds modulates tactile perception delivered via a touch screenJ. Multisens. User Interfaces 2145156.

LeggeG. E.MadisonC.VaughnB. N.CheongA. M. Y.MillerJ. C. (2008). Retention of high tactile acuity thoughout the lifspan in blindnessAtten. Percept. Psychophys. 7014711488.

LimanowskiJ.HechtH. (2011). Where do we stand on locating the self? Psychology 2312317.

LinvillJ. G.BlissJ. C. (1966). A direct translation reading aid for the blindProc. IEEE 544051.

LökenL. S.WessbergJ.MorrisonI.McGloneF.OlaussonH. (2009). Coding of pleasant touch by unmyelinated afferents in humansNat. Neurosci. 12547548.

LoomisJ. M. (1992). Distal attribution and presencePresence (Camb.) 1113119.

LoomisJ. M.KlatzkyR. L.GiudiceN. A. (2012). Sensory substitution of vision: importance of perceptual and cognitive processing in: Assistive Technology for Blindness and Low VisionManduchiR.KurniawanS. (Eds) pp.  162191. CRC PressBoca Raton, FL, USA.

LukJ.PasqueroJ.LittleS.MacLeanK.LévesqueV.HaywardV. (2006). A role for haptics in mobile interaction: initial design using a handheld tactile display prototype in: Proc. ACM Conf. Hum. Fact. Comput. Syst. (CHI ’06) CHI Lett. 8 no. 1 Montreal Canada pp. 171–180.

MarksP. (2006). Gadgets get the feel of the tactile worldNew Sci. 256027.

MartinM. G. F. (1995). Bodily awareness: a sense of ownership in: The Body and the SelfBermudezJ. L.MarcelA.EilanN. (Eds) pp.  267289. MIT PressCambridge, MA, USA.

McGloneF. P.SpenceC. (2010). Editorial: the cutaneous senses: touch, temperature, pain/itch, and pleasureNeurosci. Biobehav. Rev. 34145147.

MeijerP. B. L. (1993). An experimental system for auditory image representationsIEEE Trans. Biomed. Eng. 39112121.

MontaguA. (1971). Touching: The Human Significance of the Skin. Columbia University PressNew York, NY, USA.

MoriokaM.MaedaS. (1998). Measurement of hand-transmitted vibration of tapping the long cane for visually handicapped people in JapanInd. Health 36179190.

NagelS. K.CarlC.KringeT.MärtinR.KönigP. (2005). Beyond sensory substitution-learning the sixth senseJ. Neural Eng. 2R13R26.

NatsoulasT. (1966). Locus and orientation of the perceiver (ego) under variable, constant, and no perspective instructionsJ. Personal. Soc. Psychol. 3190196.

NatsoulasT.DubanoskiR. A. (1964). Inferring the locus and orientation of the perceiver from responses to stimulation of the skinAm. J. Psychol. 77281285.

OccelliV.SpenceC.ZampiniM. (2013). Auditoy, tactile, and audiotactile information processing following visual depravationPsychol. Bull. 139189212.

ParsonsL. M.ShimojoS. (1987). Perceived spatial organization of cutaneous patterns on surfaces of the human body in various positionsJ. Exp. Psychol. Hum. Percept. Perform. 13488505.

Pascual-LeoneA.TorresF. (1993). Plasticity of the sensorimotor cortex representation of the reading finger in Braille readersBrain 1163952.

PtitoM.MoesgaardS. M.GjeddeA.KupersR. (2005). Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blindBrain 128606614.

PutzarL.GoerendtI.LangeK.RöslerF.RöderB. (2007). Early visual deprivation impairs multisensory interactions in humansNat. Neurosci. 1012431245.

RauscheckerJ. P. (1995). Compensatory plasticity and sensory substitution in the cerebral cortexTrends Neurosci. 183643.

RévészG. (1950). The Psychology and Art of the Blind. Longmans GreenLondon, UK.

RöderB.RöslerF. (2004). Compensatory plasticity as a consequence of sensory loss in: The Handbook of Multisensory ProcessesCalvertG. A.SpenceC.SteinB. E. (Eds) pp.  719747. MIT PressCambridge, MA, USA.

RönnbergJ. (1995). Perceptual compensation in the deaf and blind: myth or reality? in: Compensating for Psychological Deficits and Declines: Managing Losses and Promoting GainsDixonR. A.BäckmanL. (Eds) pp.  251274. Lawrence ErlbaumMahwah, NJ, USA.

SadatoN.Pascual-LeoneA.GrafmanJ.IbañezV.DeiberM.-P.DoldG.HallettM. (1996). Activation of the primary visual cortex by Braille reading in blind subjectsNature 380526528.

SaitoD. N.OkadaT.HondaM.YonekuraY.SadatoN. (2007). Practice makes perfect: the neural substrates of tactile discrimination by Mah-Jong experts include the primary visual cortexBMC Neurosci. 779.

SampaioE.MarisS.Bach-y-RitaP. (2001). Brain plasticity: “Visual” acuity of blind persons via the tongueBrain Res. 908204207.

ScaddenL. (1969). A tactual substitute for sightNew Sci. 41677678.

SchenkmanB. N. (1986). Identification of ground materials with the aid of tapping sounds and vibrations of long canes for the blindErgonomics 29985998.

SchneiderS. L.HughesB.EpsteinW.Bach-y-RitaP. (1986). The detection of length and orientation changes in dynamic vibrotactile patternsPercept. Psychophys. 40290300.

ScholtzD. A. (1957). Fundamental principles of form perception in touchActa Psychol. 13299333.

SerenoM. I.DaleA. M.ReppasJ. B.KwongK. K.BelliveauJ. W.BradyT. J.RosenB. R.TootellR. B. H. (1995). Borders of multiple visual areas in humans revealed by functional magnetic resonance imagingScience 268889893.

ShiffrinR. M.CraigJ. C.CohenE. (1973). On the degree of attention and capacity limitation in tactile processingPercept. Psychophys. 13328336.

ShimizuY. (1982). Temporal effect of tactile letter recognition by a tracing modePercept. Mot. Skills 55343349.

ShimizuY. (1995). Temporal effect in vibrotactile letter readingPercept. Mot. Skills 8010691070.

SiegleJ. H.WarrenW. H. (2010). Distal attribution and distance perception in sensory substitutionPerception 39208223.

Soto-FaracoS.SpenceC.FairbankK.KingstoneA.HillstromA. P.ShapiroK. (2002). A crossmodal attentional blink between vision and touchPsychonom. Bull. Rev. 9731738.

Soto-FaracoS.SpenceC.LloydD.KingstoneA. (2004). Moving multisensory research along: motion perception across sensory modalitiesCurr. Dir. Psychol. Sci. 132932.

SpenceC. (2010). Crossmodal spatial attentionAnn. N. Y. Acad. Sci. 1191182200.

SpenceC.NichollsM. E. R.DriverJ. (2001). The cost of expecting events in the wrong sensory modalityPercept. Psychophys. 63330336.

SpenceC.PariseC.ChenY.-C. (2011). The Colavita visual dominance effect in: Frontiers in the Neural Bases of Multisensory ProcessesMurrayM. M.WallaceM. (Eds) pp.  523550. CRC PressBoca Raton, FL, USA.

Striem-AmitE.BubicA.AmediA. (2011). Neurophysiological mechanisms underlying plastic changes and rehabilitation following sensory loss in blindness and deafness in: Frontiers in the Neural Bases of Multisensory ProcessesMurrayM. M.WallaceM. (Eds) pp.  395422. CRC PressBoca Raton, FL, USA.

Van HulleL.JuravleG.SpenceC.CrombezG.Van DammeS. (2013). Attention modulates sensory suppression during back movementsConscious. Cogn. 22420429.

Von Haller GilmerB. (1961). Toward cutaneous electro-pulse communicationJ. Psychol. 52211222.

Von Haller GilmerB. (1966). Problems in Cutaneous Communication from Psychophysics to Information Processing. The American Foundation for the BlindNew York, NY, USA.

WalkerP.MoylanK. (1994). The enhanced representation of surface texture consequent on the loss of sightNeuropsychologia 32289297.

WeinsteinS. (1968). Intensive and extensive aspects of tactile sensitivity as a function of body part, sex, and laterality in: The Skin SensesKenshaloD. R. (Ed.) pp.  195222. ThomasSpringfield, IL, USA.

WhiteB. W. (1970). Perceptual findings with the vision-substitution systemIEEE Trans. Man–Machine Syst. MMS-115458.

WilliamsM.RayC.GriffithJ.De l’AuneW. (2011). The use of tactile vision sensory substitution as an augmentative tool for individuals with visual impairmentJ. Vis. Impair. Blind. 1054550.

WrightT.WardJ. (2013). The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithmsQ. J. Exp. Psychol. 6616201638.

YanagidaY.KakitaM.LindemanR. W.KumeY.TetsutaniN. (2004). Vibrotactile letter reading using a low-resolution tactor array in: Proc. 12th Int. Symp. Haptic Interfacs Virt. Env. Teleoperator Syst. Chicago IL USA pp. 400–406.


  • View in gallery

    Kajimoto et al.’s (2006) FRS. As in Bach-y-Rita’s (1972) earlier TVSS system, the idea here is that a head-mounted camera converts a visual scene into the appropriate outline image. Thereafter, an array of 512 electrodes in a display pressed onto the user’s forehead represents that image to the user. Note how any problems associated with the segmentation of distinct objects from cluttered background scenes have been bypassed by presented the cup against a uniform white background here (see Note 7). [Image taken from Marks (2006).]

  • View in gallery

    Results of a study of tactile numerosity judgments reported by Gallace et al. (2006b; Experiment 2). The results highlight the very limited ability of people to individuate more than two or three tactile stimuli, even when repeatedly presented on their skin surface. The straight line shows perfect performance, while the other two functions represent actual performance under conditions of single (solid line) versus repeated (dashed line) stimulus presentation. [Figure reprinted with permission from Gallace et al. (2006b).]

Index Card

Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 13 13 4
Full Text Views 6 6 6
PDF Downloads 1 1 1
EPUB Downloads 0 0 0