Is There a Future for Sensory Substitution Outside Academic Laboratories?

in Multisensory Research
No Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


Sensory substitution devices (SSDs) have been developed with the ultimate purpose of supporting sensory deprived individuals in their daily activities. However, more than forty years after their first appearance in the scientific literature, SSDs still remain more common in research laboratories than in the daily life of people with sensory deprivation. Here, we seek to identify the reasons behind the limited diffusion of SSDs among the blind community by discussing the ergonomic, neurocognitive and psychosocial issues potentially associated with the use of these systems. We stress that these issues should be considered together when developing future devices or improving existing ones. We provide some examples of how to achieve this by adopting a multidisciplinary and participatory approach. These efforts would contribute not solely to address fundamental theoretical research questions, but also to better understand the everyday needs of blind people and eventually promote the use of SSDs outside laboratories.

Is There a Future for Sensory Substitution Outside Academic Laboratories?

in Multisensory Research



AbboudS.HanassyS.Levy-TzedekS.MaidenbaumS.AmediA. (2014). EyeMusic: introducing a “visual” colorful experience for the blind using auditory sensory substitutionRestor. Neurol. Neurosci. 2247257.

AmediA.SternW. M.CamprodonJ. A.BermpohlF.MerabetL.RotmanS.Pascual-LeoneA. (2007). Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complexNat. Neurosci. 10687689.

ArnoP.CapelleC.Wanet-DefalqueM. C.Catalan-AhumadaM.VeraartC. (1999). Auditory coding of visual patterns for the blindPerception 2810131030.

ArnoP.VanlierdeA.StreelE.Wanet-DefalqueM. C.Sanabria-BohorquezS.VeraartC. (2001). Auditory substitution of vision: pattern recognition by the blindAppl. Cogn. Psychol. 15509519.

AuvrayM.HannetonS.LenayC.O’ReganJ. K. (2005). There is something out there: distal attribution in sensory substitution, twenty years laterJ. Integr. Neurosci. 4505521.

AuvrayM.HannetonS.O’ReganJ. K. (2007). Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with the vOICePerception 36416.

Bach-y-RitaP.CollinsC. C.SaundersF. A.WhiteB.ScaddenL. (1969). Vision substitution by tactile image projectionNature 221(5184) 963964.

Bach-y-RitaP.KaczmarekK. A.TylerM. E.Garcia-LaraJ. (1998). Form perception with a 49-point electrotactile stimulus array on the tongue: a technical noteJ. Rehab. Res. Dev. 35427430.

BrownJ. S.CollinsA.DuguidP. (1989). Situated cognition and the culture of learningEduc. Res. 183242.

CampbellN. (1992). Sensory training in: The Sound of SilenceRosenbaumR. (Ed.). The Carroll Center for the BlindNewton, MA, USA.

CapelleC.FrereB.BolleB.TrullemansC.VeraartC. (1992). Real time auditory coding of visual information in: Engineering in Medicine and Biology Society 1992 14th Annual International Conference of the IEEE Paris France Vol. 4 pp. 1660–1661.

CapelleC.TrullemansC.ArnoP.VeraartC. (1998). A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitutionIEEE Trans. Biomed. Eng. 4512791293.

ClarkN. (1995). Interactive nature of knowledge systems: some implications for the third worldSci. Publ. Pol. 22249258.

CollignonO.De VolderA. G. (2009). Further evidence that congenitally blind participants react faster to auditory and tactile spatial targetsCan. J. Exp. Psychol. 63287.

CollignonO.RenierL.BruyerR.TranduyD.VeraartC. (2006). Improved selective and divided spatial attention in early blind subjectsBrain Res. 1075175182.

CollignonO.LassondeM.LeporeF.BastienD.VeraartC. (2007). Functional cerebral reorganization for auditory spatial processing and auditory substitution of vision in early blind subjectsCereb. Cort. 17457465.

CollignonO.VossP.LassondeM.LeporeF. (2009a). Cross-modal plasticity for the spatial processing of sounds in visually deprived subjectsExp. Brain Res. 192343358.

CollignonO.CharbonneauG.LassondeM.LeporeF. (2009b). Early visual deprivation alters multisensory processing in peripersonal spaceNeuropsychologia 4732363243.

CollignonO.ChampouxF.VossP.LeporeF. (2011a). Sensory rehabilitation in the plastic brainProgr. Brain Res. 91211231.

CollignonO.VandewalleG.VossP.AlbouyG.CharbonneauG.LassondeM.LeporeF. (2011b). Functional specialization for auditory-spatial processing in the occipital cortex of congenitally blind humansProc. Natl Acad. Sci. USA 10844354440.

CollignonO.DormalG.AlbouyG.VandewalleG.VossP.PhillipsC.LeporeF. (2013). Impact of blindness onset on the functional organization and the connectivity of the occipital cortexBrain 13627692783.

CollinsC. C. (1985). On mobility aids for the blind in: Electronic Spatial Sensing for the BlindWarrenD. H.StrelowE. R. (Eds) pp.  3564. SpringerDordrecht, Netherlands.

ConnorsE. C.YazzolinoL. A.SánchezJ.MerabetL. B. (2013). Development of an audio-based virtual gaming environment to assist with navigation skills in the blindJ. Vis. Exp. 73e50272. DOI:10.3791/50272.

ConnorsE.ChrastilE.SanchezJ.MerabetL. B. (2014). Action video game play and transfer of navigation and spatial cognition skills in adolescents who are blindFront. Hum. Neurosci. 8133.

CostkyanG. (2002). I have no words and I must design: toward a critical vocabulary for games in: Computer Games and Digital Cultures Conference Proceedings F. Mäyrä (Ed.) pp. 9–33 Studies in Information Sciences Tampere University Press Tampere FL USA.

Del Marie RysavyS.SalesG. C. (1991). Cooperative learning in computer-based instructionEduc. Technol. Res. Dev. 397079.

DoucetM. E.BergeronF.LassondeM.FerronP.LeporeF. (2006). Cross-modal reorganization and speech perception in cochlear implant usersBrain 12933763383.

DouthwaiteB.KeatingeJ. D. H.ParkJ. R. (2001). Why promising technologies fail: the neglected role of user innovation during adoptionRes. Pol. 30819836.

DouthwaiteB.KeatingeJ. D. H.ParkJ. R. (2002). Learning selection: an evolutionary model for understanding, implementing and evaluating participatory technology developmentAgric. Syst. 72109131.

FineI.WadeA. R.BrewerA. A.MayM. G.GoodmanD. F.BoyntonG. M.WandellB. A.MacLeodD. I. A. (2003). Long-term deprivation affects visual perception and cortexNat. Neurosci. 6915917.

GiudiceN. A.LeggeG. E. (2008). Blind navigation and the role of technology in: Engineering Handbook of Smart Technology for Aging Disability and IndependenceHelalA.MokhtariM.AbdulrazakB. (Eds) pp.  479500. John Wiley and SonsHoboken, NJ, USA.

GiudiceN. A.WaltonL. A.WorboysM. (2010). The informatics of indoor and outdoor space: a research agenda in: Proceedings of the 2nd ACM SIGSPATIAL International Workshop on Indoor Spatial Awareness ACM San Jose CA USA pp. 47–53.

GolledgeR. G.MarstonJ. R.LoomisJ. M.KlatzkyR. L. (2004). Stated preference for components of a personal guidance system for nonvisual navigationJ. Vis. Impair. Blind. 98135147.

GotheJ.BrandtS. A.IrlbacherK.RörichtS.SabelB. A.MeyerB. U. (2002). Changes in visual cortex excitability in blind subjects as demonstrated by transcranial magnetic stimulationBrain 125479490.

GougouxF.LeporeF.LassondeM.VossP.ZatorreR. J.BelinP. (2004). Neuropsychology: pitch discrimination in the early blindNature 430(6997) 309.

GregoryR. L.WallaceJ. G. (1963). Recovery from Early Blindness: A Case Study. Heffer and SonsCambridge, UK.

HaighA.BrownD. J.MeijerP.ProulxM. J. (2013). How well do you see what you hear? The acuity of visual-to-auditory sensory substitutionFront. Psychol. 4330.

HardimanM. M. (2001). Connecting brain research with dimensions of learningEduc. Leadersh. 59(3) 5255.

HeekerenH. R.MarrettS.UngerleiderL. G. (2008). The neural systems that mediate human perceptual decision makingNat. Rev. Neurosci. 9467479. GelderB.GandhiT.GaneshS.MathurU.SinhaP. (2011). The newly sighted fail to match seen with feltNat. Neurosci. 14551553.

KimS. Y.ChoK. (2013). Usability and design guidelines of smart canes for users with visual impairmentsInt. J. Des. 799110.

KimJ. K.ZatorreR. J. (2008). Generalized learning of visual-to-auditory substitution in sighted individualsBrain Res. 1242263275.

KupersR.ChebatD. R.MadsenK. H.PaulsonO. B.PtitoM. (2010). Neural correlates of virtual route recognition in congenital blindnessProc. Natl Acad. Sci. 1071271612721.

LahavO.MioduserD. (2008). Construction of cognitive maps of unknown spaces using a multi-sensory virtual environment for people who are blindComput. Human Behav. 2411391155.

LahavO.SchloerbD. W.SrinivasanM. A. (2012). Newly blind persons using virtual environment system in a traditional orientation and mobility rehabilitation program: a case studyDisabil. Rehabil. Assist. Technol. 7420435.

LeeD. S.LeeJ. S.OhS. H.KimS. K.KimJ. W.ChungJ. K.KimC. S. (2001). Deafness: cross-modal plasticity and cochlear implantsNature 409(6817) 149150.

LenayC.GapenneO.HannetonS.MarqueC.GenouëlleC. (2003). Sensory substitution: limits and perspectives in: Touching for KnowingHatwellY.StreriA.GentazE. (Eds) pp.  275292. John BenjaminsAmsterdam, Netherlands.

LessardN.PareM.LeporeF.LassondeM. (1998). Early-blind human subjects localize sound sources better than sighted subjectsNature 395(6699) 278280.

LevinN.DumoulinS. O.WinawerJ.DoughertyR. F.WandellB. A. (2010). Cortical maps and white matter tracts following long period of visual deprivation and retinal image restorationNeuron 652131.

Levy-TzedekS.NovickI.ArbelR.AbboudS.MaidenbaumS.VaadiaE.AmediA. (2012a). Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitutionSci. Rep. 2949. DOI:10.1038/srep00949.

Levy-TzedekS.HanassyS.AbboudS.MaidenbaumS.AmediA. (2012b). Fast, accurate reaching movements with a visual-to-auditory sensory substitution deviceRestor. Neurol. Neurosci. 30313323.

LewaldJ. (2013). Exceptional ability of blind humans to hear sound motion: implications for the emergence of auditory spaceNeuropsychologia 51181186.

LoomisJ. M.GolledgeR. D.KlatzkyR. L. (2001). GPS-based navigation systems for the visually impaired in: Fundamentals of Wearable Computers and Augmented RealityBarfieldW.CaudellT. (Eds) pp.  429446. Lawrence Erlbaum Associates PublishersMahwah, NJ, USA.

LoomisJ. M.KlatzkyR. L.GiudiceN. A. (2012). Sensory substitution of vision: importance of perceptual and cognitive processing in: Assistive Technology for Blindness and Low VisionManduchiR.KurniawanS. (Eds) pp.  162191. CRC PressBoca Raton, FL, USA.

MacDonaldJ. A.HenryP. P.LetowskiT. R. (2006). Spatial audio through a bone conduction interface: audición espacial a través de una interfase de conducción óseaInt. J. Audiol. 45595599.

MaidenbaumS. (2012). Sight from the depths — Using distance information to help the blind and as a tool for exploring neurobiological questions in real and virtual environments PhD thesis Hebrew University of Jerusalem Press Jerusalem pp. 35–42.

MaidenbaumS.Levy-TzedekS.ChebatD. R.AmediA. (2013). Increasing accessibility to the blind of virtual environments, using a virtual mobility aid based on the “EyeCane”: feasibility studyPloS One 8e72555. DOI:10.1371/journal.pone.0072555.

ManduchiR.KurniawanS. (2010). Watch your head mind your step: mobility-related accidents experienced by people with visual impairment Tech. Rep. Department of Computer Engineering University of California Santa Cruz CA USA.

MarzanoR. J. (1992). A Different Kind of Classroom: Teaching with Dimensions of Learning. Association for Supervision and Curriculum DevelopmentAlexandria, VA, USA.

McCombsB. L.WhislerJ. S. (1997). The Learner-Centered Classroom and School: Strategies for Increasing Student Motivation and Achievement. The Jossey-Bass Education Series, Jossey-Bass Inc.Publishers, San Francisco, CA, USA.

MeijerP. B. (1992). An experimental system for auditory image representationsIEEE Trans. Biomed. Eng. 39112121.

MerabetL. B.RizzoJ. F.AmediA.SomersD. C.Pascual-LeoneA. (2005). What blindness can tell us about seeing again: Merging neuroplasticity and neuroprosthesesNat. Rev. Neurosci. 67177.

MerabetL. B.BattelliL.ObretenovaS.MaguireS.MeijerP.Pascual-LeoneA. (2009). Functional recruitment of visual cortex for sound encoded object identification in the blindNeuroreport 20132138.

MerabetL. B.ConnorsE. C.HalkoM. A.SánchezJ. (2012). Teaching the blind to find their way by playing video gamesPLoS One 7(9) e44958. DOI:10.1371/journal.pone.0044958.

MilgramP.KishinoF. (1994). A taxonomy of mixed reality visual displaysIEICE Trans. Inf. Syst. 7713211329.

MokyrJ. (1990). The Lever of Riches: Technological Creativity and Economic Progress. Oxford University PressOxford, UK.

MolyneuxW. (1688). Letter to John Locke 7 July in: The Correspondence of John Locke Vol. 3 No. 1064 1978 E. S. de Beer (Ed.). Clarendon Press Oxford UK.

OhtaJ. (2011). Artificial retina IC in: Bio-Medical CMOS ICsYooH.-J.van HoofC. (Eds) pp.  481514. SpringerNew York, NY, USA.

ParkH. J.LeeJ. D.KimE. Y.ParkB.OhM. K.LeeS.KimJ. J. (2009). Morphological alterations in the congenital blind based on the analysis of cortical thickness and surface areaNeuroimage 4798106.

Pascual-LeoneA.HamiltonR. (2001). The metamodal organization of the brainProg. Brain Res. 134427445.

PerryB. (2000). How the brain learns bestInstructor 1103435.

PetrieH.JohnsonV.StrothotteT.RaabA.FritzS.MichelR. (1996). MoBIC: designing a travel aid for blind and elderly peopleJ. Navig. 494552.

PoirierC.De VolderA.TranduyD.ScheiberC. (2007). Pattern recognition using a device substituting audition for vision in blindfolded sighted subjectsNeuropsychologia 4511081121.

PoneaS.SanduA. (2010a). Appreciative socialization group: a collaborative creativity model INVENTICA 2010 Editura Performantica Iasi Romania.

PoneaS.SanduA. (2010b). Appreciative socialization group. A model of personal developmentPostmodern Openings 47588.

ProulxM. J.StoerigP.LudowigE.KnollI. (2008). Seeing ‘where’ through the ears: effects of learning-by-doing and long-term sensory deprivation on localization based on image-to-sound substitutionPloS One 3e1840. DOI:10.1371/journal.pone.0001840.

PtitoM.MoesgaardS. M.GjeddeA.KupersR. (2005). Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blindBrain 128606614.

PtitoM.SchneiderF. C.PaulsonO. B.KupersR. (2008). Alterations of the visual pathways in congenital blindnessExp. Brain Res. 1874149.

ReardonS. (2011). Playing by earScience 333(6051) 18161818.

ReichL.MaidenbaumS.AmediA. (2012). The brain as a flexible task machine: implications for visual rehabilitation using noninvasive vs. invasive approachesCurr. Opin. Neurol. 258695.

ReigeluthC. M. (1999). Visioning public education in AmericaEduc. Technol. 395055.

RenierL.De VolderA. G. (2010). Vision substitution and depth perception: early blind subjects experience visual perspective through their earsDisabil. Rehabil. Assist. Technol. 5175183.

RenierL.CollignonO.PoirierC.TranduyD.VanlierdeA.BolA.VeraartC.De VolderA. G. (2005). Cross-modal activation of visual cortex during depth perception using auditory substitution of visionNeuroimage 26573580.

RicciardiE.PietriniP. (2011). New light from the dark: what blindness can teach us about brain functionCurr. Opin. Neurol. 24357363.

RiehleT. H.LichterP.GiudiceN. A. (2008). An indoor navigation system to support the visually impaired in: Engineering in Medicine and Biology Society 2008 EMBS 2008. 30th Annual International Conference of the IEEE Vancouver Canada pp. 4435–4438.

RizzoJ. F.IIISneboldL.KenneyM. (2007). Development of a visual prosthesis in: Visual Prosthesis and Ophthalmic DevicesTombran-TinkJ.BarnstableC. J.RizzoJ. F. (Eds) pp.  7193. Humana PressTotowa, NJ, USA.

RogersE. M. (1995). Diffusion of Innovation3rd edn.Revised Edition of: Communication of Innovations. The Free PressNew York, NY, USA.

RollingsA.AdamsE. (2003). Andrew Rollings and Ernest Adams on Game Design. New RidersAuckland, New Zealand.

RosenbergN. (1982). Inside the Black Box: Technology and Economics. Cambridge University PressCambridge, UK.

RussellJ. N.HendershotG. E.LeClereF.HowieL. J.AdlerM. (1997). Trends and differential use of assistive technology devices: United States, 1994Adv. Data 29219.

RuthenbergH.JahnkeH. E. (1985). Innovation Policy for Small Farmers in the Tropics: The Economics of Technical Innovations for Agricultural Development. Clarendon PressOxford, UK.

SaenzM.LewisL. B.HuthA. G.FineI.KochC. (2008). Visual motion area MT+/V5 responds to auditory motion in human sight-recovery subjectsJ. Neurosci. 2851415148.

SampaioE.MarisS.Bach-y-RitaP. (2001). Brain plasticity: ‘visual’ acuity of blind persons via the tongueBrain Res. 908204207.

SamuelsonW.ZeckhauserR. (1988). Status quo bias in decision makingJ. Risk Uncertain 1759.

SerinoA.BassolinoM.FarnèA.LàdavasE. (2007). Extended multisensory space in blind cane usersPsychol. Sci. 18642648.

ShamsL.SeitzA. R. (2008). Benefits of multisensory learningTrends Cogn. Sci. 12411417.

SinhaP.ChatterjeeG.GandhiT.KaliaA. (2013). Restoring vision through “Project Prakash”: the opportunities for merging science and servicePLoS Biol. 11e1001741. DOI:10.1371/journal.pbio.1001741.

SprengerM. (1998). Memory lane is a two-way streetEduc. Leadersh. 566567.

StrelnikovK.RougerJ.DemonetJ. F.LagleyreS.FraysseB.DeguineO.BaroneP. (2013). Visual activity predicts auditory recovery from deafness after adult cochlear implantationBrain 13636823695.

Striem-AmitE.CohenL.DehaeneS.AmediA. (2012a). Reading with sounds: sensory substitution selectively activates the visual word form area in the blindNeuron 76640652.

Striem-AmitE.GuendelmanM.AmediA. (2012b). ‘Visual’ acuity of the congenitally blind using visual-to-auditory sensory substitutionPloS One 7e33136. DOI:10.1371/journal.pone.0033136.

Striem-AmitE.DakwarO.ReichL.AmediA. (2012c). The large-scale organization of “visual” streams emerges without visual experienceCereb. Cortex 2216981709.

TjanB. S.BeckmannP. J.RoyR.GiudiceN.LeggeG. E. (2005). Digital sign system for indoor wayfinding for the visually impaired in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops 2005 CVPR Workshops pp. 30.

VelázquezR. (2010). Wearable assistive devices for the blind in: Wearable and Autonomous Biomedical Devices and Systems for Smart EnvironmentLay-EkuakilleA. (Ed.) pp.  331349. SpringerBerlin, Heidelberg, Germany.

Von SendenM. (1960). Space and Sight: The Perception of Space and Shape in the Congenitally Blind Before and After Operation. MethuenLondon, UK.

WalkerB. N.StanleyR. M.IyerN.SimpsonB. D.BrungartD. S. (2005). Evaluation of bone-conduction headsets for use in multitalker communication environments in: Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting pp. 1615–1619 Orlando FL USA.

WilsonJ. R. (2000). Fundamentals of ergonomics in theory and practiceAppl. Ergon. 31557567.

WongM.GnanakumaranV.GoldreichD. (2011). Tactile spatial acuity enhancement in blindness: evidence for experience-dependent mechanismsJ. Neurosci. 3170287037.

World Health Organization (2001). International classification of functioning disability and health (ICF).


Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 49 49 12
Full Text Views 165 165 65
PDF Downloads 9 9 2
EPUB Downloads 0 0 0