Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

in Multisensory Research
Restricted Access
Get Access to Full Text
Rent on DeepDyve

Have an Access Token?

Enter your access token to activate and access content online.

Please login and go to your personal user account to enter your access token.


Have Institutional Access?

Access content through your institution. Any other coaching guidance?


Sensory substitution devices have the potential to provide individuals with visual impairments with more information about their environments, which may help them recognize objects and achieve more independence in their daily lives. However, many of these devices may require extensive training and might be limited in the amount of information that they can convey. We tested the effectiveness and assessed some of the limitations of the BrainPort device, which provides stimulation through a 20 × 20 electrode grid array on the tongue. Across five experiments, including one with blind individuals, we found that subjects were unable to accurately discriminate between simple shapes as well as different line orientations that were briefly presented on the tongue, even after 300 trials of practice with the device. These experiments indicate that such a minimal training regimen with the BrainPort is not sufficient for object recognition, raising serious concerns about the usability of this device without extensive training.

Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

in Multisensory Research



AbboudS.HanassyS.Levy-TzedekS.MaidenbaumS.AmediA. (2014). EyeMusic: introducing a ‘visual’ colorful experience for the blind using auditory sensory substitutionRestor. Neurol. Neurosci. 32247257.

AmediA.MalachR.Pascual-LeoneA. (2005). Negative BOLD differentiates visual imagery and perceptionNeuron 48859872.

AzulayH.StriemE.AmediA. (2009). Negative BOLD in sensory cortices during verbal memory: a component in generating internal representations? Brain Topogr. 21221231.

Bach-y-RitaP.KaczmarekK.TylerM.Garcia-LauraJ. (1998). Form perception with a 49-point electrotactile stimulus array on the tongue: a technical noteJ. Rehab. Res. Dev. 35427430.

BrownD.SimpsonA.ProulxM. (2015). Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli? Front. Psychol. 61522. DOI:10.3389/fpsyg.2015.01522.

BurtonH.SnyderA. Z.ConturoT. E.AkbudakE.OllingerJ. M.RaichleM. E. (2002a). Adaptive changes in early and late blind: a fMRI study of Braille readingJ. Neurophysiol. 87589607.

BurtonH.SnyderA. Z.DiamondJ. B.RaichleM. E. (2002b). Adaptive changes in early and late blind: a FMRI study of verb generation to heard nounsJ. Neurophysiol. 8833593371.

ChebatD.RainvilleC.KupersR.PtitoM. (2007). Tactile–‘visual’ acuity of the tongue in early blind individualsNeuroReport 1819011904.

ChebatD.SchneiderF.KupersR.PtitoM. (2011). Navigation with a sensory substitution device in congenitally blind individualsNeuroReport 22342347.

CohenL.WeeksR.CelnikP.IshiiK.HallettM. (1999). Period of susceptibility for cross-modal plasticity in the blindAnn. Neurol. 45451460.

CollignonO.DormalG.AlbouyG.VandewalleG.VossP.PhillipsC.LeporeF. (2013). Impact of blindness onset on the functional organization and the connectivity of the occipital cortexBrain 13627692783.

DeibertE.KrautM.KremenS.HartJ. (1999). Neural pathways in tactile object recognitionNeurology 5214131417.

EssickG.ChenC.KellyD. (1999). A letter-recognition task to assess lingual tactile acuityJ. Oral Maxillofac. Surg. 5713241330.

GibsonJ. J. (1962). Observations on active touchPsychol. Rev. 69477491.

KauffmanT.ThéoretH.Pascual-LeoneA. (2002). Braille character discrimination in blindfolded human subjectsNeuroReport 13571574.

KendrickM. (2009). Tasting the lightSci. Am. 301(4) 2224.

KhooW.OlmschenkG.ZhuZ.RoT. (2015). Evaluating crowd sourced navigation for the visually impaired in a virtual environment in: IEEE 4th International Conference on Mobile Services (MS 2015) June 27–July 2 2015 New York NY USA pp. 431–437.

KhooW. L.SeidelE. L.ZhuZ. (2012). Designing a virtual environment to evaluate multimodal sensing for assisting the visually impaired in: 13th International Conference on Computers Helping People with Special Needs (ICCHP) July 11–13 2012 Linz Austria pp. 573–580.

KhooW. L.KnappJ.PalmerF.RoT.ZhuZ. (2013). Designing and testing wearable range-vibrotactile devicesJ. Assist. Technol. 7102117.

KlatzkyR.LedermanS.ReedC. (1987). There’s more to touch than meets the eye: the salience of object attributes for haptics with and without visionJ. Exp. Psychol. Gen. 116356369.

KupersR.ChebatD. R.MadsenK. H.PaulsonO. B.PtitoM. (2010). Neural correlates of virtual route recognition in congenital blindnessProc. Natl Acad. Sci. 1071271612721.

LambG. D. (1983). Tactile discrimination of textured surfaces: psychophysical performance measurements in humansJ. Physiol. 338551565.

LedermanS. J.KlatzkyR. L. (2009). Haptic perception: a tutorialAtten. Percept. Psychophys. 7114391459.

LeeV. K.NauA. C.LaymonC.ChanK. C.RosarioB. L.FisherC. (2014). Successful tactile based visual sensory substitution use functions independently of visual pathway integrityFront. Hum. Neurosci. 8291. DOI:10.3389/fnhum.2014.00291.

LozanoC.KaczmarekK.SantelloM. (2009). Electrotactile stimulation on the tongue: intensity perception, discrimination, and cross-modality estimationSomatosens. Mot. Res. 265063.

MaidenbaumS.AbboudS.AmediA. (2014). Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitationNeurosci. Biobehav. Rev. 41315.

MaidenbaumS.BuchsG.AbboudS.Lavi-RotbainO.AmediA. (2016). Perception of graphical virtual environments by blind users via sensory substitutionPLoS One 11e0147501. DOI:10.1371/journal.pone.0147501.

MatteauI.KupersR.RicciardiE.PietriniP.PtitoM. (2010). Beyond visual, aural and haptic movement perception: hMT+ is activated by electrotactile motion stimulation of the tongue in sighted and in congenitally blind individualsBrain Res. Bull. 82264270.

MeijerP. (1992). An experimental system for auditory image representationsIEEE Trans. Biomed. Eng. 39112121.

MylesK.BinseelM. S. (2007). The Tactile Modality: A review of tactile sensitivity and human tactile interfaces. U.S. Army Research Laboratory ARL-TR-4115 Aberdeen Proving Ground MD 21005-5425.

NauA.BachM.FisherC. (2013). Clinical tests of ultra-low vision used to evaluate rudimentary visual perceptions enabled by the BrainPort vision deviceTransl. Vis. Sci. Technol. 21.

NauA. C.PintarC.ArnoldussenA.FisherC. (2015). Acquisition of visual perception in blind adults using the BrainPort artificial vision deviceAm. J. Occup. Ther. 696901290010p1-8. DOI:10.5014/ajot.2015.011809.

PoirierC.De VolderA.TranduyD.ScheiberC. (2006). Pattern recognition using a device substituting audition for vision in blindfolded sighted subjectsNeuropsychologia 4511081121.

PtitoM.MoesgaardS.GjeddeA.KupersR. (2005). Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blindBrain 128606614.

PtitoM.MatteauI.GjeddeA.KupersR. (2009). Recruitment of the middle temporal area by tactile motion in congenital blindnessNeuroReport 20543547.

PtitoM.MatteauI.Zhi WangA.PaulsonO. B.SiebnerH. R.KupersR. (2012). Crossmodal recruitment of the ventral visual stream in congenital blindnessNeural Plast. 2012304045. DOI:10.1155/2012/304045.

SabbahN.AuthiéC.SandaN.Mohand-SaïdS.SahelJ.-A.SafranA.HabasC.AmediA. (2016). Increased functional connectivity between language and visually deprived areas in late and partial blindnessNeuroImage 136162173.

SadatoN.Pascual-LeoneA.GrafmaniJ.IbañezV.DeiberM.DoldG.HallettM. (1996). Activation of the primary visual cortex by Braille reading in blind subjectsNature 380(6574) 526528.

SadatoN.OkadaT.HondaM.YonekuraY. (2002). Critical period for cross-modal plasticity in blind humans: a functional MRI studyNeuroImage 16389400.

SampaioE.MarisS.Bach-y-RitaP. (2001). Brain plasticity: ‘visual’ acuity of blind persons via the tongueBrain Res. 908204207.

SathianK.ZangaladzeA.HoffmanJ.GraftonS. (1997). Feeling with the mind’s eyeNeuroReport 838773881.

SpanosN. P.StamH. J. (1979). The elicitation of visual hallucinations via brief instructions in a normal sampleJ. Nerv. Ment. Dis. 167488494.

SpenceC. (2014). The skin as a medium for sensory substitutionMultisens. Res. 27293312.

SteeleC. M.HillL.StokelyS.Peladeau-PigeonM. (2014). Age and strength influences on lingual tactile acuityJ. Text. Stud. 45317323.

SteevesJ.HarrisL. (2012). Plasticity in Sensory Systems. Cambridge University PressCambridge, UK.

StilesN. R.ShimojoS. (2015). Auditory sensory substitution is intuitive and automatic with texture stimuliSci. Rep. 515628. DOI:10.1038/srep15628.

TrulssonM.EssickG. K. (1997). Low-threshold mechanoreceptive afferents in the human lingual nerveJ. Neurophysiol. 77737748.

Van BovenR. W.JohnsonK. O. (1994). The limit of tactile spatial resolution in humans: grating orientation discrimination at the lip, tongue, and fingerNeurology 4423612366.

Vega-BermudezF.JohnsonK. O.HsiaoS. S. (1991). Human tactile pattern recognition: active versus passive touch, velocity effects, and patterns of confusionJ. Neurophysiol. 65531546.

VossP.GougouxF.LassondeM.ZatorreR. J.LeporeF. (2006). A positron emission tomography study during auditory localization by late-onset blind individualsNeuroReport 17383388.

WanC.WoodA.ReutensD.WilsonS. (2010a). Congenital blindness leads to enhanced vibrotactile perceptionNeuropsychologia 48631635.

WanC.WoodA.ReutensD.WilsonS. (2010b). Early but not late-blindness leads to enhanced auditory perceptionNeuropsychologia 48344348.

WardJ.MeijerP. (2010). Visual experiences in the blind induced by an auditory sensory substitution deviceConsc. Cogn. 19492500.

Wicab Inc. (2008). BrainPort® technology tongue interface characterization Tactical Underwater Navigation System (TUNS): interim report for November 2006 to June 2008. Wright-Patterson AFB OH: Air Force Research Laboratory Human Effectiveness Directorate Biosciences and Protection Division Aircrew Performance and Protection Branch.


  • View in gallery

    Stimuli for Experiments 1–5.

  • View in gallery

    Discrimination performance in Experiment 1. Chance-level performance is indicated by the dashed horizontal line. Error bars are ±1 standard error of the mean.

  • View in gallery

    Discrimination accuracy in Experiment 2. Chance-level performance is indicated by the dashed horizontal line. Error bars are ±1 standard error of the mean.

  • View in gallery

    Discrimination accuracy in Experiment 3. Chance-level performance is indicated by the dashed horizontal line. Error bars are ±1 standard error of the mean. p<0.05.

  • View in gallery

    Discrimination accuracy in Experiment 4. Chance-level performance is indicated by the dashed horizontal line. Error bars are ±1 standard error of the mean. p<0.05.

  • View in gallery

    Discrimination accuracy in Experiment 5 for lines during practice with 500 ms stimulation and after practice with continuous stimulation. Chance-level performance is indicated by the dashed horizontal line. Error bars are ±1 standard error of the mean. p<0.05.

Index Card

Content Metrics

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 27 27 18
Full Text Views 7 7 7
PDF Downloads 2 2 2
EPUB Downloads 0 0 0