Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

In: Multisensory Research
View More View Less
  • 1 The City College and Graduate Center of the City University of New York, 365 Fifth Ave., New York, NY 10016, USA

Purchase instant access (PDF download and unlimited online access):

$30.00

Sensory substitution devices have the potential to provide individuals with visual impairments with more information about their environments, which may help them recognize objects and achieve more independence in their daily lives. However, many of these devices may require extensive training and might be limited in the amount of information that they can convey. We tested the effectiveness and assessed some of the limitations of the BrainPort device, which provides stimulation through a 20 × 20 electrode grid array on the tongue. Across five experiments, including one with blind individuals, we found that subjects were unable to accurately discriminate between simple shapes as well as different line orientations that were briefly presented on the tongue, even after 300 trials of practice with the device. These experiments indicate that such a minimal training regimen with the BrainPort is not sufficient for object recognition, raising serious concerns about the usability of this device without extensive training.

  • Abboud S., Hanassy S., Levy-Tzedek S., Maidenbaum S., Amedi A. (2014). EyeMusic: introducing a ‘visual’ colorful experience for the blind using auditory sensory substitution, Restor. Neurol. Neurosci. 32, 247257.

    • Search Google Scholar
    • Export Citation
  • Amedi A., Malach R., Pascual-Leone A. (2005). Negative BOLD differentiates visual imagery and perception, Neuron 48, 859872.

  • Azulay H., Striem E., Amedi A. (2009). Negative BOLD in sensory cortices during verbal memory: a component in generating internal representations? Brain Topogr. 21, 221231.

    • Search Google Scholar
    • Export Citation
  • Bach-y-Rita P., Kaczmarek K., Tyler M., Garcia-Laura J. (1998). Form perception with a 49-point electrotactile stimulus array on the tongue: a technical note, J. Rehab. Res. Dev. 35, 427430.

    • Search Google Scholar
    • Export Citation
  • Brown D., Simpson A., Proulx M. (2015). Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli? Front. Psychol. 6, 1522. DOI:10.3389/fpsyg.2015.01522.

    • Search Google Scholar
    • Export Citation
  • Burton H., Snyder A. Z., Conturo T. E., Akbudak E., Ollinger J. M., Raichle M. E. (2002a). Adaptive changes in early and late blind: a fMRI study of Braille reading, J. Neurophysiol. 87, 589607.

    • Search Google Scholar
    • Export Citation
  • Burton H., Snyder A. Z., Diamond J. B., Raichle M. E. (2002b). Adaptive changes in early and late blind: a FMRI study of verb generation to heard nouns, J. Neurophysiol. 88, 33593371.

    • Search Google Scholar
    • Export Citation
  • Chebat D., Rainville C., Kupers R., Ptito M. (2007). Tactile–‘visual’ acuity of the tongue in early blind individuals, NeuroReport 18, 19011904.

    • Search Google Scholar
    • Export Citation
  • Chebat D., Schneider F., Kupers R., Ptito M. (2011). Navigation with a sensory substitution device in congenitally blind individuals, NeuroReport 22, 342347.

    • Search Google Scholar
    • Export Citation
  • Cohen L., Weeks R., Celnik P., Ishii K., Hallett M. (1999). Period of susceptibility for cross-modal plasticity in the blind, Ann. Neurol. 45, 451460.

    • Search Google Scholar
    • Export Citation
  • Collignon O., Dormal G., Albouy G., Vandewalle G., Voss P., Phillips C., Lepore F. (2013). Impact of blindness onset on the functional organization and the connectivity of the occipital cortex, Brain 136, 27692783.

    • Search Google Scholar
    • Export Citation
  • Deibert E., Kraut M., Kremen S., Hart J. (1999). Neural pathways in tactile object recognition, Neurology 52, 14131417.

  • Essick G., Chen C., Kelly D. (1999). A letter-recognition task to assess lingual tactile acuity, J. Oral Maxillofac. Surg. 57, 13241330.

  • Gibson J. J. (1962). Observations on active touch, Psychol. Rev. 69, 477491.

  • Kauffman T., Théoret H., Pascual-Leone A. (2002). Braille character discrimination in blindfolded human subjects, NeuroReport 13, 571574.

    • Search Google Scholar
    • Export Citation
  • Kendrick M. (2009). Tasting the light, Sci. Am. 301(4), 2224.

  • Khoo W., Olmschenk G., Zhu Z., Ro T. (2015). Evaluating crowd sourced navigation for the visually impaired in a virtual environment, in: IEEE 4th International Conference on Mobile Services (MS 2015), June 27–July 2, 2015, New York, NY, USA, pp. 431–437.

  • Khoo W. L., Seidel E. L., Zhu Z. (2012). Designing a virtual environment to evaluate multimodal sensing for assisting the visually impaired, in: 13th International Conference on Computers Helping People with Special Needs (ICCHP), July 11–13, 2012, Linz, Austria, pp. 573–580.

  • Khoo W. L., Knapp J., Palmer F., Ro T., Zhu Z. (2013). Designing and testing wearable range-vibrotactile devices, J. Assist. Technol. 7, 102117.

    • Search Google Scholar
    • Export Citation
  • Klatzky R., Lederman S., Reed C. (1987). There’s more to touch than meets the eye: the salience of object attributes for haptics with and without vision, J. Exp. Psychol. Gen. 116, 356369.

    • Search Google Scholar
    • Export Citation
  • Kupers R., Chebat D. R., Madsen K. H., Paulson O. B., Ptito M. (2010). Neural correlates of virtual route recognition in congenital blindness, Proc. Natl Acad. Sci. 107, 1271612721.

    • Search Google Scholar
    • Export Citation
  • Lamb G. D. (1983). Tactile discrimination of textured surfaces: psychophysical performance measurements in humans, J. Physiol. 338, 551565.

    • Search Google Scholar
    • Export Citation
  • Lederman S. J., Klatzky R. L. (2009). Haptic perception: a tutorial, Atten. Percept. Psychophys. 71, 14391459.

  • Lee V. K., Nau A. C., Laymon C., Chan K. C., Rosario B. L., Fisher C. (2014). Successful tactile based visual sensory substitution use functions independently of visual pathway integrity, Front. Hum. Neurosci. 8, 291. DOI:10.3389/fnhum.2014.00291.

    • Search Google Scholar
    • Export Citation
  • Lozano C., Kaczmarek K., Santello M. (2009). Electrotactile stimulation on the tongue: intensity perception, discrimination, and cross-modality estimation, Somatosens. Mot. Res. 26, 5063.

    • Search Google Scholar
    • Export Citation
  • Maidenbaum S., Abboud S., Amedi A. (2014). Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation, Neurosci. Biobehav. Rev. 41, 315.

    • Search Google Scholar
    • Export Citation
  • Maidenbaum S., Buchs G., Abboud S., Lavi-Rotbain O., Amedi A. (2016). Perception of graphical virtual environments by blind users via sensory substitution, PLoS One 11, e0147501. DOI:10.1371/journal.pone.0147501.

    • Search Google Scholar
    • Export Citation
  • Matteau I., Kupers R., Ricciardi E., Pietrini P., Ptito M. (2010). Beyond visual, aural and haptic movement perception: hMT+ is activated by electrotactile motion stimulation of the tongue in sighted and in congenitally blind individuals, Brain Res. Bull. 82, 264270.

    • Search Google Scholar
    • Export Citation
  • Meijer P. (1992). An experimental system for auditory image representations, IEEE Trans. Biomed. Eng. 39, 112121.

  • Myles K., Binseel M. S. (2007). The Tactile Modality: A review of tactile sensitivity and human tactile interfaces. U.S. Army Research Laboratory, ARL-TR-4115, Aberdeen Proving Ground, MD 21005-5425.

  • Nau A., Bach M., Fisher C. (2013). Clinical tests of ultra-low vision used to evaluate rudimentary visual perceptions enabled by the BrainPort vision device, Transl. Vis. Sci. Technol. 2, 1.

    • Search Google Scholar
    • Export Citation
  • Nau A. C., Pintar C., Arnoldussen A., Fisher C. (2015). Acquisition of visual perception in blind adults using the BrainPort artificial vision device, Am. J. Occup. Ther. 69, 6901290010p1-8. DOI:10.5014/ajot.2015.011809.

    • Search Google Scholar
    • Export Citation
  • Poirier C., De Volder A., Tranduy D., Scheiber C. (2006). Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects, Neuropsychologia 45, 11081121.

    • Search Google Scholar
    • Export Citation
  • Ptito M., Moesgaard S., Gjedde A., Kupers R. (2005). Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind, Brain 128, 606614.

    • Search Google Scholar
    • Export Citation
  • Ptito M., Matteau I., Gjedde A., Kupers R. (2009). Recruitment of the middle temporal area by tactile motion in congenital blindness, NeuroReport 20, 543547.

    • Search Google Scholar
    • Export Citation
  • Ptito M., Matteau I., Zhi Wang A., Paulson O. B., Siebner H. R., Kupers R. (2012). Crossmodal recruitment of the ventral visual stream in congenital blindness, Neural Plast. 2012, 304045. DOI:10.1155/2012/304045.

    • Search Google Scholar
    • Export Citation
  • Sabbah N., Authié C., Sanda N., Mohand-Saïd S., Sahel J.-A., Safran A., Habas C., Amedi A. (2016). Increased functional connectivity between language and visually deprived areas in late and partial blindness, NeuroImage 136, 162173.

    • Search Google Scholar
    • Export Citation
  • Sadato N., Pascual-Leone A., Grafmani J., Ibañez V., Deiber M., Dold G., Hallett M. (1996). Activation of the primary visual cortex by Braille reading in blind subjects, Nature 380(6574), 526528.

    • Search Google Scholar
    • Export Citation
  • Sadato N., Okada T., Honda M., Yonekura Y. (2002). Critical period for cross-modal plasticity in blind humans: a functional MRI study, NeuroImage 16, 389400.

    • Search Google Scholar
    • Export Citation
  • Sampaio E., Maris S., Bach-y-Rita P. (2001). Brain plasticity: ‘visual’ acuity of blind persons via the tongue, Brain Res. 908, 204207.

    • Search Google Scholar
    • Export Citation
  • Sathian K., Zangaladze A., Hoffman J., Grafton S. (1997). Feeling with the mind’s eye, NeuroReport 8, 38773881.

  • Spanos N. P., Stam H. J. (1979). The elicitation of visual hallucinations via brief instructions in a normal sample, J. Nerv. Ment. Dis. 167, 488494.

    • Search Google Scholar
    • Export Citation
  • Spence C. (2014). The skin as a medium for sensory substitution, Multisens. Res. 27, 293312.

  • Steele C. M., Hill L., Stokely S., Peladeau-Pigeon M. (2014). Age and strength influences on lingual tactile acuity, J. Text. Stud. 45, 317323.

    • Search Google Scholar
    • Export Citation
  • Steeves J., Harris L. (2012). Plasticity in Sensory Systems. Cambridge University Press, Cambridge, UK.

  • Stiles N. R., Shimojo S. (2015). Auditory sensory substitution is intuitive and automatic with texture stimuli, Sci. Rep. 5, 15628. DOI:10.1038/srep15628.

    • Search Google Scholar
    • Export Citation
  • Trulsson M., Essick G. K. (1997). Low-threshold mechanoreceptive afferents in the human lingual nerve, J. Neurophysiol. 77, 737748.

  • Van Boven R. W., Johnson K. O. (1994). The limit of tactile spatial resolution in humans: grating orientation discrimination at the lip, tongue, and finger, Neurology 44, 23612366.

    • Search Google Scholar
    • Export Citation
  • Vega-Bermudez F., Johnson K. O., Hsiao S. S. (1991). Human tactile pattern recognition: active versus passive touch, velocity effects, and patterns of confusion, J. Neurophysiol. 65, 531546.

    • Search Google Scholar
    • Export Citation
  • Voss P., Gougoux F., Lassonde M., Zatorre R. J., Lepore F. (2006). A positron emission tomography study during auditory localization by late-onset blind individuals, NeuroReport 17, 383388.

    • Search Google Scholar
    • Export Citation
  • Wan C., Wood A., Reutens D., Wilson S. (2010a). Congenital blindness leads to enhanced vibrotactile perception, Neuropsychologia 48, 631635.

  • Wan C., Wood A., Reutens D., Wilson S. (2010b). Early but not late-blindness leads to enhanced auditory perception, Neuropsychologia 48, 344348.

    • Search Google Scholar
    • Export Citation
  • Ward J., Meijer P. (2010). Visual experiences in the blind induced by an auditory sensory substitution device, Consc. Cogn. 19, 492500.

    • Search Google Scholar
    • Export Citation
  • Wicab Inc. (2008). BrainPort® technology tongue interface characterization Tactical Underwater Navigation System (TUNS): interim report for November 2006 to June 2008. Wright-Patterson AFB, OH: Air Force Research Laboratory, Human Effectiveness Directorate, Biosciences and Protection Division, Aircrew Performance and Protection Branch.

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 253 111 2
Full Text Views 138 1 0
PDF Downloads 7 0 0