Sensory substitution
From Wikipedia, the free encyclopedia
Sensory substitution is the principle to transform characteristics of one sensory modality into stimuli of another sensory modality. It is hoped that systems that base on sensory substitution can help handicapped people to restore the ability to perceive a certain sensory modality.
A sensory substitution system consists of three parts: a sensor, a coupling system, and a stimulator. The sensor records stimuli from one sensory modality and gives them to a coupling system which interpretes these signals and transmits them to a stimulator.
The research on sensory substitution raises many questions concerning human perception and the plasticity of the human brain.
Contents |
[edit] Applications
Applications are not restricted to handicapped persons, but also include artistic presentations, games, and augmented reality. Some examples are substitution of visual stimuli to audio or tactile, and of audio stimuli to tactile. Most popular are probably Paul Bach-y-Rita's Tactile Vision Sensory Substitution (TVSS), developed with Carter Collins at Smith-Kettlewell Institute and Peter Meijer's Seeing with Sound approach (The vOICe).
Technical developments, such as miniaturization and electrical stimulation help the advance of sensory substitution devices.
[edit] Tactile Vision Sensory Substitution
The TVSS converts the image from a video camera into a so-called tactile image. The tactile image is produced by four hundred activators placed either on the back, on the chest, or on the brow. The activators are solenoids of one millimeter diameter.
In experiments, blind (or blindfolded) subjects equipped with the TVSS can learn to detect shapes and to orient themselves. In the case of simple geometric shapes, it took around 50 trials to achieve 100 percent correct recognition. To identify objects in different orientations requires several hours of learning.
[edit] "Seeing with sound" sensory substitution
The vOICe vision technology is one of several approaches towards sensory substitution (vision substitution) for the blind that aims to provide synthetic vision to the user by means of a non-invasive visual prosthesis. The vOICe converts live camera views from a video camera into a soundscape.[1] Views are typically refreshed about once per second while associating height with pitch and brightness with loudness in a left-to-right stereo scan of every camera snapshot. The effective image resolution in sound is up to several thousand pixels as can be proven by spectrographic analysis. Another successful visual-to-auditory sensory substitution device is the Prosthesis Substituting Vision for Audition (PSVA)[2]
The ultimate goal is to provide synthetic vision with truly visual sensations by exploiting the neural plasticity of the human brain. Neuroscience research has shown that the visual cortex of even adult blind people can become responsive to sound, and seeingwithsound might reinforce this in a visual sense with live video from a head-mounted camera encoded in sound. The extent to which cortical plasticity indeed allows for functionally relevant rewiring or remapping of the human brain is still largely unknown and is being investigated in an open collaboration with research partners around the world.
The core behind this vision substitution system is a piece of software that converts the vertical position and brightness within a scanned source image into a sound wave that has a corresponding modulated frequency and amplitude. Other approaches to the substitution of hearing for vision use binaural directional cues, much as natural human echolocation does. An example of the latter approach is the "SeeHear" chip from Caltech.[3]
[edit] Criticism
It has been argued that the term "substitution" is misleading, as it is merely an "addition" or "supplementation" not a substitution of a sensory modality[4]
[edit] See also
- Biological Neural Networks
- Brain implant
- Human echolocation, blind people navigating by listening to the echo of sounds
[edit] References
- ^ P.B.L. Meijer, "An Experimental System for Auditory Image Representations", IEEE Trans. Biomed. Eng., 1992, 39: 112-121.
- ^ C. Capelle, C. Trullemans, P. Arno and C. Veraart, "A Real-Time Experimental Prototype for Enhancement of Vision Rehabilitation Using Auditory Substitution", IEEE Trans. Biomed. Eng., 1998, 45: 1279-1293.
- ^ L. Nielson, M. Mahowald, and C. Mead, "SeeHear," in Analog VLSI and Neural Systems, by C. Mead, Reading: Addison-Wesley, chapter 13, pp. 207–227, 1989.
- ^ Lenay C, Gapenne O, Hanneton S, Marque C, Geouelle C, 2003 "Sensory Substitution:limits and perspectives", in Touching for Knowing, Cognitive psychology of haptic manual perception, (Amsterdam /Philadelphia: Amsterdam /Philadelphia) pp 275-292.
