Blind 'see with sound'

By Lakshmi Sandhana

Michelle Thomas is learning to "see", not with her eyes but her ears.

A computer reconstruction of one second of sound as seen by the vOICe system

Now she can also use a mobile camera phone to do it.

Blind since birth, Ms Thomas is able to recognize the walls and doors of her house, discern whether the lights are on or off and even distinguish a CD from a floppy disk after only a week using a revolutionary new system.

She is "seeing with sound".

Developed by Dr Peter Meijer, a senior scientist at Philips Research Laboratories in the Netherlands, the system is called The vOICe (the three middle letters standing for "Oh I See").

It works by translating images from a camera on-the-fly into highly complex soundscapes, which are then transmitted to the user over headphones.

Watch the 'spikes'

A wearable setup consists of a head-mounted camera, stereo headphones and a notebook PC.

In total it costs about $2,500. The software is available as a free download.

A blind user wearing The vOICe goes in search of her lost rubbish bin

Meijer is bargaining on the brain's adaptive capacity.

He hopes that blind users will ultimately learn to mentally reconstruct the visual content of the live camera views, as carried by the soundscapes, so that they experience something akin to meaningful vision.

"Our assumption here is that the brain is ultimately not interested in the information 'carrier' (here sound) but only in the information 'content'," says Meijer.

"After all, the signals in the optic nerve of a normally sighted person are also 'just' neural spiking patterns. What you think you 'see' is what your brain makes of all those firing patterns."

Easy 'tongue'

Enabling users to get an audio snapshot of what is visually in front of them, The vOICe is taking a very different route from "bionic eyes" - retinal and brain implants.

It is non-invasive, offering a higher image resolution (up to several thousand pixels) and does not necessarily rely on the visual cortex.

The system's developer hopes ability will come with practice

"Everything has its own unique sound and once you learn the principles involved you can know what you're seeing," says Thomas.

Right now brighter areas sound louder, height is indicated by pitch and a built-in colour identifier speaks out colour names when activated.

While it can't track fast cars or read small print efficiently, it does allow blind users to trace out buildings, read a graph and even watch television.

Comparing it in terms of difficulty to learning a foreign language, Meijer hopes that in the long run, users will become more "fluent" in the mental translation so that it becomes more like natural perception, without conscious effort.

Mobile vision

Kevin O'Regan, of the CNRS (National Centre for Scientific Research) in Paris, France, and an expert in the area of sensory consciousness, is currently evaluating The vOICe.

This headset connects to a mobile PC

He believes that if perfected, the software could at least partially evoke vision-like sensations in even the congenitally blind.

"The problem is that vision is a very high bandwidth system, and it's not clear whether we can achieve sufficient bandwidth via other modalities," he stated.

To suit user preferences, Blue Edge Bulgaria has developed a simplified but highly portable mobile phone version of The vOICe for the Nokia 3650 camera phone.

It is available as a free download at The vOICe site.