Body movement to create music

By Jane Wakefield
BBC News Online technology staff

Using a mixing desk can be a complicated business

Sound engineers could ditch their mixing desks if the work of researchers at Leeds University becomes reality.

Scientists are developing ways of capturing human movement in three dimensions which would allow music to be created with the gesture of an arm.

It would eliminate the need for music technicians to twiddle hundreds of knobs to achieve the perfect sound.

The technique could also be used for scrolling a webpage, especially useful for people with limited mobility.

Tiny balls

The system is being developed at the school of music in the University of Leeds.

The possibility is for anybody to control a musical composition Dr Kia Ng

Dr Kia Ng of the Interdisciplinary Centre for Scientific Research in Music is leading the project, which captures 3D movements using infra-red light.

The light is projected onto tiny reflective balls attached to clothing and monitored by 12 cameras.

The computer recognises the changing positions of the balls and turns different gestures into instructions for music software.

Musical moves

"Effectively a person could play a note by blinking an eye or moving a foot. The possibility is for anybody to control a musical composition," Dr Ng told the BBC programme Go Digital.

Of course there are risks that the wrong gesture could lead to a bum note, so the system is also going to have a more pre-composed system that can intelligently guess what a series of gestures represents.

"The biggest challenge is to train the system to anticipate movement," said Dr Ng.

"To make sense of a gesture it need to know not only where an object has been and where it is, but also where it will be," he added.

He is hopeful that the system can be put to the test at a live concert by the end of next year.

You can hear more about the research on the BBC World Service programme, Go Digital