3D Mars recreated from photos

By Mark Ward
BBC News Online technology correspondent

Researchers are creating 3D images of the Martian surface using photographs taken by Nasa's robot rovers.

Nasa's rovers are taking lots of shots of Mars

Using techniques originally applied to paintings, the researchers have come up with novel machine vision techniques that work out what flat images would look like from different viewpoints.

The result is a navigable 3D model created from one or more pictures snapped by the Mars buggies.

The first fruits of the research are short video sequences that give 3D views of Mars without the need for special spectacles.

Replacing the brain

The work is being carried out by Antonio Criminisi and Andrew Blake from Microsoft's research labs in Cambridge.

Dr Criminisi has also developed algorithms that can take a single flat image or painting and turn it into an virtual environment.

This project, begun by Criminisi at the University of Oxford (under the supervision of Prof. Andrew Zisserman) and recently extended, has lead to the 3D reconstruction of beautiful Renaissance paintings

The algorithms work out how the objects depicted relate to each other and makes guesses about how to draw objects that are obscured in a flat view but are exposed in three dimensions.

"We extract 3D information out of the image," said Dr Criminisi. "We work out what's behind an object and how we can automatically fill in what's not seen by cameras or painted."

The resulting virtual recreation lets people explore what a scene looks like from different viewpoints.

Paintings have been turned into 3D models

Typically, said Dr Criminisi, anyone wanting to create a 3D version of a scene has used anaglyphs that, with the help of glasses, present different images to each eye.

"Your brain does the job of fusing these two flat images to produce a 3D model," he said. "We use computer power to do that job for you and give an impression of the 3D object in front of you."

Dr Criminisi has applied this technique to several famous paintings including The Flagellation of Christ by Piero della Francesca and Steinwick's Saint Jerome.

Dr Criminisi said it took about five minutes to generate the model from an image.

He said that the more images the computer had to work with, the easier it became to create the models.

"Because we have so much more information the whole process of reconstructing the scene can be done automatically," he said.

The first short video sequences have been generated using a couple of images sent back to Earth by the Spirit rover.

Now Dr Criminisi and colleagues are working on longer sequences.

Eventually, he said, people might be able to navigate through the 3D model of Mars from the comfort of their sofa using a joystick or gamepad.