Microsoft pictures the future

Jetstream can automatically trace contours

By Mark Ward BBC News Online technology correspondent

Microsoft is working on ways to make digital images as easy to change and improve as text.

Scientists at the software giant's Cambridge research lab in the UK are developing tools that automate many of the complex tasks needed to enhance or edit amateur digital photos or images.

The tools can automatically trace outlines, seamlessly cover marks or blemishes, and fill in backgrounds when pieces of an image are removed.

The researchers are also working on similar tools that automate the editing of video clips.

Tool talk

Professor Andrew Blake, head of the image research group at the Cambridge lab, said many popular software packages made it difficult for people to make improvements to their digital images.

Jetstream can work out the edges of objects

His group is working on the equivalent of the automatic grammar checkers available in many word-processing packages.

He said many popular image programs were versions of those used by professional graphics designers and demanded that people use complex commands or fiddly techniques to improve images.

Far better, he said, was to get the computer to help with tasks that would otherwise involve making changes one pixel at a time or hours spent mastering the menus and options in the software.

"We want something that produces a reasonable effect for really not very much work," he said.

Picture slices

One tool produced by the group is called "jetstream". It automates the drawing of contours around pieces of an image.

Often marking the edges of an element within an image has to be done pixel by pixel to ensure that the true outline of the object is traced.

An image modified with patchworks before...

By contrast jetstream works out by itself where the edges of an image lie.

"Jetstream explores all the possible slices that a knife might make when removing the excess from an image element," said Professor Blake.

By looking forwards and backwards along potential contour lines, the computer works out the most likely route for the edge.

Users can redirect the contour path with clicks of a mouse.

"It checks branches of curves and evaluates curves against the evidence of the image itself," he said.

Crop and fill

Professor Blake's group is also working on ways to seamlessly fill in the hole left behind when one part of an image is removed.

...and after

Filling these gaps can leave a "shadow" of the cropped element making it obvious that something has been removed.

Tools that help fill the holes can take a long time to work, introduce artefacts or blemishes, and perform worst when trying to replace big chunks of images.

The Microsoft team has come up with a tool called "patchwork" that tries to do a better job.

"The program looks in different areas of the image to see which piece of texture it can steal to fill in the hole," said Professor Blake.

The researchers are also working on tools that can be used to edit and improve digital home video footage.

One tool can be told to follow the movements of one element in a video clip, such as a child or a water-skier, and then crops the footage to keep that element in the centre of the footage.

Although only at the experimental stage, the tool can significantly shrink video clips, yet retain their most interesting parts, said Professor Blake.

The still image tools being worked on by the image research team will find their way into a future release of Microsoft's Picture It! software.