Microsoft holds Innovation Day

By Dan Simmons Reporter, BBC Click Online

Innovation Day is a chance for researchers at Microsoft to show off and to bounce around ideas about how we might use technology in the future.

One program which is a few years away from being used in our homes is an interactive table.

The business world may have use for the interactive table

An image is projected on to a table alongside an invisible infra-red light. Using the infra-red signal, a camera can detect movement which is not part of the projected image - our hands for example - and their position can be fed back to a computer which adapts the display accordingly.

"We can sense and detect all sorts of real objects as they're placed on the surface," said senior researcher Andy Wilson.

"We have chips marked with a special code that displays a different object for each when they are placed on the surface."

Apart from gaming, Andy hopes the technology will be used by the business world too, perhaps icons to sync phones, or to represent documents that can then be opened on the real desktop. Andy Wilson: "We can also sense a regular piece of paper as it's placed on the surface. I can then move it around the surface and the video being projected onto it moves along. You can think of it as a kind of display."

The table also boasts a mapping application, which makes use of Microsoft's Virtual Earth, allows the user to spin the map and zoom into places of interest just by moving their hands.

Getting a computer to understand what is around it is the principle behind this next project too.

Object recognition

"What we're doing is recognising objects at very high speed, and not just individual objects but also classes of objects, said researcher John Winn.

"So for example it can recognise pens it hasn't seen before, or types of fruit it's not seen before."

A banana is correctly identified

Object recognition is nothing new, but this system is extremely quick - chuck a banana its way and it knows within a few frames.

At the moment it recognises about 20 different object types.

It could be useful while shopping, because many things we buy do not have barcodes or radio tags so automated checkouts just do not know what an orange is unless we tell it. Recognising the object could speed up the process.

If it can be done with objects - how about identifying your face?

One piece of software on show creates a virtual 3D image from a flat one. Left to its own devices its developers claim an 85% success rate in getting an accurate likeness, and they are trying to improve on that.

"It takes three minutes to identify a face and to estimate the full 3D shape of one image from a single 2D image," said Sami Romdhani, PHD Supervisor at Basel University.

"In the future we hope to do that in real time for video sequences. People could use that, for instance while chatting, and use the system to exaggerate their expression or smile or do various things with their face."

The software has a serious side too: it aims to enhance security camera footage by reconstructing the parts of the face it is unable to see.

Speech search

Here is a lovely idea based on speech recognition systems which may one day allow us to search television and video clips not just by the title and metadata, but by the actual words spoken in the programme itself.

We tapped in "election results" and within seconds a video stream had been found in which those words had been said.

You can also create whole channels around search terms, which regularly scan the library for new material and update your channel while you are away.

With video-on-demand fast becoming a challenger to live TV this could be a new way to search for video content.

"The current system subscribes to about 100 RSS feeds of video podcasts. So we download those video files and go through them using automatic speech recognition to recognise the words that are spoken," said research manager Frank Seide.

That takes about three to four minutes for one minute of video. Popular websites like YouTube get up to 50,000 uploads a day, so you might think there's tremendous computational power needed there.

"But if you do the maths you actually only need something like 200 processors to do complete processing of the volume of one day."

Given that the system offers a search of what it has said, it would have been nice to include a voice activated search, but ironically its developers are not confident it would be reliable enough.

One of the nicest things about the system is the white bar that appears at the bottom of the clip and an orange mark on the clip indicates where the words you were searching for occur, allowing you to skip to the most relevant parts.

Copyright problems

Ever wanted to share a TV moment with absent friends? How about an instant record button on your mobile.

The answerphone has had an overhaul

A constantly updated set of downloads would be held on a central server, ready to be downloaded, or they could be broadcast over a mobile TV network like DV-BH, available to be recorded in one or two minute bites.

But like many new ideas in multimedia, copyright issues are slowing development.

"The interesting problem is that, from a research perspective, this is very exciting," said Tim Regan, research software development engineer.

"From a product perspective it is very exciting, it extends the brand of the broadcaster out in new and interesting ways.

"From a legal perspective it's very un-obvious that this is a good idea. It's the discussion between the researchers and the legal people that's proving the really difficult and time consuming work at the moment."

Of course software is available today that can record live TV onto your PC, PVR, or even an XBox360, but these guys want portability without any extra kit.

Future interaction

The traditional answerphone is long overdue for an update, and the bubbleboard offers one possibility. On the touch screen you can see all your messages at once, and the bigger the bubble the longer the message.

Newer messages simply float to the top while older messages head for the drain and deletion, unless you save them on the right hand side or get there first, dragging an unwanted rant to it's fate.

Talking of new interfaces for messages - how about getting physical with your email by simply connecting a dance mat familiar to game console - fans you can open, read, save, or delete.

Replying is tricky, but at least the more spam you get the fitter you become.

Other applications using the mat are being explored too. But, like all the ideas talked about on this page, nothing is ready to go to market for at least a year.

The aim for now is to simply explore new ways we can interact.