By Darren Waters
Technology editor, BBC News website, Texas
Microsoft Surface is helping re-think how we interact with computers
The success of the iPhone has given rise to a new grammar of touch control while the advent of multi-touch in Windows 7 will further accelerate the evolution of human computer interfaces, the South by SouthWest festival has been told.
The minute-long sequence in the film Minority Report in which Tom Cruise manipulated images on a screen using simple gestures has quickly become a cliche of future human computer interfaces.
While based on the real world science of John Underkoffler, who has since co-founded a company called g-Speak that sells the technology, it has become both the vision and barrier for many user interface and user experience designers.
The reality of standing at a giant screen and wearing special gloves may eventually render the Minority Report vision impractical, but it does reflect one certainty: the days of the mouse keyboard and desktop graphical interface are numbered.
The days of the mouse may be numbered.
Ben Rigby, from web firm Mobile Voter, told the conference that the "paper-like layered interface" of systems like Windows, Linux and Mac OS, did not work in today's social computing world.
"It's essentially paper. we are socialising on a flat paper-like interface. This is a 30-year-old interface.
"Hollywood is leading the way and showing what the future is going to look like."
Joe Engalan, director of development for Vectorform, which produces applications for Microsoft's Surface device, said: "Minority Report will happen eventually but with five people touching something virtual somewhere.
"It's getting there. It's getting started."
While the technology to read and interpret human gestures, and physical touching already exists part of the challenge is about developing a codified understanding of such actions.
"The big question to be answered is about building a library of what gestures mean," said web developer Dan Thompson.
We have to define that dictionary of gestures, agreed Mr Engalan.
Many agree that the success of the iPhone has initiated the first common standards around multi-touch: pinch, touch and sweep.
Touch screen devices are becoming more common and popular
Chris Bernard, a user experience evangelist for Microsoft, told BBC News: "In the real world we are working out how to build things like the Minority Report interface."
He said there had not been a lot of innovation since Xerox launched its desktop graphical user interface, the basis of almost every operating system for the last 20 years.
"We are on the tipping point when things are going to change a lot; mobility and touch and gesture, as well as ubiquity are driving that," said Mr Bernard.
Many user interface designers believe gesture based controls will be the next mainstream evolution of human computer interfaces.
Mr Bernard said: "It's hard to predict the future but it's hard not to say that touch isn't going to continue to innovate in mobile.
"Right now we talk about multi-touch screen; in the future it will be air-based gestural movements with the device.
"Shaking it, waving your hand over it in someway, putting your finger behind the device; these will all be methods to control your device
He added: "I wouldn't be surprised if in a few years every visual device had touch capabilities."
Microsoft's second generation Surface will be able to read gestures as well as physical touches to the screen thanks to infrared cameras embedded in the device.
Jeroen Lapre, a former visual effects technician at Industrial Light and Magic, has been working extensively on 3D virtual worlds in his new role at the California Academy of Sciences.
"We do want to be able to touch and gesture to what we are working with and relate it to our body," he said.
Beyond that, user experience and interface designers are experimenting with holographic projection, the use of 3D avatars and the challenges of working collaboratively in the same virtual space, or augmenting reality.
BMW has made cars featuring a crude head-up display.
BMW has already launched cars that overlay real world data and information onto the windscreen of cars, while Microsoft's second generation Surface will be able to project information not only to a screen but any surface, while MIT has been experimenting with personal projectors worn on the body.
User interfaces and user experiences were being humanised, said Mr Rigby.
The increased richness and sophistication of web browsers thanks to tools such as Flash and Silverlight is also giving designers a chance to re-write the rules of user experiences.
Websites such as Cool Iris, Zoomorama, Viewzi and Google Earth in the browser are transforming web surfing from a flat experience into something much more immersive.
Hakon Wium Lei, chief technology officer for browser maker Opera, and the creator of Cascading Style Sheets, which helped enrich the web visually, told BBC News that the web was set to become easier on the eye.
"We are going to a richer web, a more beautiful web with more stylistic options for designers, with downloadable fonts, more video and all the information that mankind produces will be on the web."
But any development and evolution in user experience had to take into account the impact it would have on users, said Mr Bernard.
"Microsoft is designing software for millions of people. A billion people use Windows and it's a challenge to do good design for an audience that diverse.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.