Gestures and Flexible Film Could Augment Touch
It might seem like multi-touch is the culmination of user interfaces (UIs), but something new could come to change the way we interact with devices. The industry will evolve to something new — we’re shifting into a touch-plus era, and corporate research and design laboratories make excellent Petri dishes for observation as to how this is happening.
Microsoft Research has unveiled several in-lab projects this week which could provide some insights into what the company has in mind for future generations of devices. Not all lab work comes to fruition, but the possibilities of intriguing demonstrations tend to excite the industry to change. Touch-screen input is already complemented with voice and contextual-based information, but other dimensions are on their way.
One Microsoft Research project has developed an algorithm which enables handsets to recognise sign language-like hand gestures in real-time. The software uses a device’s camera to monitor and interpret hand signals to either augment or replace touch input. Usage scenarios include developing advanced interfaces for the speech impaired, or allowing users to maneuverer their way through a UI in situations when touch or voice aren’t practical. Microsoft says that the in-air gestures are correctly recognized up to 98% of the time, an accuracy level which rivals touch. Expanding the user interface to three dimensions by using the space around a device isn’t unprecedented (Microsoft’s Kinect can recognize movement), but making detailed gesture recognition mainstream in smartphones, tablets and smartwatches has been a long-term industry goal.
In another project called FlexSense, Microsoft Research demonstrates how a thin, transparent film can be used as an accessory that can rest on top of a tablet screen or be twisted and turned to simulate real-world objects. Microsoft demonstrated how the film could hide information on a tablet, or be used to display information when the user lifts the film from the device. Microsoft also showed some rudimentary games using the flexible film. FlexSense is another 3D input experiment from Microsoft that provides some indication of input technologies to come. It might not be made into a product directly, but the inspiration — and the patents — are out there.
More flexibility is coming to devices, and the enablers for new forms and designs for hardware and software are headed to the industry. UI projects involving eye movement, thought processes and stress levels are coming out of the lab toward the mainstream. The beyond-touch era is approaching.