For decades we have used the keyboard and mouse to interact with computers but our homes and offices may not resound to the tapping of keys and clicking of mice for much longer. In fact, before long, these seemingly indispensable devices may well be heading for obsolescence in the same way as the 8-track tape, video cassette recorder and instant camera.
A new form of technology – gesture-based computing – is taking the gaming industry by storm and promises to revolutionise the way we all control and interact with our PCs, smartphones and other portable devices.
In 2002, Stephen Spielberg asked the inventor John Underkoffler, his science adviser on the film Minority Report, to incorporate futuristic concepts that were within the realm of possibility. Cinema audiences were spellbound as they watched Tom Cruise manipulating 3D computer images through hand gestures.
What seemed the stuff of science fiction has now become reality. Last year Underkoffler exhibited a real-life version of the 3D interface used in the film at the TED (Technology, Entertainment and Design) Conference in Long Beach, California.
Many of us are already familiar with the iPhone and iPad devices, which react to taps, swipes and shakes. However, it is the gaming industry, with a global value of more than $100 billion, that is the crucible for innovation and the development of this potentially highly-lucrative emerging technology.
The Nintendo Wii, which was released in 2006 and responds to body movement, led the way. Last November Microsoft released the Kinect for Xbox 360, which features cutting-edge gesture-based technology. The program uses a camera and infrared light to detect and analyse body movements, voice and even facial expressions, allowing the user to control game characters using their movement and voice alone.
Although the gaming industry is leading the charge, the impact of gesture-based technology is already being felt more widely. New devices have allowed doctors to manipulate images and data from X-ray, MRI and CT scans far more effectively and gesture-based interfaces are being developed to help healthcare professionals improve their dexterity and familiarise themselves with complex surgical procedures.
The technology could prove useful in any environment where manually controlling a machine or computer is difficult or dangerous. Without the need for a controlling device, the technology could be used by surgeons in operating theatres, or by scientists in laboratories, to control machines while maintaining a sterile environment. The oil and gas industry could also make use of the technology to allow offshore workers to operate computers or machines without worrying about oily or dirty hands.
Gesture-based technology can also be used to generate 3D replicas of objects, humans and environments. Using depth sensors combined with holographic display technology, Holovision recreates 3D images that users can interact with directly. Architects and builders will not be limited to two dimensions for showcasing their plans, but could allow clients to inspect the virtual model of their unbuilt house.
By bringing designs to life, the technology could play a role in urban planning, product design and structural engineering.
With Microsoft having already demonstrated its office of the future, in which the desk is a touch screen surface, walls become digital displays and computers are controlled by voice, touch and gesture recognition, it is clear that it will not be long before this intuitive way of interacting with ICT is adopted by businesses. If machines can be operated and data stored more quickly and easily then work can be done more efficiently and cost effectively.
Videoconferencing has already been given a gesture-based makeover. A new product, VenueGen, allows business executives to sit in their office with their feet up while an avatar – individually created to look like them, including facial expressions and body language – remains seated and attentive in a virtual meeting room.
The main barrier to the widespread adoption of gesture-based technology is, of course, cost. Despite being relatively inexpensive, businesses are unlikely to start investing in holograms and gadgets at a time when profits are low and the economic climate is uncertain. For business applications to take off, the market needs to be ready and willing.
There are also practical limitations to the use of gesture technology. Using hand and body gestures to control a computer interface is not (at the moment at least) as accurate as pressing a button or clicking a mouse. The quirks and flaws inherent in this early-stage technology might be acceptable in video games but not in business. While an accidental movement in a game is relatively inconsequential, in business the implications could be serious if sensitive data is deleted or sent in error, or where a false move could be costly to personal safety.
Nevertheless, as the technology improves and businesses strive for the competitive edge, these challenges are bound to be overcome. With the right applications, the technology could allow us to collaborate and share data more easily, make business communication more engaging and improve the design process and operations in creative and engineering industries.