It’s the Holy Grail of natural user interface (NUI): the ability to interact with a computer using speech, gesture and touch, seamlessly and simultaneously. Without these “multimodal” interactions, technologists say, computing will never be natural.
Over at Microsoft’s Envisioning Center, located on Microsoft’s Redmond campus, visitors can get a glimpse of computer interactions that will be as natural as breathing.
The Envisioning Center features a number of immersive, scenario-driven demos that explore how technology might change the way we work, live and play five to 10 years in the future. Most predict a new world of NUI. We’ll touch, swipe and tap away on our interactive workstations, of course. But step back a few feet, and we can still work on our computers by naturally speaking and gesturing. In meeting rooms, massive touchscreens will let multiple people interact simultaneously—all the while tracking who and where we are, what we’re doing and saying, and adapting and changing accordingly.
Anton Andrews (pictured), director of Envisioning in Office, and Jonathan Cluts, director of Microsoft's Strategic Prototyping team, planned everything at the Envisioning Center, from the physical architecture to the code that is powering its innovative scenarios.
Many of those futuristic scenarios are powered by two Microsoft technologies available today: Kinect for Windows and Perceptive Pixel (PPI) displays. (Perceptive Pixel is the company that Microsoft acquired in 2012 that makes the large, multi-touch and pen devices that you can see on CNN, for example.)
The Envisioning Center team was halfway through designing scenarios when Microsoft acquired PPI. A light bulb went off, say Jonathan Cluts and Anton Andrews, who planned everything at the Center from the physical architecture to the code powering its innovative scenarios.
“Immediately we put our heads together and started to think about how to put the two devices together because they can unlock so many scenarios,” says Andrews, director of Envisioning in Office.
Cluts and Andrews were already thinking about how the Kinect for Windows sensor and its features—depth sensing, facial recognition, voice and audio capabilities—might work across different Microsoft devices. “Once we thought about Kinect that way, the bezel of a screen the size of a PPI display became an obvious choice,” says Cluts, director of Microsoft’s Strategic Prototyping team.
They reached out to both the PPI and Kinect for Windows teams, who agreed there was a peanut-butter-and-jelly type relationship. The resulting marriage—and its convergence of natural user interactions to create a seamless user experience—is on display for the company’s engineering teams and key customers. But it only hints at the new NUI models that are starting to spread across Microsoft’s devices and services.
Just as Kinect for Windows sensors are everywhere, from walls to oven hoods, at the Envisioning Center, the team is working to bring NUI advances across Microsoft, says Michael Mott, general manager of Xbox Apps and the Developer Ecosystem. PPI was the perfect test case.
“It brings together the best of what’s happening in natural user interfaces with the best of what’s happening with large touch screens, and seeing if one plus one can’t equal three for the end user,” he says. “PPI was a great launching pad, and there’s nothing but NUI goodness ahead as we take what we’ve learned with Kinect and start to sprinkle it across the company.”
Mott and his team want to become part of the fundamental fabric of Windows, so that groups across the company can view Kinect not just as a camera, but as a way to unlock NUI experiences across Microsoft’s products and services.
The Envisioning Center is a great showcase for Microsoft’s devices and services strategy and what it could unlock in the future, Cluts says. “The approach we took with the Envisioning Center’s scenarios was always people first. What is the end user trying to achieve? When you start there, the company becomes this great shopping list of devices and services we can use to create new experiences for our users.”
Jeff Han, general manager of Perceptive Pixel, believes that Microsoft's breadth can help set the company apart from the competition.
Jeff Han, general manager of Perceptive Pixel, agrees. Even before his company was acquired by Microsoft, he attached a Kinect for Windows sensor atop a PPI display. Now he thinks Microsoft’s breadth can help set it apart from the competition.
“You can’t succeed anymore by going narrow and deep on only one piece. You need to make products and experiences with multiple components that are all best-in-class, Han said. “Not too many people can compete with us. We have the best software, the best 3D sensor, the best large screen touch + pen technology, all connected together seamlessly by the cloud.”
He’s a fan of the Envisioning Center and often takes customers there. It helps people see PPI not only as a giant touch screen but also as a canvas for Microsoft’s devices and services.
“The Envisioning Center helps customers understand we’re not just a company that sells individual widgets and software licenses,” Han said. “We have a vision for what the future of technology is going to be, and the Envisioning Center does a great job articulating it.”
You might also be interested in: