Kinect Skeletal Tracking
Momin Al-Ghosien, Matt Bronder, Robert Craig, Mark Finocchio, Alex Kipman, Samuel Mann, Parham Mohadjer, Craig Peeper, and Jamie Shotton
2012 Outstanding Technical Achievement
Kinect isn't just the fastest selling consumer electronic device, it is the best example yet of a magical experience that comes when the computer knows you instead of you having to learn about your computer.
The magic of Kinect was predicated on the amazing work of people to bring together the capacity to economically manufacture Kinect at scale, to make the Kinect understand when you talk, know who you are when you walk up to it, and be able to interpret your movements and translate into a format where developers can build experiences that were never before imagined. This magic starts with Skeletal Tracking, and Microsoft honors the Kinect Skeletal Tracking team members as the recipient of the 2012 Outstanding Technical Achievement award.
The operational envelope demands for commercially viable Skeletal Tracking is enormous. Simply put, Skeletal Tracking must ideally work for every person on the planet, in every household, and without any calibration. A dauntingly high number of dimensions describe this envelope, such as distance from the Kinect sensor and the sensor tilt angle. Entire sets of dimensions are necessary to describe the unique individuals including size, shape, hair, clothing, motions, and poses. Household environment dimensions are also necessary for lighting, furniture and other household furnishings, and pets.
The fields of machine vision and learning began decades ago, advancing greatly in the last 20 years, yet commercially viable Skeletal Tracking was an academic dream. To bring the vision and possibilities demonstrated by Alex Kipman's incubation group to shipping reality, Robert Craig formed and led the Skeleton Crew, a virtual team across the Interactive Entertainment Business and Microsoft Research divisions. Product engineers, graphics and vision experts, machine learning theorists, and mathematicians were brought together to rapidly explore solutions to Kinect's many open challenges.
Delivering Skeletal Tracking required its core team members to be scientist engineers, conducting research with rigorous application of the scientific method, while delivering production-quality solutions. The result is a tracking pipeline capable of filling Kinect's ambitious operational envelope. Leveraged by the outstanding creativity of application developers, magical never-before-seen experiences emerge.
Skeletal Tracking shipped to Xbox developers in Fall 2009, and Kinect shipped for the 2010 Holiday Season and became the fastest selling electronic device of all time. Skeletal Tracking's innovation continues since Kinect's launch. The Kinect for Windows SDK includes APIs, sample code and drivers. Skeletal Tracking's operational envelope has also expanded to enable developers to create experiences in a seated position on Xbox 360 for Holiday 2011. Improving Skeletal Tracking quality opens possibilities to more developers, applications, and industries, going well beyond Kinect's inception in living room entertainment.
