I lead the HoloLens Science team at Microsoft in Cambridge.  My research is focused at the intersection of computer vision, AI, machine learning, and graphics, with particular emphasis on systems that allow people to interact naturally with computers.


We’re hiring!  Looking for world-class engineers, post-docs, and researchers with expertise in computer vision, graphics, and machine learning.

Thrilled to be setting up a new research group to help invent the future for Microsoft HoloLens.

SIGGRAPH 2016 paper on efficient subdivision-surface based hand tracking.

CVPR 2016 paper on hand shape calibration.

Very excited to be chosen as an MIT Technology Review Innovator Under 35 2015!  More details in this article.

Research Highlights

Hand Pose Estimation.  Real-time, accurate, robust, and flexible articulated tracking of the human hand.

Decision Jungles.  Memory-efficient generalization of decision trees and forests with improved generalization.

Scene Coordinate Regression Forests.  A new approach to 6D camera pose estimation by regression 3D scene coordinates.

Human pose estimation for Kinect.  Our work on human body part recognition for Kinect.

Short Biography

Jamie Shotton is a Partner Scientist and leads the HoloLens Science team at Microsoft in Cambridge, UK, where his team focuses on the visual understanding of people to improve interaction and communication in mixed reality.  He studied Computer Science at the University of Cambridge, where he remained for his PhD in computer vision and machine learning. He joined Microsoft Research in 2008 where he was a research scientist and head of the Machine Intelligence & Perception group, before founding the HoloLens Science Cambridge team in 2016. His research focuses at the intersection of computer vision, AI, machine learning, and graphics, with particular emphasis on systems that allow people to interact naturally with computers. He has received multiple Best Paper and Best Demo awards at top-tier academic conferences. His work on machine learning for body part recognition for Kinect was awarded the Royal Academy of Engineering’s MacRobert Award 2011, and he shares Microsoft’s Outstanding Technical Achievement Award for 2012 with the Kinect engineering team. In 2014 he received the PAMI Young Researcher Award, and in 2015 the MIT Technology Review Innovator Under 35 Award (“TR35”).


Project Malmo

Established: June 1, 2015

How can we develop artificial intelligence that learns to make sense of complex environments? That learns from others, including humans, how to interact with the world? That learns transferable skills throughout its existence, and applies them to solve new, challenging…

SemanticPaint: Interactive 3D Labeling and Learning at your Fingertips

Established: June 29, 2015

We present a new interactive approach to 3D scene understanding. Our system, SemanticPaint, allows users to simultaneously scan their environment, whilst interactively segmenting the scene simply by reaching out and touching any desired object or surface. Our system continuously learns…

Fully Articulated Hand Tracking

Established: October 2, 2014

We present a new real-time articulated hand tracker which can enable new possibilities for human-computer interaction (HCI). Our system accurately reconstructs complex hand poses across a variety of subjects using only a single depth camera. It also allows for a…

Learning to be a depth camera for close-range human capture and interaction

Established: July 14, 2014

We present a machine learning technique for estimating absolute, per-pixel depth using any conventional monocular 2D camera, with minor hardware modifications. Our approach targets close-range human capture and interaction where dense 3D estimation of hands and faces is desired. We…


View by:


Efficient and Precise Interactive Hand Tracking through Joint, Continuous Optimization of Pose and Correspondences
Jonathan Taylor, Lucas Bordeaux, Thomas Cashman, Bob Corish, Cem Keskin, Eduardo Soto, David Sweeney, Julien Valentin, Benjamin Luff, Arran Topalian, Erroll Wood, Sameh Khamis, Pushmeet Kohli, Toby Sharp, Shahram Izadi, Richard Banks, Andrew Fitzgibbon, Jamie Shotton, in ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques, June 1, 2016, View abstract, Download PDF

















  • Tutorial on Decision Forests and Fields as presented at ICCV 2013.
  • 7-Scenes RGB-D camera relocalization dataset now available.
  • Decision Forests book including tutorial and software available here.

Former Interns