I lead the Machine Intelligence and Perception group at Microsoft Research Cambridge.  My research is focused at the intersection of computer vision, AI, machine learning, and graphics, with particular emphasis on systems that allow people to interact naturally with computers.


Research Highlights


Project Malmo

Established: June 1, 2015

How can we develop artificial intelligence that learns to make sense of complex environments? That learns from others, including humans, how to interact with the world? That learns transferable skills throughout its existence, and applies them to solve new, challenging…

SemanticPaint: Interactive 3D Labeling and Learning at your Fingertips

Established: June 29, 2015

We present a new interactive approach to 3D scene understanding. Our system, SemanticPaint, allows users to simultaneously scan their environment, whilst interactively segmenting the scene simply by reaching out and touching any desired object or surface. Our system continuously learns…

Fully Articulated Hand Tracking

Established: October 2, 2014

We present a new real-time articulated hand tracker which can enable new possibilities for human-computer interaction (HCI). Our system accurately reconstructs complex hand poses across a variety of subjects using only a single depth camera. It also allows for a…

Learning to be a depth camera for close-range human capture and interaction

Established: July 14, 2014

We present a machine learning technique for estimating absolute, per-pixel depth using any conventional monocular 2D camera, with minor hardware modifications. Our approach targets close-range human capture and interaction where dense 3D estimation of hands and faces is desired. We…



Efficient and Precise Interactive Hand Tracking through Joint, Continuous Optimization of Pose and Correspondences
Jonathan Taylor, Lucas Bordeaux, Thomas Cashman, Bob Corish, Cem Keskin, Eduardo Soto, David Sweeney, Julien Valentin, Benjamin Luff, Arran Topalian, Erroll Wood, Sameh Khamis, Pushmeet Kohli, Toby Sharp, Shahram Izadi, Richard Banks, Andrew Fitzgibbon, Jamie Shotton, in ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques, June 1, 2016, View abstract, Download PDF
















Short Biography

Jamie Shotton leads the Machine Intelligence & Perception group at Microsoft Research Cambridge.  He studied Computer Science at the University of Cambridge, where he remained for his PhD in computer vision and machine learning for visual object recognition. He joined Microsoft Research in 2008 where he is now a Principal Researcher. His research focuses at the intersection of computer vision, AI, machine learning, and graphics, with particular emphasis on systems that allow people to interact naturally with computers. He has received multiple Best Paper and Best Demo awards at top academic conferences. His work on machine learning for body part recognition for Kinect was awarded the Royal Academy of Engineering’s gold medal MacRobert Award 2011, and he shares Microsoft’s Outstanding Technical Achievement Award for 2012 with the Kinect engineering team. In 2014 he received the PAMI Young Researcher Award, and in 2015 the MIT Technology Review Innovator Under 35 Award (“TR35”).


  • Tutorial on Decision Forests and Fields as presented at ICCV 2013.
  • 7-Scenes RGB-D camera relocalization dataset now available.
  • Decision Forests book including tutorial and software available here.

Former Interns