Portrait of Toby Sharp

Toby Sharp

Principal Software Architect, HoloLens


Toby Sharp is the software architect for the HoloLens team at Microsoft in Cambridge, UK. Prior to this he was a developer in Microsoft Research since 2005. He has transferred several research projects into Microsoft technologies and products, and has contributed computer vision components to Microsoft’s Kinect, Office and LifeCam software. His other work includes computer graphics, image editing and GPU programming. He holds a BSc in Mathematics (1st) from the University of York.

Prior to joining Microsoft, he developed consumer photo and video editing software for Serif in the UK. In 2010 he received the Excellence in Design award from RSNA for real-time virtual colonoscopy software. In 2011 he and colleagues received the Best Paper prize at CVPR and the Royal Academy of Engineering MacRobert Award for the machine learning contribution to Kinect’s body tracking.

Toby is a Senior Member of the IEEE.


Fully Articulated Hand Tracking

Established: October 2, 2014

We present a new real-time articulated hand tracker which can enable new possibilities for human-computer interaction (HCI). Our system accurately reconstructs complex hand poses across a variety of subjects using only a single depth camera. It also allows for a high-degree of robustness, continually recovering from tracking failures. However, the most unique aspect of our tracker is its flexibility in terms of camera placement and operating range.   Screenshots Please note, we…

Decision Forests

Established: July 25, 2012

Decision Forests for Computer Vision and Medical Image Analysis A. Criminisi and J. Shotton Springer 2013, XIX, 368 p. 143 illus., 136 in color. ISBN 978-1-4471-4929-3  

KinectFusion Project Page

Established: August 9, 2011

This project investigates techniques to track the 6DOF position of handheld depth sensing cameras, such as Kinect, as they move through space and perform high quality 3D surface reconstructions for interaction. Other collaborators (missing from the list below): Richard Newcombe (Imperial College London); David Kim (Newcastle University & Microsoft Research); Andy Davison (Imperial College London)    

Human Pose Estimation for Kinect

Established: January 25, 2011

Kinect for Xbox 360 and Windows makes you the controller by fusing 3D imaging hardware with markerless human-motion capture software. Our group investigates such software. Mixing computer vision, graphics, and machine learning techniques, we look at how to build algorithms that can learn to recognize human poses quickly and reliably. Images Traditional RGB image

InnerEye – Assistive AI for Cancer Treatment

Established: October 7, 2008

InnerEye is a research project that uses state of the art artificial intelligence to build innovative image analysis tools to help doctors treat diseases such as cancer in a more targeted and effective way. "...We are pursuing AI so that we can empower every person and every institution that people build with tools of AI so that they can go on to solve the most pressing problems of our society and our economy. That’s the…


Established: February 26, 2007

Video images live in a 3D world. This project is about unlocking that 3D information to allow complex special effects with simple user interaction.       Andrew Fitzgibbon Toby Sharp Video looks at a 3D world, so users working with video should be able to interact with that 3D world. Editing and interaction with the video, however, remains based on 2D interface paradigms which have evolved…

Image and Video Editing at MSR Cambridge

Established: January 23, 2002

At Microsoft Research in Cambridge we are developing new machine vision algorithms for intelligent image and video editing and browsing. Our technology provides tools for: accurate interactive segmentation and matting, color correction, easy object removal and image restoration, and seamless object insertion. News! AutoCollage is now available as a product from Microsoft Research Cambridge. Click here to get a free trial version.  Computer Vision at MSR Cambridge

Image Understanding

Established: January 1, 2000

At Microsoft Research in Cambridge we are developing new machine vision algorithms for automatic recognition and segmentation of many different object categories. We are interested in both the supervised and unsupervised scenarios.   Research data Download labelled image databases for supervised learning in the "Downloads" link below. The data provided here may be used freely for research purposes but it cannot be used for commercial purposes. Database of thousands of weakly labelled, high-res images. Pixel-wise labelled…