SemanticPaint: Interactive 3D Labeling and Learning at your Fingertips

Established: June 29, 2015

We present a new interactive approach to 3D scene understanding. Our system, SemanticPaint, allows users to simultaneously scan their environment, whilst interactively segmenting the scene simply by reaching out and touching any desired object or surface. Our system continuously learns from these segmentations, and labels new unseen parts of the environment. Unlike offline systems, where capture, labeling and batch learning often takes hours or even days to perform, our approach is fully online.

To be presented at SIGGRAPH ’15. SemanticPaint is part of an ongoing collaboration between Microsoft Research and University of Oxford.



SemanticPaint Video:
SemanticPaint YouTube video

Supplementary Material Video:
SemanticPaint supplementary material video

Main Contacts:

  • Shahram Izadi, Microsoft Research:
    Interactive 3D Technologies, Microsoft Research: 



Portrait of Antonio Criminisi

Antonio Criminisi

Partner Research Lead Mesh Labs (Mixed Reality & AI) Microsoft Cambridge