Semantic Segmentation using Higher-Order Markov Random Fields

June 27, 2011
Lubor Ladický | Oxford Brookes University

Many scene understanding tasks are formulated as a labelling problem that tries to assign a label to each pixel of an image, that may correspond to different object classes such as car, grass or sky, or to depths or to intensity after denoising. These labelling problems are typically formulated as a pairwise Random Field, modelling the dependencies of labels of pairs of variables in the local neighborhoods. However, these pairwise models are very restricted in their expressivity and can not induce desired complex structures in the output labelling.

In this work we propose global structured formulation beyond pairwise models, that can combine features over multiple scales and enforce globally consistent label-set in the resulting labelling. We show that this complex model can still be learnt and optimised efficiently. Proposed method has been tested on several semantic segmentation data sets, where it gets (close to) state-of-the-art results.

We then show how multiple random field models can be formulated jointly across different domains. We demonstrate the usefulness of this model on the problem of joint semantic segmentation and dense 3D stereo reconstruction and show that this approach significantly outperforms existing methods for street scenes.

The complete source code is publicly available at http://cms.brookes.ac.uk/staff/PhilipTorr/ale.htm

Speaker Details

Lubor Ladický studied Automation in 1999-2005 at the Slovak University of Technology in Bratislava. He also studied Theoretical Physics in 2000-2005 at the Comenius University in Bratislava, and finished both schools with a Master degree. After a year in the industry, he started his PhD in Computer Vision at the Oxford Brookes University supervised by Phil Torr and defended his thesis in April this year. He is presently a post-doc at the University of Oxford in the group of Andrew Zisserman.