Estimating Geometric Scene Context from a Single Image


August 15, 2005


Alexei (Alyosha) Efros




Humans have an amazing ability to instantly grasp the overall 3D structure of a scene – ground orientation, relative positions of major landmarks, etc – even from a single image. This ability is completely missing in most popular recognition algorithms, which pretend that the world is flat and/or view it through a patch-sized peephole. Yet it seems very likely that having a grasp of this “geometric context” of a scene should be of great assistance for many tasks, including recognition, navigation, and novel view synthesis.

In this talk, I will describe our first steps toward the goal of estimating a 3D scene context from a single image. We propose to estimate the coarse geometric properties of a scene by learning appearance-based models of geometric classes. Geometric classes describe the 3D orientation of an image region with respect to the camera. We provide a multiple-hypothesis segmentation framework for robustly estimating scene structure from a single image and obtaining confidences for each geometric label. These confidences can then (hopefully) be used to improve the performance of many other applications. We provide a quantitative evaluation of our algorithm on a dataset of challenging outdoor images. We also demonstrate its usefulness in two applications: 1) improving object detection (preliminary results), and 2) automatic qualitative single-view reconstruction (“Automatic Photo Pop-up”, SIGGRAPH’05).

Joint work with Derek Hoiem and Martial Hebert at CMU.


Alexei (Alyosha) Efros

Alexei (Alyosha) Efros is an assistant professor at the Robotics Institute and the Computer Science Department at Carnegie Mellon University. His research is in the area of computer vision and computer graphics, especially at the intersection of the two. He is particularly interested in using data-driven techniques to tackle problems which are very hard to model parametrically but where large quantities of data are readily available. Before coming to CMU, Alyosha received his PhD from UC Berkeley and spent a year as a fine fellow at Oxford, England.