LabelMe: online image annotation and applications

March 30, 2010
Bryan Russell | INRIA

Central to the development of computer vision systems is the
collection and use of annotated images spanning our visual world.
Annotations may include information about the identity, spatial
extent, and viewpoint of the objects present in a depicted scene. Such
a database is useful for the training and evaluation of computer
vision systems. Motivated by the availability of images on the
internet, we introduced a web-based annotation tool that allows online
users to label objects and their spatial extent in images. To date, we
have collected over 500K annotations that span a variety of different
scene and object classes. In this talk, I will show the contents of
the database, its growth over time, and statistics of its usage. In
addition, we use the collected user-provided object annotations to
extract the real-world 3D coordinates of images in a variety of
scenes. Important for this task is the recovery of geometric
information that is implicit in the object labels, such as qualitative
relationships between objects (attachment, support, occlusion) and
quantitative ones (inferring camera parameters). We show that we are
able to obtain high quality 3D information by evaluating the proposed
approach on a database obtained with a laser range scanner.

Joint work with Antonio Torralba (MIT) and William T. Freeman (MIT)

Speaker Details

Bryan Russell is a postdoctoral fellow in the INRIA Willow team at the
Ecole Normale Superieure in Paris, France. His research is in the
area of computer vision, with particular interests in object
recognition and scene understanding. Bryan received his PhD in 2007
from MIT.