Semi-supervised Learning in Gigantic Image Collections


December 4, 2009


Rob Fergus


Courant Institute, New York University


With the advent of the Internet it is now possible to collect hundreds of millions of images. These images come with varying degrees of label information. “Clean labels” can be manually obtained on a small fraction, “noisy labels” may be extracted automatically from surrounding text, while for most images there are no labels at all. Semisupervised learning is a principled framework for combining these different label sources. However, it scales polynomially with the number of images, making it impractical for use on gigantic collections with hundreds of millions of images and thousands of classes. In this paper we show how to utilize recent results in machine learning to obtain highly efficient approximations for semi-supervised learning. Specifically, we use the convergence of the eigenvectors of the normalized graph Laplacian to eigenfunctions of weighted Laplace-Beltrami operators. We combine this with a label sharing framework obtained from Wordnet to propagate label information to classes lacking manual annotations. Our algorithm enables us to apply semi-supervised learning to a database of 80 million images with 74 thousand classes.

Joint work with Yair Weiss and Antonio Torralba.


Rob Fergus

Rob Fergus is an assistant professor of computer science at the Courant Institute, New York University. He recieved his PhD from University of Oxford, jointly advised by Prof. Andrew Zisserman and Prof. Pietro Perona (Caltech). Before coming to NYU he was a postdoc at MIT with Prof. William T. Freeman.