A search engine for the real world, or, a top-down approach to vision

August 11, 2006
Kevin Murphy | Departments of computer science and statistics at University of British Columbia

We consider the problem of finding instances of visual object categories (such as a cup or a pen) in cluttered, real-world environments. We propose a hierarchical approach, whereby we first categorize the scene (outdoors or indoors? kitchen or office?), then we use global image statistics (the “gist” of the image) to predict where to look within the image, and finally we run an object detector (based on boosted random fields) to localize the object, exploiting spatial constraints with other, easier-to-detect objects. We argue that this top-down approach is not only faster than standard “brute force” approaches, but also reduces the error-rate, since all decisions are made in context.

Joint work with Antonio Torralba and Bill Freeman.