From Local to Global Visual Similarity in Space and in Time


March 1, 2007


Eli Shechtman


Weizmann Institute of Science


Analysis and detection of objects in images or actions in video sequences require a complex notion of similarity across visual data. Existing approaches are often based on extracting informative parameters or models learned from many prior examples of the visual data (object or action) of interest. These approaches, however, are often restricted to a small set of pre-defined classes of visual data, and do not generalize to scenarios with unfamiliar objects/actions. Moreover, in many realistic cases and problems, one has only a single example of the object or action of interest, or even NO explicit example whatsoever of what he is looking for…

In this talk I will show how we can infer about the global similarity of different complex visual data, by employing local similarities within and across these visual data. I will demonstrate the power of this approach through several example problems. These include:

  1. Prediction of missing visual information in images and videos.
  2. Detection and retrieval of complex objects in cluttered images using a single example – often only a rough hand-sketch of the object of interest.
  3. Detection of complex actions performed by differently dressed people

against different backgrounds, based on a single example clip (without requiring fg/bg segmentation or motion estimation).

Joint work with Michal Irani (the first part also with Yonatan Wexler).


Eli Shechtman

Eli Shechtman received the BSc degree magna cum laude in electrical engineering from Tel-Aviv University in 1996, and the MSc degree in mathematics and computer science from the Weizmann Institute of Science in 2003. He received the Weizmann Institute Dean’s Prize for MSc students, the best paper award at the European Conference on Computer Vision (ECCV ’02), and a best poster award at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’04). He is currently a PhD student at the Weizmann Institute of Science. His current research focuses on video analysis and its applications.