MSR Symposium on Computational Photography: “Computational Photography and Bilateral Image Decomposition” and “Using Data to “Brute Force” Hard Problems in Computational Photography”


July 18, 2007


Fredo Durand and Alexei (Alyosha) Efros


Massachusetts Institute of Technology, Carnegie Mellon University


Computational Photography and Bilateral Image Decomposition; Frédo Durand – Massachusetts Institute of Technology: This talk describes new imaging architectures as well as software techniques that leverage computation to facilitate the extraction of information and enhance images. In particular, I will describe the use of a bilateral decomposition of images into a large-scale and a detail component using an edge-preserving approach. I will describe a variety of techniques that build on such decomposition for tone mapping, relighting, style transfer and flash photography. Finally, I will describe a new data structure, the bilateral grid, which naturally enables edge-preserving image manipulations by lifting images into a higher-dimensional space. Using Data to “Brute Force” Hard Problems in Computational Photography; Alexei (Alyosha) Efros – Carnegie Mellon University: Any automatic image manipulation task which requires reasoning about the content of a photograph is inherently ambiguous and ill-posed. This is because a single image does not carry enough information in itself to disambiguate the world that it’s depicting. Of course, humans have no problems understanding photographs because of all the prior visual experience they can bring to bare on the task. How can we help computers do the same? Our solution is to “brute force” the problem by using massive amounts of visual data, akin to how a search engine or automatic language translator uses textual data.

In this talk, I will briefly discuss our progress on a set of challenging problems including: filling holes in images, finding and segmenting objects, recovering 3D scene geometry from an image, and inserting objects into new scenes. In each case, access to a large image database proved crucial to tackling the problem. While some examples require labeled data, others just require a very large set of images, such as our recently collected dataset of 2.3 million Flick photographs. Pretty pictures will be shown.


Fredo Durand and Alexei (Alyosha) Efros

Frédo Durand is an associate professor in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology, and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL). He received his PhD from Grenoble University, France, in 1999, supervised by Claude Puech and George Drettakis. From 1999 till 2002, he was a post-doc in the MIT Computer Graphics Group with Julie Dorsey.He works both on synthetic image generation and computational photography, where new algorithms afford powerful image enhancement and the design of imaging system that can record richer information about a scene. His research interests span most aspects of picture generation and creation, with emphasis on mathematical analysis, signal processing, and inspiration from perceptual sciences. He co-organized the first Symposium on Computational Photography and Video in 2005 and was on the advisory board of the Image and Meaning 2 conference. He received an inaugural Eurographics Young Researcher Award in 2004, an NSF CAREER award in 2005, an inaugural Microsoft Research New Faculty Fellowship in 2005, and a Sloan fellowship in 2006.

Alexei (Alyosha) Efros is an assistant professor at the Robotics Institute and the Computer Science Department at Carnegie Mellon University. His research is in the area of computer vision and computer graphics, especially at the intersection of the two. He is particularly interested in using data-driven techniques to tackle problems which are very hard to model parametrically but where large quantities of data are readily available. Before coming to CMU, Alyosha received his PhD from UC Berkeley and spent a year as a fine fellow at Oxford, England.