Deep Photo and Gigapixel Images

This talk describes two projects in the area of computational photography that I have recently worked on in collaboration with Microsoft Research and Virtual Earth.
First, I will present a novel method for browsing, enhancing, and manipulating outdoor photographs by combining them with existing georeferenced digital terrain and urban models. A simple interactive registration process is used to align photographs with models. Once the photograph and the model have been registered, and abundance of information, such as depth, texture, and GIS data, becomes immediately available to our system. This information, in turn, enables a variety of operations, ranging from dehazing and relighting the photograph, to novel view synthesis, and overlaying with geographic information. I will describe the implementation of a number of these applications and discuss possible extensions. Our results show that augmenting photographs with 3D models in this manner supports a wide variety of new ways for us to experience and interact with our everyday snapshots.
In the second part of the talk, I will present a system to capture and view “Gigapixel Images” (very high resolution, high dynamic range, and wide angle imagery). I will show how we acquire such images, and then describe in more details our novel viewer that enables smooth real-time exploration of such images. Our viewer dynamically and smoothly interpolates the projection between perspective and curved projections, and simultaneously modifies the tone-mapping to ensure an optimal view of the portion of the scene being viewed. I will also describe how we deal with computational and memory costs when tone mapping Gigapixel images. In particular, I will show how we use a joint bilateral upsampling procedure to compute very good full resolution results from solutions computed at lower resolutions.