High-quality Image-based Interactive Exploration of Real-World Environments

  • Matt Uyttendaele
  • Antonio Criminisi
  • Sing Bing Kang
  • Simon Winder
  • Richard Hartley

IEEE Computer Graphics & Applications (CG&A) | , Vol 24: pp. 52-63

Interactive scene walk-throughs have long been an important computer graphics application area. Starting with Fred Brooks pioneering work, efficient rendering algorithms have emerged for visualizing large architectural databases. More recently, researchers have developed techniques for constructing photorealistic 3D architectural models from real-world images.Real-world tours based on panoramic images also exist, as we describe in the “Panoramic Imaging” sidebar. These systems all aim to create a real sense of being there, a sense of virtual presence that lets users experience a space or environment in an exploratory, interactive manner. This article presents an image-based rendering system that brings us a step closer to a compelling sense of being there. Whereas many previous systems have used still photography and 3D scene modeling, we avoid explicit 3D reconstruction because it tends to be brittle. Instead, we film a tour of an environment that we wish to explore and then use image-based rendering techniques to replay the tour interactively. We call such experiences interactive visual tours. In designing this system, our goal was to let users move freely along a set of predefined tracks, choose between different directions of motion at decision points, and look in any direction. In addition, the displayed views should have a high resolution and dynamic range. The system easily incorporates multimedia objects such as navigation maps, video textures, audio enhancements, and pop-up stills. Our system is not the .rst to propose interactive videobased tours (see the “Panoramic Imaging” sidebar). We believe, however, that our system is the first to deliver fully interactive, photorealistic image-based tours on a personal computer at or above broadcast video resolutions and frame rates. Moreover, to our knowledge, no other tour provides the same rich set of interactions or visually complex environments.