Dynamic Point Samples for Free-Viewpoint Video


July 19, 2004


Stephan Wuermlin


Institute of Computational Science ETH Zurich


In free-viewpoint video (FVV), multiple video streams are used to re-render a time-varying scene from arbitrary viewpoints. Free navigation with respect to time and space in streams of visual data might represent the next major step in terms of interactivity, allowing for instant virtual replays and freeze-and-rotate effects. In our work on free-viewpoint video we focus on representations and methodologies to capture and process dynamic scenes from sparsely arranged video cameras.

In this talk I will give an overview of the recent free-viewpoint video activities at ETH Zurich. As a fundamental primitive for FVV we employ dynamic point samples by generalizing 2D video pixels towards 3D irregular point samples. A special focus will be the real-time 3D video technology which is successfully employed in the blue-c collaborative virtual reality system. blue-c combines the advantages of a CAVE™-like projection environment with simultaneous and real-time 3D video capturing and processing of the user. As a major technical achievement, users can now become part of the visualized scene while keeping visual contact. Furthermore, an MPEG-compliant FVV coding framework is presented which is capable of streaming and displaying pre-recorded multi-view video data from arbitrary viewpoints. In its finest deployment, this coding framework provides multi-resolution, multi-rate and view-dependent decoding.


Stephan Wuermlin

Stephan Würmlin is currently a post-doctoral researcher in the Computer Graphics Laboratory and project leader of the blue-c-II project. He received a diploma degree in Computer Science at ETH Zurich in 2000. From 2000 to 2004, he was a research associate and Ph.D. candidate in the Computer Graphics Laboratory with Prof. Markus Gross. He finished his Ph.D. thesis in May 2004 on the design of the 3D video technology for the blue-c collaborative virtual reality system. His current research interests include free-viewpoint video, point-based representations and rendering, and multimedia coding. The dynamic point sample representation that he developed is adopted by MPEG as an extension of the MPEG-4 AFX standard.