Recovering Human Shape and Motion from Video Sequences


August 16, 2004


Pascal Fua




In recent years, because cameras have become inexpensive and ever more prevalent, there has been increasing interest in modeling human shape and motion from image data. Such an ability has many applications, such as electronic publishing, entertainment, sports medicine and athletic training. This, however, is an inherently difficult task, both because the body is very complex and because the data that can be extracted from images is often incomplete, noisy and ambiguous.

In this talk, I will present the approach we have developed to overcome these difficulties. We start from sophisticated 3-D animation models and reformulate them so that they can be used for data analysis. We use them, not only to represent faces and bodies in motion, but also to guide the interpretation of the image data, thereby substantially improving performance. Using complex video sequences, I will highlight the effectiveness of our approach to video-based shape and motion capture and demonstrate the applicability of our technology for Augmented Reality purposes. Finally, I will present some open research issues and discuss our plans for future developments.


Pascal Fua

Pascal Fua received a degree from Ecole Polytechnique, Paris, in 1984 and a Ph.D. in Computer Science from the University of Orsay in 1989. He joined EPFL (Swiss Federal Institute of Technology) in 1996 where he is now a Professor in the School of Computer and Communication Science. Before that, he worked at SRI International and at INRIA Sophia-Antipolis as a computer scientist. His research interests include human body modeling from images, optimization-based techniques for image analysis and synthesis, and the use of information theory in the area of model-based vision. He has (co)authored over 100 publications in refereed journals and conferences.