Video Editing Using Figure Tracking and Image-Based Rendering

We describe a new approach to video editing based on the semi-automatic segmentation of video into multiple layers and the compositing of layers using image-based rendering. Using figure tracking and background motion estimation, we can segment a moving figure and reconstruct the background. Using geometrically-correct pixel reprojection, layers can be composited on the basis of the geometry of the underlying scene and the position of the virtual camera. We have implemented out approach in an editing system called SpliceWorld. We show results from editing a Fred Astaire dance sequence.