Visual Echo Cancellation for Seamless Integration of Remote Sites
In a typical remote collaboration setup, two or more projector-camera pairs are “cross-wired” to form a full-duplex system for two-way communication. A whiteboard can be used as the projector screen, and in that case, the whiteboard serves as an output device as well as an input device. Users can write on the whiteboard to comment on what is projected or to add new thoughts in the discussion. That is, images from the projector are mixed with real objects (such as papers, writings, and hands) to create a shared space.
In such a setup, the captured video contains writings or user gesture (foreground) along with projected contents. If we simply send the image for display on the other end, there could be a feedback loop that will distort the projected image. After a few frames, some part of the image becomes saturated and some part of the real writing has ghosting effect. Analogous to audio echo in telephone communication, we call this effect visual echo, i.e., the appearance of the projected contents viewed by the camera. Our approach to visual echo cancellation includes an off-line calibration procedure that records the geometric and photometric transfer between the projector and the camera in a look-up table. During run-time, projected contents in the captured video are identified using the calibration information and suppressed, therefore achieving the goal of canceling visual echo.