VROOM (Virtual Robotic Overlay for Online Meetings) is a prototype telepresence system that puts belonging at the center of the experience. It aims to help a remote individual feel like a remote physical place belongs as much to them as the local people in it, and to help local people feel that a remote individual belongs with them in the same physical space.
To explore how to improve the social experience of robotic telepresence, we built a bi-directional asymmetric system using AR, VR, Robots, and 360 cameras.
For a person in the local activity space wearing a Hololens, an augmented-reality (AR) interface shows a life-size avatar of the remote user overlaid on a Beam telepresence robot using tracker markers. The Beam is also equipped with an extra 360° camera.
For the remote user, a head-mounted virtual-reality (VR) interface presents an immersive 360° view of the local space. The remote user also has VR controllers,which allow for both piloting and gestural expression. The remote user has separate mobile and visual autonomy in the local activity space.
The remote user’s speech, head pose, and hand movements, are applied to the avatar, along with canned blink, idle, and walking animations. The local user sees the entire avatar body in third person view. The remote user sees the avatar from the shoulders down in first person view. Local users can thus identify the remote user as a specific person, while the remote user has an identifiable embodiment of self, and both can take advantage of naturalistic verbal and gestural communication modalities.
We thank our Microsoft colleagues who helped us tremendously in developing the VROOM prototype: James Scott, Xu Cao, He Huang, Minnie Liu, Zhao Jun, Matthew Gan, and Leon Lu.
VROOM is a sub-project of the Socially Intelligent Meetings workstream, which explores how to make video meetings and other geo-distributed collaborations socially engaging, adaptive, and inclusive.