VROOM Telepresence

VROOM Telepresence

Established: July 22, 2019

Publications

News & features

News & features

News & features

Overview

A Socially Intelligent Meetings project

VROOM (Virtual Robotic Overlay for Online Meetings) is a prototype telepresence system that puts belonging at the center of the experience. It aims to help a remote individual feel like a remote physical place belongs as much to them as the local people in it,  and to help local people feel that a remote individual belongs with them in the same physical space.

To explore how to improve the social experience of robotic telepresence, we built a bi-directional asymmetric system using AR, VR, Robots, and 360 cameras.

VROOM overview

For a person in the local activity space wearing a Hololens, an augmented-reality (AR) interface shows a life-size avatar of the remote user overlaid on a Beam telepresence robot using tracker markers. The Beam is also equipped with an extra 360° camera.

For the remote user, a head-mounted virtual-reality (VR) interface presents an immersive 360° view of the local space. The remote user also has VR controllers,which allow for both piloting and gestural expression. The remote user has separate mobile and visual autonomy in the local activity space.

The remote user’s speech, head pose, and hand movements, are applied to the avatar, along with canned blink, idle, and walking animations. The local user sees the entire avatar body in third person view. The remote user sees the avatar from the shoulders down in first person view. Local users can thus identify the remote user as a specific person, while the remote user has an identifiable embodiment of self, and both can take advantage of naturalistic verbal and gestural communication modalities.

Acknowledgements

We thank our Microsoft colleagues who helped us tremendously in developing the VROOM prototype: James Scott, Xu Cao, He Huang, Minnie Liu, Zhao Jun, Matthew Gan, and Leon Lu.

Socially Intelligent Meetings

VROOM is a sub-project of the Socially Intelligent Meetings workstream, which explores how to make video meetings and other geo-distributed collaborations socially engaging, adaptive, and inclusive.

 

People

  • Portrait of Sean Rintel

    Sean Rintel

    Principal Researcher

  • Portrait of Brennan  Jones

    Brennan Jones

    Past Intern

    University of Calgary

  • Portrait of Yaying Zhang

    Yaying Zhang

    Software Engineer

    Microsoft Vancouver

  • Portrait of Priscilla Wong

    Priscilla Wong

    Research Assistant