Socially Intelligent Meetings

Socially Intelligent Meetings

Established: January 1, 2019


Microsoft Research Blog

Microsoft Research Blog


Making (tele)presence work

The Socially Intelligent Meetings program explores social and technological interventions for making work-oriented telepresence effective, comfortable, adaptable, and robust. The guiding principles are that telepresence technologies should aim to drive social value rather than attempt to replicate face-to-face experiences, and that telepresence technologies should be designed with a holistic and flexible approach to workflows rather than fitted narrowly to tasks.

This program currently has four workstreams:

  • All-Remote Meetings During COVID-19
  • Mixed Reality and Robotic Telepresence
  • Configuring Hybrid Meetings
  • Social Devices

All-Remote Meetings During COVID-19

In common with companies around the world, from March in 2020 Microsoft employees entered a prolonged period of mandatory working from home due to the COVID-19 pandemic. We wondered what would happen to employees’ effectiveness and collegiality as all meetings went remote.

From April through August 2020 we recruited almost 1000 employees to take part in a large scale study involving two month diaries and weekly polls.

While there is a long history of research into all-remote meetings, there are still few definitive answers about fundamental issues of why video meetings are not the simple “next best thing to being there” that we have all assumed they would be or could be. This ‘natural experiment’ is highlighting the everyday reality of the difference in “being there” as against “being remote” in work situations, compelling us to think about how to manage the boundaries between presence and absence. This is an opportunity to re-examine the longstanding questions with some of the most wide-scale naturalistic data ever available.

We are currently analyzing the data. Outputs will be reported here as they become available.

Current internal research

Current external research

  • Attention in remote moeetings
    • Collaborator: Brittany Duff
    • Department: Institute of Communications Research, University of Illinois at Urbana-Champaign

Mixed Reality and Robotic Telepresence: Pushing the envelope on presence and engagement

Live mediated collaboration is at an inflection point between traditional 2D video communication and Mixed reality and robotic telepresence. This research explores the boundaries of what can be achieved with high fidelity augmented or virtual reality avatars (which are hoped to provide the missing elements from the flat rectangular portals of traditional video communication) and robotic telepresence, which provides remote participants with independent mobility and physical social presence. This research also involves improving baseline understandings of geo-distributed collaboration such as attention, effectiveness, and inclusiveness.

Current internal research


  • Multi-modal turn-taking behaviours with mixed reality avatars
  • Truthfulness and influence of mixed reality avatars
  • Meeting effectiveness of mixed reality avatars

Intern project

Current external research

Microsoft PhD Scholarship: Interacting via Robotic Telepresence

Microsoft PhD Scholarship – Cambridge University: Human avatars as co-workers

  • Student: TBD
  • Supervisor: Thomas Bohné
  • Organisation: IfM Cyber-Human Lab, Cambridge University

Configuring Hybrid Meetings: Reasoning about people, the room, and collaboration patterns

The traditional video meeting stage is neither engaging nor inclusive because it is not geared to present customised and augmented social presence. Machine perception can reason about people, the room, and patterns of collaboration between people within and across connected rooms to produce meeting experiences that helps all users, local or remote, and inclusive of all abilities, understand and engage with the people and content of meetings. need to design compelling user experiences of augmented social presence, and we need to understand the communicative, organisational, and ethical fundamentals of how people will respond to and use such experiences.

Machine perception of a video meeting

Baseline research


Microsoft PhD Scholarship: Hybrid Meetings

Current collaborative external research

Microsoft Productivity Grant: Supporting Hybrid Collaboration for the Teams of Tomorrow

Collaborators: Mirjam Augstein and Thomas Neumayr, University of Applied Sciences Upper Austria

Concept: Tomorrow’s information workers are increasingly employed in flexible work settings and oftentimes come upon situations where they engage in hybrid meetings and hybrid collaboration. Although such situations, with their dynamic interplay between co-located and remote collaborators, are increasingly supported by software and hardware tools, there are still significant research gaps related to the description and analysis of such settings (which would also allow for more targeted tool support). Thus, the full potential of existing tools such as the Microsoft Surface Hub with its software solutions for co-located (e.g., Shared Whiteboard) or remote (audio and video conferencing) collaboration in the collaborative settings of the future is not yet fully exploited and requires in-depth conceptual as well as technological research. The envisioned research endeavor includes 1) thorough grounding work on a descriptive framework for hybrid collaboration, a small part of which already exists and was published at ACM CSCW 2018 (Domino: A Descriptive Framework for Hybrid Collaboration and Coupling Styles in Partially Distributed Teams) and 2) technical work on a software prototype for the support of hybrid meetings and in-depth (on-the-fly as well as post-hoc) analysis functionalities based on Microsoft hardware and software tools and APIs. To draw conclusions, we will conduct an extensive qualitative user study.

Social Devices: Adaptable cross-device configurations to support ad hoc task needs

a person standing in front of a computer

The Social Devices work stream explores how to enable devices to act as companions. We look to ways in which our computing devices can be dynamically aggregated and dis-aggregated to create user experiences that adapt to the changing social and activity demands we encounter throughout the day. Features using this research have shipped in Microsoft Teams as Mobile Sharing and Companion Experiences.

See our Garage Wall of Fame post:

See Microsoft Teams support documents at:

These features grew out of several older explorations of video-calling and meetings.


  • Beyond Presentations and Office Social: Slideware concept that enabled open access to shared interaction with slides across multiple devices.
  • Ad hoc adaptability in video calling: A position paper in which we explore ad hoc adaptability across devices in video-calling. We note the current difficulty of even simple combinations, discuss design issues, briefly report on a study of ad hoc screen mirroring, and note future directions.
  • SkypeBeam: An experimental system that enable lightweight multi-user wireless smartphone mirroring within a video call. The system enabled multiple smartphones to share both digital content as well as physical artefacts when mirroring the live view from the phone camera feed.

Current external research

HEIF Fellowship – Oxford: Multi-device configurations to support physical examinations in NHS video consultations

Collaborators: Lucas Seuren and Sara Shaw, Nuffield Department of Primary Care Health Sciences, University of Oxford

Concept: The UK National Healthcare Service (NHS) has the goal of replacing up to a third of all outpatient consultations with video consultations by 2024. With the patient and clinician no longer in the same room, physical examinations are challenging. Patients and carers in a video consultation have an active role in the examination while, at the same time must ‘show and tell’ their clinician what’s going on. Lucas Seuren and Sara Shaw at the University of Oxford are exploring how Microsoft Teams Mobile Sharing and Companion Experiences could enable a patient to talk to a clinician on a laptop or tablet and use their mobile phone as an additional camera, providing more opportunities to show their body and the environment, unlocking the potential of video consultations.



Current Cambridge UK project members

Previous Cambridge UK project members

  • Portrait of Priscilla Wong

    Priscilla Wong

    Research Assistant

  • Portrait of Kenton O'Hara

    Kenton O'Hara

    Principal Research Manager

  • Portrait of Martin Grayson

    Martin Grayson

    Senior Research Software Development Engineer

  • Portrait of Tony Wieser

    Tony Wieser

    Principal Research Software Development Engineer

  • Portrait of Anastasia Kuzminykh

    Anastasia Kuzminykh

    Past Intern

    University of Waterloo

  • Portrait of Brennan  Jones

    Brennan Jones

    Past Intern

    University of Calgary

  • Portrait of Safinaz Büyükgüzel

    Safinaz Büyükgüzel

    Past Intern

    University of Southern Denmark

  • Portrait of Roman Rädle

    Roman Rädle

    Past Intern


  • Portrait of Debaleena Chattopadhyay

    Debaleena Chattopadhyay

    Past Intern


  • Portrait of Francesca Salvadori

    Francesca Salvadori

    Past Intern


  • Portrait of Jens Emil Groenbaek

    Jens Emil Groenbaek

    Past Intern

    Aarhus University

  • Portrait of Mario Schreiner

    Mario Schreiner

    Past Intern

  • Portrait of Behnaz Yeganeh

    Behnaz Yeganeh

    Past Intern

    University of Melbourne