Socially Intelligent Meetings

Socially Intelligent Meetings

Established: January 1, 2019

Making (tele)presence work

Publications

Microsoft Research blog

Microsoft Research blog

Overview

How can we make remote and hybrid telepresence engaging, inclusive, and effective?

The Socially Intelligent Meetings program explores social and technological interventions for making organizational telepresence engaging, inclusive, and effective. The guiding principles are that telepresence technologies should drive social and organizational value rather than attempt to replicate in-person experiences, and that telepresence technologies should be designed with a holistic and flexible approach to intentional collaboration over time rather than fitted narrowly to specific tasks.

​​​​​​​Goals

  • Advance the state of the art in understanding how to make telepresence effective, comfortable, and adaptable.
  • Establish best practices for deriving the most organizational value from telepresence from a holistic view of work and the ecosystem of collaboration technologies.
  • Collaborate with product teams to develop experiences for current and future telepresence platforms.

Themes

  • Meeting (the) Pandemic: All-Remote Meetings During COVID-19.
  • Belonging There: Mixed Reality and Robotic Telepresence.
  • Configuring the Meeting: Attention and engagement in remote and hybrid collaboration.
  • Social Devices: Adaptable cross-device configurations to support ad-hoc meeting needs. [Archival]

Meeting (the) Pandemic: All-Remote Meetings During COVID-19

In common with companies around the world, from March in 2020 Microsoft employees entered a prolonged period of mandatory working from home due to the COVID-19 pandemic. We wondered what would happen to employees’ effectiveness and collegiality as all meetings went remote.

From April through August 2020 we recruited almost 1000 employees to take part in a large scale study involving two month diaries and weekly polls.

While there is a long history of research into all-remote meetings, there are still few definitive answers about fundamental issues of why video meetings are not the simple “next best thing to being there” that we have all assumed they would be or could be. This ‘natural experiment’ is highlighting the everyday reality of the difference in “being there” as against “being remote” in work situations, compelling us to think about how to manage the boundaries between presence and absence. This is an opportunity to re-examine the longstanding questions with some of the most wide-scale naturalistic data ever available.

We are currently analyzing the data. Outputs will be reported here as they become available.

Current internal research

Current external research

  • Multitasking in remote moeetings
    • Collaborator: Brittany Duff
    • Department: Institute of Communications Research, University of Illinois at Urbana-Champaign

Belonging There: Mixed Reality and Robotic Telepresence

Live mediated collaboration is at an inflection point between traditional 2D video communication and Mixed reality and robotic telepresence. This research explores the boundaries of what can be achieved with high fidelity augmented or virtual reality avatars (which are hoped to provide the missing elements from the flat rectangular portals of traditional video communication) and robotic telepresence, which provides remote participants with independent mobility and physical social presence. This research also involves improving baseline understandings of geo-distributed collaboration such as attention, effectiveness, and inclusiveness.

Current internal research

Presence

  • Multi-modal turn-taking behaviours with mixed reality avatars
  • Truthfulness and influence of mixed reality avatars
  • Meeting effectiveness of mixed reality avatars

Intern project

Current external research

Microsoft PhD Scholarship: Interacting via Robotic Telepresence

Microsoft PhD Scholarship – Cambridge University: Human avatars as co-workers


Configuring the Meeting: Attention and engagement in remote and hybrid collaboration

The traditional video meeting stage is neither engaging nor inclusive because it is not geared to present customised and augmented social presence. Machine perception can reason about people, the room, and patterns of collaboration between people within and across connected rooms to produce meeting experiences that helps all users, local or remote, and inclusive of all abilities, understand and engage with the people and content of meetings. need to design compelling user experiences of augmented social presence, and we need to understand the communicative, organisational, and ethical fundamentals of how people will respond to and use such experiences.

Machine perception of a video meeting

Baseline research

Attention

Microsoft PhD Scholarship: Hybrid Meetings

Current collaborative external research

Microsoft Productivity Grant: Supporting Hybrid Collaboration for the Teams of Tomorrow

Collaborators: Mirjam Augstein and Thomas Neumayr, University of Applied Sciences Upper Austria

Concept: Tomorrow’s information workers are increasingly employed in flexible work settings and oftentimes come upon situations where they engage in hybrid meetings and hybrid collaboration. Although such situations, with their dynamic interplay between co-located and remote collaborators, are increasingly supported by software and hardware tools, there are still significant research gaps related to the description and analysis of such settings (which would also allow for more targeted tool support). Thus, the full potential of existing tools such as the Microsoft Surface Hub with its software solutions for co-located (e.g., Shared Whiteboard) or remote (audio and video conferencing) collaboration in the collaborative settings of the future is not yet fully exploited and requires in-depth conceptual as well as technological research. The envisioned research endeavor includes 1) thorough grounding work on a descriptive framework for hybrid collaboration, a small part of which already exists and was published at ACM CSCW 2018 (Domino: A Descriptive Framework for Hybrid Collaboration and Coupling Styles in Partially Distributed Teams) and 2) technical work on a software prototype for the support of hybrid meetings and in-depth (on-the-fly as well as post-hoc) analysis functionalities based on Microsoft hardware and software tools and APIs. To draw conclusions, we will conduct an extensive qualitative user study.


Social Devices: Adaptable cross-device configurations to support ad-hoc task needs [Archival]

a person standing in front of a computer

The Social Devices work stream explores how to enable devices to act as companions. We look to ways in which our computing devices can be dynamically aggregated and dis-aggregated to create user experiences that adapt to the changing social and activity demands we encounter throughout the day. Features using this research have shipped in Microsoft Teams as Mobile Sharing and Companion Experiences.

See our Garage Wall of Fame post:

See Microsoft Teams support documents at:

These features grew out of several older explorations of video-calling and meetings.

Papers

  • Beyond Presentations and Office Social: Slideware concept that enabled open access to shared interaction with slides across multiple devices.
  • Ad hoc adaptability in video calling: A position paper in which we explore ad hoc adaptability across devices in video-calling. We note the current difficulty of even simple combinations, discuss design issues, briefly report on a study of ad hoc screen mirroring, and note future directions.
  • SkypeBeam: An experimental system that enable lightweight multi-user wireless smartphone mirroring within a video call. The system enabled multiple smartphones to share both digital content as well as physical artefacts when mirroring the live view from the phone camera feed.

External research

HEIF Fellowship – Oxford: Multi-device configurations to support physical examinations in NHS video consultations

Collaborators: Lucas Seuren and Sara Shaw, Nuffield Department of Primary Care Health Sciences, University of Oxford

Concept: The UK National Healthcare Service (NHS) has the goal of replacing up to a third of all outpatient consultations with video consultations by 2024. With the patient and clinician no longer in the same room, physical examinations are challenging. Patients and carers in a video consultation have an active role in the examination while, at the same time must ‘show and tell’ their clinician what’s going on. Lucas Seuren and Sara Shaw at the University of Oxford are exploring how Microsoft Teams Mobile Sharing and Companion Experiences could enable a patient to talk to a clinician on a laptop or tablet and use their mobile phone as an additional camera, providing more opportunities to show their body and the environment, unlocking the potential of video consultations.


Projects

The New Future of Work

At the start of the COVID-19 pandemic, researchers at Microsoft formed a cross-company initiative to coordinate their efforts to understand the impact of remote work and identifying opportunities to support new work practices. The initiative consists of over 50 research projects, conducted by teams that span a range of disciplines (including engineering, research, marketing, human resources, and facilities) and divisions (including Microsoft Research, Office, Windows, Azure, Xbox, GitHub, and LinkedIn).

The New Future of Work site features the research that has come out of this initiative, including what we believe to be the world’s largest synthesis of findings related to how people’s work practices have evolved since the start of the pandemic. You can think of this site as a place to learn the science behind WorkLab, which Microsoft recently launched to share compelling stories and research-based insights about the changing nature of work. Stay tuned as we add more peer-reviewed research papers and other scholarly content.

WorkLab

WorkLab is a new digital publication devoted to illuminating the future of work, grounded in research and the lessons of the COVID-19 pandemic.

People

Current Cambridge UK project members

Previous Cambridge UK project members

  • Portrait of Damian Borowiec

    Damian Borowiec

    Research Assistant

  • Portrait of Safinaz Büyükgüzel

    Safinaz Büyükgüzel

    Past Intern

    University of Southern Denmark

  • Portrait of Debaleena Chattopadhyay

    Debaleena Chattopadhyay

    Past Intern

    UIC

  • Portrait of Martin Grayson

    Martin Grayson

    Senior Research Software Development Engineer

  • Portrait of Jens Emil Groenbaek

    Jens Emil Groenbaek

    Past Intern

    Aarhus University

  • Portrait of Brennan  Jones

    Brennan Jones

    Past Intern

    University of Calgary

  • Portrait of Anastasia Kuzminykh

    Anastasia Kuzminykh

    Past Intern

    University of Waterloo

  • Portrait of Kenton O'Hara

    Kenton O'Hara

    Principal Research Manager

  • Portrait of Roman Rädle

    Roman Rädle

    Past Intern

    Facebook

  • Portrait of Francesca Salvadori

    Francesca Salvadori

    Past Intern

    IBM

  • Portrait of Mario Schreiner

    Mario Schreiner

    Past Intern

  • Portrait of Tony Wieser

    Tony Wieser

    Principal Research Software Development Engineer

  • Portrait of Priscilla Wong

    Priscilla Wong

    Research Assistant

  • Portrait of Behnaz Yeganeh

    Behnaz Yeganeh

    Past Intern

    University of Melbourne

Current Non-UK Affiliated Researchers