2019 Microsoft Productivity Research Collaboration Winners
Mirjam Augstein and Thomas Neumayr
University of Applied Sciences Upper Austria
Microsoft lead collaborator: Sean Rintel
Supporting Hybrid Collaboration for the Teams of Tomorrow
Abstract: Tomorrow’s information workers are increasingly employed in flexible work settings and oftentimes come upon situations where they engage in hybrid meetings and hybrid collaboration. Although such situations, with their dynamic interplay between co-located and remote collaborators, are increasingly supported by software and hardware tools, there are still significant research gaps related to the description and analysis of such settings (which would also allow for more targeted tool support). Thus, the full potential of existing tools such as the Microsoft Surface Hub with its software solutions for co-located (e.g., Shared Whiteboard) or remote (audio and video conferencing) collaboration in the collaborative settings of the future is not yet fully exploited and requires in-depth conceptual as well as technological research. The envisioned research endeavor includes 1) thorough grounding work on a descriptive framework for hybrid collaboration, a small part of which already exists and was published at ACM CSCW (receiving a best paper award) and 2) technical work on a software prototype for the support of hybrid meetings and in-depth (on-the-fly as well as post-hoc) analysis functionalities based on Microsoft hardware and software tools and APIs. To draw conclusions, we will conduct an extensive qualitative user study.
Microsoft lead collaborator: Mary Czerwinski
TeamDNA: Productivity-enhancing Tools for Diverse and Distributed Teams
Abstract: In this project, we are inspired by the question: what are the fundamental elements that impact team productivity?, for example, what constitutes the “DNA of team productivity?” Due to the necessity and prevalence of teamwork in the workplace, cracking this code and building better teams has the potential to significantly improve workplace productivity and also the teamwork experience. As one would expect, the topic has been studied extensively, especially in Organizational Psychology. However, to date, most prior research has combined human observations with self-reported data, thereby resulting in high-level insights but not deployable systems. Thanks to advances in engineering and Microsoft platforms that enable the real-time tracking of team interactions, we have a unique and unprecedented opportunity to study and improve team processes.
University of California – San Diego
Microsoft lead collaborator: Nathalie Riche
A Human-Centered Information Space
Abstract: For far too long we have conceived of thinking as something that happens exclusively in the head. Thinking happens in the world as well as the head. Thinking is a distributed, socially-situated activity that exploits the extraordinary facilities of language, media, and embodied interaction with the world. With computers becoming ubiquitous and intertwined with every sphere of life, today we increasingly think with computers. This is accelerated by a radically changing cost structure in which the cost to use a thousand computers for a second or day is not appreciably more than to use one computer for a thousand days or seconds. Yet with all the advances in capacity, speed, and connectivity, using computers too often remains difficult, awkward, and frustrating. Even after six decades of design evolution, there is little of the naturalness, spontaneity, and contextual sensitivity required for convivial interaction with information. We argue that this is a result of a legacy document and application-centered design paradigm that presupposes information is static and disconnected from the context of processes, tasks, and personal histories. We propose a new human-centered view of information: as dynamic entities whose representation and behavior are designed in accordance with the cognitive requirements of human activity.
Chris North and Doug Bowman
Microsoft lead collaborators: Rich Stoakley, March Rogers
Evaluating Physical and Virtual Large Displays for Windows Productivity Beyond the Desktop
Abstract: The fundamental space limitations of small display monitors pose significant problems for information workers’ productivity. The increased availability of low-cost, large physical displays and the coming feasibility of virtual displays (viewed through AR and VR headsets) will open fundamentally new user interface opportunities. However, little is known about the value of these modalities for desktop use, the trade-offs between physical and virtual displays, and how to best exploit them for productivity tasks. Our goal is to collect empirical data that will inform the design of future productivity hardware and software, such as Microsoft Windows and Office. These results could help to free users from the confines of current desktop environments, and lead to the next major revolution in increased productivity.
Research engagement results: The partnership with Virginia Tech explored how virtual monitors can be used to improve productivity through augmented reality. By looking at range scenarios – from mobile knowledge work to working from home to low vision contexts – the team found ways to expand beyond small screen and monitors that occupy physical space. These videos give an example of the immense flexibility virtual monitors can bring regardless of how much display space is needed:
Microsoft lead collaborator: Ahmed Hassan Awadallah
Improving Employee Productivity Using Text Summarization
Abstract: Company employees spend a large fraction of their time reading text documents such as company policies, technical manuals, patents, research papers, industry news articles, and email, among others. Reading text takes time that can be used for other work-related activities or for enjoying more leisure time. We are proposing to improve employee productivity, both during onboarding and throughout their entire careers, through automatic text summarization techniques. We will develop a generic, state of the art library, named SummerTime, that will be used on summarization tasks, such as single-document and multi-document summarization, query-based summarization, text simplification, and text re-targeting. The code base will be flexible enough to allow the introduction of new techniques, data sets, and evaluation metrics. We will also implement a number of classic and recent neural algorithms and also improve the state of the art using transfer learning and several novel neural architectures.
Research engagement results: The project with Yale focused on improving employee productivity via dialogue summarization. Dialogue summarization has become increasingly important since the COVID-19 pandemic given the growth in video conferencing, and the team created a benchmark to measure how well current models perform in the dialogue domain. Their results can enable meeting summarization models to help new employees’ onboarding process or new student’s learning process by providing a concise summary of meeting or class interactions.
- QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization (NAACL 2021). Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu and Dragomir Radev. Accepted by NAACL 2021. [GitHub, video, poster]
- An Exploratory Study on Long Dialogue Summarization: What Works and What’s Next (submitted to EMNLP 2021). Yusen Zhang, Ansong Ni, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah and Dragomir Radev.
The team also developed a new, wide-coverage summarization library named SummerTime. SummerTime targets non-expert users to expand access for state-of-the-art summarization models to a wider range of people. Users of the library do not need an NLP background and functionality is provided to help identify the best model to use for a particular case, including visualization, automatic model selection, and automatic model assembly.
- SummerTime: Text Summarization Toolkit for Non-experts (submitted to EMNLP 2021, Demo Track). Ansong Ni, Zhangir Azerbayev, Mutethia Mutuma, Troy Feng,Yusen Zhang, Tao Yu,Ahmed Hassan Awadallah, Dragomir Radev. [GitHub repository]
For more about this project, contact: Ahmed H. Awadallah (hassanam)
Microsoft lead collaborator: Mary Czerwinski
Unobtrusive Personalized Work Engagement Assistant
Abstract: Work engagement and workload/task management are an important aspect of achieving successful and productive everyday information work missions. However, work tasks/schedule and strategies to promote work engagement and well-being could vary from person to person. It is hard to adapt one strategy to all workers. In this proposal, we examine the hypothesis that multi-modal ubiquitous sensors and AI technologies help design a personalized work engagement assistant to suggest personalized productivity management strategies and provide unobtrusive personalized feedback to enhance work engagement and well-being. The aim of the proposed work is to develop and validate an unobtrusive personalized closed-loop system to measure work engagement and workload, and provide personalized real time feedback including work engagement management assistant and subtle sensory feedback based on the user’s physiological and behavioral data. We are focused on the development of unobtrusive and practical technologies, and selected the optimal sets of tools and mechanisms based on our team’s interdisciplinary work: ubiquitous and effective sensing and computing, computational imagining, machine learning, organizational psychology and human computer interaction.
University of Southern California
Microsoft lead collaborator: John Krumm
Privacy-Preserving Machine Learning Techniques for Improving Individual and Organizational Productivity
Abstract: Studying patterns of human activity (e.g., moving behaviors, daily routines, organizational workflows) can significantly improve productivity. Neural networks are a powerful tool to capture such patterns, but they need large amounts of individual data (e.g., location data) to train on, which raises significant privacy concerns. This project will design and implement differentially-private techniques to train neural networks. We will focus on skip-grams, which are suitable for sparse data, especially when used in conjunction with negative sampling. We will design algorithms that can build accurate models for human activity patterns, even under strict privacy constraints. We will also study privacy budget allocation strategies across different stages of the model, and we will perform tuning of model hyper-parameters to improve accuracy and performance.
Research engagement results: The collaboration with USC consisted of three different workstreams.
The first effort created:
- A density-aware technique for publication of OD matrices in a differential private way,
- An extension of the OD concept to multiple dimensions that allows one to quantify privately the frequency of certain trajectory segments of interest, and
- An ML-based technique for accurately answering differentially private range count queries on geospatial data. All techniques were extensively tested using large-scale real datasets, and significantly outperformed all existing state of the art approaches.
The second parallel effort identified a new problem of quantifying the intrinsic value of information of trajectories, including a technique for quantifying the intrinsic VOI of trajectories:
The third effort worked on evaluating a methodology for different diseases and transmission models and quantifying the impact of sampling bias. The team considered the spread of SARS and the flu, in addition to work on COVID-19. Results showed the robustness of the method to bias in observed trajectories.
For more about this project, contact: John Krumm (jckrumm)