July 17, 2019 – July 18, 2019

Faculty Summit 2019 | The future of work

Location: Redmond, Washington, USA

Keynotes

Speaker: Jaime Teevan, Microsoft

Productivity is the cornerstone of the experiences and devices that Microsoft builds. And yet the nature of productivity is fundamentally changing with the emergence of the intelligent cloud and edge, increasing use of digital media, and an explosion of devices. No longer is it enough for the tools we build to merely help people be faster, more efficient, and better organized. Our tools must now help people approach problems in new ways. The future of productivity is collaborative, intelligent, and deeply embedded in the world around us. This talk will explore the research breakthroughs necessary to bring this expansive view of productivity to life.
Speaker: Gloria Mark, University of California, Irvine

The future of work will involve gaining a deep understanding of people’s workplace experience and use that understanding to develop solutions that improve health, mood, and productivity. Using wearable sensors, computer logging, and experience sampling data, we can analyze digital media use and how attention varies over the workday. Our goal is to leverage this data to design custom interfaces and interventions that people can use to promote well-being and productivity.
Speaker: Mira Lane, Microsoft

Emerging technologies like artificial intelligence and mixed reality have the potential to upend the way we create, work, and interact with one another. This disruption is an opportunity and a challenge. As we see technologies begin to closely replicate aspects of creative human output, we must consider the evolution of work and our relationship to machines. New technologies alter our connections with one other, they have the potential to rapidly turn our ideas into tangibles, and yet… we all know that we must tread intentionally in this new era. Should we aim for more ambitious relationships between computers and ourselves and what does responsible innovation mean in a future of human-machine collaboration?
Speaker: Johannes Gehrke, Microsoft

Automation, digital platforms, and AI are changing the fundamental nature of work. Many activities that we do today have the potential to be automated, but there is also huge potential to dramatically enhance individual productivity. The cloud plays a key role here. Beyond multi-tenancy, elasticity, and unlimited resources, the cloud allows us to learn and to make our users more successful through data. In this session, we provide several examples of how we work across research and product groups empowering our users through data and lay out several challenges that we hope to solve together with the academic community.

Breakout Sessions

Chair: Ed Cutrell, Microsoft

Advances in hybrid intelligence, deep learning, and related artificial intelligence techniques have provided us with a remarkable opportunity to ensure the future of work will be even more inclusive to more people than ever before. Because the communication and products of work increasingly comprise images—photos, charts, maps, and the like—that are often not accessible, people who are blind or low vision face unique challenges. One promising technology is the automated understanding and captioning of images. Office 365 applications, for example, can use APIs from Microsoft Azure Cognitive Services to automatically add alt text to images. But there remain many hurdles to making these captions truly useful and usable. In this breakout session, we will explore the state of the art and potential for advancement in automated image captioning, including data capture and curation for training, caption presentation and interactivity, and computer vision.
Chair: Mary Czerwinski, Microsoft

We’re on a path to a future where artificial intelligence (AI) and humans collaborate; one could argue that time is already here. Issues of trust in teams, building rapport, and group formation will dramatically change with the infusion of AI into our social workplace. As AI begins to further augment user interactions and interfaces, there is a pressing need to think about how we want to design such AI-powered social systems and experiences. In this breakout session, we bring together experts from the social sciences, AI and machine learning, and computer science to discuss this integration, particularly how emotional intelligence can be realized to make AI-powered systems more human-like and how this may influence collaborative practices at both the practical and ethical levels.
Chair: Mar Gonzalez-Franco, Microsoft

Important questions about how the productivity of VR/AR applications is measured remain. This panel discusses ways of conceptualizing and measuring productivity. Ideally, objective measures of the experience can be developed that evaluate the realism of any mediated scenario being presented, and these measures could be developed with such precision that components of an AR or VR system could be identified for weaknesses and be improved upon. The panel brings together researchers from different backgrounds and perspectives in the context of this issue, using approaches of calibration, interaction, realism and presence, and the use of behavior as objective measures to compare across real world and mediated environments. The questions that the panel will discuss include:
- What types of productivity are there? How are they similar and different?
- What are the best ways of measuring productivity?
- How has productivity changed with new technologies?
- Is productivity different between AR and VR?
- What new areas of VR/AR will help increase productivity?
Chair: Nathalie Riche, Microsoft

The future of work is beyond the desktop, existing in a more natural environment where individuals can grab a pen to annotate a document on any device and share it effortlessly with others, including themselves. The future of work is identifying relevant information on the web or in email on a phone while commuting and seamlessly transitioning it to a large interactive whiteboard for a collaborative team meeting at the office. In this session we offer several complementary perspectives of this near future in which pen and touch play a central role in leveraging Office 365 and cross-device experiences are easy and fluid.
Co-Chairs: Andy Gordon, Shi Han, Ben Zorn, Microsoft

The spreadsheet has continually evolved to remain at the forefront of productivity tools and work practices for over forty years. For example, today’s spreadsheets embrace collaboration, serve as databases, are mobile, and encompass AI-powered interaction via natural language. Going forward, research advances in AI, program synthesis, visualization, and programming languages are being integrated into spreadsheets to improve user productivity and experience. This session examines recent innovations in spreadsheeting, from both Microsoft Research and the academic research community.
Chair: Besmira Nushi and Ece Kamar, Microsoft

Currently, there is an increasing ambition in industry and research for building artificial intelligence (AI) that will support people in their everyday life and work. Decision-making is a central aspect in this ambit. AI and machine learning algorithms are being used to assist doctors, lawyers, and governments in diagnosing diseases, making judicial decisions, and defining policies. It is more important than ever that these algorithms are designed and optimized in a way that simplifies and accounts for human-AI collaboration. Addressing these challenges requires methods and techniques at the intersection of cognition, sociology, and artificial intelligence. In this breakout session, we bring together expertise from these different perspectives and enable the speakers, as well as the audience, to share emerging theoretical results, experimental findings, and applications involving human-AI decision-making.
Co-Chairs: Sarah Bird and Markus Weimer, Microsoft

With the advent of machine learning techniques, programmer productivity is poised to significantly improve. The job of a software engineer is changing into one where they learn a model for a function by using vast amounts of data, and then apply this model to predict or infer the value of this function on new and unknown data.

But for this new model of software development to become the dominant approach, we will need advances in several areas, including in program synthesis, compilers, high-performance computer systems, and neural network architectures. This session will identify the new programmer paradigm and identify what is needed to realize the potential productivity improvements it promises.
Chair: Shamsi Iqbal, Microsoft

In today’s world, people have to attend to a number of tasks near simultaneously, and with the widespread use of mobile devices, tasks can be tackled almost anywhere at any time. It is not surprising, then, that being able to address any one task for an extended period is becoming increasingly difficult. A new research area is focusing on “microproductivity,” breaking larger tasks down into manageable components conducive to small moments throughout the day. In this breakout session, we bring together experts from academia and the product side to share their vision of a future where traditional tasks can be accomplished via both focused attention and microproductivity. We will unpack how microproductivity may manifest across different domains and scenarios, identify key challenges in designing for microproductivity, discuss how expected outcomes may be impacted, and put forward an agenda that can move the field toward real-life adaptation.
By attending the Microsoft Research Faculty Summit you have the exclusive opportunity to meet with some of our senior technical leaders from product groups across the company. Over lunch, you are encouraged to join a table hosted by one of Microsoft’s Distinguished Engineers to discuss a specific technology topic. This is your chance to ask questions, share your opinions and views on the future of the technology, and build relationships within our product teams. These frank and informal conversations with strategic leaders are sure to pique your interest and inspire you.
Chair: Neel Sundaresan, Microsoft

In this era of “Internet of Code”, data and metadata around open source projects are available in abundance. While research in program synthesis is not new, deep learning systems that take advantage of large scale code as data is starting to show new promise in improving developer productivity. The availability of GPU machines and cloud-based distributed systems help build deeper networks and scale them to production systems. In addition to passive input from open repos, crowdsourcing software expertise and integrating this with software systems has shown positive results. AI promises assistance and automation in every aspect of software development from edit and build stage to test and deploy stage. What traditional compiler and run time systems did with rules and analyzers can be replaced with AI-driven algorithmic systems. The concept of Software 2.0 is being discussed where code appears as data and where traditional software development processes give way to AI-based systems. In this panel, we explore opportunities for research and technology to improve productivity in software engineering and how AI plays a role in it.
Chair: Paul Bennett, Microsoft

Artificial intelligence has the potential to improve productivity throughout the workplace by leveraging how people communicate to proactively connect a person to the right people and information. Providing this benefit requires key system capabilities, including understanding how language in communications relates to the actions people take, how behavioral traces can be used to measure personal productivity, how we can make recommendations from the personal web, and—critical to all of these—how we can learn from each person’s data in a privacy-preserving way. This breakout session will consist of 10-minute talks to review recent progress in related areas and a panel discussion on how research can address the challenges in this arena. We will explore how machine learning methods can be applied to customer-level data to improve personalization and facilitate productivity without sacrificing privacy and address such technical issues as data sources/feedback, modeling objectives, and accurate evaluation.
Chair: Ryen White, Microsoft

Tasks—defined pieces of work ranging in scope from the specific, such as sending an email, to the broad, such as shipping a feature—are central to all aspects of personal and team productivity. Task intelligence spans technologies and experiences to extract, understand, and support the completion of short- and long-term goals. Helping people complete tasks is a key capability of search systems, digital assistants, and productivity applications. We will discuss work on task support across four Microsoft products: To-Do, Project/Planner, Bing, and Azure DevOps. Invited speakers will discuss how their respective products help users do more with less effort. We will also spotlight our ongoing collaboration with RMIT University on the Cortana Intelligence Institute, an initiative to co-develop task intelligence technologies.
Chair: Steven Drucker, Microsoft

Without good models and the right tools to interpret them, data scientists risk making decisions based on hidden biases, spurious correlations, and false generalizations. This has led to a rallying cry for model interpretability. Yet the concept of interpretability remains nebulous, such that researchers and tool designers lack actionable guidelines for how to incorporate interpretability into models and accompanying tools. This panel brings together experts on visualization, machine learning and human interaction to present their views as well as discuss these complicated issues.
Chair: Sean Rintel, Microsoft

Communication is the foundation on which our efforts toward accomplishing goals and tasks are built. We articulate needs and results and tell and react to stories about what we do and how we do it. The modern working world consists of a myriad tools and technologies by which to convey this information, providing for choice, integration, and analysis, and these options will only increase in the future. This breakout session aims to unpack the communicative backbone of the work environment to explore how communication patterns are changing and to discuss new ways to understand how communication relates to productivity. We will cover issues from different disciplines and perspectives across diverse domains and outline both challenges and opportunities.
Chair: Mary Gray, Microsoft

From Uber-like services dishing up physical labor to startups “taskifying” information services, tech companies shaping the platform-driven, on-demand economy have paid little attention to the value of connected workers. What could the future of work look like if tech facilitated new worker-centered organizational structures? How might we build these structures while both delivering job opportunities and supporting worker groups, even in resource-constrained settings? What are the opportunities for capitalizing on technology innovation to design new economic markets that benefit workers and more fairly distribute jobs to a global workforce? We will address these questions and more, drawing on their research to outline how technologies could be used to counter the trend toward isolating on-demand workers.
Chair: Ehsan Hoque, University of Rochester

The workforce of the future will have to be creative and innovative, rather than merely good at performing specific tasks. However, many individuals lack these skills, particularly if they suffer from cognitive disabilities or difficulties. Many argue that those skills are either innate or require extensive practice with human experts. We show that using technology we can improve human skills across a variety of domain including public speaking, job interviews, aging, autism, music training, negotiations, collaborations, end-of-life communication and deception.