Microsoft Research Blog

Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

The story of an Office AI feature: How AI can promote efficient meeting preparation

July 10, 2018 | By Microsoft blog editor

Adam Troy had an idea. He was spending so much time searching through old emails and attachments to prepare for meetings: Could AI help him be better prepared for meetings? Could machine learning automatically identify the most relevant emails and documents he needed for his meetings? Troy, who works on a team that incubates projects that infuse AI into Office, built an early prototype application to see.

At the same time, Paul Bennett’s Information and Data Sciences team in Microsoft Research AI was in the second year of a project to help people find the right information, effortlessly. What information was not just relevant, but useful, as people go about their days?

When Troy demonstrated his prototype, Bennett recognized the opportunity to explore the usefulness of his group’s theories in the real world, and these two independent but aligned projects came together. Bennett recruited research intern Qian Zhao, whose expertise is in developing intelligent systems and measuring their usefulness, to develop a prototype feature as part of a research project aimed at understanding how people interact with email when preparing for meetings.

After Zhao, Bennett, and Troy had proven out the core idea with research, Abhishek Arun, who works in productizing AI into Office, began building and implementing a feature that clicks on a Calendar event in Outlook and brings up related emails and documents that help the user prepare for that meeting.

This is one example of how a close partnership between research and product development can quickly turn good ideas and deep research understandings into product-ready features. Today, Arun’s team is testing the feature for Microsoft employees; if the research results scale to a broad set of customers, it may be applied in the future to commercially available versions of Outlook.

Introducing CAPERS

The feature is called a Calendar-Aware Proactive Email Recommender System (CAPERS). While the acronym is a mouthful, the concept is simple: Outlook understands meetings and their context well enough to recommend the key emails that might help someone plan for a meeting. Using AI, CAPERS proactively selects and recommends potentially useful emails to users based on their upcoming calendar events, with a particular focus on meeting preparation.

While this may sound like a trivial task, the amount of time spent in meetings and preparing for meetings is immense. According to a Brandeis University infographic, in the US, 11 million meetings are held every day and the average corporate employee spends 37 percent of their time in meetings. There’s nothing trivial about that!

Recommendation engines must accurately predict what people will want and need. AI can make the workflow of meeting preparation more productive, because it can predict which emails would actually help one prepare for a meeting. There is a relationship between content (email), location and time (calendar) that can be measured, and AI can identify those relationships with the highest predicted usefulness.

The team, with lead author Zhao, published its results in the proceedings of the 41st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. The paper covered the results of a survey, a task-based experiment, and a field experiment comparing multiple AI-based email recommenders.

Said Zhao, “Industry-wide, email and calendar recommendations are an under-explored area of AI research. There is a lot of research on algorithms, but less on practical efficiencies in workflows. That’s why this was an interesting research project.”

The results of the research showed that a large proportion of email access is related to meetings. The amount of email content, how recent the email is, the semantic similarity between the content describing the meeting and the content in the email, and the similarity between the people involved in the meeting and the people in the email thread all positively predict the usefulness of emails to meeting preparation, with the similarity in content being the most predictive. Basically, the recommender searches through emails and finds those it thinks might help one come better prepared for a meeting, because of topics, people, or calendar content.

Because of the privacy issues related to email and calendars, the machine learning models needed to work without much training data, and needed to find learning signals that were generalizable—the system had to keep the cloud data private. Four different machine learning models were tested and the results of the research showed that an online-learned linear model doubled user engagement when compared with the baselines.

Said Troy, “AI works best when it’s invisible. It shouldn’t announce itself as AI; it should just make your workflow better. CAPERS has given us the information to help create a feature that many people will find useful as they make their meeting preparation time more efficient.”

Recommendation technologies have been primarily developed for selling people products or finding better news and movies, but the underlying AI technology has unexplored potential to improve the ease of every person’s daily job. Said Bennett, “CAPERS demonstrates the rich research space available to incorporate AI directly at the point of interaction in productivity workflows. CAPERS also demonstrates how we work together at Microsoft to push the boundaries of science and the state-of-the-art in productivity software to build a better experience for every person.”

Up Next

Artificial intelligence, Data visualization, analytics, and platform, Systems and networking

Improving AI Systems with Human Feedback and no Heartburn

Humans play an indispensable role in many modern AI-enabled services – not just as consumers of the service, but as the actual intelligence behind the artificial intelligence. From news portals to e-commerce websites, it is people’s ratings, clicks, and other interactions which provide a teaching signal used by the underlying intelligent systems to learn. While […]

Tobias Schnabel

PhD Student

Artificial intelligence

Life at the Intersection of AI and Society with Dr. Ece Kamar

Episode 9, January 24, 2018 - As the reality of artificial intelligence continues to capture our imagination, and critical AI systems enter our world at a rapid pace, Dr. Ece Kamar, a senior researcher in the Adaptive Systems and Interaction Group at Microsoft Research, is working to help us understand AI’s far-reaching implications, both as we use it, and as we build it.

Microsoft blog editor

Artificial intelligence

Substance, not hype, powers AI excitement at premier machine learning conference

By Christopher Bishop, Distinguished Scientist and Director of Microsoft Research Cambridge Lab This month, I will attend the Conference and Workshop on Neural Information Processing Systems (NIPS), the premier gathering in the machine learning field. I’ve participated in this conference most years since it began in 1987 and I’m looking forward once again to catching […]

Microsoft blog editor