Microsoft Research https://www.microsoft.com/en-us/research Sun, 23 Apr 2017 15:06:01 +0000 en-US hourly 1 https://wordpress.org/?v=4.7.3 Deep Learning for Program Synthesis https://www.microsoft.com/en-us/research/blog/deep-learning-program-synthesis/ Fri, 21 Apr 2017 16:00:53 +0000 https://www.microsoft.com/en-us/research/?p=378116 By Rishabh Singh, Jacob Devlin, Abdelrahman Mohamed, and Pushmeet Kohli, Microsoft Research Despite the many advances in computing over the past decades, the actual process of writing computer software has not fundamentally changed — a programmer must manually code the exact algorithmic logic of a program in a step-by-step manner using a specialized programming language. Although programming languages […]

The post Deep Learning for Program Synthesis appeared first on Microsoft Research.

]]>
By Rishabh Singh, Jacob Devlin, Abdelrahman Mohamed, and Pushmeet Kohli, Microsoft Research

Despite the many advances in computing over the past decades, the actual process of writing computer software has not fundamentally changed — a programmer must manually code the exact algorithmic logic of a program in a step-by-step manner using a specialized programming language. Although programming languages have become much more user-friendly over the years, learning how to program is still a major endeavor that most computer users have not undertaken.

In a recent paper, we report our latest work in deep learning for program synthesis, where deep neural networks learn how to generate computer programs based on a user’s intent. The user simply provides a few input/output (I/O) examples to specify the desired program behavior, and the system uses these to generate a corresponding program.

For example, imagine that a user has a list of names that she wishes to format in a specific way, as shown below. She provides a few input-output examples, and the system auto-fills the remaining outputs (shown in light gray). In cases where users have hundreds or thousands of input strings, this can be a major time saver.

input/output string for Program Synthesis

The system performs this task by generating a program in a domain specific language (DSL). The user does not have to understand the details of the DSL, and in fact she generally does not see the program at all. In our DSL, the correct program corresponding to the above example is:

Concat(
   ToCase(
       GetToken(
           input,
           Type=Word,
           Index=-1),
       Type=Proper),
   Const(", "),
   ToCase(
       SubString(
         GetToken(
             input,
             Type=Word,
             Index=1),
         Start=0,
         End=1),
       Type=Proper),
   Const("."))

There are two key challenges in program synthesis. First, there are trillions of possible programs in our expressive DSL, and the correct program has likely never been seen before by the system. Second, because the I/O examples are hand-generated by a human, they often contain noise (e.g., typos), as in the example above (Notice Useato instead of Uesato in the second example output).

Previous approaches to solving this problem — most notably the FlashFill system in Microsoft Excel — have relied on hand-crafted rules and heuristics to guide the search for the desired program. However, this approach makes it very difficult to extend the capabilities of the DSL, requires years of manual effort, and is also extremely sensitive to any noise in the I/O examples.

Our system, called RobustFill, leverages recent advances in deep learning to take a data-driven approach to program synthesis without the need for any hand-crafted rules. Instead, it uses an attentional sequence-to-sequence neural network, first pioneered for use in language translation, to generate the program based on the I/O examples. A sketch of our network is shown below, and the details are provided.

RobustFill for Program Synthesis

We trained our system on millions of randomly generated I/O + program pairs, but determined it could perform well on real-world data, because it could learn the semantics of the DSL. Overall, our system achieved 92 percent accuracy on a set of real-world benchmarks.  What’s especially encouraging is the system is able to maintain its high accuracy even when the I/O examples contain significant noise.

Implications for Programming

The ability to successfully train neural architectures for learning to program in a rich functional language (FlashFill DSL) not only marks an exciting breakthrough in neural program synthesis, but also is a small but noteworthy step towards achieving more general artificial intelligence. It addresses the key challenges of incorporating interpretability and also touches on the important topic of connecting distributed representations with symbolic representations of knowledge.

We are now working on extensions of these architectures for learning programs in DSLs with stateful variables and control flow to generate richer program classes.  We believe work along with direction will require us to study and tackle the fundamental technical issues involved in program synthesis and induction.

More information can be found in the papers below:

RobustFill: Neural Program Learning under Noisy I/O” by Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdelrahman Mohamed, and Pushmeet Kohli, in arXiv:1703.07469

Neuro-Symbolic Program Synthesis” by Emilio Parisotto, Abdelrahman Mohamed, Rishabh Singh, Lihong Li, Denny Zhou, and Pushmeet Kohli, in 5th International Conference on Learning Representations (ICLR 2017)

Contributing interns: Emilio Parisotto (CMU), Jonathan Uesato (MIT), Surya Bhuptairaju (MIT)

The post Deep Learning for Program Synthesis appeared first on Microsoft Research.

]]>
From cancer to crop genomics — using Research as a Service at the intersection of computers and biology https://www.microsoft.com/en-us/research/blog/cancer-crop-genomics-using-research-service/ Tue, 28 Mar 2017 13:00:49 +0000 https://www.microsoft.com/en-us/research/?p=373631 By Kenji Takeda, Director, Azure for Research, AI and Research Ever since Nicola Bonzanni was a little boy playing in the tiny Italian village of Bonate Sotto, just north of Milan, he was fascinated by nature and by building things. As he grew up, he wondered how computing and nature might be intertwined. “While I […]

The post From cancer to crop genomics — using Research as a Service at the intersection of computers and biology appeared first on Microsoft Research.

]]>
By Kenji Takeda, Director, Azure for Research, AI and Research

Nicola Bonzanni, Co-Founder of ENPICOM

Nicola Bonzanni, Co-Founder, ENPICOM

Ever since Nicola Bonzanni was a little boy playing in the tiny Italian village of Bonate Sotto, just north of Milan, he was fascinated by nature and by building things. As he grew up, he wondered how computing and nature might be intertwined. “While I was studying computer science I was inspired by nature in building new algorithms. But then I thought perhaps you could do it the other way around, basically taking things that are being developed in computer science and apply them to biological problems.”

After finishing his master’s thesis, he looked for opportunities to pursue his dream. He met a Dutch researcher while at a workshop in the south of France. The team at Vrije Universiteit Amsterdam was looking for a PhD student, so they interviewed Nicola. Despite his fears about his English proficiency, his passion for combining computer science and biology came across. Professor Jakob (Jaap) Heringa was impressed. “The fact that he already seemed to be able to combine computer science and life sciences was a very good sign to us, and we decided to hire him based on that.” So Nicola packed his bags and moved to Amsterdam, even though he had never been there before.

The next stop on Nicola’s path was developing novel methods for analyzing sequences of genes to look at how they evolve through a process known as phylogenetic fingerprinting. He teamed up with Professor Berthold Göttgens at the University of Cambridge to apply computer science to better understand leukemia, particularly how different genes in blood cells interact with each other.

Nicola wanted to share his work and was inspired to build a web-based tool called ConBind.org to help others around the world studying the same thing, delivering it as Research as a Service (RaaS). He chose this model of delivery so that researchers without access to powerful computers could still do the computationally intensive work. The idea for deploying his bioinformatics service in the cloud came to Nicola after he attended a Microsoft Azure for Research training session in Amsterdam and successfully applied for an award for his project. The cloud provides the on-demand scalability needed for running his complex bioinformatic queries, and he’s now using the Azure Batch service to make it even easier.

While his research originally focused on understanding cancer, the techniques are equally applicable to all living things: viruses, bacteria, plants and animals. That led Nicola to see if his tool could help farmers achieve such benefits as creating drought-resistant plants, or increasing crop yields. So, he contacted KeyGene, one of the world’s leading AgBioTech companies.

“KeyGene’s passion is crop innovation, getting new and improved crops or actually making the technology that we provide to plant-breeding companies and food industry,” explained Jan van Oeveren, program scientist lead discovery at KeyGene. Genomics is being used to identify key traits that are of interest to the plant breeder, for example disease resistance or shelf life. “We’re excited to work with Nicola to see if the ConBind tool can be applied to plant genomics. Currently we’re only looking at genes themselves. With ConBind, we’re also looking outside the genes to see how other parts of the genome of the DNA affect a certain trait.”

Nicola isn’t stopping there. His next project, in collaboration with a friend, Alvise Trevisan, is a startup called ENPICOM, supported by the Microsoft BizSpark program. This company combines their expertise in software engineering and life sciences to build the “Internet of Living Things.” This project aims to connect individuals with their environment, their health and their genomics, using Azure IoT Suite. Said Nicola, “The next step in the Internet of Living Things is using AI and Cortana Intelligence to make complex decisions using a multitude of different data sources.”

It’s been an amazing journey for Nicola, starting in Bonate Sotto, where his passion for nature and technology was born, to his exciting new startup ENPICOM. “I think that the challenge of this century is transforming life science in an engineering process and to do that you need to bring engineering into life science. And although probably I will not see the end of this journey, I would like to participate in it.”

Learn more

The post From cancer to crop genomics — using Research as a Service at the intersection of computers and biology appeared first on Microsoft Research.

]]>
Microsoft challenge: Build a collaborative AI in Minecraft https://www.microsoft.com/en-us/research/blog/microsoft-challenge-minecraft-collaborative-ai/ Tue, 14 Mar 2017 14:00:05 +0000 https://www.microsoft.com/en-us/research/?p=367736 By John Roach, Writer, Microsoft Research A Microsoft Research team has created a competition using Project Malmo, a platform that uses Minecraft as a testing ground for advanced artificial intelligence research. The team is challenging PhD students to develop an AI that learns to collaborate with other randomly assigned players to achieve a high score […]

The post Microsoft challenge: Build a collaborative AI in Minecraft appeared first on Microsoft Research.

]]>
By John Roach, Writer, Microsoft Research

Malmo Collaborative AI Challenge

A Microsoft Research team has created a competition using Project Malmo, a platform that uses Minecraft as a testing ground for advanced artificial intelligence research. The team is challenging PhD students to develop an AI that learns to collaborate with other randomly assigned players to achieve a high score in a mini-game within the virtual world.

The field of collaborative AI research involves the development of technologies that work with, empower and augment human capabilities. The ability to collaborate is key to the development of a general AI that can mimic the nuanced and complex way humans learn and make decisions. Such an AI would represent an evolution from task-specific AI technologies that recognize speech, translate languages and caption images.

That would allow researchers to develop technology that can comprehend the intent of others, develop a shared problem-solving strategy and coordinate activity to efficiently accomplish a common task. While these problems remain unsolved, recent progress in AI research provides a foundation to begin tackling them, notes the Microsoft team hosting the Malmo Collaborative AI Challenge.

The challenge is open to PhD students worldwide. After registration, teams of one to three members are supplied a task that consists of one or more mini-games. The goal is to develop an AI solution that learns how to work with other, randomly assigned players to achieve a high score in the game.

Participants submit their solutions to GitHub, including a one-minute video that shows off the AI agent and summarizes what is interesting about their approach, all the participant’s code, and a Readme file that explains the selected approach, design decisions and instructions on how to run the code.

The solutions will be evaluated on originality, performance, code quality and GitHub stars, which are a measure of popularity. Three winning teams will receive Azure research grants of US $20,000. Members of three winning teams from the European Economic Area and Switzerland are also eligible to win spots at the Microsoft Research AI Summer School.

Related:

John Roach writes about Microsoft research and innovation. Follow him on Twitter.

The post Microsoft challenge: Build a collaborative AI in Minecraft appeared first on Microsoft Research.

]]>
Democratizing AI to improve citizen health https://www.microsoft.com/en-us/research/blog/democratizing-ai-improve-citizen-health/ Mon, 13 Mar 2017 16:00:35 +0000 https://www.microsoft.com/en-us/research/?p=370100 By Kenji Takeda, Director, Azure for Research Doctors make life-saving — and life-changing — decisions every day. But how do they know that they are making the best decisions? Can artificial intelligence (AI) help? “Before evidence-based medicine, decision-making in health care was heavily reliant on the expertise and knowledge of the health professional, usually a […]

The post Democratizing AI to improve citizen health appeared first on Microsoft Research.

]]>
By Kenji Takeda, Director, Azure for Research

Doctors make life-saving — and life-changing — decisions every day. But how do they know that they are making the best decisions? Can artificial intelligence (AI) help?

“Before evidence-based medicine, decision-making in health care was heavily reliant on the expertise and knowledge of the health professional, usually a doctor. What has happened in the last 20, 30 years is that the health care literature has exploded way beyond what any individual doctor could hope to keep up with,” explained David Tovey, editor-in-chief at Cochrane. Cochrane is a not-for-profit organization that creates, publishes and maintains systematic reviews of health care interventions, with more than 37,000 contributors working in 130 countries.

The goal is to analyze the latest medical research to find the best treatments and interventions for patients and citizens. Systematic reviews bring together the best available research evidence from individual clinical trials and study data from around the world to inform the development of guidelines, individual practice decisions and national-scale health policymaking. These are rigorous and robust studies needed to provide the best advice, but they can take more than a year or two to complete. Tovey is concerned that systematic reviews, which are a painstaking process, are not used as widely as necessary. Said Tovey, “This is one of the main limitations for using systematic reviews to guide decision-making, particularly in policy areas. Policymakers increasingly want their decisions to be made in a period of weeks or months.”

Professor James Thomas at University College London (UCL) first started exploring text mining a decade ago because it promised to automate the laborious process of identifying and classifying medical research reports. In 2006, the state of the art in text mining was to develop complex rules to try to analyze these reports in an intelligent way.

But these rules-based approaches failed.

The Cochrane Transform Project is now applying AI and machine learning to analyze thousands of reports to automatically select ones to include in systematic reviews. This new approach is successfully saving weeks of monotonous work, freeing up the expert health care reviewers to spend their time and energy on high-level analysis. Thomas is using Cortana Intelligence to quickly develop and deploy AI solutions in the cloud. He explained, “What makes this particularly efficient is the fact that I can build a classifier using the studio and then just deploy it as a web service with the click of a button, without deploying a server.” The flexibility and open nature of Azure Machine Learning also means his team can be flexible in their approach. Said Thomas, “Initially I developed these models in R, which didn’t work as well as Python. So I was able to keep the top and the bottom of the process and just swap out the machine learning section in the middle.”

The team is continuously improving the performance of the machine learning system, and engaging with more than 5,000 citizen scientists in 117 countries through the Cochrane Crowd platform. “Cochrane Crowd has really enabled anybody who wants to get involved with producing health evidence to do so,” said Anna Noel-Storr of Cochrane’s Dementia Group. The Cochrane Crowd participants have screened more than 300,000 reports and 30,000 clinical trials to create a dataset to help train James’ AI. The machine learning system has improved the efficiency of the crowd by more than 60 percent. Thomas is excited by how this benefits everyone. “So what we end up with is a virtuous circle whereby the AI makes the crowd more efficient, and then the crowd, by supplying more data, makes the AI more accurate.”

These services are deployed through cloud APIs to help clinical assessment groups look at the latest medical research in different specialties, from cardiology and dementia, to public health issues such as obesity, healthy eating and physical exercise. The flexibility of the Microsoft Cloud means that the researchers can quickly create customized systems tailored for each clinical assessment group. This is then made usable by researchers worldwide through the EPPI-Reviewer cloud service, which is now running on Microsoft Azure. The AI system augments reviewers’ capabilities, allowing them to focus on understanding data instead of the monotonous work of sifting through thousands of research studies by hand.

The system is now being used by the National Institute for Health and Care Excellence in the U.K., which provides health care delivery guidance across the U.K. National Health Service (NHS) for more than 65 million people. It will directly speed up adoption of new health care interventions at national and international scale, for example, by facilitating the clearing of new drugs for widespread use across the NHS. The result is that decisions about which interventions are best will be faster and more reliable, thanks to the intelligent cloud.

Noel-Storr is excited by the prospect of democratizing AI for health, and how it can augment what people can achieve. “I think what excites me most about this work is that it is about exploring where those boundaries lie between what the machine can do and what the human can do…. so that we can then better direct human effort where it’s most needed.”

Learn more

The post Democratizing AI to improve citizen health appeared first on Microsoft Research.

]]>
A new understanding of the world through grassroots Data Science education at UC Berkeley https://www.microsoft.com/en-us/research/blog/grassroots-data-science-education-uc-berkeley/ Thu, 09 Mar 2017 17:00:03 +0000 https://www.microsoft.com/en-us/research/?p=368999 By Vani Mandava, Director, Data Science, Microsoft Research While some may regard data science as an easy passport to a job for the tech savvy, Luis Macias has different ideas. The fourth-year undergraduate student, who is majoring in American Studies at University of California, Berkeley (UC Berkeley), wants to turn the hype of data science […]

The post A new understanding of the world through grassroots Data Science education at UC Berkeley appeared first on Microsoft Research.

]]>
By Vani Mandava, Director, Data Science, Microsoft Research

Data Science education at UC Berkeley

Students at UC Berkeley Foundations of Data Science Program

While some may regard data science as an easy passport to a job for the tech savvy, Luis Macias has different ideas. The fourth-year undergraduate student, who is majoring in American Studies at University of California, Berkeley (UC Berkeley), wants to turn the hype of data science into hope for low-income communities like the one he grew up in.

Luis was among the first students to take UC Berkeley’s innovative Foundations of Data Science, an introductory data science course designed for freshman and sophomore students of all majors. The course is a key component of a multi-year university effort to forge a broader, more diverse, and inclusive scope for the emerging discipline of data science.

Recalling an assignment to gauge how water consumption data might relate to socioeconomic conditions, Luis explained how having the power to get and analyze data about his own ZIP Code ignited an understanding that data science can yield new insights capable of solving some of society’s most complex problems.

“The income level was around $25K, a number that was powerful for me in particular, because I think it explained a lot of the social issues and problems my community had,” he said.

Berkeley’s Data Science Education Program aims to make data science an integral feature of a liberal education and a core interdisciplinary capacity available to all Berkeley undergraduates. This is a bold experiment that will equip thousands of Berkeley students across campus with a fundamental education in data-driven thinking empowered by advanced statistical and computational techniques.

“The team of educators see their role as making it possible for students to bring to bear data science in all the ways they wish to use it in the world,” notes History professor Cathryn Carson. Carson is one of the faculty members leading the effort to build a diverse curriculum that includes advanced classes as well as connector courses that provide a bridge between familiar academic subjects and newly available data science techniques. “The energy and enthusiasm of students in the courses clearly demonstrate that the initiative will put data science to work in a breadth of domains that serve society, and UC Berkeley will play a particularly powerful role as a public university in this new data rich era.”

This year, the program is enabling more than a thousand students across 56 different undergraduate majors to learn critical computational and analytical skills demanded by the projected half million jobs in data science by 2018. That’s a lot of potentially unfilled jobs, an opportunity highlighted in numerous media accounts over the past couple of years.  In 2015, Forbes wrote about the urgent need for qualified data science workers who bring different skills, expertise, and experiences to the discipline. Some of them will no doubt emerge from Berkeley’s unique program of connector courses that represent students from a diverse range of skills and disciplines.

To succeed, the program had to be accessible to students beyond the realm of computer science. One way the program does this is through a flexible and scalable technology infrastructure that enables students to quickly set up labs for hands-on practice—they don’t have to spend time installing programs or learning nuances of complicated applications.

“By hosting it in Azure, we can control the environment,” said Ryan Lovett, Systems Manager for the Department of Statistics at UC Berkeley. “Students just log in and they’re ready to go.”

David Culler, professor of Electrical Engineering and Computer Sciences at UC Berkeley, believes the program can extend computational thinking to benefit more disciplines. He anticipates the program will equip students with the ability to extract their own insights from the world’s information and build tools that benefit people in society. He likens the ability to understand an increasingly complex world to a new form of perception — combining mathematical thinking and the arts with computational tools for new forms of expression.

Such data science projects include classic and new problems like music genre classification, text analysis of famous literary works, identifying insights from bike sharing data in San Francisco, or analyzing jury selection in Alameda County.

Berkeley prides itself as a place where the world’s brightest minds explore, ask questions and improve the world. Thanks to the Data Science Education Program, thousands of Berkeley students are better critical thinkers.

Note: Microsoft partners closely with UC Berkeley in support of its Data Science Education Program. Since 2015, Microsoft Research, through its Azure for Research program provided $235K in Azure credits to enable the Foundations of Data Science course along with $260K in research credits and $5K in training credits. Microsoft also provided $75K in unrestricted gift funding towards UC Berkeley’s Data Program.

Learn more

The post A new understanding of the world through grassroots Data Science education at UC Berkeley appeared first on Microsoft Research.

]]>
New Microsoft Research Dissertation Grant provides support to under-represented groups in computing https://www.microsoft.com/en-us/research/blog/microsoft-research-dissertation-grant-support-underrepresented-groups-computing/ Tue, 07 Mar 2017 17:00:34 +0000 https://www.microsoft.com/en-us/research/?p=369245 By Dr. Meredith Ringel Morris, Principal Researcher, Microsoft Research I am pleased to announce that Microsoft Research is funding a new academic program, the Microsoft Research Dissertation Grant. This grant program offers selected doctoral students doing computing research at U.S. and Canadian universities up to US $20,000 to fund their dissertation work. This program is […]

The post New Microsoft Research Dissertation Grant provides support to under-represented groups in computing appeared first on Microsoft Research.

]]>
By Dr. Meredith Ringel Morris, Principal Researcher, Microsoft Research

Microsoft Research Dissertation Grant Program

I am pleased to announce that Microsoft Research is funding a new academic program, the Microsoft Research Dissertation Grant. This grant program offers selected doctoral students doing computing research at U.S. and Canadian universities up to US $20,000 to fund their dissertation work. This program is open to students currently under-represented in the technology sector, including women, people with disabilities, and people who are African-American/Black, Hispanic/Latino, American Indian/Alaskan Native, or Native Hawaiian/Pacific Islander, reflecting Microsoft’s commitment to growing the number of diverse students obtaining computing degrees. A key goal of this grant is to broaden participation and diversify the high-tech workforce.

This grant program targets students in their fourth year or beyond of doctoral studies. Students at this later stage of their doctoral work have a sufficiently concrete research plan that they should be able to articulate specific funding needs. Microsoft Research also offers funding programs for earlier stages of students’ doctoral studies, such as the PhD Fellowship Program, which is open to second- and third-year doctoral students.

Interested students should submit a description of their dissertation research, as well as a budget that details the amount and purpose of requested funds. Since every research project is different, Microsoft is not prescribing how the funds should be spent, but expects to see requests for equipment or data set purchases, compensation for experimental participants, travel for collecting or presenting research results, or student stipends. Proposals will be reviewed by the scientists who work for Microsoft’s global network of research labs, and evaluated based on the technical merit and potential for impact of the dissertation research.

In addition to receiving their grant, awardees will receive additional travel support to attend a two-day mentoring workshop in the autumn at the Microsoft Research Redmond Lab. During this event, grant recipients will meet one-on-one with Microsoft researchers who are doing work in related areas, receive information and advice regarding internship and post-doctoral career opportunities with Microsoft, and hear from the lab’s senior leadership about current scientific efforts at the company. Recipients will also present a talk describing their dissertation research and will receive feedback on their work from a panel of Microsoft researchers.  Attending the Dissertation Grant Workshop is an incredible learning and networking opportunity for the winners.

This year’s grant applications are due by April 7, 2017, with supporting reference letters due by April 24. Winners will be notified by June 30. Detailed information about the award, as well as the application form, can be found on the official website for the Microsoft Research Dissertation Grant.

I am excited about this opportunity to recognize and support technical innovation by students from under-represented groups; increasing the pipeline of diverse student talent is an important step toward growing a strong and diverse computing workforce.  I look forward to receiving grant applications for this program’s inaugural year.

Learn more and apply

The post New Microsoft Research Dissertation Grant provides support to under-represented groups in computing appeared first on Microsoft Research.

]]>
From improving a golf swing to reducing energy in datacenters https://www.microsoft.com/en-us/research/blog/2017-swiss-joint-research-center/ Tue, 21 Feb 2017 17:00:14 +0000 https://www.microsoft.com/en-us/research/?p=364241 2017 Swiss Joint Research Center kick off By Scarlet Schwiderski-Grosche, Senior Research Program Manager Recently, we celebrated an important milestone for our Swiss Joint Research Center (Swiss JRC). We welcomed top researchers from all partners to a workshop at the Microsoft Research Cambridge Lab, to kick off a new phase in our collaboration. This workshop […]

The post From improving a golf swing to reducing energy in datacenters appeared first on Microsoft Research.

]]>
2017 Swiss Joint Research Center kick off

By Scarlet Schwiderski-Grosche, Senior Research Program Manager

Attendees of the 2017 Swiss Joint Research Center Workshop in Cambridge, UK.

MSR Swiss Joint Research Center

(left to right) Aurelien Lucchi and Sebastian Stich, Postdoctoral Researchers at ETH Zurich and EPFL, and Martin Jaggi, Assistant Professor at EPFL, at the workshop.

Recently, we celebrated an important milestone for our Swiss Joint Research Center (Swiss JRC). We welcomed top researchers from all partners to a workshop at the Microsoft Research Cambridge Lab, to kick off a new phase in our collaboration. This workshop represented the end of a busy 10-month period for the Swiss JRC during which we ramped down projects from the first phase, and conducted a Call for Proposals for the selection of projects for the second phase. At the workshop, researchers from the Swiss JRC presented their selected proposals to kick off the collaborations in the new funding cycle.

First, a little background. The Swiss JRC is a collaborative research engagement between Microsoft Research and the two universities that make up the Swiss Federal Institutes of Technology: ETH Zurich (Eidgenössische Technische Hochschule Zürich, which serves German-speaking students) and EPFL (École Polytechnique Fédérale de Lausanne, which serves French-speaking students). The Swiss JRC is a continuation of a collaborative engagement that began back in 2009, when the same three partners embarked on ICES (Innovation Cluster for Embedded Software) and was renewed for another five years in 2014. Basically, university researchers collaborate with Microsoft researchers to solve problems in computer science.

Microsoft Research Swiss Joint Research Center

Pamela Delgado, PhD student on the EPFL project with Florin Dinu, “Towards Resources-Efficient Data Centers”

With this workshop, the Swiss JRC kicked off 10 projects, four between ETH Zurich and Microsoft and six between EPFL and Microsoft. These projects were chosen from 20 proposals assessed by the Swiss JRC steering committee for their intellectual merit, potential scientific and societal impact and evidence of strong collaborative interest between the project partners.

One compelling project uses drones that follow you around while you ski or play golf, then gives you feedback for improvement of your form—think of it as a personal trainer/GoPro/drone combo that can both figure out how to video you while you do an activity, as well as analyze your performance and make recommendations for improvements. Another drone-based project (or as we like to call them, micro-aerial vehicles, or MAVs), makes the MAV easier to control via a solution-based approach, versus movement-based controls. This project asks, “What is the goal of the MAV flight?” and solves for that, versus making the operator think about both “Where should this MAV go?” and “What should it do while it’s flying?”

Microsoft Swiss Joint Research Center

Babak Falsafi, Professor of Computer and Communication Sciences, EPFL

Other projects address new requirements in the data center, aiming at making data processing more efficient and essentially helping to reduce energy usage. One set of projects assesses data-intensive applications as are common in, for example, machine learning, graph processing, and bioinformatics. These projects explore near-memory processing, better server utilization, improved data clustering, and new approaches to transactional processing. Another set of projects leverages new hardware architectures based on, for example, FPGAs (Field Programmable Gate Arrays) and DRAM (Dynamic Random-Access Memory). Some projects address mechanisms to off-load expensive computations to achieve massive parallelism or to co-locate different stages of deep learning on the same platform. All of these projects propel the leading edge of artificial intelligence.

“Emerging silicon technologies provide an opportunity to offload data management services to near-memory accelerators for better performance. Through several Microsoft Research collaborations, including this funding round’s NeMeSys project, we are rapidly propelling the state-of-the-art in near-memory processing.” – Babak Falsafi, Professor of Computer and Communication Sciences, EPFL

Here’s the list of projects and their principal investigators:

Data Science with FPGAs in the Data Center
Gustavo Alonso, ETH Zurich
Ken Eguro, Microsoft Research, Redmond lab

Human-Centric Flight II: End-user Design of High-level Robotic Behavior
Otmar Hilliges, ETH Zurich
Marc Pollefeys, Microsoft Analog Research & Development

Tractable by Design
Thomas Hofmann and Aurelien Lucchi, ETH Zurich
Sebastian Nowozin, Microsoft Research, Cambridge lab

Enabling Practical, Efficient and Large-Scale Computation Near Data to Improve the Performance and Efficiency of Data Center and Consumer Systems
Onur Mutlu and Luca Benini, ETH Zurich
Derek Chiou, Microsoft Relevance and Intent, Research &Development

Towards Resource-Efficient Data Centers
Florin Dinu, EPFL
Christos Gkantsidis and Sergey Legtchenko, Microsoft Research, Cambridge lab

Near-Memory System Services
Babak Falsafi, EPFL
Stavros Volos, Microsoft Research, Redmond lab

Coltrain: Co-located Deep Learning Training and Inference
Babak Falsafi and Martin Jaggi, EPFL
Eric Chung, Microsoft Research, Redmond lab

From Companion Drones to Personal Trainers
Pascal Fua and Mathieu Salzmann, EPFL
Debadeepta Dey, Ashish Kapoor, and Sudipta Sinha, Microsoft Research, Redmond lab

Revisiting Transactional Computing on Modern Hardware
Rachid Guerraoui and Georgios Chatzopoulos, EPFL
Aleksandar Dragojevic, Microsoft Research, Cambridge lab

Fast and Accurate Algorithms for Clustering
Michael Kapralov and Ola Svensson, EPFL
Yuval Peres, Nikhil Devanur and Sebastien Bubeck, Microsoft Research, Redmond lab

Related:

The post From improving a golf swing to reducing energy in datacenters appeared first on Microsoft Research.

]]>
Partnership yields key breakthroughs in VR’s “grand challenge” https://www.microsoft.com/en-us/research/blog/partnership-yields-key-breakthroughs-vr-grand-challenge/ Mon, 06 Feb 2017 17:00:56 +0000 https://www.microsoft.com/en-us/research/?p=361382 By Noboru Sean Kuno, Research Program Manager, Microsoft Research Asia The potential for virtual reality (VR) to upend industrial design, medicine, and other specialized fields has now vaulted the emerging field into the ranks of what the National Academy of Engineering calls its 14 grand challenges of the 21st century, an eclectic list of endeavors […]

The post Partnership yields key breakthroughs in VR’s “grand challenge” appeared first on Microsoft Research.

]]>
By Noboru Sean Kuno, Research Program Manager, Microsoft Research Asia

The potential for virtual reality (VR) to upend industrial design, medicine, and other specialized fields has now vaulted the emerging field into the ranks of what the National Academy of Engineering calls its 14 grand challenges of the 21st century, an eclectic list of endeavors from preventing nuclear terror to securing cyberspace.

The importance of improving VR and 3D immersive communication has been a cornerstone of Microsoft’s long term investment in this technology space, resulting in multiple innovations from

Dr. Gene Cheung, associate professor, National Institute of Informatics, Japan

Dr. Gene Cheung, associate professor, National Institute of Informatics, Japan

Microsoft’s Kinect for Xbox 360 Sensor, Surface Hub and HoloLens to Windows Creator update.

Collaborating with partners

Realizing more immersive communication via 3D applications requires a quantum leap in the capture and exchange of 3D geometry that can only be achieved with an ongoing commitment to signal processing research. At the heart of this effort is our collaborative research (CORE) project with academic partners including Dr. Gene Cheung, associate professor at Japan’s National Institute of Informatics, who has been tackling this problem for years.

Breakthrough

Using depth-sensing devices such as the Kinect Sensor, researchers developed an algorithm to enable better noise reduction and restore missing details across images. Crucially, they discovered a method to utilize graph-signal smoothness prior to enhancing both natural images (see Fig. 1) and depth images.

<bFigure 1: example of original 4-bit image (left) and bit-depth enhanced image to 8 bits using our approach (right)

Figure 1: example of original 4-bit image (left) and bit-depth enhanced image to 8 bits using our approach (right)

Collaboration with Microsoft Research

Dinei Florencio, senior researcher at Microsoft Research, has been working alongside professor Cheung on research into “rate-constrained 3D surface estimation” and “precision enhancement of multiple 3D depth maps.”

“These two research lines are the most active in our recent collaboration,” Florenicio said. “As we make the needed progress toward immersive communication, I believe Gene’s research is bringing some fundamental contributions.”

Other key members of the project include Cha Zhang of Microsoft Research as well as Pengfei Wan, a former graduate student at Hong Kong University of Science and Technology.

Moving forward

Florencio and Chueng are now leading research into whether active light sensing can accurately detect informative bio-signals — such as pulse/respiratory rate and temperature changes on a face — to reveal stress and mood or indicate if subjects are lying. A key question of the research is whether active light sensing can be extended to reveal the same details for shaded or remote human subjects.

“The project is very interesting in that it tries to estimate bio-signals for more efficient face-to-face communications,” said Tao Mei, senior researcher at Microsoft Research. “The Principal Investigator (PI) proposed to use active imaging, which is entirely non-contact and noninvasive, to solve this problem with a novel idea by analyzing the constructed thermal and depth images in an indoor active image sensing system.”

Upon completion, Professor Cheung will make the research tool publicly available. I am looking forward to seeing continuous progress and achievements from this collaboration. We hope more researchers explore this area to expand the frontier of Virtual Reality technologies and realize Princess Leia’s holographic messaging in future.

The post Partnership yields key breakthroughs in VR’s “grand challenge” appeared first on Microsoft Research.

]]>
AI is getting smarter; Microsoft researchers want to ensure it’s also getting more accurate https://www.microsoft.com/en-us/research/blog/ai-getting-smarter-microsoft-researchers-ensure-ai-accuracy/ Fri, 03 Feb 2017 17:00:58 +0000 https://www.microsoft.com/en-us/research/?p=360965 By Allison Linn, Senior Writer, Microsoft Just a decade ago, the idea of using technology to do things like automatically translate conversations, identify objects in pictures — or even write a sentence describing those pictures — seemed like interesting research projects, but not practical for real-world use. The recent improvements in artificial intelligence have changed that. These […]

The post AI is getting smarter; Microsoft researchers want to ensure it’s also getting more accurate appeared first on Microsoft Research.

]]>
By Allison Linn, Senior Writer, Microsoft

Marine Carpuat working to make AI technologies accurate

Marine Carpuat, an assistant professor of computer science, works with colleague Philip Resnik, professor of linguistics, in the Computational Linguistics and Information Processing Laboratory at the University of Maryland.

Just a decade ago, the idea of using technology to do things like automatically translate conversations, identify objects in pictures — or even write a sentence describing those pictures — seemed like interesting research projects, but not practical for real-world use.

The recent improvements in artificial intelligence have changed that. These days more and more people are starting to rely on systems built with technologies such as machine learning. That’s raising new questions among artificial intelligence researchers about how to ensure that the basis for many of these systems — the algorithms, the training data and even the systems for testing the tools — are accurate and as unbiased as possible.

Ece Kamar

Ece Kamar, researcher

Ece Kamar, a researcher in Microsoft’s adaptive systems and interaction group, said the push comes as researchers and developers realize that, despite the fact that the systems are imperfect, many people are already trusting them for important tasks.

“This is why it is so important for us to know where our systems are making mistakes,” Kamar said.

At the AAAI Conference on Artificial Intelligence, which begins this weekend in San Francisco, Kamar and other Microsoft researchers will present two research papers that aim to use a combination of algorithms and human expertise to weed out data and system imperfections. Separately, another team of Microsoft researchers is releasing a corpus that can help speech translation researchers test the accuracy and effectiveness of their bilingual conversational systems.

The data underpinning artificial intelligence

When a developer creates a tool using machine learning, she generally relies on what’s called training data to teach the system to do a particular task. For example, to teach a system to recognize various types of animals, developers would likely show the system many pictures of animals so it could be trained to tell the difference between, say, a cat and a dog.

Theoretically, the system could then be shown pictures of dogs and cats it’s never seen before and still categorize them accurately.

But, Kamar said, training data systems can sometimes have some so-called blind spots that will lead to false results. For example, let’s say the system is only trained with pictures of cats that are white and dogs that are black. Show it a picture of a white dog, and it may make a false correlation and mislabel the dog as a cat.

These problems arise in part because many researchers and developers are using training sets that weren’t specifically designed for learning the task at hand. That makes sense – a set of data that already exists, such as an archive of animal pictures, is cheaper and faster than building the sets on your own – but it makes it all the more important to add these kinds of safety checks.

“Without these, we are not going to understand what kind of biases there are,” Kamar said.

In one of the research papers, Kamar and her colleagues show an algorithm that they think could be used to identify those blind spots in predictive models, allowing developers and researchers to fix the problem. It’s a research project for now, but they hope that it would eventually grow into something that developers and researchers could use to identify blind spots.

“Any kind of company or academic that’s doing machine learning needs these tools,” Kamar said.

Another research paper Kamar and her colleagues are presenting at the AAAI conference aims to help researchers figure out how different types of mistakes in a complex artificial intelligence system lead to incorrect results. That can be surprisingly difficult to parse out as artificial intelligence systems are doing more and more complex tasks, relying on multiple components that can become entangled.

For example, let’s say an automated photo captioning tool is describing a picture of a teddy bear as a blender. You might think the problem is with the component trained to recognize the pictures, only to find that it really lies in the element designed to write descriptions.

Kamar and her colleagues designed a methodology that provides guidance to researchers about how they can best troubleshoot these problems by simulating various fixes to root out where the trouble lies.

A ‘human in the loop’

For this and other research she has been conducting, Kamar said she was strongly influenced by the work she did on AI 100, a Stanford University-based study on how artificial intelligence will affect people over the next century.

Kamar said one takeaway from that work was the importance of making sure that people are deeply involved in developing, verifying and troubleshooting systems – what researchers call a “human in the loop.” That will ensure that the artificial intelligences we are creating augment human capabilities and reflect how we want them to perform.

Testing the accuracy of conversational translation

When developers and academic researchers create systems for recognizing the words in a conversation, they have well-regarded ways of testing the accuracy of their work: Sets of conversational data such as Switchboard and CALLHOME.

Christian Federmann

Christian Federmann, senior program manager

Christian Federmann, a senior program manager working with the Microsoft Translator team, said there aren’t as many standardized data sets for testing bilingual conversational speech translation systems such as the Microsoft Translator live feature and Skype Translator.

So he and his colleagues decided to make one.

The Microsoft Speech Language Translation corpus, which is being released publicly Friday for anyone to use, allows researchers to measure the quality and effectiveness of their conversational translation systems against a data set that includes multiple conversations between bilingual speakers who are speaking French, German and English.

The corpus, which was produced by Microsoft using bilingual speakers, aims to create a standard by which people can measure how well their conversational speech translation systems work.

“You need high-quality data in order to have high-quality testing,” Federmann said.

A data set that hits on the combination of both conversational speech and bilingual translation has been lacking until now.

Marine Carpuat, an assistant professor of computer science at the University of Maryland, who does research in natural language processing, said that when she wants to test how well her algorithms for conversational translation are working, she often has to rely on data that is freely available, such as official translations of European Union documents.

Those kinds of translations weren’t created to test conversational translation systems and they don’t necessarily reflect the more casual, spontaneous way in which people actually talk to each other, she said. That makes it difficult to know if the techniques she has will work when people want to translate a regular conversation, with all the attendant pauses, “ums” and other quirks of spoken language.

Carpuat, who was given early access to the corpus, said it was immediately helpful to her.

“It was a way of taking a system that I know does great on formal data and seeing what happens if we try to handle conversations,” she said.

Will Lewis

Will Lewis, principal technical program manager

The Microsoft team hopes the corpus, which will be freely available, will benefit the entire field of conversational translation and help to create more standardized benchmarks that researchers can use to measure their work against others.

“This helps propel the field forward,” said Will Lewis, a principal technical program manager with the Microsoft Translator team who also worked on the project.

Related:

Allison Linn is a senior writer at Microsoft. Follow her on Twitter

The post AI is getting smarter; Microsoft researchers want to ensure it’s also getting more accurate appeared first on Microsoft Research.

]]>
Microsoft Research PhD fellowships provide financial support to promising researchers https://www.microsoft.com/en-us/research/blog/microsoft-research-phd-fellowships-provide-financial-support/ Thu, 02 Feb 2017 17:00:32 +0000 https://www.microsoft.com/en-us/research/?p=360389 By Jim Pinkelman, Senior Director, Microsoft Research Since 2008, Microsoft Research has been awarding two-year PhD fellowships to computer science and related researchers at leading universities in the United States and Canada. These awards are designed to help promising young researchers focus on their studies, not their finances! This year’s program provides fellowships to 10, […]

The post Microsoft Research PhD fellowships provide financial support to promising researchers appeared first on Microsoft Research.

]]>
By Jim Pinkelman, Senior Director, Microsoft Research

Since 2008, Microsoft Research has been awarding two-year PhD fellowships to computer science and related researchers at leading universities in the United States and Canada. These awards are designed to help promising young researchers focus on their studies, not their finances!

This year’s program provides fellowships to 10, second- or third-year PhD students who are studying computer science, electrical engineering or mathematics. Recipients receive full tuition for their program, along with a generous living expense and a conference attendance stipend. In addition, the fellows are offered the opportunity to intern with a Microsoft researcher.

To identify these promising researchers, Microsoft Research goes through a rigorous process. First, department chairs nominate their best candidates, up to 9 from each university. Each application is vetted, and Microsoft researchers interview finalists to identify the awardees.

Microsoft Research 2017 PhD Fellowship Awardees

This year’s awardees for the 2017-19 academic years are:

Michael B. Cohen 
Massachusetts Institute of Technology
Mathematics, complexity and cryptography

Bita Darvish Rouhani
University of California, San Diego
Computer architecture and hardware

Michaelanne Dye
Georgia Institute of Technology
Human-computer interaction, social computing and collaboration

Kira Goldner
University of Washington
Algorithms and economic systems

Aditya Grover
Stanford University
Machine learning and intelligence

Silu Huang
University of Illinois at Urbana-Champaign
Data management and mining

Ethan J. Jackson
University of California, Berkeley
Mobility and networking

Saswat Padhi
University of California, Los Angeles
Software engineering and programming languages

Andrew Quinn
University of Michigan
Operating systems and distributed computing

Mengting Wan
University of California, San Diego
Data management and mining

Congratulations, 2017’s awardees! Interested in applying for next year? Applications from department chairs (sorry, no self-referrals) are due in October; ask your department chair if they are planning to participate.

Learn more

The post Microsoft Research PhD fellowships provide financial support to promising researchers appeared first on Microsoft Research.

]]>