{"id":680043,"date":"2020-08-04T09:33:34","date_gmt":"2020-08-04T16:33:34","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=680043"},"modified":"2020-10-06T15:59:25","modified_gmt":"2020-10-06T22:59:25","slug":"icml-2020-highlights-a-transformer-based-rl-agent-causal-ml-for-increased-privacy-and-more","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/icml-2020-highlights-a-transformer-based-rl-agent-causal-ml-for-increased-privacy-and-more\/","title":{"rendered":"ICML 2020 highlights: A Transformer-based RL agent, causal ML for increased privacy, and more"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

With over 50 papers from Microsoft accepted at this year\u2019s International Conference on Machine Learning (ICML 2020) (opens in new tab)<\/span><\/a>, a number of which were presented in virtual workshops, Microsoft researchers are in full summer swing when it comes to advancing machine learning in accessibility, privacy, healthcare, and other areas. As Microsoft Partner Research Manager and ICML President John Langford<\/a> puts it, \u201cICML is a very broad conference, so its specialty is in some sense \u2018all of the above.\u2019\u201d But Langford goes on to add that one of the topics that ICML has a long track record on is currently trending: reinforcement learning. A brief glance through the sessions <\/a>and workshops <\/a>presented by Microsoft researchers shows the wide influence reinforcement learning has in our world today, from natural language to robotics to infrastructure considerations like transportation.<\/p>\n\n\n\n

Beyond the research contributions, Microsoft was also a sponsor of and recruiter at the conference. Additionally, the company sponsored two events co-located with the conference, the first Women in Machine Learning Un-Workshop (opens in new tab)<\/span><\/a> and the fourth Queer in AI Workshop (opens in new tab)<\/span><\/a>. The impact of the conference\u2014now and in the future\u2014is multifaceted, according to Langford. \u201cICML is \u2018the\u2019 summer machine learning conference. As such, it\u2019s critically important to the academic discovery, review, and dissemination process, a great way to meet fellow researchers, and a natural recruiting point for the field,\u201d he says.<\/p>\n\n\n\n\t

\n\t\t\n\n\t\t

\n\t\tSpotlight: AI-POWERED EXPERIENCE<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"\"\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

Microsoft research copilot experience<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Discover more about research at Microsoft through our AI-powered experience<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tStart now\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n

Below are five selections of research presented by Microsoft. These projects highlight how broadly researchers are thinking about ML and its implications for society. But this diverse group of papers represents only a small slice of the advancements presented by Microsoft researchers. Explore the Microsoft at ICML 2020 accepted papers list<\/a> to learn about further research contributions. <\/p>\n\n\n

See sections on: ICML 2020 Overview<\/u> | How AI models reason <\/a> | Utility and privacy with causal machine learning<\/a> | Using Transformers to create RL agents<\/a> | Pretraining for bidirectional language models<\/a> | Identifying layer normalization location<\/a><\/p>\n\n\n

\"\"<\/figure>\n\n\n\n
<\/div>\n\n

Understanding how AI models reason about what they see <\/h3>\n\n\n\n

Bottom line:<\/strong> \u201cWe propose a principled approach to isolate, analyze, and interpret how visual question-answering models reason about what they see.\u201d 
\u2014Machine Learning Scientist Saeed Amizadeh<\/a><\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tPublication <\/span>\n\t\t\tNeuro-Symbolic Visual Reasoning: Disentangling \u201cVisual\u201d from \u201cReasoning\u201d<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

Quick glance:<\/strong> In \u201cNeuro-Symbolic Visual Reasoning: Disentangling \u2018Visual\u2019 from \u2018Reasoning,\u2019\u201d<\/a> researchers from the Microsoft Applied Sciences Lab and MSR AI collaborated to combine visual understanding and neuro-symbolic reasoning with natural language processing and program synthesis. \u201cWe develop a novel way to perform differentiable logical inference over visual scenes, which allows us to disentangle the processes of reasoning and perception in visual question answering (VQA) models,\u201d explains Amizadeh. The work also led to creating a methodology for evaluating state-of-the-art VQA models, and the researchers propose expanding beyond pure probabilistic logical reasoning to incorporate other contextual signals and improve visual perception of the models. <\/p>\n\n\n\n

Areas of impact: <\/strong>This research lies at the intersection of natural language and visual perception, which makes it a good candidate for systems using AI for accessibility. Key is the work\u2019s focus on interpretability. The user should understand throughout the whole process how the neural reasoning is connecting what it \u201csees\u201d to language, building trust and reliability in AI. <\/p>\n\n\n\n

Fun Fact:<\/strong> This project initially began as a project out of the Microsoft AI Residency Program. <\/p>\n\n\n\n

The research team: <\/strong>Saeed Amizadeh<\/a>, Hamid Palangi<\/a>, Alex Polozov<\/a>, Yichen Huang, Kazuhito Koishida<\/a><\/p>\n\n\n\n

Additional Resources: <\/strong> 
 
Applied Sciences homepage <\/a>
 
MSR AI homepage<\/a>  
 
Microsoft AI Residency Program<\/a><\/p>\n\n\n\n

<\/div>\n\n

Improving utility and privacy with causal machine learning<\/h3>\n\n\n\n

Bottom line: <\/strong>\u201cWhat if you can build machine learning models that are both accurate and preserve privacy of individuals?<\/em> Try causal predictive models: We show that they are more robust to privacy attacks like membership inference and have higher accuracy on new domains than typical ML models.\u201d 
\u2014Microsoft Senior Researchers Amit Sharma <\/a>and Shruti Tople<\/a><\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tPublication <\/span>\n\t\t\tAlleviating Privacy Attacks via Causal Learning<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

Quick glance: <\/strong>Privacy is paramount for institutions like hospitals and governments, which handle sensitive datasets and use ML models. Standard ML privacy approaches add noise to a model or data to protect information, but this can have the undesired effect of reducing accuracy or utility of the model. This work shows that causal learning, by which ML models are trained based on domain knowledge about causal relationships between features and outcomes, can increase both privacy and utility when compared to associational ML models with the same amount of noise. Researchers from Microsoft Research India <\/strong>provided knowledge of causal ML for this project, while researchers from Microsoft Research Cambridge brought expertise on privacy and security. Their paper is called \u201cAlleviating Privacy Attacks via Causal Learning.”<\/a><\/p>\n\n\n\n

Areas of impact: <\/strong>This work aims to improve privacy protections for institutions using sensitive data with causal ML. In addition, this direction allows for improved model sharing across institutions and allows individuals to voluntarily share their own data without risk of information being leaked by an ML model.  <\/p>\n\n\n\n

New tools: <\/strong>The researchers have released an open-source toolkit (opens in new tab)<\/span><\/a>, RobustDG, for evaluating causal ML models on privacy, robustness, and out-of-distribution accuracy.<\/p>\n\n\n\n

The research team: <\/strong>Amit Sharma<\/a>, Shruti Tople<\/a>, and Aditya Nori<\/a><\/p>\n\n\n\n

Additional Resources:<\/strong> 
 
GitHub repository including open-source toolkit (opens in new tab)<\/span><\/a>
 
Microsoft Research India homepage 

<\/a>Microsoft Research Cambridge homepage<\/a>  <\/p>\n\n\n\n

<\/div>\n\n

Using Transformers to create RL agents suited for real-world tasks<\/h3>\n\n\n\n

Bottom line: <\/strong>\u201cTransformers for RL!\u201d 
\u2014Senior Research Software Engineer Ricky Loynd<\/a><\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tPublication<\/span>\n\t\t\tWorking Memory Graphs<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

Quick glance: <\/strong>\u201cWorking Memory Graphs<\/a>\u201d presents a new reinforcement learning agent \u201cthat accelerates learning on challenging tasks by leveraging the power of Transformers in three ways,\u201d explains Loynd. These three approaches apply Transformer attention to past observations, recurrent state vectors, and factored observations, respectively. \u201cBy leveraging the power of Transformers in these ways, our Working Memory Graph (WMG) agent accelerates learning on several challenging tasks:\u202fBabyAI, Pathfinding, and Sokoban. In BabyAI, WMG achieves drastic improvements in sample efficiency when observations are factored into more succinct representations,\u201d says Loynd. The team includes members from the reinforcement learning and deep learning groups within MSR AI.<\/p>\n\n\n\n

Areas of impact: <\/strong>This work shows that WMG is effective in handling the structured, factored observations used in today\u2019s real-world applications of RL and accelerates RL so that AI agents will eventually be able to accomplish previously unattainable real-world tasks.<\/p>\n\n\n\n

Performance and novel features: <\/strong>WMG outperforms a GRU (Gated Recurrent Unit) baseline agent at complex reasoning over past observations, and WMG has a new form of \u201cshortcut recurrence\u201d that proves to be more effective than standard gated recurrence. Sokoban results demonstrate that WMG performs better on this complex domain than the state-of-the-art Deep Repeated ConvLSTM (DRC) agent (by Google DeepMind) throughout 20 million steps of training.<\/p>\n\n\n\n

The research team: <\/strong>Ricky Loynd<\/a>, Roland Fernandez<\/a>, Asli Celikyilmaz<\/a>, Adith Swaminathan<\/a>, and Matthew Hausknecht<\/a>. <\/p>\n\n\n\n

Additional Resources:<\/strong> 
 
Working Memory Graph GitHub repository 

(opens in new tab)<\/span><\/a>MSR AI homepage<\/a><\/p>\n\n\n\n

<\/div>\n\n

Efficient pretraining for bidirectional language models in one forward pass<\/h3>\n\n\n\n

Bottom line: <\/strong>\u201cOur work efficiently realizes unified pretraining of bidirectional language models (via autoencoding) and sequence-to-sequence language models (via partially autoregressive) with a pseudo-masked language model for language understanding and generation.\u201d  
\u2014Senior Principal Research Manager Furu Wei<\/a><\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tPublication<\/span>\n\t\t\tUniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

Quick glance: <\/strong>This research introduces pseudo-masked language models, allowing for efficient pretraining of bidirectional language models in natural language understanding and sequence-to-sequence language models in natural language generation in one forward pass. This work is a collaboration between Microsoft Research Asia, Microsoft Research Redmond, and both the DeepSpeed and Project Turing teams, who help scale up the pretraining to larger models and are working to implement those models in Microsoft products in an initiative called AI at Scale. The paper is titled \u201cUniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training<\/a>.\u201d<\/p>\n\n\n\n

Areas of impact: <\/strong>This novel language model improves techniques for natural language generation, including document summarization and dialog generation. It also builds on techniques for natural language understanding, which includes text classification, question answering, and information extraction.  <\/p>\n\n\n\n

New state of the art: <\/strong>Results show this model achieves state of the art on various natural language generation and understanding tasks across numerous benchmarks.<\/p>\n\n\n\n

The research team: <\/strong><\/p>\n\n\n\n

MSR Asia: <\/strong>Li Dong<\/a>, Furu Wei<\/a>, Wenhui Wang, Nan Yang<\/a>, Ming Zhou<\/a>, Hsiao-Wuen Hon<\/a>, and collaborators Hangbo Bao and Songhao Piao<\/p>\n\n\n\n

MSR Redmond: <\/strong>Xiaodong Liu,<\/a> Yu Wang<\/a>, and Jianfeng Gao <\/a><\/p>\n\n\n\n

Additional resources: <\/strong><\/p>\n\n\n\n

GitHub repository of UniLM  (opens in new tab)<\/span><\/a><\/p>\n\n\n\n

AI at Scale homepage<\/a> <\/p>\n\n\n\n

Microsoft Research Asia homepage<\/a><\/strong> <\/p>\n\n\n\n

Microsoft Research Redmond homepage <\/a><\/p>\n\n\n\n

DeepSpeed homepage (opens in new tab)<\/span><\/a> <\/p>\n\n\n\n

Project Turing homepage (opens in new tab)<\/span><\/a><\/p>\n\n\n\n

<\/div>\n\n

Correctly identifying layer normalization location for better Transformer optimization<\/h3>\n\n\n\n

Bottom line:<\/strong><\/strong> \u201cUse Pre-LN Transformer to remove the annoying warm-up stage and save greatly on converge time.\u201d
<\/strong> \u2014 Microsoft Researchers Di He<\/a> and Shuxin Zheng<\/a><\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tPublication<\/span>\n\t\t\tOn Layer Normalization in the Transformer Architecture<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

Quick glance: <\/strong>This research explores a known optimization issue with the original Transformer (BERT) that causes slowed down training, requiring hyperparameter tunings. The researchers offer theoretical proof the issue emerges from the location of layer normalization. They propose a variant of Pre-LN Transformer that correctly locates the layer normalization with easy optimization and the ability to quickly converge. This work was done by researchers affiliated with Microsoft Research Asia, the Chinese Academy of Sciences, and Peking University. The research is detailed in the paper \u201cOn Layer Normalization in the Transformer Architecture<\/a>.\u201d  <\/p>\n\n\n\n

Areas of impact: <\/strong>There are many projects already using Pre-LN Transformer to train large-scale BERT models because of its exceptional optimization stability, including training on NVIDIA\u2019s Megatron, Open AI\u2019s GPT-2, and Open AI\u2019s GPT-3 models.<\/p>\n\n\n\n

Added benefits: <\/strong>Because of the way this variant operates, it requires no additional hyperparameter tuning. This fact, combined with faster convergence, results in boosted energy efficiency.  <\/p>\n\n\n\n

The research team:<\/strong> Di He<\/a>, Shuxin Zheng<\/a>, Huishuai Zhang<\/a>, and Tie-Yan Liu (opens in new tab)<\/span><\/a>  <\/p>\n\n\n\n

Additional Resources:<\/strong> <\/strong> 
 
AI at Scale homepage<\/a> <\/p>\n\n\n\n

Microsoft Research Asia homepage<\/a> <\/p>\n\n\n\n

Neural Machine Translation <\/a><\/p>\n\n\n\n

<\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tExplore more <\/span>\n\t\t\tICML 2020 accepted papers <\/span> <\/span><\/a>\t\t\t\t\t\t\t

Check out the complete list of accepted papers from MSR at ICML 2020 <\/p>\n\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n


 
 <\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

With over 50 papers from Microsoft accepted at this year\u2019s International Conference on Machine Learning (ICML 2020) (opens in new tab), a number of which were presented in virtual workshops, Microsoft researchers are in full summer swing when it comes to advancing machine learning in accessibility, privacy, healthcare, and other areas. As Microsoft Partner Research […]<\/p>\n","protected":false},"author":38838,"featured_media":682260,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[],"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[],"research-area":[13561,13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-680043","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-algorithms","msr-research-area-artificial-intelligence","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199560,199561],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[558228],"related-events":[670011],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"\"\"","byline":"","formattedDate":"August 4, 2020","formattedExcerpt":"With over 50 papers from Microsoft accepted at this year\u2019s International Conference on Machine Learning (ICML 2020) (opens in new tab), a number of which were presented in virtual workshops, Microsoft researchers are in full summer swing when it comes to advancing machine learning in…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/680043","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/38838"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=680043"}],"version-history":[{"count":35,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/680043\/revisions"}],"predecessor-version":[{"id":696453,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/680043\/revisions\/696453"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/682260"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=680043"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=680043"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=680043"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=680043"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=680043"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=680043"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=680043"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=680043"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=680043"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=680043"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=680043"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}