{"id":722851,"date":"2021-03-05T07:50:16","date_gmt":"2021-03-05T15:50:16","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-group&#038;p=722851"},"modified":"2024-03-06T09:34:32","modified_gmt":"2024-03-06T17:34:32","slug":"microsoft-turing-academic-program","status":"publish","type":"msr-group","link":"https:\/\/www.microsoft.com\/en-us\/research\/collaboration\/microsoft-turing-academic-program\/","title":{"rendered":"Microsoft Turing Academic Program (MS-TAP)"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background- card-background--full-bleed\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1920\" height=\"720\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Turing-Academic-Prog_HLT_header_02-2021_1920x720.jpg\" class=\"attachment-full size-full\" alt=\"Microsoft Turing Academic Program header\" style=\"\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Turing-Academic-Prog_HLT_header_02-2021_1920x720.jpg 1920w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Turing-Academic-Prog_HLT_header_02-2021_1920x720-300x113.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Turing-Academic-Prog_HLT_header_02-2021_1920x720-1024x384.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Turing-Academic-Prog_HLT_header_02-2021_1920x720-768x288.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Turing-Academic-Prog_HLT_header_02-2021_1920x720-1536x576.jpg 1536w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Turing-Academic-Prog_HLT_header_02-2021_1920x720-16x6.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Turing-Academic-Prog_HLT_header_02-2021_1920x720-1600x600.jpg 1600w\" sizes=\"auto, (max-width: 1920px) 100vw, 1920px\" \/>\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 align-self-center\">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading h2\" id=\"microsoft-turing-academic-program-ms-tap\">Microsoft Turing Academic Program (MS-TAP)<\/h1>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u201c<em>There\u2019s a great deal of interest and enthusiasm about the construction and use of large-scale neural language models. We\u2019re seeing new capabilities\u2014and they\u2019re being pressed into service in exciting applications. However, these models, built in a self-supervised manner from massive corpora, can generate offensive, biased, and costly output. We need to better understand these behaviors and to develop methods for mitigating harms. Considering both the value and potential costs of AI innovations, and developing best practices for addressing the risks, is central in the responsible development and fielding of AI technologies.<\/em>\u201d<\/p>\n<cite><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/horvitz\/\">Eric Horvitz<\/a>, Chief Scientific Officer, Microsoft<\/cite><\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u201c<em>As AI models are becoming more powerful and reaching a large section of population either directly or indirectly through Microsoft\u2019s products and services, it is becoming pertinent to have the best minds of the world look at the impact these models can have and identify mechanisms to improve upon them. We believe strongly in improvements through collaboration and open research. We are excited about the potential contributions these new research collaborations can make, both to Microsoft and to the open research community interested in large-scale, language-centric models.<\/em>\u201d<\/p>\n<cite><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, Vice President & Distinguished Engineer, Microsoft <\/cite><\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"program-description\">Program Description<\/h2>\n\n\n\n<p>Microsoft is committed to the <a href=\"https:\/\/www.microsoft.com\/en-us\/ai\/responsible-ai?activetab=pivot1:primaryr6\" target=\"_blank\" rel=\"noopener\">responsible development and fielding of AI technologies<\/a> including careful deliberation about the value and costs of harnessing large-scale neural language models. These models have been delivering breakthroughs in language capabilities, but have also been found to have the ability to generate language fraught with bias, toxic language, and denigration.<\/p>\n\n\n\n<p>We have created the Microsoft Turing Academic Program (MS-TAP) as part of our program to share Microsoft advances with <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/msturing.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">Microsoft\u2019s Turing family<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> of natural language models in responsible manner. MS-TAP provides leading academics and researchers with a private preview of Turing models. Our goal is to engage our colleagues on shared interests, with a goal of better understanding model behavior, identifying novel applications, exploring and mitigating potential risks and mitigations, and improving future models.<\/p>\n\n\n\n<p>Program participants collaborate closely with Microsoft Turing scientists as well as domain experts contributing to Microsoft\u2019s on ethical and responsible AI. As concerns may come to the fore with AI advances, we seek to spend time and effort to better understand the capabilities, benefits, and costs of AI technologies in advance of general releases to the public. We take a stepwise approach to releasing the technology per our dual goals of sharing our technologies broadly and ensuring that AI models are used responsibly and safely in the open world.<\/p>\n\n\n\n<p>Specific goals of MS-TAP include the following:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Stimulate new, high-impact research on model behavior<\/strong>: Explore the application of Turing language models in a variety of downstream tasks that span various domains of machine learning. Such work expands knowledge of how the models are likely to perform when released more broadly.<\/li>\n\n\n\n<li><strong>Explore inadvertent and unintended outcomes<\/strong>: Identify and evaluate the spectrum of potential unintended outcomes posed by Turing models. Generate new insights into the different types and severity of potential unintended outcomes that the models present.<\/li>\n\n\n\n<li><strong>Develop practices and tools that can mitigate harms<\/strong>: Identify ways to mitigate potential negative, unintended outcomes. Mitigations can use existing tools and practices as well as recommendations for creation of specialized new tools and practices.<\/li>\n<\/ul>\n\n\n\n<p>This program will be accomplished in phases\u2014currently in Phase 1 we are testing the Turing Natural Language Representation model. Upon successful completion, we will add future phases that will involve larger and more complex models. <\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-outline is-style-outline--1\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"#!collaboration-projects\">Explore the research projects<\/a><\/div>\n<\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"our-collaborators\">Our collaborators<\/h3>\n\n\n\n<p>We are working closely with the following research collaborators:<\/p>\n\n\n\n<div class=\"wp-block-columns are-vertically-aligned-center is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Brown-University_logo_300x200.jpg\" alt=\"Brown University logo\" class=\"wp-image-724105\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Brown-University_logo_300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Brown-University_logo_300x200-16x12.jpg 16w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/CMU-LTI_logo_300x200.png\" alt=\"Carnegie Mellon University: Language Technologies Institute logo\" class=\"wp-image-818149\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/CMU-LTI_logo_300x200.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/CMU-LTI_logo_300x200-240x160.png 240w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/09\/EPFL_300x200.jpg\" alt=\"EPFL logo\" class=\"wp-image-691188\"\/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-columns are-vertically-aligned-center is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Georgia-Tech_logo_300x200.jpg\" alt=\"Georgia Tech logo\" class=\"wp-image-724108\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Georgia-Tech_logo_300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/Georgia-Tech_logo_300x200-16x12.jpg 16w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/Stanford-University_logo-stacked_300x200.png\" alt=\"Stanford University logo\" class=\"wp-image-818152\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/Stanford-University_logo-stacked_300x200.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/Stanford-University_logo-stacked_300x200-240x160.png 240w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/UC-Berkeley_logo_300x200.jpg\" alt=\"University of California, Berkeley\" class=\"wp-image-724111\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/UC-Berkeley_logo_300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/UC-Berkeley_logo_300x200-16x12.jpg 16w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-columns are-vertically-aligned-center is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/UCSF_sig_navy_300x200.jpg\" alt=\"University of California, San Francisco\" class=\"wp-image-724114\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/UCSF_sig_navy_300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/02\/UCSF_sig_navy_300x200-16x12.jpg 16w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/UofMichigan_logo_300x200.png\" alt=\"University of Michigan logo\" class=\"wp-image-818146\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/UofMichigan_logo_300x200.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/03\/UofMichigan_logo_300x200-240x160.png 240w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" height=\"200\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/University-Washington_logo2_300x200.png\" alt=\"University of Washington logo\" class=\"wp-image-674199\"\/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-outline is-style-outline--2\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"#!collaboration-projects\">Explore the research projects<\/a><\/div>\n<\/div>\n\n\n\n\n\n<p>The program includes collaborative projects with academia to stress test large, natural language models.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"round-1-proposals-natural-language-representation-model\">Round 1 proposals: Natural Language Representation Model<\/h3>\n\n\n\n\n\n<p><strong>University of California, Berkeley<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/binyu.stat.berkeley.edu\/\" target=\"_blank\" rel=\"noopener noreferrer\">Bin Yu<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/csinva.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Chandan Singh<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/haywse.github.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Wooseok Ha<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (Postdoc), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.stat.berkeley.edu\/~yugroup\/people\/Briton.html\" target=\"_blank\" rel=\"noopener noreferrer\">Briton Park<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.stat.berkeley.edu\/~yugroup\/people\/Robbie.html\" target=\"_blank\" rel=\"noopener noreferrer\">Robert Netzorg<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student)<\/p>\n\n\n\n<p>Departments: Department of Statistics, Department of Electrical Engineering and Computer Sciences, and Center for Computational Biology, UC Berkeley Department of Urology<\/p>\n\n\n\n<p><strong>University of California, San Francisco<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/urology.ucsf.edu\/people\/anobel-y-odisho\" target=\"_blank\" rel=\"noopener noreferrer\">Anobel Odisho<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI)<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/pepotash\/\">Peter Potash<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/krisgan\/\">Kris Ganjam<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/alinastoicabeck\/\" target=\"_blank\" rel=\"noopener noreferrer\">Alina Stoica Beck<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/dcarig\/\">Dean Carignan<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/fpoursabzi\/\">Forough Poursabzi Sangdeh<\/a><\/p>\n\n\n\n<p>Recent language models, such as Microsoft&#8217;s Turing Natural Language Representation (TNLR) models, have shown an impressive ability to capture semantic information useful for transferring to new tasks. Much of human medical intelligence is contained in medical language notes such as pathology reports. Automated data extraction from such notes will be a key driver for the delivery of precision medicine (e.g. providing patient-level cancer grade information for treatment selection). It is also important for clinical research, risk stratification, and clinical trial enrollment. We propose to evaluate these language models in comparison to our current methods to improve performance in natural language processing tasks relating to automated data extraction.<\/p>\n\n\n\n\n\n<p><strong>Brown University<\/strong>\/<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/lunar.cs.brown.edu\/\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>LUNAR<\/strong><span class=\"sr-only\"> (opens in new tab)<\/span><\/a>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/cs.brown.edu\/people\/epavlick\/\" target=\"_blank\" rel=\"noopener noreferrer\">Ellie Pavlick<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.cs.brown.edu\/~mlittman\" target=\"_blank\" rel=\"noopener noreferrer\">Michael Littman<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (faculty), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/cs.brown.edu\/people\/rpatel59\/\" target=\"_blank\" rel=\"noopener noreferrer\">Roma Patel<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student)<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/pomoradi\/\">Pooya Moradi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/dcarig\/\">Dean Carignan<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/fpoursabzi\/\">Forough Poursabzi Sangdeh<\/a><\/p>\n\n\n\n<p>Recent studies have shown that neural language models (LMs) trained on large text corpora can encode societal biases. Such models risk generating toxic text, whether in response to deliberate manipulation or seemingly innocuous natural language prompts. This fact is especially worrying in scenarios where these language models form the basis of NLP systems that are deployed in production. We propose to investigate the extent to which generative LMs are sensitive to subtle linguistic framing effects that form the basis of linguistic theories of bias in written text. In particular, we test whether prompts that contain linguistic markers of author bias (e.g., hedges, implicatives, subjective intensifiers, assertives) result in measurable differences in models&#8217; generated passages. We then propose several measures to identify linguistic markers in text that cause LMs to exacerbate this bias in their generations, as well methods that attempt to mitigate this.<\/p>\n\n\n\n\n\n<p><strong>EPFL<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/dlab.epfl.ch\/people\/west\/\" target=\"_blank\" rel=\"noopener noreferrer\">Robert West<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), Maxime Peyrard (Postdoc), Martin Josifoski (PhD student)<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/bapatra\/\">Barun Patra<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/viagar\/\">Vidhan Agarwal<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/sarvjeet-singh-ghotra-1a145898\/\" target=\"_blank\" rel=\"noopener noreferrer\">Sarv Ghotra<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/dcarig\/\">Dean Carignan<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/fpoursabzi\/\">Forough Poursabzi Sangdeh<\/a><\/p>\n\n\n\n<p>We propose to reduce the impact of spurious correlations within large language models by leveraging invariance learning principles. In particular, we will continue the training of existing language models according to the Invariant Risk Minimization (IRM) paradigm in order to enforce domain-invariant representations and achieve better out-of-domain generalization.<\/p>\n\n\n\n\n\n<p><strong>Georgia Tech<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.munmund.net\/\" target=\"_blank\" rel=\"noopener noreferrer\">Munmun De Choudhury<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/aritter.github.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Alan Ritter<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (faculty), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.cc.gatech.edu\/~dchau\/\" target=\"_blank\" rel=\"noopener noreferrer\">Duen &#8220;Polo&#8221; Horng Chau<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (faculty), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.cc.gatech.edu\/~dyang888\/\" target=\"_blank\" rel=\"noopener noreferrer\">Diyi Yang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (faculty), Mai ElSherief (Postdoc), PhD students: Caleb Ziems, Ashutosh Baheti, Yang Chen, Jay Wang, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.austinpwright.com\" target=\"_blank\" rel=\"noopener noreferrer\">Austin Wright<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>; Undergrads: Zhaoran Ma, Vincent Lieng, Omar Shaikh<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurbah Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/dcarig\/\">Dean Carignan<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/fpoursabzi\/\">Forough Poursabzi Sangdeh<\/a><\/p>\n\n\n\n<p>This project proposes a generic set of research thrusts for understanding and using large pretrained language models, specifically leveraging Microsoft\u2019s Turing project language models to a) understand and quantify biases in diverse societal contexts, and b) propose methods to mitigate biases that get encoded in neural models built using naturalistically occurring behavioral trace data.<\/p>\n\n\n\n\n\n<p><strong>University of Washington<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/homes.cs.washington.edu\/~yejin\/\" target=\"_blank\" rel=\"noopener noreferrer\">Yejin Choi<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/homes.cs.washington.edu\/~nasmith\/\" target=\"_blank\" rel=\"noopener noreferrer\">Noah Smith<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/homes.cs.washington.edu\/~msap\/\" target=\"_blank\" rel=\"noopener noreferrer\">Maarten Sap<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/sites.google.com\/uw.edu\/annajafarpour\" target=\"_blank\" rel=\"noopener noreferrer\">Anna Jafarpour<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (Postdoctoral fellow) and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/homes.cs.washington.edu\/~eaclark7\/\" target=\"_blank\" rel=\"noopener noreferrer\">Elizabeth Clark<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student)<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/pomoradi\/\">Pooya Moradi<\/a>, Zhun Liu, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/kragga\/\">Kriti Aggarwal<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/dcarig\/\">Dean Carignan<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/fpoursabzi\/\">Forough Poursabzi Sangdeh<\/a><\/p>\n\n\n\n<p>Recent computing and deep learning advances have led to significant improvements in language modelling and text generation. However, these new language models (LM)are known to generate toxic and false information with human-like fluency, prohibiting their safe deployment. We propose two lines of research to address these issues, focusing on Microsoft\u2019s Turing Natural Language Representation (TNLR) models. First, we will investigate methods for reducing toxic generations and for improving readers\u2019 ability to distinguish between human- and machine-generated text. Second, we will look at how models distinguish between real and fake events and how to best introduce new knowledge through finetuning and conditioning.<\/p>\n\n\n\n\n\n\n\n\n\n<p><strong>Carnegie Mellon University<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.cs.cmu.edu\/~max\/\" target=\"_blank\" rel=\"noopener noreferrer\">Maxine Eskenazi<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/shikib.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Shikib Mehri <span class=\"sr-only\"> (opens in new tab)<\/span><\/a>(PhD student)&nbsp;&nbsp;<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/payal-bajaj-3166a241\/\" target=\"_blank\" rel=\"noopener noreferrer\">Payal Bajaj<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/adadhika\/?originalSubdomain=ca\" target=\"_blank\" rel=\"noopener noreferrer\">Ashutosh Adhikari<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/vchaudhary\/\">Vishrav Chaudhary<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/sarvjeet-singh-ghotra-1a145898\/\" target=\"_blank\" rel=\"noopener noreferrer\">Sarv Ghotra<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/bapeng\/\" target=\"_blank\" rel=\"noreferrer noopener\">Baolin Peng<\/a>&nbsp;<\/p>\n\n\n\n<p>We propose to leverage TNLG in combination with the schema-guided paradigm to facilitate zero-shot transfer to unseen dialog tasks. While the schema-guided paradigm would handle generalization to new policies, TNLG would allow zero-shot NLU\/NLG. Concretely, we propose four different approaches for using TNLG to facilitate zero-shot transfer: (1) using TNLG naively without the schema-guided paradigm, (2) representing the dialog policy as a sequence that is input to TNLG, (3) fine-tuning TNLG to attend to a graph &#8211; based representation of the dialog policy and (4) exploring the use of fusion mechanisms (e.g., ColdFusion) with TNLG. Through these approaches, we hope to assess the impact of large-scale pre-training on zero-shot transfer in end-to-end dialog and better understand the capabilities of TNLG.<\/p>\n\n\n\n\n\n<p><strong>Carnegie Mellon University<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.cs.cmu.edu\/~katef\/\" target=\"_blank\" rel=\"noopener noreferrer\">Katerina Fragkiadaki<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.cs.cmu.edu\/~tom\/\" target=\"_blank\" rel=\"noopener noreferrer\">Tom Mitchell<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/hcii.cmu.edu\/people\/chris-atkeson\" target=\"_blank\" rel=\"noopener noreferrer\">Chris Atkeson<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), Nikos Gkanasios (PhD student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.cs.cmu.edu\/directory\/ayushj2\" target=\"_blank\" rel=\"noopener noreferrer\">Ayush Jain<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (Masters student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/yunchuzhang.github.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Yunchu Zhang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (Masters student)&nbsp;<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/kragga\/#:~:text=Kriti%20Aggarwal%20is%20an%20Applied%20Scientist%20in%20the,products%20%28Bing%2C%20M365%2C%20Outlook%2C%20Word%2C%20Teams%2C%20Powerpoint%20etc.%29.\" target=\"_blank\" rel=\"noreferrer noopener\">Kriti Aggarwal<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/vchaudhary\/\">Vishrav Chaudhary<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/sarvjeet-singh-ghotra-1a145898\/\" target=\"_blank\" rel=\"noopener noreferrer\">Sarv Ghotra<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/julia-kiseleva-24842710\/\" target=\"_blank\" rel=\"noopener noreferrer\">Julia Kiseleva<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/p>\n\n\n\n<p>We will explore and develop algorithms to simplify human collaboration with robots and make programming collaborative robots (co-robots or cobots) less expensive, with the assistance of human teachers that employ natural language descriptions paired with visual or kinesthetic demonstrations, in order to teach robotic agents new skills or help them adjust\/improve existing ones.<\/p>\n\n\n\n\n\n<p><strong>Harvard<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.hsph.harvard.edu\/junwei-lu\/\" target=\"_blank\" rel=\"noopener noreferrer\">Junwei Lu<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.hsph.harvard.edu\/tianxi-cai\/\" target=\"_blank\" rel=\"noopener noreferrer\">Tianxi Cai<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/dbmi.hms.harvard.edu\/people\/katherine-liao\" target=\"_blank\" rel=\"noopener noreferrer\">Katherine P. Liao<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Doudou Zhou (PhD student), Keming Lu (Masters student), Zebin Wang (PhD student), Shuting Sheng (PhD student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/dbmi.hms.harvard.edu\/people\/priyam-das\" target=\"_blank\" rel=\"noopener noreferrer\"><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/statistics.fas.harvard.edu\/people\/yue-liu\" target=\"_blank\" rel=\"noopener noreferrer\">Yue Liu<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student)<\/a><\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/pepotash\/\" target=\"_blank\" rel=\"noreferrer noopener\">Peter Potash<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/vchaudhary\/\">Vishrav Chaudhary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/xihlin\/\" target=\"_blank\" rel=\"noreferrer noopener\">Eric Lin<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/kpprasa\/\" target=\"_blank\" rel=\"noopener noreferrer\">Kiran Prasad<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/tristan\/\" target=\"_blank\" rel=\"noreferrer noopener\">Tristan Naumann<\/a>&nbsp;<\/p>\n\n\n\n<p>Recent studies such as Microsoft\u2019s Turing Natural Language Representation (TNLR) models has been widely applied to numerous practical tasks, including summarization and question answering.&nbsp; We propose to transfer the TNLR models to the electronic health record (EHR) datasets and handle the disparity, bias, and the privacy during such process.&nbsp; We aim to learn the representation of EHR codes from the learning procedure and reveal novel clinical insights.<\/p>\n\n\n\n\n\n<p><strong>Harvard<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/eecs.harvard.edu\/shieber\/\" target=\"_blank\" rel=\"noopener noreferrer\">Stuart M. Shieber<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/yuntiandeng.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Yuntian Deng<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/simassakenis\/\" target=\"_blank\" rel=\"noopener noreferrer\">Simas \u0160akenis<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (student)&nbsp;<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/kpprasa\/\" target=\"_blank\" rel=\"noopener noreferrer\">Kiran Prasad<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/saxenakaran\/\" target=\"_blank\" rel=\"noopener noreferrer\">Karan Saxena<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/adadhika\/?originalSubdomain=ca\" target=\"_blank\" rel=\"noopener noreferrer\">Ashutosh Adhikari<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/vchaudhary\/\">Vishrav Chaudhary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/psmo\/\" target=\"_blank\" rel=\"noreferrer noopener\">Paul Smolensky<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/rfernand\/\" target=\"_blank\" rel=\"noreferrer noopener\">Roland Fernandez<\/a> &nbsp;<\/p>\n\n\n\n<p>In recent years, large pretrained language models have demonstrated their ability to generate fluent text.&nbsp; However, they do not enjoy as much success in tasks requiring reasoning.&nbsp; Motivated by the fact that reasoning is a crucial part of natural-language understanding and generation, our goal in this proposal is to improve the reasoning ability of large pretrained language models.&nbsp; To this end, we argue that the normal end-to-end training scheme that only uses the inputs and the desired reasoning outcomes as supervision is unlikely to provide enough learning signal to the model and propose to augment the training process with instructional scaffolding, which provides intermediate reasoning steps for some of the training examples.&nbsp; The proposed approach will be evaluated on tasks that require both language fluency and logical reasoning.<\/p>\n\n\n\n\n\n<p><strong>MIT<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/songhan.mit.edu\/\" target=\"_blank\" rel=\"noopener noreferrer\">Song Han<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/lzhu.me\/\" target=\"_blank\" rel=\"noopener noreferrer\">Ligeng Zhu<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linji.me\/\" target=\"_blank\" rel=\"noopener noreferrer\">Ji Lin<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PhD student), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/han-cai.github.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Han Cai<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/hanruiwang.me\/\" target=\"_blank\" rel=\"noopener noreferrer\">Hanrui Wang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/xihlin\/\" target=\"_blank\" rel=\"noreferrer noopener\">Eric Lin<\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/sarvjeet-singh-ghotra-1a145898\/\" target=\"_blank\" rel=\"noopener noreferrer\">Sarv Ghotra<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/sakshamsinghal\/\" target=\"_blank\" rel=\"noopener noreferrer\">Saksham Singhal<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/karthikm01\/\" target=\"_blank\" rel=\"noopener noreferrer\">Karthik Mohan<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/payal-bajaj-3166a241\/\" target=\"_blank\" rel=\"noopener noreferrer\">Payal Bajaj<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/saxenakaran\/\" target=\"_blank\" rel=\"noopener noreferrer\">Karan Saxena<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/vchaudhary\/\">Vishrav Chaudhary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/yucheng1\/\" target=\"_blank\" rel=\"noreferrer noopener\">Yu Cheng<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/submukhe\/\" target=\"_blank\" rel=\"noreferrer noopener\">Subho Mukherjee<\/a>&nbsp;<\/p>\n\n\n\n<p>Natural language processing (NLP) has made tremendous progress, thanks to advanced models such as the Generative Pre-training Transformer (GPT) and Turing-NLG (TNLG). Because of their massive size, these models are difficult to fine-tune, let alone deploy to real-world applications. Given the growing demand for small model size, fast response time, and low computational cost, we propose to study (1) efficient training of large NLP models from scratch with different levels of sparsity (2) efficient fine-tuning large NLP models with limited memory budgets (3) explore real-world application of TNLG based on proposed efficient techniques.<\/p>\n\n\n\n\n\n<p><strong>University of Michigan<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/web.eecs.umich.edu\/~chaijy\/\" target=\"_blank\" rel=\"noopener noreferrer\">Joyce Chai<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/web.eecs.umich.edu\/~mihalcea\/\" target=\"_blank\" rel=\"noopener noreferrer\">Rada Mihalcea<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI), <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/web.eecs.umich.edu\/~wangluxy\/\" target=\"_blank\" rel=\"noopener noreferrer\">Lu Wang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI)&nbsp;<\/p>\n\n\n\n<p>Students: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.laurabiester.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Laura Biester<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/shuyang-cao-b07429227\/\" target=\"_blank\" rel=\"noopener noreferrer\">Shuyang Cao<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/mukhal.github.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Muhammad Khalifa<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/andrew-lee-81819a103\/\" target=\"_blank\" rel=\"noopener noreferrer\">Andrew Lee<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/ziqiao-ma\/\" target=\"_blank\" rel=\"noopener noreferrer\">Ziqiao Ma<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/mindojune.github.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Do June Min<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/scholar.google.com\/citations?user=3ZxqnhwAAAAJ\" target=\"_blank\" rel=\"noopener noreferrer\">Joseph Peper<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Siqi Shen, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/sled.eecs.umich.edu\/author\/shane-storks\/\" target=\"_blank\" rel=\"noopener noreferrer\">Shane Storks<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/sled.eecs.umich.edu\/author\/keunwoo-peter-yu\/\" target=\"_blank\" rel=\"noopener noreferrer\">Peter Yu<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/scholar.google.com\/citations?user=-uGCT5QAAAAJ\" target=\"_blank\" rel=\"noopener noreferrer\">Frederick Xinliang Zhang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.si.umich.edu\/people\/yichi-zhang\" target=\"_blank\" rel=\"noopener noreferrer\">Yichi Zhang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/p>\n\n\n\n<p>Postdocs: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/jkk.name\/bio_and_cv\/\" target=\"_blank\" rel=\"noopener noreferrer\">Jonathan Kummerfeld<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/ianbstewart.github.io\/\" target=\"_blank\" rel=\"noopener noreferrer\">Ian Stewart<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/p>\n\n\n\n<p>Research scientists: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/eecs.engin.umich.edu\/people\/perez-rosas-veronica\/\" target=\"_blank\" rel=\"noopener noreferrer\">Veronica Perez-Rosas<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/saxenakaran\/\" target=\"_blank\" rel=\"noopener noreferrer\">Karan Saxena<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/kpprasa\/\" target=\"_blank\" rel=\"noopener noreferrer\">Kiran Prasad<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/kragga\/#:~:text=Kriti%20Aggarwal%20is%20an%20Applied%20Scientist%20in%20the,products%20%28Bing%2C%20M365%2C%20Outlook%2C%20Word%2C%20Teams%2C%20Powerpoint%20etc.%29.\" target=\"_blank\" rel=\"noreferrer noopener\">Kriti Aggarwal<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/vchaudhary\/\">Vishrav Chaudhary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/dcarig\/\" target=\"_blank\" rel=\"noreferrer noopener\">Dean Carignan<\/a>&nbsp;<\/p>\n\n\n\n<p>Large pre-trained Transformer models have achieved state-of-the-art performance in a wide range of natural language processing tasks.\u202f However, due to Transformer\u2019s entangled multi-head multi-layer architecture, it is hard to interpret why a prediction is made.\u202f While large models are used for tasks with great societal impacts, it is critical to understand the rationales behind model decisions along with their reasoning process, model limitations, and their flaws. In this project, we aim to improve the understanding and transparency of Turing Models through two major research tasks: T1) investigating whether they can reason logically and coherently similar to how humans do, and T2) examining and quantifying the prevalence of bias (especially the less-studied racial bias) based on a new benchmark dataset and a novel entity-focused approach.<\/p>\n\n\n\n\n\n<p><strong>Stanford University<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/cs.stanford.edu\/~pliang\/\" target=\"_blank\" rel=\"noopener noreferrer\">Percy Liang<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> (PI)&nbsp;<\/p>\n\n\n\n<p><strong>Microsoft<\/strong>: <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/christiancosgrove\/\" target=\"_blank\" rel=\"noopener noreferrer\">Christian Cosgrove<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.linkedin.com\/in\/payal-bajaj-3166a241\/\" target=\"_blank\" rel=\"noopener noreferrer\">Payal Bajaj<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/bapatra\/\" target=\"_blank\" rel=\"noreferrer noopener\">Barun Patra<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/vchaudhary\/\">Vishrav Chaudhary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/hassanam\/\" target=\"_blank\" rel=\"noreferrer noopener\">Ahmed H. Awadallah<\/a>&nbsp;<\/p>\n\n\n\n<p>As large language models become more ubiquitous, it is important to characterize their properties so that downstream users of these models can have a better sense of what their capabilities and risks are.\u202f We will develop metrics that capture properties such as coverage across domains and dialects, robustness to perturbations, propensity to memorize (with copyright and privacy implications), faithfulness of generated text,\u202frisks for disinformation, ability to augment humans in interactive tasks, and others.<\/p>\n\n\n\n\n","protected":false},"excerpt":{"rendered":"<p>Our efforts are aimed at advancing principles of learning and reasoning, exploring novel applications, and pursuing better understanding of challenges and opportunities with regard to the ethical and responsible use of large-scale neural language models.<\/p>\n","protected":false},"featured_media":723157,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":true,"_classifai_error":"","msr_group_start":"","footnotes":""},"research-area":[13545],"msr-group-type":[243721],"msr-locale":[268875],"msr-impact-theme":[],"class_list":["post-722851","msr-group","type-msr-group","status-publish","has-post-thumbnail","hentry","msr-research-area-human-language-technologies","msr-group-type-collaboration","msr-locale-en_us"],"msr_group_start":"","msr_detailed_description":"","msr_further_details":"","msr_hero_images":[],"msr_research_lab":[],"related-researchers":[{"type":"user_nicename","display_name":"Eric Horvitz","user_id":32033,"people_section":"Office of the Chief Science Officer","alias":"horvitz"},{"type":"user_nicename","display_name":"Dean Carignan","user_id":40087,"people_section":"Office of the Chief Science Officer","alias":"dcarig"},{"type":"user_nicename","display_name":"Forough Poursabzi","user_id":40264,"people_section":"Office of the Chief Science Officer","alias":"fpoursabzi"},{"type":"user_nicename","display_name":"Jianfeng Gao","user_id":32246,"people_section":"Microsoft Research","alias":"jfgao"},{"type":"user_nicename","display_name":"Ahmed Awadallah","user_id":31979,"people_section":"Microsoft Research","alias":"hassanam"},{"type":"user_nicename","display_name":"Emre Kiciman","user_id":31739,"people_section":"Microsoft Research","alias":"emrek"},{"type":"user_nicename","display_name":"Tristan Naumann","user_id":37929,"people_section":"Microsoft Research","alias":"tristan"},{"type":"user_nicename","display_name":"Mike Shepperd","user_id":32920,"people_section":"Microsoft Research","alias":"mikeshep"},{"type":"user_nicename","display_name":"Ali Alvi","user_id":38919,"people_section":"Microsoft Turing","alias":"alialvi"},{"type":"user_nicename","display_name":"Kate Cook","user_id":40093,"people_section":"Microsoft Turing","alias":"kacoo"},{"type":"user_nicename","display_name":"Barun Patra","user_id":39099,"people_section":"Microsoft Turing","alias":"bapatra"},{"type":"user_nicename","display_name":"Alina Stoica Beck","user_id":40072,"people_section":"Microsoft Turing","alias":"alstoic"}],"related-publications":[903717],"related-downloads":[],"related-videos":[],"related-projects":[691494],"related-events":[876363],"related-opportunities":[],"related-posts":[801178,914184,924495,1017150],"tab-content":[{"id":0,"name":"Collaboration projects","content":"The program will include collaborative projects with academia to stress test large, natural language models.\r\n<h3>Round 1 Proposals: Natural Language Representation Model<\/h3>\r\n[accordion]\r\n[panel header=\"Leveraging large language models for transfer learning in medical notes\"]\r\n\r\n<strong>University of California, Berkeley<\/strong>: <a href=\"https:\/\/binyu.stat.berkeley.edu\/\" target=\"_blank\" rel=\"noopener\">Bin Yu<\/a> (PI), <a href=\"https:\/\/csinva.io\/\" target=\"_blank\" rel=\"noopener\">Chandan Singh<\/a> (PhD student), <a href=\"https:\/\/haywse.github.io\/\" target=\"_blank\" rel=\"noopener\">Wooseok Ha<\/a> (Postdoc), <a href=\"https:\/\/www.stat.berkeley.edu\/~yugroup\/people\/Briton.html\" target=\"_blank\" rel=\"noopener\">Briton Park<\/a> (PhD student), <a href=\"https:\/\/www.stat.berkeley.edu\/~yugroup\/people\/Robbie.html\" target=\"_blank\" rel=\"noopener\">Robert Netzorg<\/a> (PhD student)\r\n\r\nDepartments: Department of Statistics, Department of Electrical Engineering and Computer Sciences, and Center for Computational Biology, UC Berkeley Department of Urology\r\n\r\n<strong>University of California, San Francisco<\/strong>: <a href=\"https:\/\/urology.ucsf.edu\/people\/anobel-y-odisho\" target=\"_blank\" rel=\"noopener\">Anobel Odisho<\/a> (PI)\r\n\r\n<strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/pepotash\/\">Peter Potash<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/krisgan\/\">Kris Ganjam<\/a>, <a href=\"https:\/\/www.linkedin.com\/in\/alinastoicabeck\/\" target=\"_blank\" rel=\"noopener\">Alina Stoica Beck<\/a>, Dean Carignan, Forough Poursabzi Sangdeh\r\n\r\nRecent language models, such as Microsoft's Turing Natural Language Representation (TNLR) models, have shown an impressive ability to capture semantic information useful for transferring to new tasks. Much of human medical intelligence is contained in medical language notes such as pathology reports. Automated data extraction from such notes will be a key driver for the delivery of precision medicine (e.g. providing patient-level cancer grade information for treatment selection). It is also important for clinical research, risk stratification, and clinical trial enrollment. We propose to evaluate these language models in comparison to our current methods to improve performance in natural language processing tasks relating to automated data extraction.\r\n\r\n[\/panel]\r\n\r\n[panel header=\"The extent to which large language models exacerbate bias when given different types of biased and unbiased inputs\"]\r\n\r\n<strong>Brown University<\/strong>\/<a href=\"https:\/\/lunar.cs.brown.edu\/\" target=\"_blank\" rel=\"noopener\"><strong>LUNAR<\/strong><\/a>: <a href=\"https:\/\/cs.brown.edu\/people\/epavlick\/\" target=\"_blank\" rel=\"noopener\">Ellie Pavlick<\/a> (PI), <a href=\"http:\/\/www.cs.brown.edu\/~mlittman\" target=\"_blank\" rel=\"noopener\">Michael Littman<\/a> (faculty), <a href=\"http:\/\/cs.brown.edu\/people\/rpatel59\/\" target=\"_blank\" rel=\"noopener\">Roma Patel<\/a> (PhD student)\r\n\r\n<strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/pomoradi\/\">Pooya Moradi<\/a>, Dean Carignan, Forough Poursabzi Sangdeh\r\n\r\nRecent studies have shown that neural language models (LMs) trained on large text corpora can encode societal biases. Such models risk generating toxic text, whether in response to deliberate manipulation or seemingly innocuous natural language prompts. This fact is especially worrying in scenarios where these language models form the basis of NLP systems that are deployed in production. We propose to investigate the extent to which generative LMs are sensitive to subtle linguistic framing effects that form the basis of linguistic theories of bias in written text. In particular, we test whether prompts that contain linguistic markers of author bias (e.g., hedges, implicatives, subjective intensifiers, assertives) result in measurable differences in models' generated passages. We then propose several measures to identify linguistic markers in text that cause LMs to exacerbate this bias in their generations, as well methods that attempt to mitigate this.\r\n\r\n[\/panel]\r\n\r\n[panel header=\"Enhancing the robustness of massive language models via invariant risk minimization\"]\r\n\r\n<strong>EPFL<\/strong>: <a href=\"https:\/\/dlab.epfl.ch\/people\/west\/\" target=\"_blank\" rel=\"noopener\">Robert West<\/a> (PI), Maxime Peyrard (Postdoc), Martin Josifoski (PhD student)\r\n\r\n<strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/bapatra\/\">Barun Patra<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/viagar\/\">Vidhan Agarwal<\/a>, <a href=\"https:\/\/www.linkedin.com\/in\/sarvjeet-singh-ghotra-1a145898\/\" target=\"_blank\" rel=\"noopener\">Sarv Ghotra<\/a>, Dean Carignan, Forough Poursabzi Sangdeh\r\n\r\nWe propose to reduce the impact of spurious correlations within large language models by leveraging invariance learning principles. In particular, we will continue the training of existing language models according to the Invariant Risk Minimization (IRM) paradigm in order to enforce domain-invariant representations and achieve better out-of-domain generalization.\r\n\r\n[\/panel]\r\n\r\n[panel header=\"Analyzing and using large pretrained language models for societal good\"]\r\n\r\n<strong>Georgia Tech<\/strong>: <a href=\"http:\/\/www.munmund.net\/\" target=\"_blank\" rel=\"noopener\">Munmun De Choudhury<\/a> (PI), <a href=\"http:\/\/aritter.github.io\/\" target=\"_blank\" rel=\"noopener\">Alan Ritter<\/a> (faculty), <a href=\"https:\/\/www.cc.gatech.edu\/~dchau\/\" target=\"_blank\" rel=\"noopener\">Duen \"Polo\" Horng Chau<\/a> (faculty), <a href=\"https:\/\/www.cc.gatech.edu\/~dyang888\/\" target=\"_blank\" rel=\"noopener\">Diyi Yang<\/a> (faculty), Mai ElSherief (Postdoc), PhD students: Caleb Ziems, Ashutosh Baheti, Yang Chen, Jay Wang, <a href=\"https:\/\/www.austinpwright.com\" target=\"_blank\" rel=\"noopener\">Austin Wright<\/a>; Undergrads: Zhaoran Ma, Vincent Lieng, Omar Shaikh\r\n\r\n<strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurbah Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, Dean Carignan, Forough Poursabzi Sangdeh\r\n\r\nThis project proposes a generic set of research thrusts for understanding and using large pretrained language models, specifically leveraging Microsoft\u2019s Turing project language models to a) understand and quantify biases in diverse societal contexts, and b) propose methods to mitigate biases that get encoded in neural models built using naturalistically occurring behavioral trace data.\r\n\r\n[\/panel]\r\n\r\n[panel header=\"Analyzing toxicity, factuality and memory\"]\r\n\r\n<strong>University of Washington<\/strong>: <a href=\"https:\/\/homes.cs.washington.edu\/~yejin\/\" target=\"_blank\" rel=\"noopener\">Yejin Choi<\/a> (PI), <a href=\"https:\/\/homes.cs.washington.edu\/~nasmith\/\" target=\"_blank\" rel=\"noopener\">Noah Smith<\/a> (PI), <a href=\"https:\/\/homes.cs.washington.edu\/~msap\/\" target=\"_blank\" rel=\"noopener\">Maarten Sap<\/a> (PhD student), <a href=\"https:\/\/sites.google.com\/uw.edu\/annajafarpour\" target=\"_blank\" rel=\"noopener\">Anna Jafarpour<\/a> (Postdoctoral fellow) and <a href=\"https:\/\/homes.cs.washington.edu\/~eaclark7\/\" target=\"_blank\" rel=\"noopener\">Elizabeth Clark<\/a> (PhD student)\r\n\r\n<strong>Microsoft<\/strong>: <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/satiwary\/\">Saurabh Tiwary<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/pomoradi\/\">Pooya Moradi<\/a>, Zhun Liu, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/alialvi\/\">Ali Alvi<\/a>, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/kragga\/\">Kriti Aggarwal<\/a>, Dean Carignan, Forough Poursabzi Sangdeh\r\n\r\nRecent computing and deep learning advances have led to significant improvements in language modelling and text generation. However, these new language models (LM)are known to generate toxic and false information with human-like fluency, prohibiting their safe deployment. We propose two lines of research to address these issues, focusing on Microsoft\u2019s Turing Natural Language Representation (TNLR) models. First, we will investigate methods for reducing toxic generations and for improving readers\u2019 ability to distinguish between human- and machine-generated text. Second, we will look at how models distinguish between real and fake events and how to best introduce new knowledge through finetuning and conditioning.\r\n\r\n[\/panel]\r\n[\/accordion]"}],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group\/722851","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-group"}],"version-history":[{"count":41,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group\/722851\/revisions"}],"predecessor-version":[{"id":1012332,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group\/722851\/revisions\/1012332"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/723157"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=722851"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=722851"},{"taxonomy":"msr-group-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group-type?post=722851"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=722851"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=722851"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}