The AI landscape has been transformed by the advent of large-scale models like BERT, Turing, and GPT. Researchers have brought language models to new heights in terms of performance, propelling advancements in search, translation, code generation and more. As models grow in size and capability, their applications are also expanding.
At Microsoft Research, we are approaching large-scale AI from different perspectives, which include not only creating bigger, more powerful models, but also developing new algorithms and approaches to AI development with the goal of delivering meaningful improvements to the way people interact with their worlds, while accounting for the effects they will have on people and society.
The workshop will facilitate knowledge sharing and in-depth discussions about ongoing and future work aiming to develop new technology to bring the benefits of large-scale AI to an expanding set of people in an expanding set of scenarios.
Speakers
Agenda
| Time (Eastern Time) | Session |
|---|---|
| 10:00 AM | Welcome Remarks |
| Session 1: Skills acquisition and new capabilities | video (opens in new tab) AI today only covers a small number of the skills compared to humans; to bring the benefits of AI to a broader set of scenarios, we need to develop AI that can learn to quickly accomplish new tasks and adapt to new and changing environments. | |
| 10:05 AM | Infrastructure and Progress Towards the First Community-Built and Continually-Improved Model | video (opens in new tab) Colin Raffel (opens in new tab), University of North Carolina at Chapel Hill |
| 10:35 AM | Open discussion |
| 11:00 AM | Short break |
| 11:05 AM | Combining modular skills in multitask learning | video (opens in new tab) Edoardo M. Ponti (opens in new tab), University of Edinburgh |
| 11:35 AM | Open discussion |
| 12:00 PM | Mitigating the Order Sensitivity of Pretrained Language Models | video (opens in new tab) Cristina Garbacea (opens in new tab), University of Michigan Ann Arbor |
| 12:15 PM | Live Q&A |
| 12:20 PM | Short break |
| Session 2: Training and inference efficiency | video (opens in new tab) To bring AI to more people, models need to be cheaper to train and run, in terms of both computational and human resources. Thus, we will focus on increasing efficiency across various parts of the training and inference pipeline. | |
| 12:50 PM | Efficient Vision Transformer | video (opens in new tab) Song Han (opens in new tab), Massachusetts Institute of Technology |
| 1:20 PM | Open discussion |
| 1:45 PM | Short break |
| 1:50 PM | Large Scale MoE Models into Cloud Scale Production with Highly Efficient Inference and Training | video (opens in new tab) Young Jin Kim, Microsoft Translator Hany Awadalla, Azure AI Cognitive Services |
| 2:20 PM | Open discussion |
| 2:45 PM | LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models | video (opens in new tab) Mojan Javaheripi (opens in new tab), Microsoft Research Redmond and University of California San Diego |
| 3:00 PM | Live Q&A |
| 3:05 PM | Short break |
| Session 3: Aligning models with human intent | video (opens in new tab) For the AI we develop to benefit people, we need to facilitate human intervention and feedback in the training and use of these models. | |
| 3:35 PM | Continual Language Learning in Collaborative Systems | video (opens in new tab) Yoav Artzi (opens in new tab), Cornell Tech at Cornell University |
| 4:05 PM | Open discussion |
| 4:30 PM | Short break |
| 4:35 PM | How New Datasets Can Allow AI to Make Progress: A few Observations | video (opens in new tab) Julia Hockenmaier (opens in new tab), University of Illinois at Urbana-Champaign |
| 5:05 PM | Open discussion |
| 5:30 PM | Interactive grounded language understanding in a collaborative environment (IGLU) | video (opens in new tab) Julia Kiseleva (opens in new tab), Microsoft Research Redmond |
| 5:45 PM | Live Q&A |
| 5:50 PM | Closing Remarks |
Workshop organizers
Alessandro Sordoni, Microsoft Research Montréal
Subho Mukherjee, Microsoft Research Redmond
Xiaodong Liu, Microsoft Research Redmond
Yu Cheng, Microsoft Research Redmond
Julia Kiseleva (opens in new tab), Microsoft Research Redmond
Nicolas Le Roux, Microsoft Research Montréal
Ahmed Awadallah, Microsoft Research Redmond
Microsoft’s Event Code of Conduct
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. This includes events Microsoft hosts and participates in, where we seek to create a respectful, friendly, and inclusive experience for all participants. As such, we do not tolerate harassing or disrespectful behavior, messages, images, or interactions by any event participant, in any form, at any aspect of the program including business and social activities, regardless of location.
We do not tolerate any behavior that is degrading to any gender, race, sexual orientation or disability, or any behavior that would violate Microsoft’s Anti-Harassment and Anti-Discrimination Policy, Equal Employment Opportunity Policy, or Standards of Business Conduct. In short, the entire experience at the venue must meet our culture standards. We encourage everyone to assist in creating a welcoming and safe environment. Please report (opens in new tab) any concerns, harassing behavior, or suspicious or disruptive activity to venue staff, the event host or owner, or event staff. Microsoft reserves the right to refuse admittance to or remove any person from company-sponsored events at any time in its sole discretion.









