Reinforcement Learning Day 2021

Reinforcement Learning Day 2021


Reinforcement learning is the study of decision making with consequences over time. The topic draws together multi-disciplinary efforts from computer science, cognitive science, mathematics, economics, control theory, and neuroscience. The common thread through all of these studies is: how do natural and artificial systems learn to make decisions in complex environments based on external, and possibly delayed, feedback.

This virtual workshop featured talks by a number of outstanding speakers whose research covers a broad swath of the topic, from statistics to neuroscience, from computer science to control. A key objective was to bring together the research communities of all these areas to learn from each other and build on the latest knowledge.

Committee Chairs

Akshay Krishnamurthy, Microsoft Research
Ching-An Cheng, Microsoft Research
Dipendra Misra, Microsoft Research
Ida Momennejad, Microsoft Research
Robert Loftin, Microsoft Research

Microsoft’s Event Code of Conduct

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. This includes virtual events Microsoft hosts and participates in, where we seek to create a respectful, friendly, and inclusive experience for all participants. As such, we do not tolerate harassing or disrespectful behavior, messages, images, or interactions by any event participant, in any form, at any aspect of the program including business and social activities, regardless of location.

We do not tolerate any behavior that is degrading to any gender, race, sexual orientation or disability, or any behavior that would violate Microsoft’s Anti-Harassment and Anti-Discrimination Policy, Equal Employment Opportunity Policy, or Standards of Business Conduct. In short, the entire experience must meet our culture standards. We encourage everyone to assist in creating a welcoming and safe environment. Please report any concerns, harassing behavior, or suspicious or disruptive activity. Microsoft reserves the right to ask attendees to leave at any time at its sole discretion.


This event has now concluded.

Thursday, January 14, 2021

Time (EST) Session Speaker
10:00 AM-10:15 AM Welcome Remarks Portrait of Akshay Kristhnamurthy Akshay Krishnamurthy, Microsoft Research
10:15 AM-11:00 AM New Advances in Hierarchical Reinforcement Learning Portrait of Doina Precup Doina Precup, McGill University
11:00 AM-11:45 AM Reinforcement Learning Debate: The State of RL and The Theory-Practice Divide Portrait of John Langford

Portrait of Yoshua Bengio

John Langford, Microsoft Research

Yoshua Bengio, Mila (Quebec AI Institute)

11:45 AM-12:15 PM Break
12:15 PM-1:45 PM Virtual Poster Presentations
Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective Yunzong Xu, MIT
Taylor Expansion Policy Optimization Yunhao Tang, Columbia University
Provably Efficient Policy Optimization with Thompson Sampling Haque Ishfaq, McGill University
Active Imitation Learning with Noisy Guidance Kianté Brantley, University of Maryland
Finite-Time Analysis of Decentralized Stochastic Approximation with Applications in Multi-Agent and Multi-Task Learning Sihan Zeng, Georgia Tech
META-Q-LEARNING Rasool Fakoor, Amazon Web Services
Toward the Fundamental Limits of Imitation Learning Nived Rajaraman, UC Berkeley
Multitask Bandit Learning through Heterogeneous Feedback Aggregation Zhi Wang, UC San Diego
“It’s Unwieldy and it Takes a Lot of Time.” Challenges and Opportunities for Creating Agents in Commercial Games Mikhail Jacob, Microsoft Research, Cambridge UK
A Framework for Robust Learning and Control of Nonlinear Systems with Large Uncertainty Hoang Le, Microsoft Research, Redmond
Learning Dynamic Belief Graphs to Generalize on Text-Based Games Eric Yuan, Microsoft Research, Montreal
Frugal Optimization for Cost-Related Hyperparameters Qingyun Wu, Microsoft Research, NYC
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels Denis Yarats, New York University
Self Supervised Policy Adaptation During Deployment Nicklas Hansen, Technical University of Denmark
Multi-Task Reinforcement Learning with Soft Modularization Ruihan Yang, UC San Diego
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning Rishabh Agarwal, Google Research, and Mila Research
A Regret Minimization Approach to Iterative Learning Control Karan Singh, Princeton University
RMP2: A Differentiable Policy Class for Robotic Systems with Control-Theoretic Guarantees Anqi Li, University of Washington
Generating Adversarial Disturbances for Controller Verification Udaya Ghai, Princeton University

Call for papers

This event has now concluded.

Call for virtual poster session

Reinforcement learning as a field that studies the problem of sequential decision making with unknown and potentially long-term consequences. Reinforcement learning is a multi-disciplinary topic, bringing together diverse fields of study including computer science, cognitive science, mathematics, psychology, economics, control theory, and neuroscience. The common theme that connects these fields, and the core goal of reinforcement learning is the question: How do natural and artificial systems learn to make decisions in complex, unknown environments based on limited, noisy, and possibly delayed feedback?

This virtual workshop aims to bring together researchers from industry and academia to share and discuss recent advances, challenges, and future research directions for reinforcement learning. Our goal is to highlight emerging research opportunities for the reinforcement learning community, particularly those driven by the evolving need for robust decision making in practical applications. Reinforcement Learning Day 2021 will provide an opportunity for different research communities to learn from each other and build on the latest knowledge in reinforcement learning and related disciplines.

Invited speakers

Reinforcement Learning Day 2021 will feature invited talks and conversations with leaders in the field, including Yoshua Bengio and John Langford, whose research covers a broad array of topics related to reinforcement learning. For more details please see the agenda page.

Virtual poster session

In addition to our speaker program, Reinforcement Learning Day 2021 will include a virtual poster session, showcasing recent and ongoing research in all areas of reinforcement learning.

We invite you to submit posters on all topics related to reinforcement learning. Suggested topics include (but are certainly not limited to):

  • Deep Reinforcement Learning
  • Reinforcement Learning Theory
  • Bandit Algorithms
  • Multi-Agent Reinforcement Learning
  • Reinforcement Learning Benchmarks and Datasets
  • Reinforcement Learning with Natural Language
  • Human-in-the-Loop Reinforcement Learning
  • Imitation Learning
  • Control Theory
  • Cross-Disciplinary Research with Reinforcement Learning: Structured Prediction, Game Theory, Operation Research, Fairness, Active Learning, Causality, Privacy, etc.
  • Applications of Reinforcement Learning: Recommender Systems, Robotics, Healthcare, Education, Conversational AI, Gaming, Finance, Neuroscience, Manufacturing etc.

What to submit

We invite the submission of extended abstracts (1-4 pages) on topics related to reinforcement learning. Authors of accepted abstracts will be invited to present their work at our virtual poster session (via Microsoft Teams), giving authors the opportunity for in-depth discussions with other Reinforcement Learning Day 2021 participants, presenters, and Microsoft researchers. Abstract reviewing will be single-blind. From the applications, we will be accepting 10-15 presenters only. Accepted presenters will be asked to prepare pre-recorded video presentations to complement the live discussion during the virtual poster session.

Please submit your abstract to

Important dates

  • December 11, 2020: Abstract submission deadline
  • December 22, 2020: Author notification
  • January 14, 2021: Reinforcement Learning Day 2021 – virtual workshop!


Career opportunities

Microsoft Research blog