Reinforcement Learning Day 2021

Reinforcement learning is the study of decision making with consequences over time. The topic draws together multi-disciplinary efforts from computer science, cognitive science, mathematics, economics, control theory, and neuroscience. The common thread through all of these studies is: how do natural and artificial systems learn to make decisions in complex environments based on external, and possibly delayed, feedback.

This virtual workshop featured talks by a number of outstanding speakers whose research covers a broad swath of the topic, from statistics to neuroscience, from computer science to control. A key objective was to bring together the research communities of all these areas to learn from each other and build on the latest knowledge.

Committee Chairs

Akshay Krishnamurthy, Microsoft Research
Ching-An Cheng, Microsoft Research
Dipendra Misra, Microsoft Research
Ida Momennejad, Microsoft Research
Robert Loftin, Microsoft Research

Microsoft’s Event Code of Conduct

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. This includes virtual events Microsoft hosts and participates in, where we seek to create a respectful, friendly, and inclusive experience for all participants. As such, we do not tolerate harassing or disrespectful behavior, messages, images, or interactions by any event participant, in any form, at any aspect of the program including business and social activities, regardless of location.

We do not tolerate any behavior that is degrading to any gender, race, sexual orientation or disability, or any behavior that would violate Microsoft’s Anti-Harassment and Anti-Discrimination Policy, Equal Employment Opportunity Policy, or Standards of Business Conduct. In short, the entire experience must meet our culture standards. We encourage everyone to assist in creating a welcoming and safe environment. Please report any concerns, harassing behavior, or suspicious or disruptive activity. Microsoft reserves the right to ask attendees to leave at any time at its sole discretion.

This event has now concluded.

Thursday, January 14, 2021

Time (EST)	Session		Speaker
10:00 AM-10:15 AM	Welcome Remarks		Akshay Krishnamurthy, Microsoft Research
10:15 AM-11:00 AM	New Advances in Hierarchical Reinforcement Learning		Doina Precup, McGill University
11:00 AM-11:45 AM	Reinforcement Learning Debate: The State of RL and The Theory-Practice Divide		John Langford, Microsoft Research Yoshua Bengio, Mila (Quebec AI Institute)
11:45 AM-12:15 PM	Break
12:15 PM-1:45 PM	Virtual Poster Presentations
	Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective		Yunzong Xu, MIT
	Taylor Expansion Policy Optimization		Yunhao Tang, Columbia University
	Provably Efficient Policy Optimization with Thompson Sampling		Haque Ishfaq, McGill University
	Active Imitation Learning with Noisy Guidance		Kianté Brantley, University of Maryland
	Finite-Time Analysis of Decentralized Stochastic Approximation with Applications in Multi-Agent and Multi-Task Learning		Sihan Zeng, Georgia Tech
	META-Q-LEARNING		Rasool Fakoor, Amazon Web Services
	Toward the Fundamental Limits of Imitation Learning		Nived Rajaraman, UC Berkeley
	Multitask Bandit Learning through Heterogeneous Feedback Aggregation		Zhi Wang, UC San Diego
	“It’s Unwieldy and it Takes a Lot of Time.” Challenges and Opportunities for Creating Agents in Commercial Games		Mikhail Jacob, Microsoft Research, Cambridge UK
	A Framework for Robust Learning and Control of Nonlinear Systems with Large Uncertainty		Hoang Le, Microsoft Research, Redmond
	Learning Dynamic Belief Graphs to Generalize on Text-Based Games		Eric Yuan, Microsoft Research, Montreal
	Frugal Optimization for Cost-Related Hyperparameters		Qingyun Wu, Microsoft Research, NYC
	Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels		Denis Yarats, New York University
	Self Supervised Policy Adaptation During Deployment		Nicklas Hansen, Technical University of Denmark
	Multi-Task Reinforcement Learning with Soft Modularization		Ruihan Yang, UC San Diego
	Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning		Rishabh Agarwal, Google Research, and Mila Research
	A Regret Minimization Approach to Iterative Learning Control		Karan Singh, Princeton University
	RMP2: A Differentiable Policy Class for Robotic Systems with Control-Theoretic Guarantees		Anqi Li, University of Washington
	Generating Adversarial Disturbances for Controller Verification		Udaya Ghai, Princeton University

This event has now concluded.

Call for virtual poster session

Reinforcement learning as a field that studies the problem of sequential decision making with unknown and potentially long-term consequences. Reinforcement learning is a multi-disciplinary topic, bringing together diverse fields of study including computer science, cognitive science, mathematics, psychology, economics, control theory, and neuroscience. The common theme that connects these fields, and the core goal of reinforcement learning is the question: How do natural and artificial systems learn to make decisions in complex, unknown environments based on limited, noisy, and possibly delayed feedback?

This virtual workshop aims to bring together researchers from industry and academia to share and discuss recent advances, challenges, and future research directions for reinforcement learning. Our goal is to highlight emerging research opportunities for the reinforcement learning community, particularly those driven by the evolving need for robust decision making in practical applications. Reinforcement Learning Day 2021 will provide an opportunity for different research communities to learn from each other and build on the latest knowledge in reinforcement learning and related disciplines.

Invited speakers

Reinforcement Learning Day 2021 will feature invited talks and conversations with leaders in the field, including Yoshua Bengio and John Langford, whose research covers a broad array of topics related to reinforcement learning. For more details please see the agenda page.

Virtual poster session

In addition to our speaker program, Reinforcement Learning Day 2021 will include a virtual poster session, showcasing recent and ongoing research in all areas of reinforcement learning.

We invite you to submit posters on all topics related to reinforcement learning. Suggested topics include (but are certainly not limited to):

• Deep Reinforcement Learning
• Reinforcement Learning Theory
• Bandit Algorithms
• Multi-Agent Reinforcement Learning
• Reinforcement Learning Benchmarks and Datasets
• Reinforcement Learning with Natural Language
• Human-in-the-Loop Reinforcement Learning
• Imitation Learning
• Control Theory
• Cross-Disciplinary Research with Reinforcement Learning: Structured Prediction, Game Theory, Operation Research, Fairness, Active Learning, Causality, Privacy, etc.
• Applications of Reinforcement Learning: Recommender Systems, Robotics, Healthcare, Education, Conversational AI, Gaming, Finance, Neuroscience, Manufacturing etc.

What to submit

We invite the submission of extended abstracts (1-4 pages) on topics related to reinforcement learning. Authors of accepted abstracts will be invited to present their work at our virtual poster session (via Microsoft Teams), giving authors the opportunity for in-depth discussions with other Reinforcement Learning Day 2021 participants, presenters, and Microsoft researchers. Abstract reviewing will be single-blind. From the applications, we will be accepting 10-15 presenters only. Accepted presenters will be asked to prepare pre-recorded video presentations to complement the live discussion during the virtual poster session.

Please submit your abstract to msrrlday@microsoft.com.

Important dates

• December 11, 2020: Abstract submission deadline
• December 22, 2020: Author notification
• January 14, 2021: Reinforcement Learning Day 2021 – virtual workshop!