Katja Hofmann
Senior Principal Researcher
About
I am a Senior Principal Researcher within the Machine Intelligence theme at Microsoft Research Cambridge. I lead a team that focuses on Deep Reinforcement Learning for Games, with our mission to advance the state of the art in reinforcement learning, driven by current and future applications in video games. We share the belief that games will drive a transformation of how we interact with AI technology. My long-term goal is to develop AI systems that learn to collaborate with people, to empower their users and help solve complex real-world problems.
As part of the Microsoft Research PhD Scholarship program, I currently co-supervise the following PhD students:
- David Lindner (ETH Zurich, Switzerland and Microsoft Joint Research Center) – co-supervision with Andreas Krause
- Rémy Portelas (Inria, Bordeaux, France) – co-supervision with Pierre-Yves Oudeyer
- Steindor Saemundsson (Imperial College London, UK) – co-supervision with Marc Deisenroth
- Laetitia Teodorescu (Inria, Bordeaux, France) – co-supervision with Pierre-Yves Oudeyer
- Luisa Zintgraf (University of Oxford, UK) – co-supervision with Shimon Whiteson
One of the projects developed by my team is Project Malmo, which uses the popular game Minecraft as an experimentation platform for developing intelligent technology.
Before joining Microsoft Research, I completed my PhD in Computer Science as part of the ILPS group at the University of Amsterdam. I worked with Maarten de Rijke and Shimon Whiteson on smart search engines that learn directly from their users. For a list of my publications before joining MSR, please see the ILPS (Information and Language Processing Systems) list of publications, MSR Academic, or dblp.
Featured content
Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation
A key challenge on the path to developing agents that learn complex human-like behavior is the need to quickly and accurately quantify human-likeness. While human assessments of such behavior can be highly accurate, speed and scalability are limited. We address…
Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning
To rapidly learn a new task, it is often essential for agents to explore efficiently -- especially when performance matters from the first timestep. One way to learn such behaviour is via meta-learning. Many existing methods however rely on dense…
Strategically Efficient Exploration in Competitive Multiagent Reinforcement Learning
High sample complexity remains a barrier to the application of reinforcement learning (RL), particularly in multi-agent systems. A large body of work has demonstrated that exploration mechanisms based on the principle of optimism under uncertainty can significantly improve the sample…
Designer-centered reinforcement learning
In video games, nonplayer characters, bots, and other game agents help bring a digital world and its story to life. They can help make the mission of saving humanity feel urgent, transform every turn of a corner into a gamer’s…
Malmo, Minecraft and machine learning with Dr. Katja Hofmann
Episode 39, August 29, 2018 - Dr. Hofmann talks about her vision of a future where machines learn to collaborate with people and empower them to help solve complex, real-world problems. She also shares the story of how her early years in East Germany, behind the Iron Curtain, shaped her both personally and professionally, and ultimately facilitated a creative, exploratory mindset about computing that informs her work to this day.
Teacher Algorithms for Deep Reinforcement Learning Students | JRC Workshop 2021
Artificial Intelligence (AI) 20 May 2021 Speaker: Rémy Portelas, INRIA (collaboration with Pierre-Yves Oudeyer, INRIA and Katja Hofmann, Microsoft) This virtual event brought together the PhD students and postdocs working on collaborative research engagements with Microsoft via the Swiss Joint…
Optimistic Actor Critic avoids the pitfalls of greedy exploration in reinforcement learning
One of the core directions of Project Malmo is to develop AI capable of rich interactions. Whether that means learning new skills to apply to challenging problems, understanding complex environments, or knowing when to enlist the help of humans, reinforcement…
The road less traveled: With Successor Uncertainties, RL agents become better informed explorers
Imagine moving to a new city. You want to get from your new home to your new job. Unfamiliar with the area, you ask your co-workers for the best route, and as far as you can tell ... they’re right!…
Project Malmo competition returns with student organizers and a new mission: To democratize reinforcement learning
When I was asked about my favorite movie in a game with friends after my wedding ceremony, I replied Star Wars. That was about two decades ago, and, yes, it’s still the case. I especially like Return of the Jedi.…
The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
Learning in multi-agent scenarios is a fruitful research direction, but current approaches still show scalability problems in multiple games with general reward settings and different opponent types. The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) competition is a new challenge that…