Project Malmo
How can we develop artificial intelligence that learns to make sense of complex environments? That learns from others, including humans, how to interact with the world? That learns transferable skills throughout its existence, and applies them to solve new, challenging problems?
Project Malmo sets out to address these core research challenges, addressing them by integrating (deep) reinforcement learning, cognitive science, and many ideas from artificial intelligence.

The Malmo platform is a sophisticated AI experimentation platform built on top of Minecraft, and designed to support fundamental research in artificial intelligence.
The Project Malmo platform consists of a mod for the Java version, and code that helps artificial intelligence agents sense and act within the Minecraft environment. The two components can run on Windows, Linux, or Mac OS, and researchers can program their agents in any programming language they’re comfortable with.
Minecraft is ideal for artificial intelligence research for the same reason it is addictively appealing to the millions of fans who enter its virtual world every day. Unlike other computer games, Minecraft offers its users endless possibilities, ranging from simple tasks, like walking around looking for treasure, to complex ones, like building a structure with a group of teammates.
See what’s possible with Project Malmo using Minecraft to build intelligent technology >
Read about our open source release >
Are you interested in accelerating the path of innovation in AI? Then make the Malmo platform your choice.
Join the community Follow @Project_Malmo
MineRL Competition 2019
Multi-Agent Reinforcement Learning in Malmo (MARLO) Competition 2018
Malmo Collaborative AI Challenge 2017
People
Research team
Oliver Kilian
Research Engineer
Malmo alumni
Publications
Publications
Publications by Year
-
20th International Conference on Autonomous Agents and Multiagent Systems - Extended Abstract | May 2021
Difference Rewards Policy Gradients
Jacopo Castellini, Sam Devlin, Frans A. Oliehoek, Rahul Savani20th International Conference on Autonomous Agents and Multiagent Systems - Extended Abstract | May 2021The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
William H. Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Ruslan Salakhutdinov, John Schulman, Shinya Shiroshita, Nicholay Topin, Avinash Ummadisingu, Oriol VinyalsJanuary 2021 -
Variational Integrator Networks for Physically Meaningful Embeddings
Steindór Sæmundsson, Alexander Terenin, Katja Hofmann, Marc Peter DeisenrothTwenty-Third International Conference on Artificial Intelligence and Statistics (AISTATS) | June 2020Combining No-regret and Q-learning
Ian A. Kash, Michael Sullins, Katja Hofmann2020 Adaptive Agents and Multi-Agents Systems | May 2020A Novel Individually Rational Objective In Multi-Agent Multi-Armed Bandit: Algorithms and Regret Bounds
Aristide Tossou, Christos Dimitrakakis, Jaroslaw Rzepecki, Katja HofmannInternational Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2020AMRL: Aggregated Memory For Reinforcement Learning
Jacob Beck, Kamil Ciosek, Sam Devlin, Sebastian Tschiatschek, Cheng Zhang, Katja HofmannEighth International Conference on Learning Representations (ICLR) | April 2020Conservative Uncertainty Estimation By Fitting Prior Networks
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, Richard TurnerEighth International Conference on Learning Representations (ICLR) | April 2020VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, Shimon WhitesonEighth International Conference on Learning Representations (ICLR) | April 2020 -
The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors
William H Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Sean Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, Manuela Veloso, Philip WangThirty-third Conference on Neural Information Processing Systems (NeurIPS) Competition track | December 2019Minecraft as AI Playground and Laboratory
Katja HofmannProceedings of the Annual Symposium on Computer-Human Interaction in Play (ChiPlay) | November 2019Opening keynote (extended abstract)The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noboru Sean Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel IonitaPublished by Arxiv -
How Players Speak to an Intelligent Game Character Using Natural Language Messages
Fraser Allison, Ewa Luger, Katja HofmannTransactions of the Digital Games Research Association | December 2018, Vol 4(2): pp. 1-49Combining No-regret and Q-learning (2018)
Ian Kash, Katja HofmannThe 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018Also presented at the AAAI-19 workshop on Reinforcement Learning in Games. -
A New AI Evaluation Cosmos: Ready to Play the Game?
José Hérnandez-Orallo, Marco Baroni, Jordi Bieger, Nader Chmait, David L. Dowe, Katja Hofmann, Fernando Martínez-Plumed, Claes Strannegård, Kristinn R. ThórissonAI Magazine | September 2017, Vol 38Asynchronous Data Aggregation for Training End to End Visual Control Networks
Mathew Monfort, Matthew Johnson, Aude Oliva, Katja HofmannAAMAS '17 Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems | May 2017Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft
Fraser Allison, Ewa Luger, Katja HofmannCHI EA '17 Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems | May 2017 -
Memory Lens – How Much Memory Does an Agent Use?
Christoph Dann, Katja Hofmann, Sebastian NowozinThe 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016Also presented at the Interpretable ML for Complex Systems Workshop at NeurIPS 2016Decoding multitask DQN in the world of Minecraft
Lydia T. Liu, Urun Dogan, Katja HofmannThe 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016Also presented at the 11th Women in Machine Learning Workshop and the Deep Reinforcement Learning Workshop at NeurIPS 2016.Experimental and Causal View on Information Integration in Autonomous Agents
Philipp Geiger, Katja Hofmann, Bernhard Schölkopf6th International Workshop on Combinations of Intelligent Methods and Applications | August 20166th International Workshop on Combinations of Intelligent Methods and ApplicationsThe Malmo Platform for Artificial Intelligence Experimentation
Matthew Johnson, Katja Hofmann, Tm Hutton, David Bignell, Katja Hofmann25th International Joint Conference on Artificial Intelligence (IJCAI-16) | July 2016
Publications by Research Area
-
Difference Rewards Policy Gradients
Jacopo Castellini, Sam Devlin, Frans A. Oliehoek, Rahul Savani20th International Conference on Autonomous Agents and Multiagent Systems - Extended Abstract | May 2021Deep Interactive Bayesian Reinforcement Learning via Meta-Learning
Luisa Zintgraf, Sam Devlin, Kamil Ciosek, Shimon Whiteson, Katja Hofmann20th International Conference on Autonomous Agents and Multiagent Systems - Extended Abstract | May 2021The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
William H. Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Ruslan Salakhutdinov, John Schulman, Shinya Shiroshita, Nicholay Topin, Avinash Ummadisingu, Oriol VinyalsJanuary 2021Variational Integrator Networks for Physically Meaningful Embeddings
Steindór Sæmundsson, Alexander Terenin, Katja Hofmann, Marc Peter DeisenrothTwenty-Third International Conference on Artificial Intelligence and Statistics (AISTATS) | June 2020Combining No-regret and Q-learning
Ian A. Kash, Michael Sullins, Katja Hofmann2020 Adaptive Agents and Multi-Agents Systems | May 2020A Novel Individually Rational Objective In Multi-Agent Multi-Armed Bandit: Algorithms and Regret Bounds
Aristide Tossou, Christos Dimitrakakis, Jaroslaw Rzepecki, Katja HofmannInternational Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2020VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, Shimon WhitesonEighth International Conference on Learning Representations (ICLR) | April 2020Conservative Uncertainty Estimation By Fitting Prior Networks
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, Richard TurnerEighth International Conference on Learning Representations (ICLR) | April 2020AMRL: Aggregated Memory For Reinforcement Learning
Jacob Beck, Kamil Ciosek, Sam Devlin, Sebastian Tschiatschek, Cheng Zhang, Katja HofmannEighth International Conference on Learning Representations (ICLR) | April 2020The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors
William H Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Sean Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, Manuela Veloso, Philip WangThirty-third Conference on Neural Information Processing Systems (NeurIPS) Competition track | December 2019Minecraft as AI Playground and Laboratory
Katja HofmannProceedings of the Annual Symposium on Computer-Human Interaction in Play (ChiPlay) | November 2019Opening keynote (extended abstract)The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noboru Sean Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel IonitaPublished by ArxivHow Players Speak to an Intelligent Game Character Using Natural Language Messages
Fraser Allison, Ewa Luger, Katja HofmannTransactions of the Digital Games Research Association | December 2018, Vol 4(2): pp. 1-49Combining No-regret and Q-learning (2018)
Ian Kash, Katja HofmannThe 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018Also presented at the AAAI-19 workshop on Reinforcement Learning in Games.A New AI Evaluation Cosmos: Ready to Play the Game?
José Hérnandez-Orallo, Marco Baroni, Jordi Bieger, Nader Chmait, David L. Dowe, Katja Hofmann, Fernando Martínez-Plumed, Claes Strannegård, Kristinn R. ThórissonAI Magazine | September 2017, Vol 38Asynchronous Data Aggregation for Training End to End Visual Control Networks
Mathew Monfort, Matthew Johnson, Aude Oliva, Katja HofmannAAMAS '17 Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems | May 2017Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft
Fraser Allison, Ewa Luger, Katja HofmannCHI EA '17 Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems | May 2017Memory Lens – How Much Memory Does an Agent Use?
Christoph Dann, Katja Hofmann, Sebastian NowozinThe 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016Also presented at the Interpretable ML for Complex Systems Workshop at NeurIPS 2016Decoding multitask DQN in the world of Minecraft
Lydia T. Liu, Urun Dogan, Katja HofmannThe 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016Also presented at the 11th Women in Machine Learning Workshop and the Deep Reinforcement Learning Workshop at NeurIPS 2016.Experimental and Causal View on Information Integration in Autonomous Agents
Philipp Geiger, Katja Hofmann, Bernhard Schölkopf6th International Workshop on Combinations of Intelligent Methods and Applications | August 20166th International Workshop on Combinations of Intelligent Methods and ApplicationsThe Malmo Platform for Artificial Intelligence Experimentation
Matthew Johnson, Katja Hofmann, Tm Hutton, David Bignell, Katja Hofmann25th International Joint Conference on Artificial Intelligence (IJCAI-16) | July 2016 -
Asynchronous Data Aggregation for Training End to End Visual Control Networks
Mathew Monfort, Matthew Johnson, Aude Oliva, Katja HofmannAAMAS '17 Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems | May 2017 -
How Players Speak to an Intelligent Game Character Using Natural Language Messages
Fraser Allison, Ewa Luger, Katja HofmannTransactions of the Digital Games Research Association | December 2018, Vol 4(2): pp. 1-49Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft
Fraser Allison, Ewa Luger, Katja HofmannCHI EA '17 Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems | May 2017
Publications by Type
-
Difference Rewards Policy Gradients
Jacopo Castellini, Sam Devlin, Frans A. Oliehoek, Rahul Savani20th International Conference on Autonomous Agents and Multiagent Systems - Extended Abstract | May 2021Deep Interactive Bayesian Reinforcement Learning via Meta-Learning
Luisa Zintgraf, Sam Devlin, Kamil Ciosek, Shimon Whiteson, Katja Hofmann20th International Conference on Autonomous Agents and Multiagent Systems - Extended Abstract | May 2021Variational Integrator Networks for Physically Meaningful Embeddings
Steindór Sæmundsson, Alexander Terenin, Katja Hofmann, Marc Peter DeisenrothTwenty-Third International Conference on Artificial Intelligence and Statistics (AISTATS) | June 2020Combining No-regret and Q-learning
Ian A. Kash, Michael Sullins, Katja Hofmann2020 Adaptive Agents and Multi-Agents Systems | May 2020A Novel Individually Rational Objective In Multi-Agent Multi-Armed Bandit: Algorithms and Regret Bounds
Aristide Tossou, Christos Dimitrakakis, Jaroslaw Rzepecki, Katja HofmannInternational Conference on Autonomous Agents and Multiagent Systems (AAMAS) | May 2020AMRL: Aggregated Memory For Reinforcement Learning
Jacob Beck, Kamil Ciosek, Sam Devlin, Sebastian Tschiatschek, Cheng Zhang, Katja HofmannEighth International Conference on Learning Representations (ICLR) | April 2020VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, Shimon WhitesonEighth International Conference on Learning Representations (ICLR) | April 2020Conservative Uncertainty Estimation By Fitting Prior Networks
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, Richard TurnerEighth International Conference on Learning Representations (ICLR) | April 2020The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors
William H Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Sean Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, Manuela Veloso, Philip WangThirty-third Conference on Neural Information Processing Systems (NeurIPS) Competition track | December 2019Minecraft as AI Playground and Laboratory
Katja HofmannProceedings of the Annual Symposium on Computer-Human Interaction in Play (ChiPlay) | November 2019Opening keynote (extended abstract)Combining No-regret and Q-learning (2018)
Ian Kash, Katja HofmannThe 14th European Workshop on Reinforcement Learning (EWRL 2018) | October 2018Also presented at the AAAI-19 workshop on Reinforcement Learning in Games.Asynchronous Data Aggregation for Training End to End Visual Control Networks
Mathew Monfort, Matthew Johnson, Aude Oliva, Katja HofmannAAMAS '17 Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems | May 2017Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft
Fraser Allison, Ewa Luger, Katja HofmannCHI EA '17 Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems | May 2017Memory Lens – How Much Memory Does an Agent Use?
Christoph Dann, Katja Hofmann, Sebastian NowozinThe 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016Also presented at the Interpretable ML for Complex Systems Workshop at NeurIPS 2016Decoding multitask DQN in the world of Minecraft
Lydia T. Liu, Urun Dogan, Katja HofmannThe 13th European Workshop on Reinforcement Learning (EWRL) 2016 | December 2016Also presented at the 11th Women in Machine Learning Workshop and the Deep Reinforcement Learning Workshop at NeurIPS 2016.Experimental and Causal View on Information Integration in Autonomous Agents
Philipp Geiger, Katja Hofmann, Bernhard Schölkopf6th International Workshop on Combinations of Intelligent Methods and Applications | August 20166th International Workshop on Combinations of Intelligent Methods and ApplicationsThe Malmo Platform for Artificial Intelligence Experimentation
Matthew Johnson, Katja Hofmann, Tm Hutton, David Bignell, Katja Hofmann25th International Joint Conference on Artificial Intelligence (IJCAI-16) | July 2016 -
How Players Speak to an Intelligent Game Character Using Natural Language Messages
Fraser Allison, Ewa Luger, Katja HofmannTransactions of the Digital Games Research Association | December 2018, Vol 4(2): pp. 1-49A New AI Evaluation Cosmos: Ready to Play the Game?
José Hérnandez-Orallo, Marco Baroni, Jordi Bieger, Nader Chmait, David L. Dowe, Katja Hofmann, Fernando Martínez-Plumed, Claes Strannegård, Kristinn R. ThórissonAI Magazine | September 2017, Vol 38 -
The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noboru Sean Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel IonitaPublished by Arxiv -
The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
William H. Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Ruslan Salakhutdinov, John Schulman, Shinya Shiroshita, Nicholay Topin, Avinash Ummadisingu, Oriol VinyalsJanuary 2021
Downloads
Videos
Groups
Events
News & features