Microsoft Research Blog

Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Presenting the winners of the Project Malmo Collaborative AI Challenge

June 5, 2017

By Noburu Kuno, Senior Research Program Manager; Scarlet Schwiderski-Grosche, Principal Research Program Manager

As we move from narrow AI to more general AI, it will be important to instill machines with the ability to work together with both other agents and humans. Project Malmo, which is built on the popular multiplayer game Minecraft, is an AI research tool for investigating how to train intelligent agents to collaborate. Our recent Project Malmo Collaborative AI Challenge asked teams to solve a game using collaborative agents, a project designed to push the state of the art of collaborative AI.

Spotlight: Microsoft research newsletter

Microsoft Research Newsletter

Stay connected to the research community at Microsoft.

Katja Hofmann, the lead and visionary for Project Malmo, summarizes the intent of this challenge: “In Minecraft, the possibilities for creation are endless. Project Malmo adds the ability to try different methods and approaches to teaching an agent to work within the Minecraft framework. The challenge then gave teams a specific task to demonstrate an agent’s ability to predict or learn whether to collaborate and how to collaborate successfully. From this challenge, we learned lots about various strategies for developing collaborative AI, such as planning-based approaches, deep neural network-based approaches, and co-evolution approaches.”

We were gratified that more than 80 teams comprising postgraduate students from 26 countries entered the challenge which required the teams to create and train agents to play a collaborative minigame — catch the pig — in which players work together to achieve a common goal. We selected this challenge because it mirrors the game theoretic “stag hunt,” a classic example that models the trade-offs between choosing to work together or go solo. We wanted to see how participants would approach this problem, what algorithms and strategies perform well today, and identify promising directions for future research.

Each team had to submit their code to GitHub, send in a write-up about their approach, and create a video showing their agent in action. Prizes for winning the challenge included a placement at the invitation-only Microsoft Research AI Summer School and/or Microsoft Azure for Research grants, worth up to $20,000 USD. Winners were selected based upon several criteria, including the ability of their agents to achieve high game scores consistently, and the novelty and creativity of the teams’ approaches.

Today we’re pleased to announce the winners with three teams winning prizes in both categories.

Microsoft Research AI Summer School placement prize winners

This prize provides winners with a slot at our Microsoft Research AI Summer School, where PhD students work alongside Microsoft Research scientists in Cambridge. Attendees learn general research skills, get to know the MSR Cambridge Lab and enjoy the opportunity to network with leading AI research leaders.

First place:

Second place:

Third place:

Runner up:

Microsoft Azure for Research grant prize winners

This prize provides additional computing resources to the student teams for their future research efforts. Microsoft Azure for Research supports hundreds of researchers worldwide when they need cloud-based memory, a cluster with thousands of cores, a big data platform, an Internet of Things solution or open-source machine learning at scale.

First place:

Second place (tie):

Third place (tie):

  • The Danish Puppeteers (Denmark)
  • Village People (Romania)

Said Bo An, assistant professor of Nanyang Technological University and supervisor of the HogRider team: “The Malmo testbed reflects all the important characteristics of collaborative AI:  incomplete information, partial/noisy observation, co-existence of cooperation and competition, sequential decision making, and more. To solve the challenge, we needed to coherently integrate techniques from different sub-fields together. We’re looking forward to applying our Azure for Research grant to future efforts.”

We thank the teams that entered, and congratulate the winners on their achievement.

What we’ve learned from this challenge

The Project Malmo challenge was designed to move the understanding of collaborative AI forward. Did it do so? The answer is an unqualified yes. Here’s some of what we learned:

  • There is no single approach to this problem that was clearly superior to the others. If you look at the leaderboard performance (game scores) alone, many different approaches achieved a very similar result.
  • Don’t devalue the work that’s gone before. Many teams achieved their success through a combination of both classic AI approaches and incorporating newer heuristics and models.
  • Sometimes the biggest learning is what question to ask next. One of the biggest problems in collaborative AI is the ability for an agent to, on-the-fly, adjust to new information in a dynamic environment. By introducing agents of unknown character and goals that the agent had to collaborate with, the teams were required to solve for this level of dynamism in real time. As a result, this challenge helped identify promising directions for additional research in helping AI agents adapt to unknown conditions.
  • Everything matters. Some teams spent more time on problem analysis, others on coding efficiency and still others on tuning their models. All three of these areas showed promise for future research efforts.

Related links:

Up Next

Artificial intelligence

Winners announced in multi-agent reinforcement learning challenge

In Learning to Play: The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition, we invited programmers into this digital world to help tackle multi-agent reinforcement learning. This challenge, the second competition using the Project Malmo platform, tasked participants with designing learning agents capable of collaborating with or competing against other agents to complete tasks across three different games within Minecraft.

Noboru Sean Kuno

Senior Research Program Manager

Artificial intelligence

Challenge accepted—MARLÖ competition among conference highlights

With the latest Project Malmo competition, we’re calling on researchers and engineers to test the limits of their thinking as it pertains to artificial intelligence, particularly multi-task, multi-agent reinforcement learning. Last week, a group of attendees at the 14th Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE’18) participated in a one-day workshop featuring the […]

Noboru Sean Kuno

Senior Research Program Manager

Artificial intelligence

Project Malmo: Reinforcement learning in a complex world

France’s victory over Croatia in the 2018 FIFA World Cup was as thrilling as sports competition gets. If you’re as much a fan of the game as I am, you enjoyed watching 32 national teams vie for the title over a beautiful month across 11 cities in Russia. The riveting action taking place on the […]

Noboru Sean Kuno

Senior Research Program Manager