{"id":720673,"date":"2021-02-05T13:01:57","date_gmt":"2021-02-05T21:01:57","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=720673"},"modified":"2023-03-06T09:13:27","modified_gmt":"2023-03-06T17:13:27","slug":"research-collection-shall-we-play-a-game","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/research-collection-shall-we-play-a-game\/","title":{"rendered":"Research Collection \u2013 Shall we play a game?"},"content":{"rendered":"<aside id=accordion-c182c1d2-b051-4ea6-ab01-f77c582aa25d class=\"msr-table-of-contents-block accordion mb-5 pb-0\" data-bi-aN=\"table-of-contents\">\n\t<button class=\"btn btn-collapse bg-gray-100 mb-0 display-flex justify-content-between\" type=\"button\" data-mount=\"collapse\" data-target=\"#accordion-collapse-c182c1d2-b051-4ea6-ab01-f77c582aa25d\" aria-expanded=\"true\" aria-controls=\"accordion-collapse-c182c1d2-b051-4ea6-ab01-f77c582aa25d\">\n\t\t<span class=\"msr-table-of-contents-block__label subtitle\">In this article<\/span>\n\t\t<span class=\"msr-table-of-contents-block__current mr-4 text-gray-600 font-weight-normal\" aria-hidden=\"true\"><\/span>\n\t<\/button>\n\t<div id=\"accordion-collapse-c182c1d2-b051-4ea6-ab01-f77c582aa25d\" class=\"msr-table-of-contents-block__collapse-wrapper collapse show\" data-parent=\"#accordion-c182c1d2-b051-4ea6-ab01-f77c582aa25d\">\n\t\t<div class=\"accordion-body bg-gray-100 border-top pt-4\">\n\t\t\t<ol class=\"msr-table-of-contents-block__list\">\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#games-as-a-research-tool\" class=\"msr-table-of-contents-block__list-item-link\">Games as a research tool<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t<li class=\"msr-table-of-contents-block__list-item\">\n\t\t\t\t\t\t<a href=\"#research-to-improve-the-gaming-experience\" class=\"msr-table-of-contents-block__list-item-link\">Research to improve the gaming experience<\/a>\n\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t<\/ul>\n\t\t<\/div>\n\t<\/div>\n\t<span class=\"msr-table-of-contents-block__progress-bar\"><\/span>\n<\/aside>\n\n\n\n<blockquote class=\"wp-block-quote is-style-default is-layout-flow wp-block-quote-is-layout-flow\">\n<p><em>From a research point of view, games offer an amazing environment in which to develop new machine learning algorithms and techniques. And we hope, in due course, that those new algorithms will feed back not just into gaming, but into many other domains. Beyond the very technical machine learning techniques themselves, gaming is an environment in which we can explore the relationship between AI and people, and see how they can work in partnership. It\u2019s a very rich environment in which to drive new research ideas.<\/em><\/p>\n<cite>Dr. Chris Bishop, Technical Fellow and Lab Director, Microsoft Research Cambridge<\/cite><\/blockquote>\n\n\n\n<p>The history of computer gaming goes back nearly as far as the history of computer science itself. As early as the 1950s, researchers were using games to push the boundaries of computing, driving many advances in computer graphics and human-computer interaction. Games played a role in the earliest research into artificial intelligence, as researchers trained the ancestors of modern AI systems to learn and play games like chess and tic-tac-toe. From the 1960s campus phenomenon of <em>Spacewar!<\/em> to the insatiable demand for the latest generation of consoles, games endure as a popular and engaging pastime \u2013 and, for many future researchers and developers, as a gateway to a career in computer science.<\/p>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/the-human-side-of-ai-for-chess\/\" data-bi-cN=\"The human side of AI for chess\" data-external-link=\"false\" data-bi-aN=\"margin-callout\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>The human side of AI for chess<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<p>Today, researchers are working to advance the state of the art in gaming in many different ways \u2013 enabling more realistic and immersive sound and graphics, using AI to improve gameplay and make gaming more engaging for spectators, and making games more accessible, inclusive and safer to play. Games continue to play a role in advancing artificial intelligence, offering up new challenges for AI models to solve, and using game environments to train language models and autonomous systems. Chess also continues to be fertile ground for AI research; most recently, researchers from Microsoft and their collaborators explore how to better match AI to human skill levels and playing styles.<\/p>\n\n\n\n<p>Listen to <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/game-on-with-dr-chris-bishop-and-phil-spencer\/\">this podcast<\/a> with Dr. Chris Bishop and Microsoft Executive Vice President of gaming Phil Spencer about the intersection of research and gaming at Microsoft.<\/p>\n\n\n\n<p>In February 2021, Microsoft Research hosted the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/aiandgaming2020\/\">AI and Gaming Research Summit<\/a>, which brought together researchers and innovators from academia, creators from the gaming industry, Microsoft Research (MSR), and Gaming teams at Microsoft, to share ideas and discuss how AI and machine learning is transforming the landscape of gaming. All recorded talks are now available on the Videos tab at the <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/aiandgaming2021\/#!videos\">event site<\/a>. <\/p>\n\n\n\n<p>This collection highlights much of the research happening across Microsoft to advance the state of the art in gaming, and put games to work to enable new breakthroughs in artificial intelligence.<\/p>\n\n\n\n\t<div class=\"border-bottom border-top border-gray-300 mt-5 mb-5 msr-promo text-center text-md-left alignwide\" data-bi-aN=\"promo\" data-bi-id=\"999693\">\n\t\t\n\n\t\t<p class=\"msr-promo__label text-gray-800 text-center text-uppercase\">\n\t\t<span class=\"px-4 bg-white display-inline-block font-weight-semibold small\">Spotlight: Event Series<\/span>\n\t<\/p>\n\t\n\t<div class=\"row pt-3 pb-4 align-items-center\">\n\t\t\t\t\t\t<div class=\"msr-promo__media col-12 col-md-5\">\n\t\t\t\t<a class=\"bg-gray-300 display-block\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/microsoft-research-forum\/?OCID=msr_researchforum_MCR_Blog_Promo\" aria-label=\"Microsoft Research Forum\" data-bi-cN=\"Microsoft Research Forum\" target=\"_blank\">\n\t\t\t\t\t<img decoding=\"async\" class=\"w-100 display-block\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/05\/Research-Forum-hero_1400x788.jpg\" alt=\"Research Forum | abstract background with colorful hexagons\" \/>\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"msr-promo__content p-3 px-5 col-12 col-md\">\n\n\t\t\t\t\t\t\t\t\t<h2 class=\"h4\">Microsoft Research Forum<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<p id=\"microsoft-research-forum\" class=\"large\">Join us for a continuous exchange of ideas about research in the era of general AI. Watch the first four episodes on demand.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"wp-block-buttons justify-content-center justify-content-md-start\">\n\t\t\t\t\t<div class=\"wp-block-button\">\n\t\t\t\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/event\/microsoft-research-forum\/?OCID=msr_researchforum_MCR_Blog_Promo\" aria-describedby=\"microsoft-research-forum\" class=\"btn btn-brand glyph-append glyph-append-chevron-right\" data-bi-cN=\"Microsoft Research Forum\" target=\"_blank\">\n\t\t\t\t\t\t\tWatch on-demand\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div><!--\/.msr-promo__content-->\n\t<\/div><!--\/.msr-promo__inner-wrap-->\n\t<\/div><!--\/.msr-promo-->\n\t\n\n\n<h2 id=\"games-as-a-research-tool\">Games as a research tool<\/h2>\n\n\n\n<h3 id=\"airsim\">AirSim<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"480\" height=\"266\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/frame_54_delay-0.07s.png\" alt=\"simulated aerial drone navigating course\" class=\"wp-image-660678\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/frame_54_delay-0.07s.png 480w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/frame_54_delay-0.07s-300x166.png 300w\" sizes=\"auto, (max-width: 480px) 100vw, 480px\" \/><\/figure>\n\n\n\n<p>Microsoft AirSim (<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/aerial-informatics-robotics-platform\/\">Aerial Informatics and Robotics Simulation<\/a>) is an open-source robotics simulation platform. From ground vehicles, wheeled robotics, aerial drones, and even static IoT devices, AirSim can capture data for models without costly field operations. AirSim works as a plug-in to Epic Games\u2019 Unreal Engine 4 editor, providing control over building environments and simulating difficult-to-reproduce, real-world events to capture meaningful data for AI models.<\/p>\n\n\n\n<p>Using AirSim, researchers can capture data for machine learning models in a virtual environment, without costly field operations.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Tool<\/span>\n\t\t\t<a href=\"https:\/\/github.com\/microsoft\/AirSim-Drone-Racing-VAE-Imitation\" data-bi-cN=\"AirSim Drone Racing VAE Imitation\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>AirSim Drone Racing VAE Imitation<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/game-of-drones-at-neurips-2019-simulation-based-drone-racing-competition-built-on-airsim\/\" data-bi-cN=\"Game of Drones at NeurIPS 2019: Simulation-based drone-racing competition built on AirSim\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Game of Drones at NeurIPS 2019: Simulation-based drone-racing competition built on AirSim<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/autonomous-systems-aerial-robotics-and-game-of-drones-with-gurdeep-pall-and-dr-ashish-kapoor\/\" data-bi-cN=\"Autonomous systems, aerial robotics and Game of Drones with Gurdeep Pall and Dr. Ashish Kapoor\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Autonomous systems, aerial robotics and Game of Drones with Gurdeep Pall and Dr. Ashish Kapoor<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<p><\/p>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"hybrid-reward-architecture\">Hybrid Reward Architecture<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"582\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/06\/download-1024x582.gif\" alt=\"Hybrid Reward Architecture for Ms. Pac-Man\" class=\"wp-image-455751\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/06\/download-1024x582.gif 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/06\/download-300x171.gif 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/06\/download-768x437.gif 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Games are popular as a testbed for new machine learning techniques because they can be very challenging and allow for easy analysis of new learning techniques in a controlled environment. For reinforcement learning (RL), where the goal is to learn good behavior in a data-driven way, the Arcade Learning Environment (ALE), which provides access to a large number of Atari 2600 games, has been a popular testing environment. The&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/hybrid-reward-architecture\/\">Hybrid Reward Architecture<\/a>&nbsp;project combines standard reinforcement learning techniques with deep neural networks, with the aim of outperforming humans in Arcade Learning Environment (ALE) games. In 2017, it achieved a perfect score on Ms. Pac-Man.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/hybrid-reward-architecture-achieving-super-human-ms-pac-man-performance\/\" data-bi-cN=\"Hybrid Reward Architecture (HRA) Achieving super-human performance on Ms. Pac-Man\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Hybrid Reward Architecture (HRA) Achieving super-human performance on Ms. Pac-Man<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/hybrid-reward-architecture-reinforcement-learning\/\" data-bi-cN=\"Hybrid Reward Architecture for Reinforcement Learning\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Hybrid Reward Architecture for Reinforcement Learning<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/hybrid-reward-architecture-fall-ms-pac-man-dr-harm-van-seijen\/\" data-bi-cN=\"Hybrid Reward Architecture and the Fall of Ms. Pac-Man with Dr. Harm van Seijen\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Hybrid Reward Architecture and the Fall of Ms. Pac-Man with Dr. Harm van Seijen<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"project-malmo\">Project Malmo<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"247\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/malmo_human_ai_interaction-web.png\" alt=\"Project malmo\" class=\"wp-image-235757\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/malmo_human_ai_interaction-web.png 800w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/malmo_human_ai_interaction-web-300x93.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/06\/malmo_human_ai_interaction-web-768x237.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/figure>\n\n\n\n<p>The Malmo platform is a sophisticated AI experimentation platform built on top of Minecraft, and designed to support fundamental research in artificial intelligence.<\/p>\n\n\n\n<p>The <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-malmo\/\">Project Malmo<\/a> platform consists of a mod for the Java version, and code that helps artificial intelligence agents sense and act within the Minecraft environment. The two components can run on Windows, Linux, or Mac OS, and researchers can program their agents in any programming language they\u2019re comfortable with.<\/p>\n\n\n\n<p>Minecraft is ideal for artificial intelligence research for the same reason it is appealing to the millions of fans who enter its virtual world every day. Unlike other computer games, Minecraft offers its users endless possibilities, ranging from simple tasks, like walking around looking for treasure, to complex ones, like building a structure with a group of teammates.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/project-malmo-reinforcement-learning-in-a-complex-world\/\" data-bi-cN=\"Project Malmo: Reinforcement learning in a complex world\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Project Malmo: Reinforcement learning in a complex world<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/minerl-sample-efficient-reinforcement-learning-challenge-back-for-a-second-year-benefits-organizers-as-well-as-larger-research-community\/\" data-bi-cN=\"MineRL sample-efficient reinforcement learning challenge\u2014back for a second year\u2014benefits organizers, as well as larger research community\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>MineRL sample-efficient reinforcement learning challenge\u2014back for a second year\u2014benefits organizers, as well as larger research community<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/malmo-minecraft-and-machine-learning-with-dr-katja-hofmann\/\" data-bi-cN=\"Malmo, Minecraft and machine learning with Dr. Katja Hofmann\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Malmo, Minecraft and machine learning with Dr. Katja Hofmann<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/malmo-platform-artificial-intelligence-experimentation\/\" data-bi-cN=\"The Malmo Platform for Artificial Intelligence Experimentation\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>The Malmo Platform for Artificial Intelligence Experimentation<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/video\/project-malmo-enabling-ai-technology-can-collaborate-humans\/\" data-bi-cN=\"Project Malmo: Enabling AI technology that can collaborate with humans\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Project Malmo: Enabling AI technology that can collaborate with humans<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"super-phoenix-suphx\">Super Phoenix (Suphx)<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1024x576.jpg\" alt=\"Suphx - Mahjong board\" class=\"wp-image-720721\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-16x9.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong.jpg 1400w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Mahjong is a popular multi-player imperfect-information game, but it is very challenging for AI research due to its complex playing\/scoring rules and rich hidden information. In 2019, Microsoft researchers designed <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/suphx-mastering-mahjong-with-deep-reinforcement-learning\/\">Super Phoenix (Suphx)<\/a>, an AI for Mahjong based on deep reinforcement learning with some newly introduced techniques including global reward prediction, oracle guiding, and run-time policy adaptation. Suphx has demonstrated stronger performance than most top human players in terms of stable rank and is rated above 99.99% of all the officially ranked human players in the Tenhou platform. This is the first time that a computer program outperformed most top human players in Mahjong.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/news.microsoft.com\/apac\/features\/mastering-mahjong-with-ai-and-machine-learning\/\" data-bi-cN=\"More than a game: Mastering Mahjong with AI and machine learning - Microsoft Stories Asia\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>More than a game: Mastering Mahjong with AI and machine learning &#8211; Microsoft Stories Asia<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/suphx-mastering-mahjong-with-deep-reinforcement-learning\/\" data-bi-cN=\"Suphx: Mastering Mahjong with Deep Reinforcement Learning\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Suphx: Mastering Mahjong with Deep Reinforcement Learning<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"textworld\">TextWorld<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"528\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2019\/08\/fig_1_textworld-1024x528.png\" alt=\"screen shot of the TextWorld interface\" class=\"wp-image-603234\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2019\/08\/fig_1_textworld-1024x528.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2019\/08\/fig_1_textworld-300x155.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2019\/08\/fig_1_textworld-768x396.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2019\/08\/fig_1_textworld.png 1376w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Text-based games may seem primitive next to the beautifully rendered graphics of today, but to succeed at even the simplest such game, humans use a special set of skills. We seamlessly and simultaneously comprehend language descriptions, plan our next moves, visualize the environment, remember important information, and generalize past experiences to new situations. AI agents don\u2019t yet possess these capabilities, but they are key to general intelligence. Microsoft&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/textworld\/\">TextWorld<\/a>&nbsp;is an open-source, extensible platform that both generates and simulates text-based games. It is a popular testing ground for AI, used by researchers to train, evaluate, analyze, and improve reinforcement learning (RL) agents that learn from and interact with the world through language.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Tool<\/span>\n\t\t\t<a href=\"https:\/\/github.com\/Microsoft\/TextWorld\" data-bi-cN=\"TextWorld\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>TextWorld<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/first-textworld-problems-the-competition-using-text-based-games-to-advance-capabilities-of-ai-agents\/\" data-bi-cN=\"First TextWorld Problems, the competition: Using text-based games to advance capabilities of AI agents\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>First TextWorld Problems, the competition: Using text-based games to advance capabilities of AI agents<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/textworld-a-learning-environment-for-training-reinforcement-learning-agents-inspired-by-text-based-games\/\" data-bi-cN=\"TextWorld: A learning environment for training reinforcement learning agents, inspired by text-based games\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>TextWorld: A learning environment for training reinforcement learning agents, inspired by text-based games<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/video\/getting-started-with-textworld\/\" data-bi-cN=\"Getting started with TextWorld\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Getting started with TextWorld<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h2 id=\"research-to-improve-the-gaming-experience\">Research to improve the gaming experience<\/h2>\n\n\n\n<h3 id=\"project-paidia\">Project Paidia<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"410\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560-1024x410.png\" alt=\"example game screens used in Project Paidia\" class=\"wp-image-678297\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560-1024x410.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560-300x120.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560-768x307.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/07\/ProjPaidia-game-screens_AI_07-2020_1400x560.png 1400w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-paidia\/\">Project Paidia<\/a> is a research project in close collaboration between Microsoft Research Cambridge and&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/ninjatheory.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Ninja Theory<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. Its focus is to drive state of the art research in reinforcement learning to enable novel applications in modern video games, in particular: agents that learn to collaborate with human players.<\/p>\n\n\n\n<p>In contrast to traditional approaches to crafting the behavior of bots, non-player characters, or other in-game characters, reinforcement learning does not require a game developer to anticipate a wide range of possible game situations and map out and code all required behaviors. Instead, with reinforcement learning, game developers control a reward signal which the game character then learns to optimize while responding fluidly to all aspects of a game\u2019s dynamics. The result is nuanced situation and player-aware emergent behavior that would be challenging or prohibitive to achieve using traditional Game AI.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/developer.microsoft.com\/en-us\/games\/blog\/supercharge-games-with-azure-ai-and-reinforcement-learning\/\" data-bi-cN=\"Game Stack - Supercharge games with Azure AI and reinforcement learning | Microsoft Developer\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Game Stack &#8211; Supercharge games with Azure AI and reinforcement learning | Microsoft Developer<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/three-new-reinforcement-learning-methods-aim-to-improve-ai-in-gaming-and-beyond\/\" data-bi-cN=\"Three new reinforcement learning methods aim to improve AI in gaming and beyond\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Three new reinforcement learning methods aim to improve AI in gaming and beyond<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/podcast\/inside-the-microsoft-ai-residency-program-with-dr-brian-broll\/\" data-bi-cN=\"Inside the Microsoft AI Residency Program with Dr. Brian Broll\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Inside the Microsoft AI Residency Program with Dr. Brian Broll<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"roomalive-toolkit\">RoomAlive Toolkit<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"933\" height=\"555\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Roomalive-Toolkit_example-scene.png\" alt=\"RoomAlive example scene\" class=\"wp-image-720949\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Roomalive-Toolkit_example-scene.png 933w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Roomalive-Toolkit_example-scene-300x178.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Roomalive-Toolkit_example-scene-768x457.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Roomalive-Toolkit_example-scene-16x10.png 16w\" sizes=\"auto, (max-width: 933px) 100vw, 933px\" \/><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/roomalive-toolkit\/\">RoomAlive Toolkit<\/a>&nbsp;is an open source SDK that transforms any room into an immersive, augmented, magical entertainment experience. RoomAlive Toolkit presents a unified, scalable approach for interactive projection mapping that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Tool<\/span>\n\t\t\t<a href=\"https:\/\/github.com\/Microsoft\/RoomAliveToolkit\" data-bi-cN=\"RoomAlive Toolkit\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>RoomAlive Toolkit<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/illumiroom-peripheral-projected-illusions-for-interactive-experiences\/\" data-bi-cN=\"IllumiRoom: Peripheral Projected Illusions for Interactive Experiences\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>IllumiRoom: Peripheral Projected Illusions for Interactive Experiences<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"seeingvr\">SeeingVR<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"349\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/12\/seeingvr-1024x349.png\" alt=\"SeeingVR's low vision tools\" class=\"wp-image-556662\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/12\/seeingvr-1024x349.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/12\/seeingvr-300x102.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2018\/12\/seeingvr-768x262.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/seeingvr\/\">SeeingVR<\/a>&nbsp;is a research toolkit for making virtual reality more accessible to people with low vision \u2013 moderate to severe visual impairments that cannot be corrected fully by glasses. It includes a number of enhancements that provide visual and audio augmentations to existing VR experiences \u2013 including magnification, edge enhancement, brightness and contrast adjustment, text augmentation and text-to-speech. The toolkit also enables VR developers to add capabilities such as object descriptions and object labeling and highlighting. SeeingVR is designed to be used with the Unity VR development platform.<\/p>\n\n\n\n<p>SeeingVR includes enhancements that help developers make VR more accessible to people with low vision.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Tool<\/span>\n\t\t\t<a href=\"http:\/\/aka.ms\/seeingvropensource\" data-bi-cN=\"SeeingVR Toolkit\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>SeeingVR Toolkit<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/inclusive-design-for-all-or-ict4d-and-4u-with-dr-ed-cutrell\/\" data-bi-cN=\"Inclusive design for all, or ICT4D and 4U! with Dr. Ed Cutrell\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Inclusive design for all, or ICT4D and 4U! with Dr. Ed Cutrell<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"http:\/\/aka.ms\/seeingvrpaper\" data-bi-cN=\"SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"http:\/\/aka.ms\/seeingvrvideo\" data-bi-cN=\"SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"project-triton\">Project Triton<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"683\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-1024x683.jpg\" alt=\"virtual world example in Project Acoustics\" class=\"wp-image-720952\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-1024x683.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-300x200.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-768x512.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels-16x12.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/ProjectAcoustics_virtual-world_gears-with-voxels.jpg 1098w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-triton\/\">Project Triton<\/a>&nbsp;renders immersive acoustics that responds to the shape and materials of 3D environments in games and augmented\/virtual reality (AR\/VR). It is unique in modeling the true wave physics of sound, including diffraction, producing natural effects like smooth occlusion and scene-dependent reverberation. Detailed simulation is performed offline, yielding a lightweight runtime that scales from desktop to mobile phones. Incubated over a decade in Microsoft Research, Project Triton ships in major game titles like Gears of War, Sea of Thieves, and Borderlands 3. Plugin integrations for Unity and Unreal Engine are available as part of Project Acoustics.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Tool<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/download\/details.aspx?id=57346\" data-bi-cN=\"Project Acoustics Unity plugin and samples\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Project Acoustics Unity plugin and samples<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Documentation<\/span>\n\t\t\t<a href=\"https:\/\/docs.microsoft.com\/en-us\/gaming\/acoustics\/what-is-acoustics\" data-bi-cN=\"Project Acoustics\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Project Acoustics<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Podcast<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/project-triton-and-the-physics-of-sound-with-dr-nikunj-raghuvanshi\/\" data-bi-cN=\"Project Triton and the physics of sound with Dr. Nikunj Raghuvanshi\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Project Triton and the physics of sound with Dr. Nikunj Raghuvanshi<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"truematch\">TrueMatch<\/h3>\n\n\n\n<p>Developed by Microsoft researchers and based on reinforcement learning, the TrueMatch matchmaking system uses AI to decide which people should play together in online multiplayer games.<\/p>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/truematch\/\">TrueMatch<\/a> dynamically adjusts players\u2019 matchmaking allowances based on their location and skill level. If there are many players nearby, then TrueMatch shrinks the allowances so that they only get fair matches with low ping. If there are few players nearby, TrueMatch expands the allowances. Previous systems expand the allowance only after a player has waited a long time in the queue. TrueMatch starts the search pre-expanded, cutting wait time in half in low population regions. TrueMatch can also give players a personalized estimate of their wait time, before they queue up to play.<\/p>\n\n\n\n<p>TrueMatch has launched in Gears of War 4, Gears of War 5, and Halo 5.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Project<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/truematch\/\" data-bi-cN=\"TrueMatch Matchmaking System\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>TrueMatch Matchmaking System<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/levelup-a-thin-cloud-approach-to-game-livestreaming\/\" data-bi-cN=\"LevelUp: A thin-cloud approach to game livestreaming\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>LevelUp: A thin-cloud approach to game livestreaming<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/switchboard-a-matchmaking-system-for-multiplayer-mobile-games\/\" data-bi-cN=\"Switchboard: A Matchmaking System for Multiplayer Mobile Games\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Switchboard: A Matchmaking System for Multiplayer Mobile Games<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Video<\/span>\n\t\t\t<a href=\"https:\/\/youtu.be\/Q8BX0nXfPjY\" data-bi-cN=\"Machine Learning for Optimal Matchmaking\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Machine Learning for Optimal Matchmaking<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"trueskill\">TrueSkill<\/h3>\n\n\n\n<p>The <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/trueskill-ranking-system\/\">TrueSkill ranking system<\/a> is a skill based ranking system for&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.xbox.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Xbox Live<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;developed at&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\">Microsoft Research<\/a>. The purpose of a ranking system is to both identify and track the skills of gamers in a game (mode) in order to be able to match them into competitive matches. TrueSkill has been used to rank and match players in many different games, from Halo 3 to Forza Motorsport 7.<\/p>\n\n\n\n<p>The classic TrueSkill ranking system only uses the final standings of all teams in a match in order to update the skill estimates (ranks) of all players in the match. The TrueSkill 2 ranking system also uses the individual scores of players in order to weight the contribution of each player to each team. As a result, TrueSkill 2 is much faster at figuring out the skill of a new player.<\/p>\n\n\n\n<p>An improved version of the TrueSkill ranking system, named&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/trueskill-2-improved-bayesian-skill-rating-system\/\" target=\"_blank\" rel=\"noreferrer noopener\">TrueSkill 2<\/a>, launched with&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/gearsofwar.com\/en-us\/community\/news\/web-seasons\" target=\"_blank\" rel=\"noopener noreferrer\">Gears of War 4<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;and was later incorporated into&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.halowaypoint.com\/en-us\/news\/may-playlist-updates\" target=\"_blank\" rel=\"noopener noreferrer\">Halo 5<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.<\/p>\n\n\n\n<h4 id=\"explore-more\" class=\"alignwide\">Explore more<\/h4>\n\n\n\n<div class=\"wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Project<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/trueskill-ranking-system\/\" data-bi-cN=\"TrueSkill\u2122 Ranking System\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>TrueSkill\u2122 Ranking System<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/score-based-bayesian-skill-learning-2\/\" data-bi-cN=\"Score-based Bayesian Skill Learning - Microsoft Research\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Score-based Bayesian Skill Learning &#8211; Microsoft Research<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/trueskilltm-a-bayesian-skill-rating-system-2\/\" data-bi-cN=\"TrueSkill: A Bayesian Skill Rating System\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>TrueSkill: A Bayesian Skill Rating System<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"citation\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 \">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/trueskill-2-improved-bayesian-skill-rating-system\/\" data-bi-cN=\"TrueSkill 2: An improved Bayesian skill rating system\" data-external-link=\"false\" data-bi-aN=\"citation\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>TrueSkill 2: An improved Bayesian skill rating system<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-text-color has-teal-color has-css-opacity has-teal-background-color has-background is-style-dots\"\/>\n\n\n\n<h3 id=\"watch-for\">Watch For<\/h3>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<article class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<div class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Blog<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/garage\/wall-of-fame\/watch-for\/\" data-bi-cN=\"Watch For - Microsoft Garage\" data-external-link=\"false\" data-bi-aN=\"margin-callout\" data-bi-type=\"annotated-link\" class=\"annotations__link font-weight-semibold text-decoration-none\"><span>Watch For &#8211; Microsoft Garage<\/span>&nbsp;<span class=\"glyph-in-link glyph-append glyph-append-chevron-right\" aria-hidden=\"true\"><\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n<p>Developed by Microsoft researchers as part of a 2017 internal company hackathon, Watch For identifies and surfaces the most interesting parts of a livestream in real time. It recognizes text, symbols, objects, players and other elements that signal exciting moments in a game, and uses AI to learn over time as more streams and content are fed into the model. After winning the hackathon, the researchers worked with product groups to integrate the technology into HypeZone, which was part of the company\u2019s Mixer game streaming service. The technology was subsequently incorporated into MSN\u2019s Esports hub, and was also used by the Cloud9 professional esports organization to derive insights to support match preparation. Based on customer feedback and the team\u2019s combined AI expertise and end-to-end distributed systems knowledge, the Watch For team is continuing their exploration in this area, and working on tools for enabling non-experts to efficiently extract video data.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>From a research point of view, games offer an amazing environment in which to develop new machine learning algorithms and techniques. And we hope, in due course, that those new algorithms will feed back not just into gaming, but into many other domains. Beyond the very technical machine learning techniques themselves, gaming is an environment [&hellip;]<\/p>\n","protected":false},"author":38004,"featured_media":720721,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[],"msr_hide_image_in_river":0,"footnotes":""},"categories":[244017],"tags":[],"research-area":[13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[244005,243996,243993],"msr-podcast-series":[],"class_list":["post-720673","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-collection","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-promo-type-newsletter","msr-promo-type-podcast","msr-promo-type-webinar"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144923,237595,283244,395930,583324,663258,863034],"related-projects":[635574,707674,669597,607743,589879,577638,546345,442191,604002,359810,235753,171462,169873],"related-events":[632643],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-960x540.jpg\" class=\"img-object-cover\" alt=\"Suphx - Mahjong board\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-16x9.jpg 16w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/01\/Collection_Gaming_Suphx-Mahjong.jpg 1400w\" sizes=\"auto, (max-width: 960px) 100vw, 960px\" \/>","byline":"","formattedDate":"February 5, 2021","formattedExcerpt":"From a research point of view, games offer an amazing environment in which to develop new machine learning algorithms and techniques. And we hope, in due course, that those new algorithms will feed back not just into gaming, but into many other domains. Beyond the&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/720673","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/38004"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=720673"}],"version-history":[{"count":34,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/720673\/revisions"}],"predecessor-version":[{"id":924948,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/720673\/revisions\/924948"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/720721"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=720673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=720673"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=720673"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=720673"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=720673"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=720673"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=720673"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=720673"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=720673"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=720673"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=720673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}