Launched a year ago, Code Hunt (opens in new tab) is a coding game that challenges players first to deduce a hidden problem from clues presented as unit tests and then to write a program to solve it. The game has been enormously successful, attracting more than 150,000 players from around the world and achieving amazing stickiness—99 percent of players return to the game within a week. In the process, it has become much more than an intriguing game. It has inspired Code Hunt competitions around the world and has become a vehicle for identifying top coding talent. It has also established itself as an educational presence, both as a tool for teaching coding and as a resource for analyzing how students learn to code.
Code Hunt challenges players to display their coding skills.
As bridge fanatics and crossword puzzle fans know, intellectual games breed competition, and Code Hunt was no exception. Soon, we were organizing weekend Code Hunt contests for students. Under the banner of Imagine Cup (opens in new tab), we advertised the contests worldwide and offered them monthly from September 2014 through April 2015. Interest and involvement ran high.
Microsoft Research Podcast
AI Frontiers: Models and Systems with Ece Kamar
Ece Kamar explores short-term mitigation techniques to make these models viable components of the AI systems that give them purpose and shares the long-term research questions that will help maximize their value.
Adam Żychowski of the Warsaw University of Technology typifies many contestants. “I really love Code Hunt Challenge and every month I wait for it with excitement,” says Żychowski. “In Poland, the challenge starts at 1:00 A.M., so it is very busy night for me.” Microsoft Imagine program lead John Scott Tynes adds, “Code Hunt has been very good for Imagine Cup—it drove thousands of new students to the main competition.” Our next set of challenges has a regional and local flavor, with contests specifically for China and India, as well as universities—starting with the University of Washington.
Imagine Cup Code Hunt Contest winners and new Puzzle Masters Dawid Bugajewski (Military University of Technology, Warsaw) and Vincent Hsiao (University of Maryland) with Puzzle Master Nigel Horspool in background
Code Hunt has an integrated scoring system, which rewards better-written programs. Top scores attract attention, and one successful player, Alexey Kolesnichenko from ETH Zurich, has recently been hired by the Microsoft Trustworthy Computing group. Other tech companies have contests to find top coders, but Code Hunt is unique in that it mixes the game with real learning, teaching students how to manage perplexing coding problems. But how does one help students who are baffled by a problem? While others have conducted research into building hint systems, most of these systems provide specific, built-in hints—which may not address each of the thousands of players’ different needs.
By contrast, the Code Hunt hint system, built by Microsoft intern Daniel Perelman, takes advantage of the game’s foundation of symbolic execution and data in the cloud. Since we have access to close to a million attempts from our players, we use data mining and program synthesis to generate hints that are relevant to the student’s progress. We thus take advantage of all the solutions in the cloud to assist students working on any solution strategy, including ones the teacher might not have chosen as the reference—or may not even have been aware of. Of course, hints are turned off during contests!
Example of the hint system in action
Believe it or not, scale and success bring their own problems. It is a challenge for us to continually devise new problems. To help us, we invited the top coders themselves to start submitting puzzles, and we now have a cohort of eager young Puzzle Masters. When first approached to be a Puzzle Master, Josua Meier from Karlsruhe University of Technology responded, “Using a gaming approach to teach coding paradigms is a very amazing concept. Making a contribution to this, however small, would be great.”
Faculty and researchers design new puzzles and games for Code Hunt.
Pictured from left to right: Rishabh Singh (Microsoft Research), Armando Solar-Lezama (MIT), Amey Karkare (IIT Kanpur), Alexey Kolesnichenko (ETH Zurich), Alessandro Orso (Georgia Tech), Nikolai Tillmann (Microsoft), and Willem Visser (University of Stellenbosch)
Now, with all the data we have, we can analyze it in order to find out how students learn and improve. In early February at the Code Hunt Workshop (opens in new tab), we presented our first results showing that such simple concepts as division can be real obstacles that discourage students from continuing to code. Now, we have publicly released a data set of 13,000 programs on GitHub (opens in new tab) so that academics can continue to analyze the coding process.
So don’t wait for a contest to come to your school or region. Join the community at one of our upcoming workshops (opens in new tab), try Code Hunt (opens in new tab) on your own, or download the data set (opens in new tab) and get into some deep analysis of how students learn to code.
—Judith Bishop (opens in new tab), Director of Computer Science, Microsoft Research
Learn more