Diagnostic Questions: Predicting Student Responses and Measuring Question Quality


Digital technologies are becoming increasingly prevalent in education, enabling personalized, high quality education resources to be accessible by students across the world. Importantly, among these resources are diagnostic questions: the answers that the students give to these questions reveal key information about the specific nature of misconceptions that the students may hold.

Analyzing the massive quantities of data stemming from students’ interactions with these diagnostic questions can help us more accurately understand the students’ learning status and thus allow us to automate learning curriculum recommendations. In this competition, participants will focus on the students’ answer records to these multiple-choice diagnostic questions, with the aim of 1) accurately predicting which answers the students provide; 2) accurately predict which questions have high quality; and 3) determine a personalized sequence of questions for each student that best predicts the student’s answers. These tasks closely mimic the goals of a real-world educational platform and are highly representative of the educational challenges faced today. We provide data from the last school year (2018-2019) of students’ answers to mathematics questions from Eedi, a leading educational platform which millions of students interact with daily around the globe. Successful competition entrants have the potential to make a lasting, real-world impact on the quality of personalized education for millions of students across the world.


The competition is now open on CodaLab!


Simon Woodhead, Eedi
Craig Barton, Eedi
José Miguel Hernández-Lobato, University of Cambridge
Richard Turner, University of Cambridge
Jack Wang, Rice University
Richard G. Baraniuk, Rice University
Angus Lamb, Microsoft Research
Evgeny Saveliev, Microsoft Research
Pashmina Cameron, Microsoft Research
Yordan Zaykov, Microsoft Research
Simon Peyton-Jones, Microsoft Research
Cheng Zhang, Microsoft Research

Prizes and support

Competition Prizes

Microsoft and Eedi will provide more than $5,000 cash prize in total for the competition. There will be $1,000 prizes awarded to the winning team for each task. In addition, a $1,000 prize for the overall winner across all tasks will be awarded, as determined by the team’s average rank across each competition task (smallest average rank wins). If a team hasn’t submitted a working solution to a particular task, their rank for that task will be considered to be equal to the number of entrants across all tasks in total.

In the event of a tie, this prize will be split evenly between the tied teams.

Restriction of eligibility for the prize may apply. More information to come.

Azure Cloud Computating Credits

Microsoft made available 50 grants of $250 Azure cloud computing credits to students participating in the contest. Students are not required to use Azure to compete. Please fill in the application form here to apply. The credits will be allocated to the first 50 valid applications.