Counterfactual Evaluation and Learning from Logged User Feedback


September 26, 2016


Adith Swaminathan


Cornell University


Interactive systems like search engines, recommender systems, and ad placement platforms are ubiquitous. Evaluating and optimizing these systems is hard – users’ feedback governs system performance, and gathering their feedback in repeated randomized experiments is costly. I study how we can use logs collected from deployed systems to perform offline evaluation and learning. I will outline two projects (Evaluation: Recommendations as Treatments, ICML’16 and Learning: Counterfactual Risk Minimization, ICML’15) that advance the state of the art for these problems. I will also briefly reference the tutorial Thorsten Joachims and I taught at SIGIR’16 ( that explores these counterfactual evaluation and learning problems in more detail.