Decision-Theoretic Crowdsourcing


September 10, 2012


Crowdsourcing continues to rise in popularity today and is considered as the future of labor by many. Presently, crowdsourcing marketplaces (e.g., Amazon Mechanical Turk) have enabled the construction of scalable applications for tasks ranging from product categorization and photo tagging to audio transcription and language translation. These vertical applications are typically realized with complex, self-managing workflows that guarantee quality results. But constructing and controlling such workflows is challenging, with a huge number of alternative decisions for the designer to consider. We argue the thesis that artificial intelligence methods can greatly simplify the process of creating and managing complex crowdsourced workflows. We present the design of CLOWDER, which uses decision-theoretic techniques to dynamically optimize the workflows. Preliminary evaluations suggest that these optimized workflows are significantly more economical and return a much higher quality output than those generated by human designers.


Mausam Mausam

Mausam is a Ph.D. candidate at the University of Washington, Seattle advised by Dan Weld. His research interests include Automated Planning, Markov Decision Processes, Machine Learning, Heuristic Search. His thesis is on the extending the expressiveness of Markov Decision Processes to formulate complex planning problems. He received his Masters of Science from University of Washington in 2004 and Bachelors of Technology from Indian Institute of Technology, Delhi in 2001.