Constraints Driven Learning for Natural Language Understanding

June 28, 2011
Dan Roth | University of Illinois at Urbana-Champaign

Intelligent Information Access and Extraction suggest significant challenges for Natural Language analysis. Tasks of interest include semantic role labeling (determining who did what to whom, when and where), information extraction (identifying entities, relations and events), following natural language instructions, and textual entailment (determining whether one utterance is a likely consequence of another). A computational approach to these challenges often involves assigning values to sets of interdependent variables and thus frequently necessitates performing global inference that accounts for these interdependencies.

This talk presents research on Constrained Conditional Models (CCMs), a framework that augments probabilistic models with declarative constraints as a way to support such decisions. We will present a framework we introduced a few years ago, formulating decision problems in NLP as Integer Linear Programming problems, but focus on new algorithms for training these global models using indirect supervision signals. Learning models for structured tasks is difficult partly since generating supervision signals is costly. We show that it is often easy to obtain a related indirect supervision signal, and discuss several options for deriving this supervision signal, including inducing it from the world’s response to the model’s actions. Our learning framework is “Constraints Driven” in the sense that it allows and even gains from global inference that guided by expressive declarative knowledge (encoded as constraints). Experimental results show the significant contribution of easy-to-get indirect supervision on NLP tasks such as information extraction, Transliteration and Textual Entailment.

Speaker Details

Dan Roth is a Professor in the Department of Computer Science and the Beckman Institute at the University of Illinois at Urbana-Champaign and a University of Illinois Scholar. He is the director of a DHS Center for Multimodal Information Access & Synthesis (MIAS) and has faculty positions also at Statistics, Linguistics and at the School of Library and Information Sciences.

Roth is a Fellow of AAAI for his contributions to the foundations of machine learning and inference and for developing learning centered solutions for natural language processing problems. He has published broadly in machine learning, natural language processing, knowledge representation and reasoning, and learning theory, and has developed advanced machine learning based tools for natural language applications that are being used widely by the research community. Prof. Roth has given keynote talks in major conferences, including AAAI, EMNLP and ECML and presented several tutorials in universities and conferences including at ACL and EACL. Roth was the program chair of AAAI’11, CoNLL’02 and of ACL’03, and is or has been on the editorial board of several journals in his research areas and has won several teaching and best paper awards. Prof. Roth got his B.A Summa cum laude in Mathematics from the Technion, Israel, and his Ph.D in Computer Science from Harvard University in 1995.