Multimodal Learning from Bespoke Data

  • Kenneth D. Forbus | Northwestern University

Big Data and Deep Learning have rightly received a lot of attention, since their combination has led to breakthroughs in performance on some very hard problems. But there are many learning problems for which people don’t need even a hundred examples, much less a million – learning the rules of tic-tac-toe, for example, or teaching a new assistant how to fill out a form. Understanding how to learn rapidly from a small number of examples is crucial both for understanding human cognition and to create software assistants that can function as collaborators rather than tools. This talk will describe how we have been using our Companion cognitive architecture to learn from small amounts of interaction, using natural language and sketching, about playing games and identifying sketched concepts. We rely heavily on analogical processing and qualitative representations, which enable our systems to learn with only around 10 examples, which, for some problems, is several orders of magnitude fewer examples than needed by traditional machine learning techniques.

Speaker Details

Kenneth D. Forbus is the Walter P. Murphy Professor of Computer Science and Professor of Education at Northwestern University. He received his degrees from MIT (Ph.D. in 1984). His research interests include qualitative reasoning, analogical reasoning and learning, spatial reasoning, sketch understanding, natural language understanding, cognitive architecture, reasoning system design, intelligent educational software, and the use of AI in interactive entertainment. He is a Fellow of the Association for the Advancement of Artificial Intelligence, the Cognitive Science Society, and the Association for Computing Machinery. He has received the Humboldt Award and has served as Chair of the Cognitive Science Society

    • Portrait of Jeff Running

      Jeff Running

Series: Microsoft Research Talks