An Overview of Microsoft Deep QA System on Stanford WebQuestions Benchmark

Question answering (QA) over an existing knowledge base (KB) such as Microsoft Satori or open Freebase is one of the most important natural language processing applications. There are approaches based on web-search motivated statistic techniques as well as linguistically oriented knowledge engineering. Both methods face the key challenge on how to handle diverse ways of naturally expressing predicates and entities existing in the KB. The domain independent web information extracted from the massive amount of web usage data can be used with traditional semantic parsing through a unified framework. We provide such a unified framework utilizing both statistically motivated information-theoretic embeddings and logically driven proof-theoretic decoding to significantly improve Stanford’s WebQuestions QA benchmark. In comparison to Stanford’s state of the art ParaSempre 39.9 (F1-score), our Deep QA system achieves 45.3 on the same test data and protocol.