Deep Learning for Spoken and Text Dialog Systems

Conversational agents (goal based, chatbots, or information seeking bots) have been the most invested and sought technologies of the last decade. A huge part of this spike is due to the recent developments in the advanced machine learning methods being used in spoken dialog systems, the technology behind the conversational systems. Recent research investigates deep neural networks for dialog applications specifically focusing on understanding complex spoken or text utterances to implement human-like conversational bots. Especially the last decade has encountered a large variety of deep learning models powering spoken or text-based dialog systems. Specifically, the generative encoder-decoder (i.e., sequence-to-sequence) models have shown promising results for non-goal oriented dialog systems, such as word-level dialogue response generation. The hope is that such models will be able to leverage massive amounts of data to learn meaningful natural language representations and response generation strategies, while requiring a minimum amount of domain knowledge and hand-crafting.

Despite the huge success of deep learning in many other fields, some important challenges for deep learning researchers focusing on conversational dialog systems such as generating meaning and diverse responses and learning better representations for long and short term dialog context is still an open research area.

This chapter will provide an extensive summary of deep learning methods for dialog modeling tasks and different applications. Specifically, a deep excursion into cutting-edge research in deep learning applied to dialog modeling, including end-to-end learning and deep reinforcement learning, neural encoder-decoder networks as well as very novel models involving a memory component. We will also provide a list of open source software on some of the research necessary for making neural networks work on practical dialog modeling problems.