Deep Reinforcement Learning with a Natural Language Action Space

  • Ji He
  • Jianshu Chen
  • Xiaodong He
  • Lihong Li
  • Li Deng
  • Mari Ostendorf

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL) |

Published by ACL - Association for Computational Linguistics

This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games. Termed a deep reinforcement relevance network (DRRN), the architecture represents action and state spaces with separate embedding vectors, which are combined with an interaction function to approximate the Q-function in reinforcement learning. We evaluate the DRRN on two popular text games, showing superior performance over other deep Q-learning architectures. Experiments with paraphrased action descriptions show that the model is extracting meaning rather than simply memorizing strings of text.