Greedy Transition-Based Dependency Parsing with Stack-LSTMs


October 14, 2015


Miguel Ballesteros


Pompeu Fabra University, in Barcelona; visiting lecturer at Carnegie Mellon University


We propose a technique for learning representations of parser states in transition-based dependency parsers. Our primary innovation is a new control structure for sequence-to-sequence neural networks—the stack LSTM. Like the conventional stack data structures used in transition-based parsing, elements can be pushed to or popped from the top of the stack in constant time, but, in addition, an LSTM maintains a continuous space embedding of the stack contents. This lets us formulate an efficient parsing model that captures three facets of a parser’s state:

  1. Unbounded look-ahead into the buffer of incoming words, (ii) the complete history of transition actions taken by the parser, and (iii) the complete contents of the stack of partially built tree fragments, including their internal structures.

In addition, we discuss different word representations, by modelling words and by modelling characters, the former is useful for all languages while the latter improves the way of handling out of vocabulary words without a pretraining regime and improves the parsing of morphologically rich languages.


Miguel Ballesteros

Miguel Ballesteros is a Postdoctoral Researcher in TALN research group (Leo Wanner’s group), at Pompeu Fabra University, in Barcelona.

Until April 2016, he is a visiting postdoc/lecturer in Carnegie Mellon University in Pittsburgh working in Chris Dyer’s lab and Noah’s Ark.

He works in the intersection of natural language processing and machine learning. He is especially interested in syntactic analysis, dependency parsing and phrase structure parsing. He publishes papers about computational linguistics. He teaches and advises students. He was born in Madrid. He has lived in Uppsala (visiting with Joakim Nivre), Singapore (visiting Yue Zhang), Madrid, Barcelona and Pittsburgh.