Automatic Construction and Ranking of Topical Keyphrases on Collections of Short Documents

  • Marina Danilevsky ,
  • Chi Wang ,
  • Nihit Desai ,
  • Xiang Ren ,
  • Jingyi Guo ,
  • Jiawei Han

Proceeding of 2014 SIAM International Conference on Data Mining |

Published by SIAM – Society for Industrial and Applied Mathematics

We introduce a framework for topical keyphrase generation and ranking, based on the output of a topic model run on a collection of short documents. By shifting from the unigram-centric traditional methods of keyphrase extraction and ranking
to a phrase-centric approach, we are able to directly compare and rank phrases of different lengths. Our method defines a function to rank topical keyphrases so that more highly ranked keyphrases are considered to be more representative phrases for that topic. We study the performance of our framework on multiple real world document collections, and also show that it is more scalable than comparable phrase-generating models.