Diverse Retrieval via Greedy Optimization of Expected 1-call@k in a Latent Subtopic Relevance Model

  • Scott Sanner ,
  • Shengbo Guo ,
  • Thore Graepel ,
  • Sadegh Kharazmi ,
  • Sarvnaz Karimi

In Proceedings of CIKM, 20th ACM Conference on Information and Knowledge Management |

Published by ACM

It has been previously observed that optimization of the 1-call@k relevance objective (i.e., a set-based objective that is 1 if at least one document is relevant, otherwise 0) empirically correlates with diverse retrieval. In this paper, we proceed one step further and show theoretically that greedily optimizing expected 1-call@k w.r.t. a latent subtopic model of binary relevance leads to a diverse retrieval algorithm sharing many features of existing diversification approaches. This new result is complementary to a variety of diverse retrieval algorithms derived from alternate rank-based relevance criteria such as average precision and reciprocal rank. As such, the derivation presented here for expected 1-call@k provides a novel theoretical perspective on the emergence of diversity via a latent subtopic model of relevance — an idea underlying both ambiguous and faceted subtopic retrieval that have been used to motivate diverse retrieval.