The Intelligent Surfer: Probabilistic Combination of Link and Content Information in PageRank

Matthew Richardson, Matthew Richardson, Pedro Domingos

Advances in Neural Information Processing Systems 14 |

Published by MIT Press

The PageRank algorithm, used in the Google search engine, greatly
improves the results of Web search by taking into account the link
structure of the Web. PageRank assigns to a page a score proportional
to the number of times a random surfer would visit that page,
if it surfed indefinitely from page to page, following all outlinks
from a page with equal probability. We propose to improve Page-
Rank by using a more intelligent surfer, one that is guided by a
probabilistic model of the relevance of a page to a query. Efficient
execution of our algorithm at query time is made possible by precomputing
at crawl time (and thus once for all queries) the necessary
terms. Experiments on two large subsets of the Web indicate
that our algorithm significantly outperforms PageRank in the (human-
rated) quality of the pages returned, while remaining efficient
enough to be used in today’s large search engines.