Enhancing Text Representation through Knowledge-Based Feature Generation

June 15, 2006
Evgeniy Gabrilovich | Israel Institute of Technology

Imagine an automatic news filtering system that tracks company news. Given the news item “FDA approves ciprofloxacin for victims of anthrax inhalation”, how can the system know that the drug mentioned is an antibiotic produced by Bayer? Or consider an information professional searching for data on RFID technology – how can a computer understand that the item “Wal-Mart supply chain goes real time” is relevant for the search? Algorithms we present can do just that.

When humans approach text processing tasks, such as text categorization, they interpret documents in the context of their background knowledge and experience. On the other hand, conventional information retrieval systems represent documents as bags of words, and are restricted to learning from individual word occurrences in the (necessarily limited) training set. We propose to enrich document representation through automatic use of vast repositories of human knowledge. To this end, we use knowledge concepts derived from the Open Directory Project and Wikipedia, the largest Web directory and encyclopedia, respectively. In the preprocessing phase, a feature generator analyzes the input documents and maps them onto relevant concepts. The latter give rise to a set of generated features that augment the standard bag of words. Feature generation is accomplished through contextual analysis of document text, thus implicitly performing word sense disambiguation. Coupled with the ability to generalize from words to concepts, this approach addresses the two main problems of natural language processing synonymy and polysemy.

Categorizing documents with the aid of knowledge-based features leverages information that cannot be deduced from the training documents alone. Empirical results confirm that this knowledge-intensive representation brings text categorization to a qualitatively new level of performance across a diverse collection of datasets. We also propose a new, knowledge-based approach for computing the degree of semantic relatedness of texts.

Speaker Details

Evgeniy Gabrilovich is currently a Ph.D. student at the Computer Science Department at the Technion – Israel Institute of Technology. Upon completing an M.Sc. degree in computational linguistics, he worked in the industry for five years as senior software engineer and team leader. Then, he returned to the Technion to pursue a Ph.D. degree in the fields of information retrieval and machine learning. His research focuses on using large-scale knowledge repositories, such as Wikipedia and the Open Directory Project, for improving text representation in information retrieval. The methodology he developed enriches the conventional bag-of-words text representation with concepts based on world knowledge, and addresses the two fundamental problems of natural language processing – polysemy and synonymy. Evgeniy was on the program committee of the WWW’06 workshop on collaborative tagging and COLING-ACL’06 track on text categorization and sentiment analysis.