Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

The Quest for Quality Searches

July 19, 2010 | By Microsoft blog editor

By Janie Chang, Writer, Microsoft Research

When the Association for Computing Machinery’s (ACM’s) Special Interest Group on Information Retrieval (SIGIR) holds a conference, it must be difficult for participants to decide which sessions to attend, because creating easy, effective search experiences these days involves challenges potentially as diverse as dealing with multimedia, social media, relevance judgments, unstructured searches, and massive scalability. The 33rd annual ACM SIGIR Conference, being held at the University of Geneva from July 19-23, features a busy schedule of tutorials, workshops, and presentations of research papers that explore these topics.

The increasingly multidisciplinary nature of this subject is reflected in the eighty-seven papers accepted for this year’s conference. Fifteen submissions from Microsoft alone represent 10 groups from four research facilities—Microsoft Research Redmond, Microsoft Research Cambridge, Microsoft Research Asia, and Microsoft Research India—as well as the Internet Services Research Center and Bing.

Image Search by Concept Map—by Hao Xu of the University of Science and Technology of China and Jingdong Wang, Xian-Sheng Hua, and Shipeng Li of Microsoft Research Asia—is an example of how the use of multimedia has increased the complexity of information-retrieval problems. Digital images are, after text, the second-most prevalent media on the Web. The challenge for these researchers was to devise a more intuitive way for users to query for images.

Hua, lead researcher with the Media Computing Group, wants to overcome the limitations of existing image-search engines, which depend on the metadata of web images—and which rarely contain spatial information. Although the Image Search by Color Sketch feature in Bing addresses spatial relationships between colors in an image, his team wanted to convey semantic intention.

“This is a totally new way of searching web images,” Hua says, “when compared to text-box-based image searches. In this model, we allow users to specify the spatial positions of the query terms. The typed keywords indicate the desired visual concepts, or objects, within the image. The spatial relation of the keywords indicates the desired layout of the visual contents. We translate from a concept map to a visual instance map.”

sample query

This sample query illustrates how a user would search for “images containing a butterfly on the top left of a flower.”

With his interest in multimedia search, it`s not surprising that Hua is also part of a team researching music-information retrieval. Along with fellow researchers Jialie Shen and HweeHwa Pang of Singapore Management University, Meng Wang of Microsoft Research Asia, and Shuicheng Yan of the National University of Singapore, Hua has authored a paper entitled Effective Music Tagging Through Advanced Statistical Modeling. The work addresses the challenge of managing large music archives through knowledge representation of music documents.

Music search and recommendation requires compact but comprehensive textual annotation—tags—to describe a musical piece’s content and semantic notion. But as with all searches, retrieved results are only as good as the tags that describe the item’s content. Manual tagging of music files in a large collection is expensive and time-consuming, making automated music tagging an important research area. But high-level semantic concepts such as genre and mood are difficult to derive from the physical properties of music, so the researchers combined advanced musical feature-extraction techniques with high-level semantic-concept modeling.

Xian-Sheng Hua

Xian-Sheng Hua

“We propose a multilayer approach to modeling the content of music.” Hua says. “It bridges the gap between music content and semantic tags. Musical content is very rich, and we need to capture features such as timbral texture, harmony, rhythm structure, instrument, and pitch. Low-level acoustic characteristics are too simplistic for accurate representation.”

The results proved that their approach delivered substantial improvements in accuracy and robust annotation over existing methods. Even so, Hua concludes this is a challenging line of research that still has a long way to go.

“Several tags involve domain knowledge, such as the instrument and mood,” he explains. “Even when you employ professional musicians to label a data set manually, they may need to listen to the music multiple times. In some cases, the labelers may need to have a discussion before establishing the final tags of a piece.”

Search and retrieval by color, image content, spatial relationships, and musical mood and genre are only a few of the technical challenges that SIGIR 2010 will address, and these difficult topics are what drive some of the most novel work in computing research today.

A Track Record of Support

This prestigious conference showcases the most innovative thinking in information retrieval and draws significant support from Microsoft Research, a gold sponsor of the event. In addition to the 15 accepted papers, Gary Flake, technical fellow at Microsoft, will deliver the first keynote address of SIGIR 2010. Microsoft researchers also have committed time to SIGIR in other capacities, chairing three of the technical-paper sessions, giving four tutorials, and organizing three workshops.

Kuansan Wang, principal researcher at Microsoft Research Redmond is pleased with an opportunity to run the Web N-gram Workshop, organized with colleagues Chengxiang Zhai from the University of Illinois at Urbana-Champaign, David Yarowsky of Johns Hopkins University, Evelyne Viegas of Microsoft Research Redmond, and Stephan Vogel of Carnegie Mellon University.

Wang hopes the workshop will encourage researchers to use the Web N-gram Services hosted by Microsoft Research, which comprises algorithms, implementation, and petabytes of data regularly updated.

“The SIGIR Web N-gram Workshop,” he says, “will feature some of the research coming from using the Microsoft Web N-gram Services which went in public beta worldwide on April 28. The service-based distribution model enables us to update the data to keep up with the fast pace at which the web is changing. At the workshop we will announce several new features about the service.”

The effort that Microsoft Research applies to advancing the state of the art in information retrieval is reflected by the fact that four of the eight nominations this year for the SIGIR best paper award were written all or in part by Microsoft Research scientists.

Papers from Microsoft Research accepted for SIGIR 2010 (* best-paper nominee):

Adaptive Near-Duplicate Detection via Similarity Learning Hannaneh Hajishirzi, University of Illinois at Urbana-Champaign; Wen-tau Yih, Microsoft Research Redmond; and Aleksander Kolcz, Microsoft

Assessing the Scenic Route: Measuring the Value of Search Trails in Web Logs* Ryen White, Microsoft Research Redmond; and Jeff Huang, University of Washington

Collecting High Quality Overlapping Labels at Low Cost Hui Yang, Carnegie Mellon University; Anton Mityagin, Microsoft; Krysta Svore, Microsoft Research Redmond; and Sergey Markov, Microsoft

Comparing the Sensitivity of Information Retrieval Metrics* Filip Radlinski, Microsoft Research Cambridge; and Nick Craswell, Microsoft Research Redmond

Context-Aware Ranking in Web Search Biao Xiang, University of Science and Technology of China; Daxin Jiang, Microsoft Research Asia; Jian Pei, Simon Fraser University; Xiaohui Sun, Microsoft; Enhong Chen, (University of Science and Technology of China; and Hang Li, Microsoft Research Asia

Effective Music Tagging Through Advanced Statistical Modeling Jialie Shen, Singapore Management University; Meng Wang, Microsoft Research Asia; Shuicheng Yan, National University of Singapore; HweeHwa Pang, Singapore Management University; and Xian-Sheng Hua, Microsoft Research Asia

Extending Average Precision to Graded Relevance Judgments* Stephen Robertson, Microsoft Research Cambridge; Evangelos Kanoulas, University of Sheffield; and Emine Yilmaz, Microsoft Research Cambridge

How Good Is a Span of Terms?  Exploiting Proximity to Improve Web Retrieval Krysta Svore, Microsoft Research Redmond; Pallika Kanani, University of Massachusetts Amherst; and Nazan Khan, Microsoft

Image Search by Concept Map Hao Xu, University of Science and Technology of China; Jingdong Wang, Microsoft Research Asia; Xian-Sheng Hua, Microsoft Research Asia; and Shipeng Li, Microsoft Research Asia

Incorporating Post-Click Behaviors Into a Click Model Feimin Zhong, Tsinghua University; Dong Wang, Tsinghua University; Gang Wang, Microsoft Research Asia; Weizhu Chen, Microsoft Research Asia; Yuchen Zhang, Microsoft Research Asia; Zheng Chen, Microsoft Research Asia; and Haixun Wang, Microsoft Research Asia

Multi-Style Language Model for Web Scale Information Retrieval* Kuansan Wang, Microsoft Research Redmond; Jianfeng Gao, Microsoft Research Redmond; and Xiaolong Li, Microsoft Research Redmond

Studying Trailfinding Algorithms for Enhanced Web Search Adish Singla, Microsoft; Ryen White, Microsoft Research Redmond; and Jeff Huang, University of Washington

The Good, the Bad, and the Random: An Eye-Tracking Study of Ad Quality in Web Search Georg Buscher, Deutsches Forschungszentrum für Künstliche Intellitgenz; Susan Dumais, Microsoft Research Redmond; and Edward Cutrell, Microsoft Research India

Understanding Web Browsing Behaviors Through Weibull Analysis of Dwell Time Chao Liu, Microsoft Research Redmond; Ryen White, Microsoft Research Redmond; and Susan Dumais, Microsoft Research Redmond

Visual Summarization of Web Pages Binxing Jiao, University of Science and Technology of China; Linjun Yang, Microsoft Research Asia; Jizheng Xu, Microsoft Research Asia; and Feng Wu, Microsoft Research Asia

Up Next

Search and information retrieval

In between the lines: Broadening access to web search by understanding the needs of people with dyslexia

People with dyslexia perceive the world in different ways when it comes to the arrangement of letters and words on paper – or, these days, on web pages. Indeed, so much of modern life and accessibility to the information that enables us to participate in it depends on our ability to perceive and process online […]

Microsoft blog editor

Artificial intelligence, Search and information retrieval

Adversarial and reinforcement learning-based approaches to information retrieval

Traditionally, machine learning based approaches to information retrieval have taken the form of supervised learning-to-rank models. Recent advances in other machine learning approaches—such as adversarial learning and reinforcement learning—should find interesting new applications in future retrieval systems. At Microsoft AI & Research, we have been exploring some of these methods in the context of web […]

Bhaskar Mitra

Principal Applied Scientist

Data platforms and analytics, Search and information retrieval

Microsoft researchers unlock the black box of network embedding

At the ACM Conference on Web Search and Data Mining 2018, my team will introduce research that, for the first time, provides a theoretical explanation of popular methods used to automatically map the structure and characteristics of networks, known as network embedding. We then use this theoretical explanation to present a new network embedding method […]

Kuansan Wang

Managing Director, MSR Outreach Academic Services