Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Yong Rui wins IEEE Computer Society 2016 Technical Achievement Award

February 22, 2016 | By Microsoft blog editor


By Allison Linn, Senior Writer at Microsoft

When Yong Rui first began doing computer vision research in 1995, the Internet was still in its infancy and the idea of using technology to automatically search for images seemed almost absurd.

“That [notion] was really quite advanced at the time,” said Rui, assistant managing director of Microsoft Research Asia.

Rui, who was a doctorate student at the University of Illinois in Champaign-Urbana when he became one of the first researchers to work on the National Science Foundation’s digital libraries project, an ambitious and forward-thinking attempt to catalog information including visuals electronically. He considers himself lucky to have landed at a university that was chosen for the project—but, it turns out, the computer vision field also was lucky that he fell into this area of research.

On February 22 the Institute of Electrical and Electronics Engineers (IEEE), announced that Rui was among the recipients of the IEEE Computer Society 2016 Technical Achievement Award. The award recognizes his “pioneering contributions to multimedia analysis and retrieval.”

Hsiao-Wuen Hon, the managing director of Microsoft Research Asia, called the award “a true testament to Yong’s research and development achievements in the fields of multimedia analysis and retrieval over the years.”

“As a brilliant scientist, Yong’s research results in multimedia have inspired a lot of researchers in the academic community,” Hon added.

Rui’s contribution to the field goes back to those days as a Ph.D. student, when he was one of the first researchers to hit on the idea of using a concept called relevance feedback in image-based search. Relevance feedback is the idea of refining the results a person gets back from a search engine based on how that person responds to earlier search results. In essence, it’s a way of making search results better based on user behavior.

At the time, lots of people were thinking about using relevance feedback in text search, but he was among the first to introduce it in image search. His early research papers on the topic have since been cited thousands of times. The work also had a big impact on people outside the research community, because it made it easier for people to search for images and find them.

“That actually turned out to be a paradigm-shifting framework because once we had that [function], the retrieval precision and recall increased significantly,” he said.

In that initial research, Rui used computer vision to try to identify features—such as the shape or color of an object—that would help a search engine identify the image as a vase, a pot, or a bowl. In the years since, his research has expanded into the field of image understanding, which uses computer vision and other technology to both recognize and describe images.

One recent research project created a framework that can automatically describe what is going on in short video clips, using natural language. Rui also has contributed to a number of Microsoft products, including the digital assistant Cortana, the popular chatbot XiaoIce and the set of artificial intelligence-based tools for developers known as Project Oxford.

Rui also has been actively involved in the research community, serving as editor in chief of the research magazine IEEE MultiMedia, the founding chair of the China chapter of special interest group ACM SIGMultimedia, and an executive member of the China Computing Federation.

Rui, who has worked for Microsoft for nearly 17 years, said one of the advantages of working for the company has been the opportunity to do everything from conducting fundamental scientific research to shipping products to millions of people.

“That’s a huge, huge benefit of working for Microsoft,” he said.

Learn More

Up Next

microsoft ability team stands in front of sign at university of texas at austin

Artificial intelligence, Human language technologies, Search and information retrieval

Microsoft Ability Initiative: A collaborative quest to innovate in image captioning for people who are blind or with low vision

Microsoft is committed to pushing the boundaries of technology to improve and positively influence all parts of society. Recent advances in deep learning and related AI techniques have resulted in significant strides in automated image captioning. However, current image captioning systems are not well-aligned with the needs of a community that can benefit greatly from […]

Meredith Ringel Morris

Principal Researcher and Research Manager

Bei Liu and Jianlong Fu of Microsoft research

Computer vision, Graphics and multimedia

The poet in the machine: Auto-generation of poetry directly from images through multi-adversarial training – and a little inspiration

As the means of expressing thoughts and feelings too sublime and elusive to be conveyed in everyday language, poetry across all cultures occupies a most sophisticated and mysterious of realms, just beyond the outskirts of creativity. Along the language-based avenues of expression available to people, poetry represents a departure from casual speech; few possess the […]

Microsoft blog editor

Artificial intelligence, Computer vision, Graphics and multimedia

ChatPainter: Improving text-to-image generation by using dialogue

Generating realistic images from a text description is a challenging task for a bot. A solution to this task has potential applications in the video game and image editing industries, among many others. Recently, researchers at Microsoft and elsewhere have been exploring ways to enable bots to draw realistic images in defined domains, such as […]

Microsoft blog editor