Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Yong Rui wins IEEE Computer Society 2016 Technical Achievement Award

February 22, 2016 | By Microsoft blog editor

Yong_Rui

By Allison Linn, Senior Writer at Microsoft

When Yong Rui first began doing computer vision research in 1995, the Internet was still in its infancy and the idea of using technology to automatically search for images seemed almost absurd.

“That [notion] was really quite advanced at the time,” said Rui, assistant managing director of Microsoft Research Asia.

Rui, who was a doctorate student at the University of Illinois in Champaign-Urbana when he became one of the first researchers to work on the National Science Foundation’s digital libraries project, an ambitious and forward-thinking attempt to catalog information including visuals electronically. He considers himself lucky to have landed at a university that was chosen for the project—but, it turns out, the computer vision field also was lucky that he fell into this area of research.

On February 22 the Institute of Electrical and Electronics Engineers (IEEE), announced that Rui was among the recipients of the IEEE Computer Society 2016 Technical Achievement Award. The award recognizes his “pioneering contributions to multimedia analysis and retrieval.”

Hsiao-Wuen Hon, the managing director of Microsoft Research Asia, called the award “a true testament to Yong’s research and development achievements in the fields of multimedia analysis and retrieval over the years.”

“As a brilliant scientist, Yong’s research results in multimedia have inspired a lot of researchers in the academic community,” Hon added.

Rui’s contribution to the field goes back to those days as a Ph.D. student, when he was one of the first researchers to hit on the idea of using a concept called relevance feedback in image-based search. Relevance feedback is the idea of refining the results a person gets back from a search engine based on how that person responds to earlier search results. In essence, it’s a way of making search results better based on user behavior.

At the time, lots of people were thinking about using relevance feedback in text search, but he was among the first to introduce it in image search. His early research papers on the topic have since been cited thousands of times. The work also had a big impact on people outside the research community, because it made it easier for people to search for images and find them.

“That actually turned out to be a paradigm-shifting framework because once we had that [function], the retrieval precision and recall increased significantly,” he said.

In that initial research, Rui used computer vision to try to identify features—such as the shape or color of an object—that would help a search engine identify the image as a vase, a pot, or a bowl. In the years since, his research has expanded into the field of image understanding, which uses computer vision and other technology to both recognize and describe images.

One recent research project created a framework that can automatically describe what is going on in short video clips, using natural language. Rui also has contributed to a number of Microsoft products, including the digital assistant Cortana, the popular chatbot XiaoIce and the set of artificial intelligence-based tools for developers known as Project Oxford.

Rui also has been actively involved in the research community, serving as editor in chief of the research magazine IEEE MultiMedia, the founding chair of the China chapter of special interest group ACM SIGMultimedia, and an executive member of the China Computing Federation.

Rui, who has worked for Microsoft for nearly 17 years, said one of the advantages of working for the company has been the opportunity to do everything from conducting fundamental scientific research to shipping products to millions of people.

“That’s a huge, huge benefit of working for Microsoft,” he said.

Learn More

Up Next

Bei Liu and Jianlong Fu of Microsoft research

Computer vision, Graphics and multimedia

The poet in the machine: Auto-generation of poetry directly from images through multi-adversarial training – and a little inspiration

As the means of expressing thoughts and feelings too sublime and elusive to be conveyed in everyday language, poetry across all cultures occupies a most sophisticated and mysterious of realms, just beyond the outskirts of creativity. Along the language-based avenues of expression available to people, poetry represents a departure from casual speech; few possess the […]

Microsoft blog editor

Artificial intelligence, Computer vision, Graphics and multimedia

Microsoft HoloLens facilitates computer vision research by providing access to raw image sensor streams with Research Mode

Microsoft HoloLens is the world’s first self-contained holographic computer. Remarkably, in Research Mode, available in the newest release of Windows 10 for HoloLens, it’s also a potent computer vision research device. Application code can not only access video and audio streams but can also at the same time leverage the results of built-in computer vision […]

Marc Pollefeys

Partner Director of Science

Artificial intelligence, Computer vision, Graphics and multimedia

ChatPainter: Improving text-to-image generation by using dialogue

Generating realistic images from a text description is a challenging task for a bot. A solution to this task has potential applications in the video game and image editing industries, among many others. Recently, researchers at Microsoft and elsewhere have been exploring ways to enable bots to draw realistic images in defined domains, such as […]

Microsoft blog editor