Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Researchers team up with Chinese botanists on machine learning, flower-recognition project

July 25, 2016 | By Microsoft blog editor

By Guobin Wu, Senior Research Program Manager, Microsoft Research Asia

Has this ever happened to you? You’re out walking with your daughter. She finds a beautiful flower, quizzes you on it, but you’re stumped — you have no idea what it is. Instead of having to admit you don’t know, what if you could quickly identify the flower or any other plant wherever you happen to be? But how? At least 250,000 species of flowers exist and even experienced botanists have trouble identifying them all. Now there’s a way thanks to the rising power and sophistication of image recognition and the ease of taking pictures with your smartphone.

Smart Flower Recognition System

It’s called the Smart Flower Recognition System but it might never have happened were it not for a chance encounter last year between Microsoft researchers and botanists at the Institute of Botany, Chinese Academy of Sciences (IBCAS). Yong Rui, assistant managing director of Microsoft Research Asia (MSRA), was explaining image-recognition technology at a seminar — much to the delight of IBCAS botanists whose own arduous efforts to collect data on regional flower distribution were experiencing poor results. The IBCAS botanists soon realized the potential of MSRA’s image-recognition technology. At the same time, Yong Rui knew he had found the perfect vehicle to improve image recognition while addressing a reality-based problem that benefits society. It also helped that IBCAS had accumulated a massive public store of 2.6 million images. Since anyone in the world could upload pictures to this flower photo dataset — and no human could possibly supervise the uploads — the MSRA team had to create algorithms to filter out the “bad” pictures. That was the first of many difficult problems facing researcher Jianlong Fu and his team in building a tool capable of discerning tiny anomalies among the many species of flowers.

To do so they trained a deep neural network to recognize images using a set of learnable filters. In a nutshell, it works like this:

During the forward pass, each filter is convolved across the width and height of the input volume, computing the dot product between the entries of the filter and the input. This produces a 2-dimensional activation map of that filter. As a result, the network learns filters that activate per specific types of features at a given spatial position in the input.

Inputting millions of pictures into the deep-learning framework, MSRA researchers eventually enabled the engine to accurately identify images more than 90 percent of the time, an astonishing rate just shy of human capabilities.

Caffe deep-learning framework

And the project greatly helped the Chinese botanists in meeting their goals. “The flower-recognition engine enables domain experts to acquire plant distribution in China in an efficient way,” said Zheping Xu, assistant director of IBCAS. “Not only that, this engine can help ordinary people who have a strong interest in flowers to gain more knowledge.”

flower project3.png

In the future, MSRA and IBCAS will continue the collaboration, hoping to create applications based on the flower-recognition engine, so that botanists can conduct their research, parents can appear infallible to their kids, and everyone can appreciate flowers on an even deeper level.

Learn more

Up Next

Artificial intelligence, Computer vision, Graphics and multimedia

Teaching computers to see with Dr. Gang Hua

Episode 28, June 13, 2018 - Dr. Hua talks about how the latest advances in AI and machine learning are making big improvements on image recognition, video understanding and even the arts. He also explains the distributed ensemble approach to active learning, where humans and machines work together in the lab to get computer vision systems ready to see and interpret the open world.

Microsoft blog editor

Artificial intelligence, Computer vision

AI with creative eyes amplifies the artistic sense of everyone

By Gang Hua, Principal Researcher, Research Manager Recent advances in the branch of artificial intelligence (AI) known as machine learning are helping everyone, including artistically challenged people such as myself, transform images and videos into creative and shareable works of art. AI-powered computer vision techniques pioneered by researchers from Microsoft’s Redmond and Beijing research labs, […]

Microsoft blog editor

Artificial intelligence, Graphics and multimedia

Bots generate video titles and tags to bring AI researchers one step closer to visual intelligence

By Winnie Cui, Senior Research Program Manager, Microsoft Research Asia When your grandma posts a video to the cloud, there it lies, lonely and unwatched, unless your grandmother has more tagging and titling skills than mine does. My nana loves taking family videos with her cellphone, but while she’s great at creating content, she’s not […]

Microsoft blog editor