Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Researchers team up with Chinese botanists on machine learning, flower-recognition project

July 25, 2016 | By Microsoft blog editor

By Guobin Wu, Senior Research Program Manager, Microsoft Research Asia

Has this ever happened to you? You’re out walking with your daughter. She finds a beautiful flower, quizzes you on it, but you’re stumped — you have no idea what it is. Instead of having to admit you don’t know, what if you could quickly identify the flower or any other plant wherever you happen to be? But how? At least 250,000 species of flowers exist and even experienced botanists have trouble identifying them all. Now there’s a way thanks to the rising power and sophistication of image recognition and the ease of taking pictures with your smartphone.

Smart Flower Recognition System

It’s called the Smart Flower Recognition System but it might never have happened were it not for a chance encounter last year between Microsoft researchers and botanists at the Institute of Botany, Chinese Academy of Sciences (IBCAS). Yong Rui, assistant managing director of Microsoft Research Asia (MSRA), was explaining image-recognition technology at a seminar — much to the delight of IBCAS botanists whose own arduous efforts to collect data on regional flower distribution were experiencing poor results. The IBCAS botanists soon realized the potential of MSRA’s image-recognition technology. At the same time, Yong Rui knew he had found the perfect vehicle to improve image recognition while addressing a reality-based problem that benefits society. It also helped that IBCAS had accumulated a massive public store of 2.6 million images. Since anyone in the world could upload pictures to this flower photo dataset — and no human could possibly supervise the uploads — the MSRA team had to create algorithms to filter out the “bad” pictures. That was the first of many difficult problems facing researcher Jianlong Fu and his team in building a tool capable of discerning tiny anomalies among the many species of flowers.

To do so they trained a deep neural network to recognize images using a set of learnable filters. In a nutshell, it works like this:

During the forward pass, each filter is convolved across the width and height of the input volume, computing the dot product between the entries of the filter and the input. This produces a 2-dimensional activation map of that filter. As a result, the network learns filters that activate per specific types of features at a given spatial position in the input.

Inputting millions of pictures into the deep-learning framework, MSRA researchers eventually enabled the engine to accurately identify images more than 90 percent of the time, an astonishing rate just shy of human capabilities.

Caffe deep-learning framework

And the project greatly helped the Chinese botanists in meeting their goals. “The flower-recognition engine enables domain experts to acquire plant distribution in China in an efficient way,” said Zheping Xu, assistant director of IBCAS. “Not only that, this engine can help ordinary people who have a strong interest in flowers to gain more knowledge.”

flower project3.png

In the future, MSRA and IBCAS will continue the collaboration, hoping to create applications based on the flower-recognition engine, so that botanists can conduct their research, parents can appear infallible to their kids, and everyone can appreciate flowers on an even deeper level.

Learn more

Up Next

Two guys writing equations on a window in Asia

Artificial intelligence, Graphics and multimedia, Human language technologies

Growing a generation of computer scientists – Microsoft Research Asia at 20 and going beyond technical achievement

Microsoft Research Asia celebrates its 20th anniversary this year, and the milestone provided an occasion for many in the industry to reflect on an amazing journey, one not only replete with excellence and technological achievement, but also significant in its profound influence as it cultivated a generation of computer scientists and engineers, catalyzed collaboration between […]

Microsoft blog editor

Artificial intelligence, Computer vision, Graphics and multimedia

Teaching computers to see with Dr. Gang Hua

Episode 28, June 13, 2018 - Dr. Hua talks about how the latest advances in AI and machine learning are making big improvements on image recognition, video understanding and even the arts. He also explains the distributed ensemble approach to active learning, where humans and machines work together in the lab to get computer vision systems ready to see and interpret the open world.

Microsoft blog editor

Microsoft Pix before and after panoramic photo of Miners Landing

Artificial intelligence, Graphics and multimedia

New Microsoft Pix features let you take bigger, wider pictures and turns your videos into comics

Microsoft has released two new features with today’s update to Microsoft Pix for iOS, an app powered by a suite of intelligent algorithms developed by Microsoft researchers to take the guesswork out of getting beautiful photos and videos. The first of these features, Photosynth, helps create photos that take in more of the perspective or […]

Nicky Budd-Thanos