AI Lab stories

Get inspired with stories, new lab projects, and examples from developers and partners.

Celebs Like Me Microsoft Math Snow Leopard Trust Gen Studio Clean Water AI Pix2Story Spektacom Angel Eyes PoseTracker

The need

Snow leopards are apex predators in Central Asia, known as “ghosts of the mountains” due to their elusive nature. Their population is a key indicator for the health of the whole ecosystem, but they can be difficult to spot and track in the wild.

The idea

Scientists use camera traps to spot snow leopards in their natural habitats with minimal disruption. Camera traps capture hundreds of thousands of images that need identification and analysis, a labor-intensive task that can be streamlined with AI services.

The solution

Microsoft Machine Learning for Apache Spark (MMLSpark) and the Azure Cognitive Services are used to automate image classification, allowing researchers to find well-camouflaged snow leopards within image sets more quickly, saving over 300 hours per camera survey.

Technical details for Snow Leopard Trust

Snow Leopards are a highly threatened species, native to the steppes and mountainous terrain of Asia. Despite their pivotal importance as this biome’s apex predator, we know very little about their numbers and behavior. Due the cats’ remote habitat, expansive range and extremely elusive nature, researchers use motion-triggered camera traps to observe snow leopards in the wild. Since the cameras trigger on any type of movement, most of the images are of goats, birds, and grass blowing in the wind. Only about 5 percent of the pictures actually contain a leopard, which can be hard to spot due to their camouflage. Over 1 million images have been gathered, and camera traps add 500,000 images each year. Manually reviewing all images to find a snow-leopards could take thousands of hours of time.

The Snow Leopard Trust used Microsoft AI to build a scalable image recognition program that is roughly 95 percent accurate in identifying snow leopards in camera trap photos. The team additionally created a live dashboard that highlights snow leopard hot spots. These spots serve as social meeting points for leopards and play important roles in their communication.

Deep Unsupervised Object Detection with Microsoft ML for Apache Spark

To create a leopard classifier, we used a technique called transfer learning where we specialize a large general-purpose vision network for a more specific classification task. In our workflow, we leverage ResNet50, a 50-layer deep convolutional network with residual connections that has been trained on the ImageNet classification challenge. Using Microsoft ML for Apache Spark, we can combine the accuracy and flexibility of deep models with the elastic scalability of Apache Spark to quickly featurize all images in the dataset and learn a classifier based on these features.

We augment our basic pipeline with several additional features to improve performance. First, we use the Azure Cognitive Services on Spark to embed large scale Bing Image Searches directly into Apache Spark. We can use some of Bing’s collective intelligence by searching for images of leopards and images of empty hillsides to augment our dataset. Additionally, we add horizontal flips to our dataset to further improve robustness. Lastly, we aggregate results over camera trap photo bursts to give the algorithm additional chances to spot a leopard in a batch of photos.

Simply classifying images of leopards is not enough to determine the number of leopards in the ecosystem. More specifically, it is tough to distinguish between an ecosystem with many shy leopards, and one with a few curious leopards that like to take selfies. To tackle this problem, we use tools like HotSpotter to identify individual leopards based on their spot patterns. However, these tools often require well-behaved, cropped images of the target animal. More explicitly, these methods require not just a leopard classifier, but a leopard detector. To transform our classifier into something that could highlight the patterns of the leopard, we created a distributed implementation of the black box model interpretability technique, LIME. Using LIME, we can refine our classifier into a model that can detect the actual patterns of the leopard, without requiring human-annotated bounding boxes.

Microsoft AI for Earth invests in environmental science

Microsoft has devoted 50 million dollars in grants to fund wildlife conservation. The AI for Earth program connects researchers in environmental science with the AI and computing resources they need to accomplish their goals. The program has also developed open-source tools to accelerate camera trap image analysis.

Resources:

Explore the possibilities of AI

Find demos to get more ideas or learn about AI technology to jumpstart your own development.