Snow Leopard Trust
AI, machine learning, and cognitive services are helping researchers find and protect threatened snow leopard populations.Try the Snow Leopard demo
Snow leopards are apex predators in Central Asia, known as “ghosts of the mountains” due to their elusive nature. Their population is a key indicator for the health of the whole ecosystem, but they can be difficult to spot and track in the wild.
Scientists use camera traps to spot snow leopards in their natural habitats with minimal disruption. Camera traps capture hundreds of thousands of images that need identification and analysis, a labor-intensive task that can be streamlined with AI services.
Microsoft Machine Learning for Apache Spark (MMLSpark) and the Azure Cognitive Services are used to automate image classification, allowing researchers to find well-camouflaged snow leopards within image sets more quickly, saving over 300 hours per camera survey.
Identifying snow leopards with AI
Camera traps capture hundreds of thousands of photos of snow leopards in the wild. Spotting well-camouflaged leopards within these photos is a labor-intensive task that AI can accomplish in minutes. We use MMLSpark, the Azure Cognitive Services, and Microsoft Cognitive Toolkit to automate image classification at scale.
Technical details for Snow Leopard Trust
Snow Leopards are a highly threatened species, native to the steppes and mountainous terrain of Asia. Despite their pivotal importance as this biome’s apex predator, we know very little about their numbers and behavior. Due the cats’ remote habitat, expansive range and extremely elusive nature, researchers use motion-triggered camera traps to observe snow leopards in the wild. Since the cameras trigger on any type of movement, most of the images are of goats, birds, and grass blowing in the wind. Only about 5 percent of the pictures actually contain a leopard, which can be hard to spot due to their camouflage. Over 1 million images have been gathered, and camera traps add 500,000 images each year. Manually reviewing all images to find a snow-leopards could take thousands of hours of time.
The Snow Leopard Trust used Microsoft AI to build a scalable image recognition program that is roughly 95 percent accurate in identifying snow leopards in camera trap photos. The team additionally created a live dashboard that highlights snow leopard hot spots. These spots serve as social meeting points for leopards and play important roles in their communication.
Deep Unsupervised Object Detection with Microsoft ML for Apache Spark
To create a leopard classifier, we used a technique called transfer learning where we specialize a large general-purpose vision network for a more specific classification task. In our workflow, we leverage ResNet50, a 50-layer deep convolutional network with residual connections that has been trained on the ImageNet classification challenge. Using Microsoft ML for Apache Spark, we can combine the accuracy and flexibility of deep models with the elastic scalability of Apache Spark to quickly featurize all images in the dataset and learn a classifier based on these features.
We augment our basic pipeline with several additional features to improve performance. First, we use the Azure Cognitive Services on Spark to embed large scale Bing Image Searches directly into Apache Spark. We can use some of Bing’s collective intelligence by searching for images of leopards and images of empty hillsides to augment our dataset. Additionally, we add horizontal flips to our dataset to further improve robustness. Lastly, we aggregate results over camera trap photo bursts to give the algorithm additional chances to spot a leopard in a batch of photos.
Simply classifying images of leopards is not enough to determine the number of leopards in the ecosystem. More specifically, it is tough to distinguish between an ecosystem with many shy leopards, and one with a few curious leopards that like to take selfies. To tackle this problem, we use tools like HotSpotter to identify individual leopards based on their spot patterns. However, these tools often require well-behaved, cropped images of the target animal. More explicitly, these methods require not just a leopard classifier, but a leopard detector. To transform our classifier into something that could highlight the patterns of the leopard, we created a distributed implementation of the black box model interpretability technique, LIME. Using LIME, we can refine our classifier into a model that can detect the actual patterns of the leopard, without requiring human-annotated bounding boxes.
Microsoft AI for Earth invests in environmental science
Microsoft has devoted 50 million dollars in grants to fund wildlife conservation. The AI for Earth program connects researchers in environmental science with the AI and computing resources they need to accomplish their goals. The program has also developed open-source tools to accelerate camera trap image analysis.
- Machine Learning Blog: Saving Snow Leopards with Deep Learning and Computer Vision on Spark
- Learn about image services at AI School
- Learn about machine learning at AI School
- Deep Learning Without Labels: The Challenges of Snow Leopard Conservation
- Academic Paper on Unsupervised Snow Leopard Detection on Spark
- Academic Paper on Deep Learning on Spark
- Microsoft Machine Learning for Apache Spark at GitHub
- The Azure Cognitive Services on Spark
- AI for Earth Camera Trap Initiative
- Learn about AI on Azure
- Learn about The Microsoft Cognitive Toolkit
- ResNet at arXiv
Gen Studio is a prototype created with collaborators from The Metropolitan Museum of Art, Microsoft, and MIT. Gen Studio uses AI to visually navigate The Met’s art collection.
Clean Water AI
Clean Water AI uses deep learning to detect dangerous bacteria and particles in water. The device analyzes drinking water with real-time detection and contamination mapping.
Angel Eyes is an IOT device that monitors a baby’s sleeping position and environment. Caregivers can view a live stream from anywhere and receive notifications if the device detects any issues.
PoseTracker uses deep learning to track the position and orientation of objects. This solution will use your phone camera to measure and track the angle, orientation, and distance of an item in real time.
Explore the possibilities of AI
Jumpstart your own AI innovations with learning resources and development solutions from Microsoft AI.
Innovation Tech Hub
Explore insights and behind-the-scenes technology for breakthrough AI innovations. From Tech Minutes videos to Technology Deep Dives, learn about the engineering that powers the future of AI.
Learn to create your own AI experiences with learning paths in conversational AI, machine learning, AI for devices, cognitive services, autonomous systems, AI strategy, and more.
Start building AI solutions with powerful tools and services. Microsoft AI is a robust framework for developing AI solutions in conversational AI, machine learning, data sciences, robotics, IoT, and more.