AI AirSim Drones
AirSim creates a 3D version of a real environment. A simulated drone captures imagery then creates a custom vision model. AI services then uses the model to identify objects or people in the images.
The need
It’s often costly and risky to collect imagery to train a custom model during disaster recovery or search and rescue operations. Deploying custom vision AI models on edge devices was challenging without advanced processing resources.
The idea
AirSim lets us create a 3D version of the real environment to process the custom vision model. A simulated drone takes pictures, and then the Custom Vision service trains a custom model to find objects or people in the images.
The solution
In one test, a piloted drone took photos of stuffed animals on a simulated football field. Next, Custom Vision service trained a model to identify each animal – and the drone sent an alert as it found the animals.
Create and monitor a virtual environment
Discover how AirSim simulates an environment and then creates a 3D version of the real environment.
Technical details for AirSim – Drones
For this search and rescue scenario, we created a 3D-generated environment in AirSim to simulate the football field on Microsoft campus and placed stuffed animals on the field.
We then created a Python script to fly the drone around the simulated environment and take many pictures of the animals. We then pushed the images into Custom Vision service and trained a model to identify each type of animal in the field.
From there we exported the trained model into TensorFlow format and pushed it into Docker containers.
These containers were then deployed to Azure IoT Edge and then pushed to a drone running a custom board and a Nvidia GPU.
The drone is then able to fly around and send an alert to Azure IoT Hub every time it successfully identifies an animal.
This is a great showcase of how real-time custom AI can run on edge devices, such as a drone.
Resources:
Intelligent Robotics
Intelligent robotics uses AI to increase collaboration between people and devices. Microsoft AI enables the next generation of robots to adapt to dynamic situations and communicate naturally with people.
Machine Teaching
Explore the concepts of machine teaching, allowing developers or subject matter experts with little AI expertise to provide abstract concepts to an intelligent system.
PoseTracker
PoseTracker uses deep learning to track the position and orientation of objects. This solution will use your phone camera to measure and track the angle, orientation and distance of an item in real time.
Spektacom
Spektacom uses a mini sticker sensor on a cricket bat to gather data on the quality, speed, twist and swing of the bat. This data is used to analyse the quality of the shot to help professionals, amateurs and coaches to improve their game.
Explore the possibilities of AI
Jump-start your own AI innovations with learning resources and development solutions from Microsoft AI.
Innovation Developer Hub
Explore insights and behind-the-scenes technology for breakthrough AI innovations. From Tech Minutes videos to Technology Deep Dives, learn about the engineering that powers the future of AI.
AI School
Learn to create your own AI experiences with learning paths in conversational AI, machine learning, AI for devices, cognitive services, autonomous systems, AI strategy, and more.
AI platform
Start building AI solutions with powerful tools and services. Microsoft AI is a robust framework for developing AI solutions in conversational AI, machine learning, data sciences, robotics, IoT, and more.
