{"id":359810,"date":"2017-02-15T06:00:48","date_gmt":"2017-02-15T14:00:48","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=359810"},"modified":"2025-02-14T11:41:19","modified_gmt":"2025-02-14T19:41:19","slug":"aerial-informatics-robotics-platform","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/aerial-informatics-robotics-platform\/","title":{"rendered":"Aerial Informatics and Robotics Platform"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background- card-background--full-bleed\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1920\" height=\"720\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/Aerial-Informatics_header_1920x720.jpg\" class=\"attachment-full size-full\" alt=\"aerial informatics robotics - AirSim - three men posing with a drone\" style=\"object-position: 78% 50%\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/Aerial-Informatics_header_1920x720.jpg 1920w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/Aerial-Informatics_header_1920x720-300x113.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/Aerial-Informatics_header_1920x720-1024x384.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/Aerial-Informatics_header_1920x720-768x288.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/Aerial-Informatics_header_1920x720-1536x576.jpg 1536w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/Aerial-Informatics_header_1920x720-1600x600.jpg 1600w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/Aerial-Informatics_header_1920x720-240x90.jpg 240w\" sizes=\"auto, (max-width: 1920px) 100vw, 1920px\" \/>\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 \">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading h2\" id=\"aerial-informatics-and-robotics-platform\">Aerial Informatics and Robotics Platform<\/h1>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<p>In 2017 Microsoft Research created AirSim (Aerial Informatics and Robotics Simulation) \u2013 an open-source robotics simulation platform. From ground vehicles, wheeled robotics, aerial drones, and even static IoT devices, AirSim enabled data capture data for models without costly field operations.\u200b<\/p>\n\n\n\n<p>Over the span of five years, the open-source AirSim research project served its purpose and is now archived in anticipation of a new aerial autonomy simulation platform. Users can still access the original AirSim code, but no further updates will be made. For more information about migrating to the new platform, please visit the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/microsoft\/AirSim\" target=\"_blank\" rel=\"noopener noreferrer\">GitHub<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> repo.\u200b<\/p>\n\n\n\n<p>Read on to learn more about the AirSim research project.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"bridging-the-sm-to-real-gap-with-airsim\">Bridging the sm-to-real gap with AirSim<\/h2>\n\n\n\n<p>Microsoft AirSim (Aerial Informatics and Robotics Simulation) is an open-source robotics simulation platform. From ground vehicles, wheeled robotics, aerial drones, and even static IoT devices, AirSim can capture data for models without costly field operations.<\/p>\n\n\n\n<p>AirSim works as a plug-in to Epic Games\u2019 Unreal Engine 4 editor, providing control over building environments and simulating difficult-to-reproduce, real-world events to capture meaningful data for AI models.<\/p>\n\n\n\n<p>Machine learning&nbsp;has become an increasingly important artificial intelligence approach in building autonomous and robotic systems. One of the key challenges with machine learning&nbsp;is the need for massive data sets\u2014and the amount of data needed to learn useful behaviors can be prohibitively high. Since a new robotic system&nbsp;is often non-operational during the training phase, the development and debugging phases with real-world experiments face an unpredictable robot.<\/p>\n\n\n\n<p>AirSim solves these two problems: the need for large data sets for training and the ability to debug in a simulator. It provides a realistic simulation tool for designers and developers for seamless generation of the amount of training data they require. In addition, AirSim leverages current game engine rendering, physics, and perception computation to create accurate, real-world simulations. Together, this realism, based on efficiently generated ground-truth data, enables the study and execution&nbsp;of complex missions that are time-consuming and\/or risky in the real-world. For example, collisions in a simulator cost virtually nothing, yet provide actionable information for improving the design of the system.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Aerial Informatics and Robotics Platform\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/GB-sBpXvM3s?feature=oembed&rel=0\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"a-toolbox-for-rapid-prototyping-testing-and-deployment\">A toolbox for rapid prototyping, testing, and deployment<\/h2>\n\n\n\n<figure class=\"wp-block-image alignright size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"400\" height=\"225\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/DroneGIF-02a.gif\" alt=\"drone animation\" class=\"wp-image-363695\"\/><\/figure>\n\n\n\n<p>Building a data-driven robotic system such as the AirSim platform is not a trivial task. First, it must support a wide variety of software and hardware. Second, given the breakneck speed of innovation in hardware, software, and algorithms, it must be flexible enough to extend easily in multiple dimensions. The AirSim framework addresses these challenges by using a modular design.<\/p>\n\n\n\n<p>The platform interfaces with common robotic platforms, such as a Robot Operating System&nbsp;(ROS), and comes pre-loaded with a commonly used&nbsp;aerial robotic&nbsp;model, a generic sports utility vehicle for autonomous driving simulation, and several sensors.&nbsp;In addition,&nbsp;the platform enables high-frequency simulations supporting hardware and software-in-the-loop simulations with widely supported protocols such as MavLink. Its cross-platform (Linux and Windows) and open-source architecture is easily extensible to accommodate diverse new types of autonomous vehicles, hardware platforms, and software protocols. This architecture allows users to add custom autonomous system models and new sensors to the simulator quickly.<\/p>\n\n\n\n<p>The platform&nbsp;is also designed to integrate with existing machine learning frameworks, including Microsoft\u2019s newly acquired Bonsai, to generate new algorithms for perception and control tasks. Methods such as reinforcement and imitation learning, learning-by-demonstration, and transfer learning can leverage simulations and synthetically generated experiences to build realistic models.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"from-perception-to-safe-control\">From perception to safe control<\/h2>\n\n\n\n<p>AirSim launched with aerial drone support and is used for applications such as precision agriculture, pathogen surveillance, and weather monitoring. These are systems that typically use a camera to perceive the world and plan and execute missions.<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"416\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/drone_one_ground-1-1024x416.png\" alt=\"AirSim drone hovering just above the street\" class=\"wp-image-363710\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/drone_one_ground-1-1024x416.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/drone_one_ground-1-300x122.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/drone_one_ground-1-768x312.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"428\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/drone_front_camera_view-1024x428.png\" alt=\"AirSim drone front-facing camera view of a street\" class=\"wp-image-363689\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/drone_front_camera_view-1024x428.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/drone_front_camera_view-300x125.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2017\/02\/drone_front_camera_view-768x321.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p>The platform enables seamless training and testing of cameras and other perception systems by using realistic renderings of the environment.&nbsp;These synthetically generated images&nbsp;can produce orders of magnitude more perception and control data than are possible with real-world data alone. If needed, custom sensors, such as infrared (IR), can be enabled.<\/p>\n\n\n\n<p>Since its launch, AirSim has grown to support autonomous cars, various wheeled robots, and even static IoT devices such as camera traps and facial expression recognition. Because AirSim is a plug-in for the Unreal Engine 4 game platform, users can construct their own scenery and vehicles.<\/p>\n\n\n\n<p>This open-source, high-fidelity physics, and photo-realistic robotic simulator can help verify control and perception software, and potentially provide certification compliance when those requirements arise. With common gaming skills matched with robotic system designers and developers, almost any creation and scenario can transfer from sim to real world with the fewest possible changes.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"learn-more\">Learn more<\/h2>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/airsim-high-fidelity-visual-physical-simulation-autonomous-vehicles\/#\" target=\"_blank\" rel=\"noreferrer noopener\">Read the Field&nbsp;and&nbsp;Service&nbsp;Robotics (FSR)&nbsp;2017&nbsp;paper ><\/a><\/p>\n\n\n\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/wp.me\/p5qjnp-ior\" target=\"_blank\" rel=\"noopener noreferrer\">Read the Next at Microsoft blog post ><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/toward-ai-operates-real-world\/\">Read the Microsoft Research blog post ><\/a><\/p>\n\n\n\n<p><em>Photo by Scott Eklund\/Red Box Pictures<\/em><\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p>Microsoft AirSim (Aerial Informatics and Robotics Simulation) is an open-source robotics simulation platform. From ground vehicles, wheeled robotics, aerial drones, and even static IoT devices, AirSim can capture data for models without costly field operations.<\/p>\n","protected":false},"featured_media":885576,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13561,13556,13562],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-359810","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-algorithms","msr-research-area-artificial-intelligence","msr-research-area-computer-vision","msr-locale-en_us","msr-archive-status-complete"],"msr_project_start":"","related-publications":[396932,396914,607764,642336,729763],"related-downloads":[],"related-videos":[363905,440325,440337,549993,550005,550017,550029,550044,502781,550053,550062,550074,550089,550098,861960,739828,670266,667962,608556],"related-groups":[237595,395930],"related-events":[],"related-opportunities":[],"related-posts":[440229,386729,550065,577041,608385,624378,638862,709156,720673],"related-articles":[],"tab-content":[],"slides":[{"attachment_id":255018,"headline":"Try the aerial informatics and robotics platform today","cta":"Available on GitHub","url":"https:\/\/github.com\/Microsoft\/AirSim","cta_style":"","slideshow_type":"feature"}],"related-researchers":[{"type":"user_nicename","display_name":"Vivan Amin","user_id":43431,"people_section":"Section name 0","alias":"aminvivan"},{"type":"user_nicename","display_name":"Rogerio Bonatti","user_id":41419,"people_section":"Section name 0","alias":"rbonatti"},{"type":"user_nicename","display_name":"Chris Lovett","user_id":36027,"people_section":"Section name 0","alias":"clovett"},{"type":"user_nicename","display_name":"Shital Shah","user_id":35435,"people_section":"Section name 0","alias":"shitals"}],"msr_research_lab":[199565],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/359810","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":19,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/359810\/revisions"}],"predecessor-version":[{"id":1129842,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/359810\/revisions\/1129842"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/885576"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=359810"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=359810"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=359810"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=359810"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=359810"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}