Ashish Kapoor at Microsoft Research

Partner Research Manager

About

I am the Partner Research Manager for the Autonomous Systems Research Group (new incarnation of Aerial Informatics and Robotics (AIR) Group) at Microsoft, Redmond. These days I am thinking a lot of safety, pre-training of deep models and data-driven simulation for Autonomous Systems.

Most of my research these days are driven by my deep desire to build intelligent and autonomous flying agents that are safe and enable applications that can positively influence our society. The core technology builds upon cutting edge research in machine intelligence, robotics and human-centered computation, in order to enable an entire fleet of flying robots that range from micro-UAVs to commercial jetliners. Various application scenarios include Weather Sensing, Monitoring for Precision Agriculture, Safe Cyber-Physical Systems, etc.

More broadly, I am interested in Machine Learning, Quantum Computation and Computer Vision, with applications in User Modeling, Affective Computing and HCI scenarios.

I…

More broadly, I am interested in Machine Learning, Quantum Computation and Computer Vision, with applications in User Modeling, Affective Computing and HCI scenarios.

I am also an avid aviator and hold FAA Commercial Pilot certificate (SEL), FAA Flight Instructor certificate (Airplane Single Engine and Instrument Airplane). I am an active amateur aircraft builder and built an RV-8 Aircraft (see: RV-8 build blog ).

——-

Please note that I update this page maybe once a year. If you need more information then the best way is to email me.

Google Scholar has most up to date list of my publications.

If I don’t know you, then the best way to communicate is via LinkedIn.

If I know you, then the best way to communicate is on email.

If I know you really well, then I’d love to talk to you on the phone.

——–

Eons ago I did a PhD at the MIT Media Lab, and my Doctoral thesis looked at building Discriminative Models for Pattern Recognition with incomplete information (semi-supervised learning, imputation, noisy data etc.). A significant part of that work involved automatic analysis of non-verbal behavior (e.g. Facial Action Analysis, see Master’s thesis) and physiological responses. The earliest part of my research career tackled the problem of photorealistic facial animations.

Featured content

Autonomous systems, aerial robotics and Game of Drones with Gurdeep Pall and Dr. Ashish Kapoor

Episode 100 | November 27, 2019 - There’s a lot of excitement around self-driving cars, delivery drones, and other intelligent, autonomous systems, but before they can be deployed at scale, they need to be both reliable and safe. That’s why Gurdeep Pall, CVP of Business AI at Microsoft, and Dr. Ashish Kapoor, who leads research in Aerial Informatics and Robotics, are using a simulated environment called AirSim to reduce the time, cost and risk of the testing necessary to get autonomous agents ready for the open world. On the podcast, Gurdeep and Ashish discuss life at the intersection of machine learning, simulation, and autonomous systems, and talk about the challenges we face as we transition from a world of automation to a world of autonomy. They also tell us about Game of Drones, an exciting new drone racing competition where the goal is to imbue flying robots with human-level perception and decision making skills… on the fly.

Machine learning and the incredible flying robot with Dr. Ashish Kapoor

Episode 22 | May 2, 2018 - Autonomous flying agents – or flying robots – may seem like the stuff of sci-fi to the average person, but to Dr. Ashish Kapoor, Principal Researcher and Research Manager of the Aerial Informatics and Robotics Group at Microsoft Research, they’re much closer to science than to fiction. And, having built – and flight tested – his own airplane, complete with state-of-the-art avionics designed to run AI and ML algorithms, he has the street cred – or should we say flight cred – to prove it. Today, Dr. Kapoor talks about how cutting-edge machine learning techniques are empowering a new generation of autonomous vehicles, and tells us all about AirSim, an innovative platform that’s helping bridge the simulator-to-reality gap, paving the way for safer, more robust real-world AI systems of all kinds.

AirSim

Microsoft AirSim (Aerial Informatics and Robotics Simulation) is an open-source robotics simulation platform. From ground vehicles, wheeled robotics, aerial drones, and even static IoT devices, AirSim can capture data for models without costly field operations. AirSim works as a plug-in to Epic Games’ Unreal Engine 4 editor, providing control over building environments and simulating difficult-to-reproduce, real-world events to capture meaningful data for AI models. Machine learning has become an increasingly important artificial intelligence approach in building autonomous and robotic systems. One of the key challenges with machine learning is the need for massive data sets—and the amount of data needed to learn useful behaviors can be prohibitively high. Since a new robotic system is often non-operational during the training phase, the development and debugging phases with real-world experiments face an unpredictable robot. AirSim solves these two problems: the need for large data sets for training and the ability to debug in a simulator. It provides a realistic simulation tool for designers and developers for seamless generation of the amount of training data they require. In addition, AirSim leverages current game engine rendering, physics, and perception computation to create accurate, real-world simulations. Together, this realism, based on efficiently generated ground-truth data, enables the study and execution of complex missions that are time-consuming and/or risky in the real-world. For example, collisions in a simulator cost virtually nothing, yet provide actionable information for improving the design of the system.