Wayve’s AV2.0 builds a brighter future with Azure Machine Learning and PyTorch

Welcome to Autonomous Vehicles 2.0

Someday soon, autonomous vehicles will pull up to intersections everywhere. They will know exactly where they are, understand exactly what to do next, be able to easily and safely distinguish pedestrians from parking meters, and manage change second by second. They’ll do it all in places they’ve never been, under skies bright, dark, clear, or blurred with fog, rain, or driving snow.

Even in this digital age, it’s easy to wonder, how in the world? Ask Wayve. As innovators of vision-based machine learning for the next wave of autonomous vehicles—what it calls AV2.0—Wayve wants to reimagine mobility and be the first company to deploy autonomous driving technology in 100 cities based on a pure AI approach.

To meet these ambitions, Wayve is going beyond hand-coded rules and high-definition maps that have typified traditional approaches to AV development. The company recognized the transformative potential of deep learning to solve the complexities of self-driving, a way that solves sensing and motion planning jointly with driving data in a neural network, frequently depicted as the end-to-end approach to self-driving. This approach brings together Microsoft Azure Machine Learning and the PyTorch machine learning framework to develop a self-driving system that is data-driven at every layer, allowing for continuous optimization without re-engineering.

Hurly-burly data

Self-driving cars are notoriously difficult to develop. The sheer number of variables involved in a simple trip to the store is staggering—traffic, pedestrians, weather, time of day and year, and even local driving culture, just to name a few. That complexity typically proves too great a challenge for conventional robotics and hand-coded, rules-based approaches. Developers also tend to design and build self-driving vehicles in ideal and more structured environments, like sunny, dry areas with wide boulevards and generally stable, predictable traffic flows.

Wayve is taking a different approach. “Rather than hand coding how a system should behave,” states Chief Executive Officer and Cofounder Alex Kendall, “we’re able to teach a system how to drive using data, through deep learning driving experiences that train our models on the patterns of how humans behave and react.”

To that end, Wayve gathers its data on the hurly-burly and often drizzly streets of London. “London has chaotic traffic, busy scenes, and really complex junctions that require true driving intelligence,” says Kendall. “That’s why we’re teaching our AV models to drive there.”

Rather than avoiding the complexity of an urban environment, Wayve saw the opportunity to capture city driving’s rich troves of data from all those roundabouts, arterials, and alleyways. The company is working to prove that if its AV2.0 models can learn to drive in London, they can learn to drive anywhere.

Teaching machines to drive

To scale its AV models to any environment, including highly complex scenarios, Wayve needs the capacity for a massive body of data, accessibility to vast computation resources, and advanced formulas for the models. The Wayve driving model inputs include visual camera data, satellite navigation data, and other sensor data to produce driving outputs such as left to right, acceleration, deceleration, and stopping. 

Wayve will collect and process millions of hours of driving data to fully realize a market-ready solution. It uses Azure Machine Learning to amass and process ever greater amounts of data, which it applies to train its driving models. Wayve engineers rely on the PyTorch open-source machine learning framework to help understand and interpret the models that the data produces. This solution has given Wayve the powerful capability to experiment, innovate, and iterate at scale.

“Autonomy 2.0 is one of the hardest problems to solve in terms of applied machine learning, but with Azure Machine Learning, we can seamlessly scale to the data and the compute requirements of our featured driving challenges,” says Sameen Jalal, Director of Engineering at Wayve. “PyTorch is the air our machine learning model breathes—we use it to step through the driving model to explain and understand various driving maneuvers undertaken by our autonomous driving system and to iterate quickly so we can take new driving features directly to the road.”

The right cloud

Wayve moved from on-premises to cloud-enabled model training in just a few days, largely because of the interoperation between Azure Machine Learning and PyTorch. The company distributes learning outcomes from individual machines across its entire network, and it has developed the capacity to horizontally scale its machine learning to incorporate as many nodes as it might require for training cycles. As a result, Wayve has increased its training data throughput by 50 times and dramatically reduced its model training time even while it trains massive computer vision systems with petabytes of data at an unprecedented scale.

“Through our use of Azure Machine Learning, we have the flexibility to train our AV2.0 models 90 percent faster compared to our previous datacenter environment and depending on the experiment, and we’ve moved from using millions of training examples to many billions. It’s given us the scale we needed to experiment, iterate, and nimbly change models,” Jalal states. “Azure Machine Learning is the platform that’s pushing the boundaries of machine learning and really the only AI technology capable of supporting this level of innovation.”

Adds Jalal, “Moving to Azure Machine Learning rapidly improved our iteration speed and our innovation for new autonomy features, which, in turn, helps our car to continuously drive better.”

An autonomous future

The scale and challenges involved with developing autonomous driving may be considerable, but so too are its potential applications and benefits. Autonomous vehicles could very well eliminate traffic congestion from the cities of tomorrow and help drastically reduce the number of collisions, injuries, and deaths related to driving. Moreover, autonomous driving has the potential to provide safe, accessible, and sustainable transportation, improve mobility for everyone, everywhere, and promote the development of smarter, more thoughtful, and more livable communities.

“Autonomous driving is the hardest engineering challenge of our generation,” says Kendall. ”We chose Azure Machine Learning because it gives us the compute resources and scale—from a partner that’s truly an enabler of deep learning technology—to build AV2.0 technology that is safe, scalable, and amenable to the needs of people worldwide.”

Find out more about Wayve on Twitter, YouTube, and LinkedIn.