Virtual inspections enhance security of the energy supply: E.ON inspects power lines with drones and artificial intelligence based on Microsoft Azure

The challenge: Inspect 700,000 km of power lines as efficiently as possible

“We make sure people have light.” With this statement, Sam Julian, Head of Data Engineering & AI Solutions at E.ON, cuts right to the heart of the responsibility his company bears as Germany’s largest power supplier. After all, electricity is essential for the functioning of our society. In 2021 alone, electricity consumption in Germany amounted to 508 terawatt hours (TWh)—enough power to run more than five billion refrigerators. In many parts of the country, this electricity is supplied by E.ON: its power grid measures 700,000 km and serves locations throughout Germany. Managing, operating, and maintaining the lines that carry electricity to homes and businesses is the responsibility of the distribution system operators. E.DIS, MITNETZ STROM, and Westnetz are 3 of the total of 9 E.ON distribution system operators in Germany.

“Responsibility is our core principle—if lines fail, people are left in the dark,” explains Jens Hache, Head of the project Drone@E.ON at Mitteldeutsche Netzgesellschaft Strom mbH, or MITNETZ STROM in short. As the largest regional distribution system operator of eastern Germany, it supplies electricity via high-, medium-, and low-voltage grids to customers in Saxony, Saxony-Anhalt and parts of Brandenburg for E.ON. “The maintenance process is enormously important to us,” Hache continues. Previously, inspections were carried out every five years. The technicians walk along the lines and look for defects or signs of wear. To do this, they inspect the lines themselves, climbing the tall poles to check that all parts are working properly, if necessary. Helicopters are also used yearly to get a better view of larger sections of the grid. “This is very hands-on work: looking for defects with the naked eye, subjectively assessing them, noting them down, and then entering them into our system. Only after all that do we make a decision about whether repairs are necessary or possible,” Hache says. “It’s difficult to identify defect patterns on the power lines in such a way as to make reliable predictions.” The three E.ON distribution system operators wanted to improve their process, and they opted for aerial drone footage and virtual image analysis using artificial intelligence (AI) based on Microsoft Azure.

The solution: Virtual inspections with drone images and artificial intelligence

At that point, E.ON had already been present in the world of the Microsoft cloud for some time, as had its partner eSmart Systems. Julian saw this as a key advantage: “Together with the three distribution system operators we were able to start developing our virtual inspection solution very quickly.” There are two elements at play here: drones take the pictures, and then artificial intelligence analyzes the images, sorts them, and evaluates them. To get the images, technicians fly the drones over the power lines according to a set pattern and take pictures at important points. “This saves our teams some strenuous climbing during the purely visual inspection of poles,” Hache says.

Back in the office, the drone images are loaded into SharePoint before being moved to Azure Data Lake Gen2. Azure Logic Apps orchestrates the input from various distribution system operators to keep the files separate and assigns them to the correct power lines. The images are then cleaned up using Azure Functions and imported to Grid Vision^® via Azure Event Hubs as an instance. A wide variety of Azure services come into play here, where Azure Kubernetes Service, Azure Database for PostgreSQL and Cosmos DB play a key role.

“Grid Vision^® is our cloud-based Software-as-a-Service, developed in Microsoft Azure, for inspecting transmission and distribution power grids,” explains Erik Åsberg, CTO of partner eSmart Systems. “AI determines whether the distribution system operator images that E.ON sends us show, for example, signs of wear on individual components or major damage to the poles. An expert user then verifies the AI findings before the output is exported back to the E.ON cloud, where it is converted into a checklist.”

The collected data improves the quality of the distribution system operators’ inspections: “In addition to a line’s current status or a potential need for maintenance, we also use the checklists to identify defect patterns that we can use in making predictions,” Hache explains. “This allows us to move from periodic maintenance to precise, predictive maintenance.” And these predictions become more accurate with each image analyzed. “Grid Vision^® is a software tool used by utilities all over Europe and North America. The AI models are developed to support our utility customers and the number of defects detected is growing rapidly,” Åsberg says. “As the expert users provide input to Grid Vision^® by validating the AI’s findings, the input is used to train and improve the AI. This happens across our entire user base world-wide, leading to a giant feedback loop for continuous improvement. We call this collaborative AI.”

E.DIS, MITNETZ and Westnetz aim to make inspections and maintenance even more efficient in the future with automation: “Right now, we are only testing beyond visual line of sight (BVLOS) flights. That means flights where the drones fly autonomously along lengthy stretches of power lines out of the pilots’ view. But soon, such flights could eliminate the need for driving a car from one line section to another during regular inspections. The technology is already available, but the necessary approvals are still quite elaborate and time-consuming.”