Editor’s note: We’ve republished this blog with a new companion video.
Sometimes you outgrow the capabilities of a well-loved tool—that’s exactly what happened to Microsoft and its on-premises Security Information Event Management (SIEM) system. Thanks to a timely assist from Microsoft Azure Sentinel, the company hasn’t missed a beat.
Our old SIEM was expected to cap out at 10 billion events daily. We had already begun to leverage other solutions to keep increasing our security monitoring coverage.
– Mei Lau, senior program manager, Microsoft Digital
As an enterprise, Microsoft’s footprint is massive. The company sees a lot of malicious traffic, which results in more than 20 billion cybersecurity events per day. This massive wave of noise was hard to sort through to find real threats—until the company’s internal security team turned to Microsoft Azure Sentinel, which, thanks to the cloud and AI, has the power to keep up with that volume.
“Our old SIEM was expected to cap out at 10 billion events daily,” says Mei Lau, a senior program manager for Microsoft Digital, the organization that powers, protects, and transforms Microsoft. Lau is responsible for leading the migration of Microsoft’s legacy SIEM to the cloud-based Microsoft Azure Sentinel. “We had already begun to leverage other solutions to keep increasing our security monitoring coverage.”
Because running out of capacity could lead to a worst-case scenario, Lau’s team works with the Microsoft Azure Sentinel product group to test and pilot the new security monitoring system, which includes several time-saving and modern solutions that empower security analysts to connect to and query datasets quickly and easily.
“Ingesting data into our legacy SIEM took hours,” Lau says. “In Sentinel, it takes around 10 minutes, which is 18 times faster.”
Now, they’re ready to deploy the cloud-based version of SIEM throughout Microsoft’s internal Security Operation Centers (SOC). In partnering with Microsoft Digital, which provides enterprise IT capabilities across Microsoft (including security), the Microsoft Azure Sentinel team introduced several time-saving and modern solutions that empower security analysts to connect and query datasets quickly and easily. Best of all, they’re using the power of cloud computing at scale.
[Learn how Microsoft is moving to next-generation SIEM with Azure Sentinel. Discover how Microsoft protects its network with Zero Trust. Find out how Microsoft uses elevated-privilege accounts for security.]
Getting it right with the right partners
The Microsoft Azure Sentinel product team tapped the expertise of the company’s internal security team in Microsoft Digital for insights about how to improve the product. Their input helped shape Microsoft Azure Sentinel into a SIEM that dramatically improved how efficiently it responds to threats.
If we can help them be successful, we’re also helping our large customers, who often have the same challenges, requirements, and needs.
– Laura Machado de Wright, principal PM manager, Microsoft Azure Sentinel product team
“Azure Sentinel uses all the automation and scalability capabilities available in the Azure platform,” Lau says.
Microsoft Digital’s engagement with the Microsoft Azure Sentinel team addressed two sets of needs at once.
“They get the benefits of Azure Sentinel for incident response, but we get the benefit as the product team of working with customers, like our own internal digital security team,” says Laura Machado de Wright, a principal PM manager on the Microsoft Azure Sentinel product team. “If we can help them be successful, we’re also helping our large customers, who often have the same challenges, requirements, and needs.”
The collaboration meant the product team could identify what enterprise-scale customers were looking for at a faster rate.
“We can work closely and iterate more rapidly with internal teams,” Machado de Wright says. “We can get their requirements and feedback before moving into formal previews with external customers.”
These early interactions allowed the product team to work through a few nuances that could have disrupted users. In an early version of Microsoft Azure Sentinel, for example, some of Microsoft Digital’s security analysts noticed that they were getting a lot of long notifications.
“When you start testing, you realize you need certain capabilities,” Lau says. “We were able to point out the business impact of noisy alerts that are too long.”
In response, the product team introduced suppression and aggregation support to avoid alert fatigue, reducing the amount of noise generated by Microsoft Azure Sentinel.
“Now we have a better product that meets our needs at an enterprise level,” Lau says.
Always a group effort
One objective of Microsoft Digital is to unify security operations teams onto a single SIEM—Microsoft Azure Sentinel. “Depending on the scope, there are different teams responsible for protecting Microsoft,” Machado de Wright says. “There are some common solutions between them, but many security operations teams built their own solutions or relied on third-party solutions to manage security events. With Azure Sentinel, we think there’s an opportunity for them to be the first and best customers of Microsoft.”
With Microsoft Azure Sentinel, it’s easier for SOCs to develop a tactical and coordinated response to security threats and incidents.
“Even though they might look at different pieces of the puzzle, data from different internal teams can be brought into Azure Sentinel and create detections,” Machado de Wright says. “Then, automation can assign it to the right group.”
These multiple sources can be connected for rich, multifactor detections.
“Multifactor allows us to grab from multiple sources and compare them together,” Lau says. “We can see if someone is attacking us in several different ways. Between detection and hunt, it’s very simple to track down what’s happening.”
Unifying security operations teams onto the Microsoft Azure Sentinel platform also allowed the company’s internal security team in Microsoft Digital to align on a deployment strategy.
“It was great to work with other SOCs within Microsoft,” Lau says. “We have the shared goal of protecting the entire enterprise, which enabled us to identify key requirements for parity to retire the legacy SIEM.”
Steps had already been taken to retire the legacy SIEM, so deploying Microsoft Azure Sentinel in a timely manner was critical.
To move to Azure Sentinel, the product team needed to verify that equivalent features and capabilities were live in the new security environment. Making sure the various teams’ needs were aligned helped ensure that.
“Some of these teams had fairly mature monitoring systems,” Machado de Wright says. “We had to work on prioritization and work closely to understand their scenarios to meet the requirements of their timeline.”
To build new detection systems, you need connected data sources. But first, you have to find each source and connect it to your analytics engine.
“Before, you had to understand how the data was structured and then build software to connect to your events management system,” Lau says. “Sentinel’s broad ecosystem allows many out-of-the-box data connectors to be connected up to 18 times faster.”
This is one of the major ways Microsoft Azure Sentinel accelerates and empowers engineers and analysts.
“Finding access to data can be ponderous across large volumes of data,” Lau says. “When security analysts go in and perform open-ended queries to find access to data in the repository, Azure Sentinel is extremely fast.”
Now tracking down a new connector or data source in Microsoft Azure Sentinel takes just a few seconds. This free time has allowed the security team in Microsoft Digital to reprioritize engineering resources previously dedicated to scaling the infrastructure. Plus, the time-saving automations introduced with Microsoft Azure Sentinel have improved the lives of Microsoft Digital’s SOC analysts.
Some of these time savings manifest in how quickly code can be written and deployed.
“It all happens at the speed of pushing code to the cloud,” Lau says. “So, a matter of minutes.”
This streamlined process gives Microsoft Digital much better change control, enabling a continuous integration and continuous detection pipeline.
Transforming the future of security
Microsoft Digital isn’t the only group benefiting from Microsoft Azure Sentinel.
During development, Microsoft Digital and the Microsoft Azure Sentinel product team also solicited input from other enterprise customers. These partners, including a global retailer that experiences more than 9 billion security events per day, helped shape the final product.
“Sometimes we get conflicting feedback from customers,” Machado de Wright says. “We can’t always address it, but we can dive deeper by asking the internal team if they have the same pain point or scenario.”
Thanks to the contributions of Microsoft Digital and its partners, the Microsoft Azure Sentinel team has quickly developed and released a product that can handle the scale and security needs of modern enterprises.
“We have access to different personas, like analysts, engineers, managers, and different security operations teams,” Machado de Wright says. “The ability to just sit with them accelerated everything.”
And there’s still more to discover with Microsoft Azure Sentinel.
For example, with new ways to engage and interact with connected datasets, Microsoft Digital is now using machine learning with the new tool. “We are moving some of our most complex detections into Azure Sentinel,” Lau says.
For enterprise customers like Microsoft who already have the Microsoft Azure stack, using cloud-based security tools made a lot of sense.
“We’re already using Azure,” Lau says. “Now we have a better product that meets our security needs at an enterprise level. Our security operations teams don’t need to leave Sentinel. They can query different Azure Data Explorer clusters and other workspaces with permission. It’s a single pane of glass to complete an investigation.”