Datadog enhances AI application monitoring with Microsoft

Organizations are adapting cutting-edge technology to support their growing customer base. With AI at the forefront, there are more innovative ways to build, use, and monitor AI-based applications to scale their businesses. Once these apps are built, organizations depend on the AI solution to continuously run smoothly so that they can optimize costs and find new ways to help customers. However, these applications demand routine maintenance to perform business-critical tasks, which can take time away from focusing on the customer’s needs. One organization created an integration with Microsoft Azure OpenAI Service to solve this problem to alleviate mundane work processes, like performance monitoring and compute optimization, to focus on product development that drives the future forward.

Building without optimization

Datadog is a cloud-scale observability solution for hybrid and multicloud organizations that leverage Microsoft Azure. The company provides security and monitoring solutions for cloud-based applications to help DevSecOps and Business teams to deliver responsive, resilient, and secure digital experiences.

In 2015, Datadog released its first monitoring solution for Azure, beginning with Virtual Machines. Then it brought it's observability solution natively into the Azure portal to embed Datadog natively into customers’ Azure workflows. Today, Datadog has more than 600 out-of-the-box integrations that are all built, managed, and maintained in-house.

Since the Microsoft partnership continues to grow and hundreds of Datadog’s customers leverage Azure OpenAI Service to scale their businesses, building its new integration on Azure made sense.

Datadog’s customers use Azure OpenAI Service to help drive innovation, but they wanted to speed up the process of monitoring its solutions to create more efficient operations. So Datadog created an out-of-the-box Azure OpenAI Service integration to ease this workflow across five areas of focus:

Fast, low-friction onboarding and configuration to ensure enterprise customers can get set up, started, and realize value right away when onboarding to Datadog’s observability platform
Comprehensive monitoring for cloud-native workloads built on Azure, including support for key Azure services when building generative AI applications
Comprehensive monitoring for Azure migrations and hybrid workloads so organizations can access the information they need to accelerate their cloud adoption journeys using the Microsoft Cloud Adoption Framework for Azure
Innovative new capabilities for monitoring Azure to ensure always-on access to the latest capabilities for Azure customers
Best-in-class quality of service to continue reducing Datadog’s costs to customers and ensuring reliability and efficiency of their infrastructure, services, and applications.

Monitored and secured

Datadog integrates with all major Azure services, including Azure OpenAI Service. Now customers can better optimize costs, troubleshoot issues, and monitor the performance of their AI-powered applications. The integration requires no additional setup, and it provides the same comprehensive visibility for Azure OpenAI Service, enabling its customers to secure instant visualizations.

With this integration, customers can track token consumption by prompt and completion tokens that help organizations understand the primary cost drivers for their Azure OpenAI Service usage.

Customers can also use the built-in integration dashboard for an overview of performance across all instances, along with usage trends for Azure OpenAI Service. Also, Datadog automatically collects performance data that is unique to Azure OpenAI Service instances. Unique organization IDs can be assigned to individual teams, enabling customers to track where and to what extent AI models are used in their organization.

Empowering Fortune 50 organizations

Since this solution launched at Microsoft Build 2023 in late May, hundreds of organizations, from Fortune 50 to large global multinational companies, have started using Datadog to monitor their AI applications. Now enterprise-scale companies can build, track, and monitor analytics. The out-of-the-box solution helps enterprises optimize costs, troubleshoot issues, and monitor performance of their AI-powered applications while giving valuable time back to development teams to focus on building customer-centric products.

With the help of the integration, Datadog’s customers can connect to Microsoft Azure to:

Get metrics from Azure Virtual Machines with or without installing the Datadog Agent
Collect standard Azure Monitor metrics for all Azure services
Tag Azure metrics with Azure-specific information about the associated resource, such as region, resource group, and custom Azure tags
Get Datadog-generated metrics to provide unique insights into their Azure environment
Correlate data from their Azure applications across logs, metrics, APM tracing, user activity, and more within their Datadog organization

Datadog believes the ongoing partnership is beneficial not only for itself but for its customers as well. From infrastructure to applications to the logs emitted by various Azure services, its engineering, product, and go-to-market partnerships with Microsoft have been critical to ensuring Datadog can continue to deliver best-in-breed innovation and observability capabilities to Azure customers.