AT&T creates digital coworkers with Azure to scale AI that works

Every day at AT&T, scale drives the challenge. Millions of customers depend on AT&T’s services. Thousands of engineers monitor networks. Developers maintain business-critical systems. That same scale created a unique challenge: employees were navigating layers of complexity. In a company where seconds of downtime can affect millions, AT&T saw AI not as a tool, but as a necessity to keep pace with its own scale.

Employees were spending too much time investigating across many search links to find an answer. Developers spent hours debugging code, line by line. In the middle of it all, data flowed faster than humans could keep up. That imbalance between data velocity and human capacity became the core challenge—one that AI needed to solve.

AI promised relief, but leadership needed proof. Could AI handle sensitive data at enterprise scale? Would it meet compliance standards, and could it deliver measurable impact?

From pilots to platform

For AT&T, pilots were never the finish line—they were the proving ground. Early wins showed potential, but that wasn’t enough. AT&T needed transformation. It needed AI that could move from isolated demos to enterprise-grade operations.

In 2023, AT&T made that leap with Ask AT&T, a generative AI platform built on Microsoft Azure, turning lessons from the lab into large-scale impact. It wasn’t another pilot. It was AI in production that set up a foundation for even more tools and generative AI solutions.

AT&T chose Azure for the fundamentals that matter most to a global enterprise: security, scalability, interoperability, and control. These principles guided every architectural decision, from data flow to agent orchestration.

Behind the interface is a data architecture purpose-built for reliability at enterprise scale. What began as small-scale automation experiments evolved into a cohesive, multi-agent system designed for enterprise-grade reliability. At the core was an agentic framework, coordinated with Microsoft Foundry and Azure OpenAI in Foundry Models to orchestrate specialized AI agents that work together in near real time.

These agents collaborate like teammates: one retrieves verified content through Azure AI Search, another summarizes the customer’s current and historical context, and a third automatically updates call notes. Together, these solutions deliver instant, accurate answers tailored to the customer’s situation—all within AT&T’s highly secure cloud environment.

“We chose Azure Cosmos DB because it is optimized for high-performance chat and agent apps, super-responsive user experiences, and scalability,” says Mark Austin, Vice President of Data Science at AT&T.

The data layer that supports these interactions is equally sophisticated. Azure DocumentDB stores long-term chat history and master configurations, providing the backbone for Azure AI Search and other applications with high availability and disaster recovery. Azure Cache for Redis keeps interactions responsive in real time. Azure Storage holds long-term records for governance and analytics.

Together, these choices created more than a platform—they built a foundation for AI that could evolve.

What sets AT&T apart is that it didn’t stop at pilots. AT&T built a living AI ecosystem where agents collaborate, scale, and evolve as the business grows, ultimately delivering more than double year-over-year ROI. To deploy and manage this system with consistency, AT&T uses Azure Kubernetes Service (AKS) for orchestration and Azure API Management for governance and monitoring of containerized agents across the enterprise. This approach helps ensure that every AI interaction remains compliant, traceable, and performance-optimized. With the foundation in place, AT&T’s attention turned to scale and stewardship.

Governance and safeguards

With the technical foundation in place, AT&T turned its focus to governance and scale.

“With Microsoft, we’ve deployed 71 unique generative AI solutions serving over 100,000 employees—every one of them governed, trusted, and delivering ROI,” adds Austin.

Every new AI agent went through AT&T’s legal, security, and finance reviews before launch. The built-in guardrails in Azure, such as Azure AI Content Safety filters, data governance policies, and model protection controls, kept the system highly secure and transparent at every stage.

To maintain visibility across such a large footprint, AT&T is using Azure Monitor and Log Analytics with OpenTelemetry-compliant Arize AI. Using this combination, engineers can trace activity, monitor agent performance, and visualize data flow across the platform in real time. Every decision, response, and outcome can be audited, giving teams confidence that AI behaves as expected. AT&T’s implementation is more than a solution. It’s a repeatable pattern for how enterprises can operationalize generative AI with governance intact.

Executives also pushed for scalability without complexity. Skills developed for one workflow could be redeployed to another, saving months of development time. AT&T’s AI stack spans applications, data, and infrastructure, providing an adaptable, scalable, and governed foundation for future innovation.

How it works

In less than two years, AI agents have become part of daily life at AT&T.
 

The next chapter

AT&T’s leaders know this is only the beginning. The company plans to reuse and scale its AI framework across new business units, with Ask AT&T as the unified platform for enterprise AI.

AT&T is redefining what enterprise AI looks like in practice. As one of the first frontier firms to scale agentic AI across thousands of users, it has set a new benchmark for innovation, reliability, and responsible transformation.

AT&T didn’t just adopt AI. It industrialized it.

Discover more about AT&T on Facebook, Instagram, LinkedIn, and X/Twitter.