Zero Trust for AI: Rebuilding security controls for autonomous and agentic systems

AI is no longer just answering questions or making recommendations. It is beginning to take action at the direction of users through autonomous agents, whether they're SOC analysts, researchers, developers, or threat actors. More than 80% of Fortune 500 companies are already using agents built with Microsoft Copilot or Agent Builder.¹ Active agents across Microsoft 365 have grown 15x year over year, with that number rising to 18x in large enterprises.² Agents are quickly becoming a part of how we work and moving into real business processes faster than security controls can adapt. The pace of adoption is creating a gap between what organizations can deploy and what they can confidently govern.

As organizations adopt AI agents, autonomous workflows, and AI-powered development tools, security teams are being asked to protect AI systems that can make decisions, access resources, and complete tasks with limited or no human oversight. Securing these systems requires continuously verifying access, enforcing least privilege, and evaluating risk through a Zero Trust approach.

This overall shift is forcing security professionals to rethink some long held assumptions. What does least privilege mean when software can decide what to do next? How do you govern the information an AI system remembers and relies on? How do you know an agent is acting within the authority it was given? How do you apply Zero Trust to systems that can take action without waiting for human approval at every step? Applying Zero Trust to AI means extending these principles to systems that can make decisions, access resources, and take action on behalf of users through continuous verification, appropriate permissions, and ongoing evaluation of risk.

These questions led us to write the book, Zero Trust for AI: Rebuilding Security Controls for Autonomous and Agentic Systems. It explores how Zero Trust principles apply to the realities of cybersecurity in the AI era, covering topics including AI agents, secure software development, AI memory, identity, and access control. It provides practical guidance for securing AI systems as they move from experimentation into production.

When the human checkpoints disappear

For years, enterprise AI behaved like an assistant. It answered questions, summarized meetings, drafted messages, and suggested code, but it stopped short of doing anything on its own. A person still clicked send, approved the change, or granted the exception. That pause did more than most people realized. It was where review, accountability, judgment, and many security controls naturally fit into the process.

Agentic AI changes that model. Instead of suggesting the next step, agents can now determine how to best achieve a goal without following a straight path. They call APIs, retrieve data, write to memory, hand work to other agents, and perform tasks using identities and credentials that carry real authority inside the organization. The productivity gains are real, with 58% of AI users saying that they are producing work they would not have been able to complete a year ago, climbing to 80% among advanced Frontier Professionals.²

However, once that human checkpoint disappears, some familiar security assumptions begin to break. Security teams can no longer rely on human review as the primary control point; controls must follow the agent, its identity, and the actions it is authorized to perform. An agent can complete its work exactly as expected while still acting on bad instructions. Poisoned memory can look like useful context. A tool with more permissions than it needs can perform actions that appear completely legitimate. The system still works but the safeguards many organizations depend on are no longer sufficient. A safeguard that holds 80% of the time still leaves one in five actions at risk when an autonomous agent is operating with real credentials.³

The assumptions that no longer hold

The challenge isn't that AI introduces entirely new security problems. It's that it undermines many of the assumptions existing security controls were built around.

In Zero Trust for AI, we take a closer look at three assumptions that have shaped enterprise security for decades and explain why they no longer hold in the age of AI agents.

A person is always in the loop: Approval workflows, separation of duties, and change reviews all assume someone pauses before something important happens. AI agents don't necessarily pause. They continue from one step to the next until the work is finished.

Instructions come from trusted sources: Agents consume information from documents, emails, web pages, APIs, and other systems. If any of those inputs are manipulated, the agent may faithfully follow instructions it was never supposed to receive.

A threat actor has one chance to succeed: Persistent memory changes that equation. Instead of relying on a single interaction, a threat actor may influence future decisions by planting information that remains long after the original event has passed.

Together, these three assumptions point to the same conclusion: organizations need a security model built for systems that can act on their own.

That's the purpose of Zero Trust for AI. Rather than treating AI as just another application, the book shows how Zero Trust principles can be applied to AI identities, agent permissions, memory, software development, and autonomous workflows so organizations can adopt AI without giving up the security principles they already trust.

The same capabilities can strengthen defense

There is another side to this shift. The same capabilities that make agents powerful also create new opportunities for security teams. AI-powered security tools can analyze signals, identify threats, and respond at a speed and scale that human teams cannot match alone.

In one recent case, Microsoft Defender's automatic attack disruption shut down a live ransomware operation by isolating the affected device and cutting off the attack in under 129 seconds, which was far faster than a response team could typically assemble manually. The opportunity is not to slow down AI adoption, but to apply the same principles of identity, least privilege, and governance to defensive AI capabilities.

Inside Zero Trust for AI

Drawing on Microsoft's own experience securing AI across engineering, identity, data, and operations, it brings together real-world implementations, research, and lessons learned to provide a practical view of what Zero Trust means when software can plan, remember, and act. One of the book's central themes is that when these controls fail, they often don't fail loudly. The agent keeps working.

The book explores four shifts security leaders need to understand:

Zero Trust principles must extend to agentic AI itself. Security can no longer stop at users, devices, or applications. It also has to account for each agent's identity, permissions, tools, memory, and actions.

The safest agent isn't the one with the best prompt. It's the one whose access, credentials, and ability to act are scoped so tightly that failures remain contained.

Memory becomes the new security boundary. Once information persists, it can influence future actions, making memory something that must be governed and protected, not simply treated as a feature.

Speed demands more control, not less. The same capabilities that help teams move faster can also increase the impact of a mistake or manipulation if the right controls are not in place.

It all comes down to one question: if one of your AI agents were turned against its intended purpose today, how far could it go before security controls stopped it?

Applying Zero Trust in the age of AI agents

Zero Trust remains foundational in the agentic era, but it must extend to the agents themselves. Zero Trust for AI helps security leaders and technology teams understand how AI changes security requirements and apply Zero Trust principles to agent identities, permissions, memory, and actions. Read the eBook to explore practical guidance for securing AI systems and building a path forward as AI moves from experimentation into production.

For additional insights, read our latest blog on Zero Trust assessments and Developer Security updates, including how Microsoft is helping organizations address the evolving security challenges introduced by AI-powered development tools, agents, and autonomous workflows.

Read the e-book