KPMG adds model security to its AI app lifecycle with Microsoft Defender for Cloud

KPMG needed seamless data governance and protection across multiple member tenants and the company’s SaaS subscriptions, to safeguard both data and enterprise-built AI apps across their lifecycles.

KPMG deployed Microsoft Defender for Cloud and Microsoft Purview to help ensure proper controls for its Azure OpenAI Service apps and to protect and govern content being migrated into M365 and AI products, like Microsoft 365 Copilot.

KPMG enhanced security controls, visibility, and data governance for its AI-driven workflows. It automated data loss prevention, freeing up time for critical security use cases. The firm prevents security incidents before they occur.

Data protection and governance in the age of generative AI

KPMG is a “Big Four” financial audit, tax, and advisory firm headquartered in New York. Approximately 40,000 employees, 23,000 vendors, and 522,000 guests use or visit the firm’s 95 physical locations across the United States, Israel, Mexico, and Latin America.

KPMG LLP professionals use generative AI tools daily in software as a service (SaaS) and custom applications. While generative AI helps surface data that employees might not otherwise find and improves their ability to analyze it, KPMG had to prepare for safe adoption.

KPMG was concerned about the top 10 new security threats arising from generative AI applications and Large Language Models (LLMs) as identified by the Open Worldwide Application Security Project (OWASP), such as prompt injections, insecure output handling, data poisoning, and sensitive information disclosure. This made safeguarding sensitive data and implementing strong access controls and encryption paramount. To prevent attacks that could manipulate model outputs, the KPMG cybersecurity team considered model security.

The firm also had to help ensure its model training data is accurate, complete, and representative, to guard against data poisoning attacks. As part of its approach, KPMG created a Responsible AI framework to address core issues, including data retention and data privacy (how AI prompt data is collected, stored, and used, to prevent sensitive data from being included in training data).

The value of solution integration

KPMG decided on Microsoft products. “The KPMG complex, multi-firm structure presents unique challenges that demand innovative solutions. Collaborating with Microsoft, we've prototyped features to deploy generative AI solutions like Microsoft Security Copilot. This strategic alliance meets our specific needs and broadens security visibility beyond endpoint, cloud, and identity through extensive use of Microsoft Security solutions,” says Zack Yarmolovich, Cyber Ops Senior Director at KPMG.

KPMG relies on Microsoft Defender for Cloud to help establish baselines to detect configuration and/or model drift. Defender for Cloud offers model security through AI detections and Cloud Security Posture Management (CSPM) capabilities. Defender is also designed to help protect the LLM in Azure OpenAI Service and will eventually cover all vulnerabilities outlined in the OWASP Top 10 for LLM applications. Defender already addresses 8 of the 10 risks, including prompt injections, insecure output handling, training data poisoning, sensitive information disclosure, and insecure plugin design. The Microsoft Purview suite addresses a Presidential Executive Order regarding content and traceability on AI materials, for extra peace of mind.

The firm uses Microsoft Purview Information Protection to apply sensitivity labels for a “secure by default” approach, to address data oversharing. Sensitivity labels are the basis for Purview Data Loss Prevention policies for Copilot interactions. “Microsoft gave us a blueprint for using Microsoft Purview to identify what data is sensitive, classify it, and identify risks to that data,” says Greg Schellenberg, Director of Cybersecurity at KPMG.

Microsoft Purview Data Security Posture Management for AI, a feature of Microsoft Purview, provides centralized visibility into AI interactions and analytics, as well as a single place to view and manage policies to secure and govern AI. These insights come via the Microsoft Purview browser extension built into Microsoft Edge.

Choosing Microsoft security tools has also saved the company money by reducing the need for multiple licenses, simplifying management overhead and providing a unified security platform with integrated features. This ultimately lowers the total cost of ownership.

“Microsoft products provide the necessary protection controls and visibility into how our business teams are using a common trust framework. This allows our lines of business, whether Audit, Tax, or Advisory, to develop an AI approach with security built in rather than developing new security controls each time.”

Greg Schellenberg, Director of Cyber Security, KPMG

Extensive data controls for governance and security

KPMG uses Microsoft Purview Data Security Posture Management (DSPM) to see a detailed inventory of its AI models and services, get proactive insights on generative AI vulnerabilities, and view potential attack paths to generative AI applications, to better manage security. “DSPM for AI is the reporting console where we see all our policies and controls, along with how data across the organization has been classified. It shows us the data types of sensitive information, locations, and levels of risk. This is how we discover generative AI activity, including sensitive data usage and total users using AI. It’s a one-stop shop portal to report on detection of business risks and regulatory violations,” says Schellenberg.

With Purview, KPMG discovers, classifies, and manages data while helping ensure compliance with industry regulations. KPMG takes advantage of DSPM for AI to get visibility into sensitive data that could be overshared with AI applications, and to mitigate and manage use risks for Microsoft 365 Copilot and other generative AI applications.

The firm uses Microsoft Purview to improve data discovery and classification by automatically scanning and classifying data across multiple tenants, making it easier for KPMG to apply security policies. “Microsoft Purview streamlines data discovery. This helps us generate reports for internal audits and client assurances. Its detailed audit logging enhances our ability to track data access and usage across a complex environment,” says Schellenberg. “Out of the box, Purview uses sensitivity information tags to classify data, which gives us the ability to surface what the data’s being used for.”

The firm protects its IT workflows with Microsoft Security Copilot, which is embedded across Microsoft Security tools, including Microsoft Defender and Microsoft Purview. The firm’s cybersecurity team uses Microsoft Defender for Cloud to monitor and secure Azure OpenAI Service apps, detecting threats throughout the application lifecycle.

“Purview and Defender help us prevent incidents before they occur. That's the name of the game in security: resolving problems before they affect our business.”

Greg Schellenberg, Director of Cyber Security, KPMG

Reducing risks while increasing safeguards

By taking a platform approach to AI security, protection, and response, KPMG benefits from a scalable standard that meets its diverse generative AI needs. “Microsoft products provide the necessary protection controls and visibility into how our business teams are using a common trust framework. This allows our lines of business, whether Audit, Tax, or Advisory, to develop an AI approach with security built in rather than developing new security controls each time,” says Schellenberg.

With CSPM in Microsoft Defender for Cloud, KPMG developers can correct AI misconfigurations that could result in data loss. When developers are building AI models for apps, they use CSPM settings in the cloud environment to help ensure they align with policies and procedures that support safe deployment of AI. Defender for Cloud also has real-time monitoring to detect and respond to threats to cloud resources, which works in concert with Purview data security policies. The firm is also on track to address all 10 OWASP vulnerabilities in the future.

KPMG runs Microsoft Defender for Endpoint agents on developers’ laptops, sending signals back to Defender for Cloud. This way, as developers enter prompts into AI models, Defender for Cloud monitors for security risks, like jailbreak attempts. This information feeds into Microsoft Sentinel, which analyzes security events and either performs an automated remediation or alerts KPMG security teams to take action if there is a threat.

The firm uses Defender for Cloud connectors to third-party cloud service providers—and connectors built into Azure services—to monitor threat signals in real time. “Microsoft Purview data loss prevention features and integration with Microsoft tools frees us up to do critical work around security use cases instead of trying to get the tool to work across tenants,” says Schellenberg. “Purview and Defender help us prevent incidents before they occur. That's the name of the game in security: resolving problems before they affect our business.”

Discover more about KPMG on Facebook, Instagram, LinkedIn, X/Twitter, and YouTube.