The AI Revolution: How Microsoft Digital (IT) is Responding with an AI Center of Excellence

The AI revolution is here, so what are you going to do about it?

This question is for IT leaders, IT practitioners, and others out there who need to decide how your companies will respond to the onslaught of AI products and solutions that are coming your way.

We had the same question here at Microsoft, and to make sure we responded in the right way, we—Microsoft Digital, the company’s IT organization—created an AI Center of Excellence (CoE) to guide us.

Here’s the story of how we did that and how our CoE is now helping us navigate the AI revolution and figure out how to deploy it internally across Microsoft.

Evaluating AI for Microsoft

For us, it started with evaluating what our people want from AI.

Next-generation AI is transformative, and as it does for all enterprises, it presents a huge opportunity for us at Microsoft. One of the fundamental steps our CoE is taking is to accept this and not get in the way. We’re encouraging a culture of disruption while also living up to our obligation to do so responsibly.

Getting AI right is about empowering your people to do their best work. We’re off to a good start—now that we’re underway, we’re laser focused on making sure everything that we do empowers our employees be their best, most creative, selves while also protecting them and the company as well.

—Rajamma Krishnamurthy, principal program manager architect, MSD

We know that integrating AI into everything we do will never be a matter of stitching AI features or capabilities into our existing systems and processes, but rather a process of reexamining how we do things. We want it to do three important things—amplify human ingenuity, deliver transformative experiences, and safeguard our people, business, and data.

Our CoE encourages our teams to think about how AI can help their work and to rethink their work with AI in mind. To respond adequately to this AI revolution, we took a holistic view of how each of our employees can achieve their full potential and how each team, department, and the entire organization can benefit from AI.

“Getting AI right is about empowering your people to do their best work,” says Rajamma Krishnamurthy, a principal program manager architect in MSD and one of the leaders of our CoE. “We’re off to a good start—now that we’re underway, we’re laser focused on making sure everything that we do empowers our employees be their best, most creative, selves while also protecting them and the company as well.”

Meeting needs and answering important questions

We’re using feedback from our employees and leaders to guide how we invest in AI.

Our employees are telling us they want to simplify and offload mundane tasks and focus on productive, creative work. They view AI as a tool to find information and answers, summarize meetings and action items, perform administrative tasks, and plan their day. However, their focus is on more than administration and tedium. Employees also want to use AI to improve and inform their creative work and enable them to produce deeper and more insightful analytical work.

Drilling down, we asked our employees for specifics on what they want out of AI to improve the experience they have at work.

What employees want from AI: Find info and answers, summarize actions, analytical work, admin tasks, creative work, plan their day. — According to Microsoft’s 2023 Work Trend Index Annual Report, employees want AI to give them time back to be more strategic and creative.

Our leaders want AI to empower employees, not replace them.

Leaders want AI to create an environment for employees that increases their productivity, improves their well-being, decreases the time they spend on low-value activities, and improves their skills.

“We want to empower employees to find time for more innovative and rewarding work,” Krishnamurthy says.

What leaders want from AI, highlighted by increased productivity, reducing mundane tasks, improving wellbeing, and eliminating time spent on low-value activities. — Leaders see AI making their employees more productive, not replacing them, according to Microsoft’s 2023 Work Trend Index Annual Report. Amid fears of AI job losses, the report found that business leaders are two times more likely to choose ‘increasing employee productivity’ than ‘reducing headcount’ when asked what they would most value about AI in the workplace.

Transforming Microsoft with the AI Center of Excellence

Now that we’ve shared how our CoE is listening to our employees and leaders, we’ll share more detail on the CoE itself.

Our AI CoE team is comprised of experts across Microsoft in various fields, including data science, machine learning, business intelligence development, product development, experience design and research, accessibility, and program management.

Working under the AI 4 ALL (Accelerate, Learn, and Land) tagline, the team is responsible for planning, designing, implementing, and championing how we use AI internally at Microsoft.

Our CoE uses these pillars to guide their work:

Strategy. They work with product and feature teams to determine what we want to achieve with AI. They define business goals and prioritize the most important implementations and investments.
Architecture. They enable infrastructure, data, services, security, privacy, scalability, accessibility, and interoperability for all our AI use cases.
Roadmap. They build and manage implementation plans for all our AI projects, including tools, technologies, responsibilities, targets, and performance measurement.
Culture. They foster collaboration, innovation, education, and responsible AI among our stakeholders.

“These pillars are helping us stay focused on the right things,” Krishnamurthy says. “It’s about using AI to grow and nourish a culture of innovation and excellence across the company.”

Strategy

Our CoE Strategy team evaluates what we’re doing with AI at Microsoft. The most fundamental perspective for the strategy pillar is examining AI as a catalyst for transforming our tools and processes, not as an addition or augmentation to existing tools and processes. While AI is designed to augment and improve human capabilities, it can’t be approached as only an augmentation or improvement to the tools and processes we use. We must be willing to start over if that achieves the best outcome for our employees and business.

From the beginning, the strategy team examined the projects and business goals through positive disruption—a willingness to refine each idea to its core. To capture the full value of AI in our organization, we knocked down boundaries. We examined every one of our business processes, reimagining them and how AI could improve them, often in revolutionary ways.

The pace of AI technology is incredibly fast. We’re moving into implementations quickly to capture value and stay relevant to developments in AI technology. Our digital transformation has made this possible in many ways. At the same time, governance and control have to be at the forefront of our strategy and consider and respect responsible AI tenets in everything we do.

—David Finney, director of IT Service Management

Our strategy is driven from the organization’s top level, and executive sponsorship is crucial to executing our implementation well. When our transformation mandate comes from the organization’s leader, it resonates in every corner of the organization, every piece of work, and every task that could be changed. Simultaneously, we have encouraged and welcomed ideas from every level of the organization, empowering individuals from across our organization to contribute their AI insights.

We’re moving quickly, thanks to our digital transformation. David Finney, director of IT Service Management, is excited about the rapid progress the CoE is making.

“The pace of AI technology is incredibly fast,” Finney says. “We’re moving into implementations quickly to capture value and stay relevant to developments in AI technology. Our digital transformation has made this possible in many ways. At the same time, governance and control have to be at the forefront of our strategy and consider and respect responsible AI tenets in everything we do.”

Capturing strategy with an idea pipeline

The strategy pillar captures all the ideas and all the ongoing work that’s happening within AI at Microsoft. Idea capture involves the entire organization, and every employee is invited to contribute ideas for how AI can transform how we work, from the most straightforward task to the broadest organizational policies and processes. No element of our business processes is off-limits. Our pipeline contains ideas for the next year or so range from AI-powered career planning, to intelligent helpdesk and troubleshooting tools, to fully automated issue detection and remediation, to AI-powered codebase migration.

One of the CoE Strategy team’s most significant responsibilities is prioritizing the idea pipeline for AI solutions. All employees can feed the pipeline through a form that records important pipeline details. The strategy team evaluates each idea in the pipeline, analyzing two primary metrics: business value and implementation effort.

Business value. How important is the solution to our business? Potential cost reduction, market opportunity, and user impact all factor into business value. As our business value increases, so does the idea’s position in the pipeline priority queue.
Implementation effort. How much effort is required to implement the idea? We evaluate the implementation effort based on data gaps for modeling, the complexity of the solution, and the resources required. Low-effort ideas can result in quick wins, while ideas requiring more significant effort must be further evaluated.

Chart showing how Microsoft will activate AI by focusing on a mix of quick wins and long-term projects. — We’re well positioned to move quickly on deploying AI internally at Microsoft while also working on major projects that will transform the way we provide IT services to the company.

Business value and implementation effort supply the two axes for the simple four-quadrant matrix we use to determine the overall priority for the ideas and their order in the pipeline. High-value, low-effort ideas are at the start of the pipeline and our highest priority. Low-value, high-effort projects are sent to the back of the pipeline. High-value ideas are all essential and deserve focus, and our strategy team used this evaluation matrix to determine which projects should be started and when.

Architecture

Our architecture pillar focuses on the readiness and design of the infrastructure and services that support AI at Microsoft. It also encompasses data readiness and the reusability of enterprise assets used for AI capabilities.

The CoE’s Architecture team manages these systems’ supporting infrastructure and ensures that our environment adheres to best practices for standards and governance. Architecture dependencies and interactions are critical to establishing sound architecture practices.

When a product team is developing a service within the architecture, like storage, compute, or an API, decisions about design and architecture are influenced by the dependencies across these services.

The architecture we build is focused on open and liberal architecture standards. We know that if our engineers and developers use the tools they’re most comfortable and fluent with, they’ll be able to create quickly with competency and confidence. With more than hundreds of potential projects in the pipeline, rapid iteration and agility are critical.

We’ve made our teams aware of AI playgrounds and aggregators that they can use to explore supported AI tools and machine-learning models to test their scenarios and validate data-handling practices. Standardizing these playgrounds and aggregators provides freedom for our developers to experiment and innovate while staying within the bounds of our AI best practices and approved technologies.

We ask questions. What are the touch points? What is the impact? How do we achieve cost and performance balance? Where are we going to invest? Which of these services is going to get the capabilities? How are we organizing services? What platform-level capability will all services use, and what will be native to a particular area or service within a smaller group?

—Faisal Nasir, principal architect, MSD

We’re also enabling our architecture communities to collaborate and share ideas about developing the most optimal microservice architecture, cloud service-based architecture, or hybrid infrastructure architecture. We have presentations, on-demand engagements, and groups that use Microsoft Viva and Microsoft Teams to encourage and facilitate collaboration. This enables our architecture teams to move quickly and in concert with each other, which is necessary with the rapid pace of AI technology advancement.

A comprehensive architectural view involves understanding the nuances of the infrastructure and how new infrastructure will affect the current architectural state. We achieve this view by gathering data on systems and dependencies across the existing architecture and super-imposing new ideas or new architecture on top of it.

“We ask questions,” says Faisal Nasir, a principal architect on the CoE Architecture team. He stresses the importance of continual self-examination. “What are the touch points? What is the impact? How do we achieve cost and performance balance? Where are we going to invest? Which of these services is going to get the capabilities? How are we organizing services? What platform-level capability will all services use, and what will be native to a particular area or service within a smaller group?”

The answers help us make sure we take the right approach.

“Determining dependencies and working through the implications has to be done, and it has to be continually evaluated,” Nasir says.

Roadmap

The CoE Roadmap team examines our employee experience in the context of our AI solutions and governs how we achieve the optimal experience in and throughout AI projects. One of the most critical aspects of implementing AI is how our employees will interact with it. Getting the roadmap right ensures these user experiences are cohesive and align with our broader employee experience goals.

AI isn’t a traditional product, so there isn’t a traditional path for user experience. We’re using AI to level the playing field for all Microsoft employees.

—Aria Fredman, senior user experience researcher, MSD

The Roadmap team leans heavily on research to confirm and test capabilities and the potential for AI-based services and processes. The user experience involves many considerations, including how employees interact with a service, ordinary use case scenarios, accessibility needs, etc.

We’ve recognized AI’s potential to impact how our employees get their work done and what level of satisfaction and positive experience comes from the interactions with AI services and tools. The roadmap pillar is designed to encourage experiences across all these services and tools that are complementary and cohesive.

“AI isn’t a traditional product, so there isn’t a traditional path for user experience,” says Aria Fredman, a senior user experience researcher on the CoE Roadmap team. “We’re using AI to level the playing field for all Microsoft employees.”

AI is an excellent tool for leveling the playing field for everyone.

We’re moving away from the legacy models of interaction and navigation of static product topologies. We’re transforming the user experience through AI and copilot-based experiences and creating new paradigms of interaction and navigation across a complex topology of services and products.

—Yannis Paniaras, principal designer, MSD

“We’re using next-generation AI to transform how everyone interacts with the products we’re building,” Fredman says. “Natural language interfaces and predictive interactions remove the barriers of traditional input and user interface design. Accessibility becomes not something that we build into a user interface but something that the interface natively is. Our goal is universal accessibility, to use AI to empower and include everyone.”

We’re focusing on the open nature of AI interaction. We’re surfacing AI capabilities and information when the user needs them, according to their context. It makes the user experience and user interface for an AI service less important than how the service allows other applications or user interfaces to interact with it and harness its power.

“We’re moving away from the legacy models of interaction and navigation of static product topologies,” says Yannis Paniaras, a principal designer at the Microsoft Digital Studio who collaborates with other designers to create entirely new user experiences with AI and ML. “We’re transforming the user experience through AI and copilot-based experiences and creating new paradigms of interaction and navigation across a complex topology of services and products.”

Collage of portrait photos showing Krishnamurthy, Finney, Fredman, Paniaras, Nasir, Pancholi, Kumar Jain, Awal, Sengar, Avram, and Jaysingh. — The Microsoft Digital AI Center of Excellence (CoE) team includes (top row, L-R) Rajamma Krishnamurthy, David Finney, Aria Fredman, Yannis Paniaras, Faisal Nasir, Nitul Pancholi, (bottom row, L-R) Ajay Kumar Jain, Anupam Awal, Urvi Sengar, Gigel Avram, and Biswa Jaysingh.

It’s about design that simplifies employee workflows for speed and efficiency.

“Our goal is to enable instant point-to-point access to all employee services with minimal or no navigation,” Paniaras says. “It’s like experiencing UX-teleportation, where accessing a service becomes instantaneous.”

This concept is what we call “Just-In-Time UX.” Furthermore, we use AI to facilitate continuous relationships between our employees and the various services that they use, which ensures that the experiences are always on and available to them.

“In our studio, the designers are also reinventing how we design for the era of AI,” Paniaras says. “We are transforming our discipline as much as the experiences and products we create for everyone at Microsoft.”

Culture

Our long-running focus on fostering a culture of innovation within our organization is now helping us embrace this new opportunity with AI. It’s enabling us to empower our employees to learn the skills they need to lead us through this transformation and to help us build a vision for what we can do with it as a group.

AI provides so many potential capabilities, but we must always ask, ‘are we using it in the right way? We’re building governance and guardrails around our systems to ensure we don’t misuse it. Our mandate to use AI responsibly underpins everything we do in this space.

—Urvi Sengar, software engineer, MSD

Our CoE Culture team focuses on two key areas of our AI implementation: responsibility and education. Culture moves into and influences the other pillars more than any other pillar. Culture underpins everything we do in the AI space. Ensuring our employees can increase their AI skillsets and access guidance for using AI responsibly are critical to AI at Microsoft.

AI’s opportunities are immense, and our implementation must be carried out with a growth mindset and responsible approach.

The Culture team has published training, recommended practices, and our shared learnings on next-generation AI capabilities and worked with individual business groups at Microsoft to determine the needs of all the disciplines across the organization, including groups as diverse as engineering, facilities and real estate, human resources, legal, sales, and marketing, among many. A telling example of how we’re rallying around AI is how quickly we created a Data and AI curriculum that everyone in our larger organization can take—employees of all roles are using it learn about and roll AI into their individual work.

We’re weaving responsible AI into the fabric of everything we do with AI, so our employees understand the importance and implications of responsible AI for their work, their teams, and the organization. We’re continually asking questions about our AI practices and evaluating the answers through diverse lenses to ensure our AI capabilities are fair and unbiased.

Urvi Sengar is a leading voice on the cultural team. She highlights the critical role responsible AI plays in developing AI at Microsoft.

“AI provides so many potential capabilities, but we must always ask, ‘are we using it in the right way?’” says Sengar, a software engineer on the CoE and in MSD. “We’re building governance and guardrails around our systems to ensure we don’t misuse it. Our mandate to use AI responsibly underpins everything we do in this space.”

Whatever applications we produce, whatever experiences we create, whatever productivity and efficiency we want to bring to our employees, we always ask the question: “How will this contribute to their engagement and involvement and enable them to thrive within the company.” The answer to this question is found in the moments that matter to our employees, in which they meaningfully contribute to the teams around them and move forward toward our collective vision, from the beginning of their time with Microsoft and all the way through their journey.

—Rajamma Krishnamurthy, principal program manager architect, MSD

Our aim is to weave AI and education together in ways that enhance but don’t overrun our company culture. Our aim is to show our employees how they can transform the work they do while also making sure they protect the social and cultural considerations of the rest of our employees, our partners, our customers, and our larger organization. To do this important work, we’re implementing listening systems throughout our organization that are enabling us to adjust and adapt our approach to make sure we stay focused on the right things.

Moving forward

We’re at the beginning of our journey toward harnessing the transformative power of AI at Microsoft. Our AI CoE will provide the guidance and governance we need to foster innovation and encourage positive disruption in all lines of business. We’re ushering in a new vision for creativity, productivity, and personal growth for each of our employees, and we’re excited to capture those benefits within our organization and share them with our customers.

“Whatever applications we produce, whatever experiences we create, whatever productivity and efficiency we want to bring to our employees, we always ask the question: ‘How will this contribute to their engagement and involvement and enable them to thrive within the company,’” Krishnamurthy says. “The answer to this question is found in the moments that matter to our employees, in which they meaningfully contribute to the teams around them and move forward toward our collective vision, from the beginning of their time with Microsoft and all the way through their journey.”

Use AI to fuel organizational transformation and to improve your employee experience.
Approach AI as a tool that can help your employees boost their creativity, enhance their productivity, and grow their skills.
Provide personalized and contextualized information to increase employee satisfaction and productivity.
Use AI to improve both the on-site and remote experiences for your employees—it can help you get hybrid work right.
Use AI to improve your infrastructure management, compliance monitoring, governance, and real estate and space planning.
Give your employees good guardrails—take a responsible and responsive approach in each area where you use AI.
Encourage your employees to contribute ideas on how AI can improve their work processes and evaluate ideas that are most valuable and feasible.
Foster a culture of continuous learning and adaptability around AI and data, where failures become steppingstones, and cross-functional collaboration drives innovation.

Learn about the Microsoft 365 Copilot Early Access program and new capabilities in Copilot.