Building a foundation for AI success: AI strategy and experience

This is part four of a six-part blog series. See part one, part two, part three, and download the white paper.

Building your organization’s understanding and experience are fundamental to any successful AI strategy. In this post, we’ll focus on emerging best practices that can help you position your AI projects for success.

Start with a diverse team

Assembling a team that brings a diverse set of roles and experiences is a crucial first step toward realizing value with AI. A combination of technical, business, finance, marketing, security, data privacy, responsible AI and other experts is key, because diverse viewpoints tend to surface potential issues early on, reducing the need for re-work later in the project. A diverse team also helps to build the institutional knowledge that is so critical to your organization’s ability to scale AI projects successfully over time.

As your organization deploys more use cases and learns from deployments, you will be better able to anticipate and address potential barriers to implementation and success. One of the most common examples is the “perpetual proof of concept” loop, which tends to point to issues related to data, infrastructure, or lack of alignment between projects and valued business outcomes.

Think—and act—like a scientist

AI relies on probabilities and statistical models to identify patterns and relationships, unlike computing systems of the past that used precise rules that generated predictable outputs. The probabilistic nature of AI requires a different approach to development—one that is more geared toward testing and learning—than some organizations may be accustomed to.

“The most successful organizations tend to have a mindset of experimentation and learning so they can see what’s working and systematically tackle any issues that arise,” says Eric Boyd, Corporate Vice President, Azure AI Platform, at Microsoft. “That said, you really have to have a clear vision of what you’re trying to achieve with your AI model to determine how well it is performing.”

Pairing experimentation with structured, repeatable processes is very much in line with scientific method; developers know it as agile development. Whatever you call it, a focus on iteration and continual learning, combined with incremental planning, team collaboration, repeatable processes, and measurement rigor are characteristics of organizations that tend to see the most benefit from AI.

Use the right tool for the right job

The AI landscape is evolving rapidly, and a critical driver of success is to apply the right model to your use case—in other words, use the right tool for the right job. There are many types of AI models, including models that can find patterns and generate recommendations, understand languages and handle complex queries, summarize and translate text, recognize visual objects and scenes, and produce natural language, images, and code, among others.

To best position your organization to realize value, it’s critical to establish clear communication between developers and subject matter experts in the business so that developers know exactly what they are solving for and can choose the model best suited to the data and the use case. This means clearly articulating the business challenge you’d like to address with AI, your desired outcomes, and how you will measure success.

Measure the impact of AI projects holistically

Measuring the impact of AI projects should encompass a range of stakeholders and objectives and include both quantitative and qualitative methods. Following are a few suggestions on potential metrics to help you get started.

Business	Customer-centric	Technical	Qualitative
Business value: Increased revenue, brand lift, insights that lead to growth opportunities, risk reduction, cost savings, and improved productivity and efficiency.	Customer satisfaction (CSAT): Conduct surveys and gather feedback to understand how customers perceive the AI experience. Are they finding it helpful, efficient, and personalized?	Model performance: Track accuracy, precision, and recall of your AI models. Are they making correct predictions or recommendations?	Feedback: Gather feedback from employees who interact with the AI system in their daily work. How is it affecting their productivity and workflow?
Operational efficiency: Efficiency gains from automated tasks, reduced errors, and streamlined processes.	Analytics/ telemetry: Monitor how customers interact with the AI system. Measure metrics such as click-through rates, chat session lengths, and use of specific features.	Data quality: Monitor data quality, accuracy, completeness, and representativeness against your target audiences or business objectives.	A/B testing: Compare different versions of your AI model or user interface to see which one performs better with customers.

Next steps for successful AI development

Successful AI development is a blend of diverse teams, continuous learning, and a healthy tolerance for ambiguity. But the most important step is the first one.

“You’ve got to get in the game,” says Eric Boyd. “Try something. Iterate and learn, try different things, and see what works for your application. Empower everyone in your organization to discover how AI can transform your business.”

Stay tuned for the next post in our series: “Building a foundation for AI success: Organization and culture,” in which we will explore additional best practices that are frequently cited as critical to AI success.

Download a copy of the “Building a Foundation for AI Success: A Leader’s Guide” whitepaper.