You don’t know what you don’t know. In the world of IT, illuminating those hidden areas helps stave off nasty surprises.

When elements of IT infrastructure are shrouded in mystery, it can lead to security vulnerabilities, non-compliance, and poor budget management. That’s the trouble with shadow IT—a term for any technical infrastructure that conventional IT teams and engineers don’t govern.

At Microsoft, we’re on a journey to increase our shadow IT maturity, resulting in fewer vulnerabilities and increased efficiencies. To get there, we’re leveraging tools and techniques we’ve developed through our core discipline of Microsoft Azure optimization.

[See how we’re doing more with less internally at Microsoft with Microsoft Azure. Learn how we’re transforming our internal Microsoft Azure spend forecasting.]

The challenges of shadow IT

Shadow IT is the set of applications, services, and infrastructure that teams develop and manage outside of defined company standards.

It typically crops up when engineering teams are unable to support their non-engineering partners. That situation may arise from a lack of available engineering capacity or the need for specialized domain solutions. On top of those circumstances, modern tools enable citizen developers to stand up low-code/no-code solutions that enable businesses to reduce their dependency on traditional engineering organizations.

Six corporate function teams have been involved in creating shadow IT environments: business development, legal, finance, human resources, and our consumer and commercial marketing and sales organizations.

Many of the solutions they’ve developed make strong business sense—as long as they’re secure and efficient. That’s where our Microsoft Digital (MSD) team comes in.

Three years ago, our biggest driver was getting visibility into the shadow IT estate and finding ways to secure it. Now we’re at a point where we’re looking for cost savings—that’s a natural progression.

—Myron Wan, principal product manager, Infrastructure and Engineering Services team

Over the last few years, our IT experts have been working with the shadow IT divisions to increase the maturity of the solutions they’ve developed, taking them from unsanctioned toolsets lurking in the shadows to well-governed, compliant, and secure assets they can safely use to advance our business goals.

Now that these shadow IT solutions are more secure and compliant, we’ve turned our attention to efficiency and optimization to ensure we’re able to do as much as possible with the least necessary budget expenditure.

“Three years ago, our biggest driver was getting visibility into the shadow IT estate and finding ways to secure it,” says Myron Wan, principal product manager within the Infrastructure and Engineering Services (IES) team. “Now we’re at a point where we’re looking for cost savings—that’s a natural progression.”

Because many of our shadow IT solutions leverage Microsoft Azure subscriptions, that was a natural place to start the optimization work.

Azure best practices, shadow IT efficiency

Fortunately, we have robust discipline around optimizing Microsoft Azure spend in conventional IT and engineering settings. Microsoft Azure Advisor, available through the Microsoft Azure Portal, has been providing optimization recommendations and identifying overspend for subscribers both within Microsoft and in our customers’ organizations for years.

The plan was to take applicable recommendations that we use in our core engineering organizations and distribute them to the shadow IT divisions.

—Trey Morgan, principal product manager, MSD FinOps

Internally, we’ve added layers that help streamline the optimization process. One, called CloudFit, draws from a library of optimization recommendations, which are tailored to the specific needs of the teams we support. Then we use Service 360, our internal notification center that flags actions in need of addressing for our engineering teams, to route those recommendations to subscription owners within MSD, product groups, and business groups.

Optimization tickets then enter their queue and progress through open, active, and resolved statuses. It’s a standard method for creating and prioritizing engineering tasks, and Microsoft customers could accomplish a similar result by building a bridge between Microsoft Azure Advisor and their own ticketing tool, whether that’s Jira, ServiceNow, or others.

“We have an existing set of cost optimization recommendations that we use for a variety of different technologies like Azure Cosmos DB and SQL,” says Trey Morgan, principal product manager for MSD FinOps. “The plan was to take applicable recommendations that we use in our core engineering organizations and distribute them to the shadow IT divisions.”

Getting there was a matter of establishing visibility and building culture.

Shining a light on shadow IT spend

Many of the optimization issues within shadow IT divisions came about because of non-engineers’ and non-developers’ unfamiliarity or lack of training with subscription-based software. They might not have the background or expertise to set them up or even ensure that their subscriptions would terminate after they had served their purpose.

In some cases, vendors or contractors may have set up processes and then moved on once their engagement was complete. Each of these scenarios had the potential for suboptimal Azure spend.

Providing visibility into these issues was relatively simple. Because all Microsoft Azure subscriptions across our organization are searchable through our company-wide inventory management system and sortable by department, engineers were able to locate all the subscriptions belonging to shadow IT divisions. From there, they simply had to apply CloudFit recommendations to those subscriptions and loop them through Service 360.

Our people now have the information they need to act—our organizational leaders can visit their Service 360 dashboard or can review their action summary report to see what they can do to cut their costs. That’s where culture and education came into the equation.

“Culture is always the number-one challenge when items aren’t actually owned by a core engineering team,” Wan says. “When you have teams that are more about generating revenue or managing corporate processes, a lot of what we have to deal with is education.”

It wasn’t just educating teams about Microsoft Azure optimization techniques. CloudFit and Service 360 provided a lot of the guidance those teams would need to get the job done. To a great degree, non-engineering employees needed to build the discipline of receiving and resolving tickets like a developer or engineer would.

But through direct communications from FinOps tools and support from Wan’s colleagues in engineering, we’ve been meeting our goal of optimizing Azure spend in shadow IT divisions. In the first six months of this solution’s availability, we’ve saved $1 million thanks to various optimizations.

Microsoft Azure savings and organizational discipline

Shadow IT will always exist in some form or another, so this journey isn’t just about remedying past inefficiencies. It’s also about building a culture of optimization and best practices across shadow IT divisions as they use their Microsoft Azure subscriptions moving forward.

With these solutions and practices in place, we’ve moved on from a “get clean” and “stay clean” culture to one where we “start clean.”

—Qingsu Wu, principal program manager, IES

“As we get more mature and divisions build up their muscles, we’re actually getting to an ongoing state of optimization,” says Feng Liu, principal product manager with IES. “As we build up that culture and that practice, folks are becoming more aware and taking more ownership and accountability.”

Some shadow IT divisions are even going beyond FinOps recommendations. For example, our commercial sales and marketing organization uses shadow IT solutions so extensively and is so keen to optimize their budget that they’ve automated the implementation of recommendations and created their own internal FinOps team.

“The whole vision of our shadow IT program is helping business teams to be self-accountable and sustainable,” says Qingsu Wu, principal program manager for the Infrastructure and Engineering Services (IES) team. “With these solutions and practices in place, we’ve moved on from a ‘get clean’ and ‘stay clean’ culture to one where we ‘start clean.’”

It’s all part of building a more effective culture and practice to do more with less.

• Understand your inventory. Spend time linking your organizational hierarchy to your Azure resources.
• Get to a confident view of your estate and your data. It’s crucial.
• Don’t be overly prescriptive. Be open to how you’re going to approach the situation.
• Build sustainability into your efforts by getting non-engineering teams more comfortable with regular engineering practices and learning from each other.
• Don’t overlook small wins. When they scale out across an entire organization, they can produce substantial savings.

• See how we’re doing more with less internally at Microsoft with Microsoft Azure.
• Learn how we’re transforming our internal Microsoft Azure spend forecasting.
• See how we’re enabling launch optimization at Microsoft.
• Find out how SAP and Azure combine to forge a trusted path to innovation in the cloud.
• Discover how we’re optimizing network performance for Microsoft Office 365.

Tags: cloud transformation, digital transformation, Microsoft Azure