[Editor’s note: This content was written to highlight a particular event or moment in time. Although that moment has passed, we’re republishing it here so you can see what our thinking and experience was like at the time.]
Can unsupervised machine learning in Microsoft Azure be used to find errors and anomalies inside financial data?
Shilpa Tiwari thinks so.
Tiwari had a vision of using Microsoft’s complex volumes of financial data to develop new methods of identifying risk. Tiwari is a principal group engineering manager with the Financial Services group in Microsoft Digital, the engineering organization at Microsoft that builds and manages the products, processes, and services that Microsoft runs on.
But first, Tiwari and their team would have to solve the problem of traditional modeling practices.
“In the past, we’ve been looking at data in an ‘if this, then this’ way,” Tiwari says. “That doesn’t scale. Microsoft’s transaction volume is going up ten-fold, but the financial data is much larger than that. Plus, it’s all done in silos. If a wrong payment is made, you’re only looking at payment data, not the connected datasets.”
To innovate on the way information is extracted from financial data, Tiwari and the Finance Data Insights Team, a subset of the Financial Data Services group, looked to artificial neural networks, an unsupervised machine learning technique, to identify anomalies.
With the help of business partners throughout Microsoft, Tiwari and their team are improving the way errors and anomalies are recognized in historically disparate datasets, breaking down silos and proactively predicting risks.
[Learn how Microsoft finance professionals make data-driven decisions on cash flow. Read more about Microsoft’s connected and discoverable data. Read about Microsoft’s digital transformation with a modern data foundation.]
Changing financial data support
It should come as no surprise, but Microsoft deals with a lot of data and more is added every day.
The Financial Data Services group supports Microsoft’s efforts to understand financial data, including corporate data. Several teams, including the Finance Data Insights Team, help oversee and understand the complex and sizable datasets that Microsoft manages.
“These teams are all managing, governing, providing access and insights, and reporting for all of Microsoft’s financial data,” Tiwari says.
The amounts of data the Finance Data Insights Team sees exposes the very heart of the problem.
Not only are they facing enormous datasets and sources spread across all of Microsoft, but traditional supervised approaches to machine learning cannot keep pace with the complexity or volume.
“Rules-based systems will not scale with the amount of financial data we receive, nor will it give us richer insights through connected datasets,” Tiwari says.
This makes it incredibly difficult to extract useful intelligence from sources and easy for anomalies to go undetected.
There are human limitations in terms of thinking several permutations ahead. It’s not like anomalies are going to be able to communicate how they’ll appear.
– Shilpa Tiwari, principal group engineering manager, Microsoft Digital
Traditional models for parsing, such as rules-based machine learning, require long sprints and heavy involvement from data scientists, engineers, and subject-matter experts (SMEs).
“It takes months to launch a rules-based system,” Tiwari says. “You have to develop rules, test, and then launch them. The SME expertise to do this is finite, and there are human limitations in terms of thinking several permutations ahead. It’s not like anomalies are going to be able to communicate how they’ll appear.”
The Finance Data Insights Team saw an opportunity to use new technology and Microsoft’s vast network of SMEs to gain richer insights from the company’s growing financial data.
Identifying risks with artificial neural networks
Instead of running from the large amount of data, the team leaned into it.
“As you get more data, you want to be able to turn it on in a semi-automated fashion,” Tiwari says. “You don’t want to spend months going to SMEs. Unsupervised machine learning learns from the data without human labelling and engineering. It’s complex, but it’s well suited for this kind of application.”
Unsupervised machine learning not only performs better as it scales, it’s also faster. Unlike a rules-based approach, unsupervised machine learning develops its own model, benefitting from large amounts of data.
In addition to quickly processing the information, unsupervised machine learning is able to learn from a variety of sources, which further improves the model as it identifies patterns a rules-based approach might not account for.
This makes Microsoft’s financial data a good use case to demonstrate impact.
“Financial data is prone to risk,” Tiwari says. “It’s also prone to regulatory controls. You have to be proactive about the risks. The more insights we can extract, the better we get at predicting these risks.”
But where to start?
“We picked a paper based on artificial neural networks,” Tiwari says. “It’s not easy to introduce a model through a research paper, but a lot of these ideas are totally new.”
Tiwari needed the Finance Data Insight Team to apply the knowledge within the research, so Tiwari tasked Joy Chepkwony and Abhishek Mehra, two software engineers from the team, to make the unsupervised machine learning vision a reality.
“It was a different process for me, starting from a research paper like this,” Chepkwony says. “Initial research leads you to a lot of other papers. Some help solve certain things, but it becomes a lot of trial and error.”
For Mehra, the process was a complicated balancing act with lots of moving parts.
“A good insight project needs engineers, scientists, a good dataset, and a feedback loop,” Mehra says. “That’s a challenge. If you’re running things with only one, you’ll fail.”
Having read through several papers, Chepkwony built the unsupervised machine learning model in Azure Databricks. Unlike a traditional model, the artificial neural network could be engineered without the input of SMEs. Chepkwony designed the model to extract insights from a series of algorithms and services that sit atop data and compute planes.
“The pipelines are all on the Azure stack,” Chepkwony says.
To find anomalies, data passes through an autoencoder, a type of artificial neural network. According to Chepkwony, the autoencoder compresses data to its latent state, then the decoder decompresses the latent representation. The decompressed output, a reconstruction of the original, is compared to its initial source and if there’s a high enough error rate the data is flagged as “not normal.”
Items that fall below the “not normal” threshold could be a variety of things, including erroneous or suspicious activity. Additionally, the model could just be spotting something that’s simply outside of a normal pattern.
“It’s hard for humans to look at datasets and follow patterns or explain what’s wrong,” Mehra says. “We want a feedback loop that includes auditor feedback.”
Mehra, who also helped acquire the connected data sets, reached out to several SME business partners across Microsoft to help explain the findings of the model.
They had a map but needed to make sense of it.
Getting the most out of an unsupervised machine learning model
In trying to understand anomalies, context matters.
That’s why the team relies heavily on SME partners to help explain the anomalies, including Simaan Huda, principal program manager with Microsoft’s Corporate Functions Engineering team.
“My team was looking to use transaction level insights,” Huda says. “We thought about doing it on our own, but we saw the benefits of partnering with the Finance Data Insight Team.”
Initially, Huda used the neural networks to read through Microsoft’s general ledger.
“We were getting a lot of flagged items that weren’t suspicious,” Huda says. “They were often entries of less significance or manually entered. Anomalies, but not actionable.”
But it was useful for the Finance Data Insight Team. As the neural network identified deviations, Huda and their team could help explain the anomalies. This feedback fine-tuned the model for a variety of circumstances.
The large size of the general ledger was initially a challenge.
“There are so many practices at Microsoft that it takes a lot of SMEs to understand the general ledger,” Huda says. “But there are many subprocesses that make up the general ledger, and by learning on these sets we can use machine learning to uncover knowledge, especially what’s not well documented or understood beyond the SMEs.”
By looking at portions of the general ledger, such as manual entries, Huda and their team were able to hone the model with Chepkowny and Mehra.
“Machine learning is about running scenarios as expected, but there’s also surrounding data that’s interesting,” Mehra says. “We can enrich data by running the model again, taking the smaller subsets plus the general ledger to extract insights.”
With each iteration, the model improves itself.
This entire approach still comes with a steep learning curve, especially for SME partners who don’t understand how the unsupervised machine learning model identifies data as not normal.
“It’s a journey to help people understand machine learning,” Huda says. “We’re in finance and compliance. We need to have some rationale as to why we’re taking an action. Explainability is important; when you’ve caught an anomaly, you need to show how your model will catch all of the anomalies.”
Tiwari agrees, which is why the team is taking extra steps to try and explain findings.
“Unsupervised is a different experience than rules-based,” Tiwari says. “It’s not as easy to explain why the machine is telling you something, but we’re relying on responsible AI practices to create transparency.”
An interconnected Microsoft based on rich insights
Insights now have the capability of being predictive.
“There are lots of risks associated with finance and operations,” Tiwari says. “We want to proactively identify them based on data alone, then begin to reduce risks overall.”
The agnostic nature of the model means that financial, compliance, and operational risks can all be recognized and addressed.
“We’re able to generalize the inference and training of our pipeline, allowing us to generate models specific to the different facets of the datasets,” Chepkwony says. “This allows us to scale and provides insights we would never see in a rules-based model.”
The ability to harness interconnected datasets allows rich insights to be drawn from across Microsoft.
“Risk used to be viewed in a silo,” Tiwari says. “Now it can be viewed holistically, at scale.”
Huda sees even more opportunities by changing the review process.
“Today, really big and key accounts are reviewed first, insignificant entries are moved down the line in priority, and then there’s the audit,” Huda says. “With anomaly detection, everything is reviewed upstream and continuous.”
A bright future
What started as a research paper is now transforming the way Microsoft approaches anomaly detection.
Artificial neural networks and unsupervised insights might appear daunting, but it’s the right time to leverage new technology for innovative solutions.
– Shilpa Tiwari, principal group engineering manager, Microsoft Digital
Now that the model is able to successfully identify anomalies, the Finance Data Insight Team will continue to hone their insights with SME input on other datasets. They’re only in the second leg of the journey, but for Tiwari and the Finance Data Insights Team, it’s all upside.
“Artificial neural networks and unsupervised insights might appear daunting, but it’s the right time to leverage new technology for innovative solutions,” Tiwari says. “Newer technology not only gives better outcomes it’s also simpler.”
