Microsoft Unified supercharges EY tax services with AI document intelligence

The process of preparing business taxes requires absolute attention to detail in a fast-paced environment. Tax seasons are of short durations, and the work of proper filing involves carefully evaluating data from a myriad of documents and forms under immense deadline pressure.

EY, already the recognized leader in global tax services, is further improving results for its customers by pioneering the use of Microsoft Azure AI Document Intelligence on its Global Tax Platform (GTP-FS) in handling import data from forms. By significantly reducing the amount of manual transcription, its tax professionals can get to work much faster on the important work of preparing and improving its customers’ returns.

“By integrating Microsoft OpenAI within a secure, cloud-native elastic Azure architecture, we automated both structured and unstructured K-1 tax data processing at scale,” says Johann Weder, Chief Architect at Ernst and Young LLP. “This capability significantly improved data extraction speed, accuracy, and operational efficiency, supporting high-volume tax workflows with minimal manual intervention.”

From tedium to timeless

The first step in preparing a return used to involve a flurry of manual labor. Teams of people would have to open every form, receipt, invoice, or other relevant document and manually enter the key values into EY’s systems. The forms were especially complicated. Most spanned more than 50 pages. Some were nearly 300 pages.

EY saw the potential to save time and money by being an early adopter of Azure AI Document Intelligence, a powerful cloud-based service that uses advanced language learning models to extract key data automatically. The service allows businesses to quickly and accurately transform unstructured data into structured, actionable information.

While the service comes with prebuilt models that work in some cases, tax forms are especially complicated and varied, so EY began building its own models—work that would have been prohibitively expensive just a few years ago.

“If you want to train a machine-learning model, that’s a lot of manual labor,” said Melinda Thielbar, Chief AI Strategist at Ernst and Young LLP. “You have to annotate forms. You have to tell it what kind of data is in the form. You had to create synthetic data. Before, it was really expensive to build a document intelligence model because of the automation burden.”

To solve that challenge, EY also leveraged the power of generative AI. One person can annotate a page and then use generative AI to produce synthetic data that’s used to train the document intelligence model. This also ensures security, as no sensitive client data is needed for training.

With clients spanning the world, there are thousands upon thousands of document types, each of which need their own model. EY has been able to build off its initial success, developing a repeatable process for crafting new models efficiently. In the first year of the project, EY was able to build about 20 models. Already this year, it has 340 new models in production.

“Instead of taking a whole day, you can annotate a single page in about an hour,” Thielbar said. “Instead of every model being this precious work of art that was handcrafted over hours and hours, it’s now more like a factory. It goes very quickly. And we are able to pass this work to a trained annotation team. We don’t need to have developers spending their time.”

Unified helps accelerate results

As an early adopter, EY frequently found itself in uncharted territory, but through the expertise available to it through Microsoft Unified, it was never alone. 

“Given the newness of the technology, it helped that we had a fantastic partnership with Microsoft,” said Vinod Kevaliya, Senior Manager at Ernst and Young LLP. “Through our strategic partnership, we were able to scale in a quick amount of time by leveraging the capabilities of Microsoft.”

One especially valuable output of the partnership was ensuring that EY had the computational resources needed to build its models. Training is resource-intensive, and because of the bleeding-edge nature of the work, EY ran into some capacity constraints early on. But because of its close working relationship with Microsoft, EY was able to get rapid help to provision and balance its loads, allowing it to rapidly scale its work.

“Having to research on my own to find solutions slows production,” Thielbar said. “With Unified, I am able to get answers and respond in a lightning-fast way, which is critically important in a tax season that may only be three months long.” 

Delivering real business value

The results from Azure AI Document Intelligence are exceeding EY’s hopes for the effort. Its document models have reduced manual interventions by 80 percent. The team gets continuous feedback on the performance of the models, accuracy that is calculated without needing to store any client data.

“The accuracy is pretty amazing,” Thielbar says. “State-of-the-art form extraction is actually only around 50 percent. And we’re still improving. We would love to get pinpoint accuracy.”

So far, EY has used this approach to successfully process about 16,000 federal and state tax forms spanning 200,000 pages of both structured and unstructured data. There have been substantial savings. Automation has significantly improved file processing efficiency and reduced operational costs, delivering faster results compared to manual handling. 

But for EY, this transformative technology is not just about saving money. It’s saving its client’s valuable time. 

“If I have a new client, they can send me all their forms and last year’s taxes and I can get them up and running in a couple of days as opposed to a few weeks,” Thielbar says. “The tax business is a timely business. This is huge.” 

Time is money … and so much more

EY has a rich tradition of deploying groundbreaking technologies for the benefit of its clients. Working with its Microsoft Unified team, it is an industry leader in using AI to drive substantial and measurable improvements in efficiency and responsiveness.

“Through the power of Microsoft OpenAI within a scalable Azure framework, we transformed and improved our K-1 tax processing pipeline,” says Nate Hanafy, FSO Tax CTO, Ernst and Young LLP. “This AI-driven solution not only streamlined operations but also solidified GTP-FS as a cutting-edge, intelligent platform—delivering a market-leading capability that provides us with a strong competitive edge.”