Crayon boosts speed, accuracy of healthcare auditing process using Azure Machine Learning and PyTorch

Hilda Kosorus is Director of the Data and AI Center of Excellence in Vienna, the organization in Crayon that delivers Data & AI solutions for customers from a wide variety of industries and government organizations, bringing tough problems and seeking innovative opportunities to address those problems. Kosorus also directs the energy and expertise of skilled data scientists and engineers toward solving those problems and identifying opportunities to introduce highly effective, AI-driven solutions that deliver real, bottom-line value. “What we offer is our expertise,” she says. “We combine our data, AI, and machine learning skills with customers’ existing domain expertise to help identify opportunities and develop groundbreaking solutions.”

A major resource in that effort is Chief Data Scientist Alexander Vaagan. 

The problem: rules, regulations, and reports

Vaagan has been working on a recent project in the healthcare sector using Microsoft Azure Machine Learning that provides a great example of how Crayon can now use proven AI technologies to build powerful and unprecedented responses to hard problems. He says, “Sometimes, customers can be hesitant around AI given all the overblown claims about it they’ve read in the press. They wonder if it’s just hype. But a project as successful as this shows that for some problems, it’s exactly the right answer.” 

It was the right answer to a long-term problem that had proved resistant to more conventional approaches. Healthcare providers need to be able to verify that they’re maintaining the highest operating safety and efficacy standards. Those standards are set by a national accreditation organization whose surveyors, often healthcare professionals themselves, regularly visit facilities and document situations that might need to be corrected or brought back in line with the latest rules and policies. Anything from a misplaced fire extinguisher to a mislabeled equipment locker or mishandled patient information can indicate an issue that must be reported, and subsequently corrected according to current rules and procedures, before that facility’s local, state, or national accreditation can be confirmed. 

That assessment and accreditation process generates a huge amount of data, and even the most experienced surveyors struggle to keep ahead of the ongoing development of thousands of policy rules that might be relevant in any particular scenario. The rules change. New rules get introduced. Some are superseded, and others are combined. Reported incidents can be hard to interpret and might involve multiple potential rule violations. And yet, the highest level of accuracy is essential when matching rules with reports if the resulting assessment is to be valid and comprehensive. “The solution that customers used previously was a simple scale search,” says Vaagan. “They received hundreds of results when they tried to match reports to rules. They had to read through them all, and the most accurate might be number 499.” Vaagan and his team took on the task of fixing the issue by building a machine learning solution that could ingest text from those reports and return a top ten list of the latest associated rules with unprecedented accuracy. 

The solution: AI, expertise, and Azure

They used Azure technology, development tools, and services to bring that solution to fruition. “Everything had to be built from scratch—the whole development, test, production environment,” Vaagan says. “Azure was the obvious choice. We have deep experience with Azure and the PyTorch Enterprise on Microsoft Azure program. It’s an excellent and preferred platform for doing AI work with first-class usability. We’ve been through multiple iterations and can confirm it’s the strongest it’s ever been.” 

And Azure embraces PyTorch, which is Vaagan’s deep learning framework of choice. He notes, “I’ve been doing machine learning for more than 10 years, and PyTorch works with Azure Machine Learning and all of our virtual machines and services. It’s a powerful environment that’s also easy to use.” 

Azure Data Factory, too, helps ingest and manage all that information. “We upload data to Azure, and we built pipelines using Data Factory to extract, reformat, and reassemble that data into structures we can work with,” Vaagan continues. The data is cleaned up, and compliance policies remove any personal health information before datasets are created for testing and model training. “And of course, we needed to take care of versioning there because that’s an integral part of the solution. The accreditation rules change every six months, so those changes feed into the Azure Machine Learning training pipelines while we perform test and validation of the models on a separate pipeline,” he adds. When a model achieves a high enough test score, it’s automatically deployed as a Docker image into an Azure Kubernetes Service cluster in the production environment, where it serves as the most up-to-date back end to a set of user-facing service APIs. 

Vaagan describes how his infrastructure choices help simplify a complex process: “Beyond the model itself, there’s the complexity of the infrastructure surrounding it and of managing the overall environment.” He and his team addressed this using HashiCorp Terraform on Azure along with other support in Azure for infrastructure as code (IaC) tools. “Having all the infrastructure in Terraform makes it easy for us to build out and replicate it.” It also makes the entire development, test, and deployment cycle much more efficient for Crayon because, says Vaagan, “Azure will automatically provision everything for you, run the training, and then shut everything down and discard it. After you have that process up and running, you save a lot of cost and time using the Azure infrastructure—it’s a big plus for us.” 

The advantage for Crayon and its customers

Crayon customers report clear time savings with the new healthcare solution. Just as important, the solution provides consistent responses that aren’t subject to the vagaries of individual interpretation or potentially out-of-date data. The organization’s most skilled surveyors use it for training, blending their experience with the solution to boost the productivity of even the most recent hires. And finally, because using the solution helps eliminate errors, all participants in the accreditation process have greater confidence in the overall system that only genuine issues are being identified and that the latest policy rules are driving efforts to make associated improvements. 

Vaagan sums up the experience from the development side of the solution: “I would recommend the Azure environment to other developers. It’s user-friendly, easy to develop with, and very importantly, it follows best practices for AI and machine learning work.” According to Vaagan, things change fast in the world of AI, and the widespread developer community also wants to move quickly. He views Microsoft as helping lead the charge while making significant investments in supporting and embracing outside innovators. “Microsoft supports and promotes the AI community, including popular standards and tools, bringing them into the Azure environment. Microsoft is doing great work with Azure.” 

Find out more about Crayon on Twitter, Facebook, and LinkedIn.