Scandinavian Airlines reduces loyalty program fraud with Microsoft Azure Machine Learning

Scandinavian Airlines—commonly known as SAS—transports approximately 30 million passengers a year within Scandinavia and to the rest of the world. The company aims to make life better for its travelers, its employees, and the planet by implementing safe, efficient, and sustainable operating platforms. Beyond just carrying passengers, SAS has a goal to connect people and countries by bringing Scandinavian culture and lifestyles around the globe.

Like many industries, aviation produces huge volumes of data every day. Companies within the industry are digitally transforming their businesses to turn that data into valuable insights that help them streamline operations and provide customer service innovations. For SAS, that transformation has included embracing the potential of cloud computing and exploring the possibilities of AI and machine learning.

“We believe that AI and machine learning offer vast potential to drive efficiencies within our organization,” says Daniel Engberg, Head of Data Analytics and Artificial Intelligence at Scandinavian Airlines. “It’s not just a case of IT wanting to use leading-edge technology—the business side of SAS fully supports the development of AI technologies and machine learning models to solve real-world operational challenges and drive everyday benefits for our customers.”

Taking the first steps into machine learning

A longtime Microsoft customer, SAS made its first foray into the cloud in 2017 with Microsoft 365 (known at the time as the Office 365 productivity suite). When SAS decided to get out of the datacenter business and move its IT resources to the cloud, it chose Microsoft Azure as the best fit. SAS now runs more than 90 percent of its applications in Azure.

The company’s Azure experience made it only natural that SAS would start with Azure Machine Learning when it came time to explore possible applications of AI. And SAS developers liked what they found.

“Our developers tested a lot of technologies, and they came back saying that they wanted to use Azure Machine Learning,” says Peter Gustavsson, Senior Systems Specialist – EuroBonus/Loyalty at Scandinavian Airlines. “The ease of use, the maturity of the product, and the support available from Microsoft and through the online developer community have been superb. The inherent scalability of Azure means that we can easily build out solutions without worrying about resources.”

Before any coding could begin, though, SAS needed to determine which projects to pursue. “We made it a key factor to first identify and show the actual business value that individual projects could produce,” says Engberg. “So we spent extensive time laying the groundwork for our machine learning initiatives and proving that they could provide business benefits when we put them into production.”

The developers had no shortage of potential projects—the initial list that SAS produced included 150 possible use cases for Azure Machine Learning and its automated machine learning capabilities. The company narrowed that list down to 11 prime candidates and, after further evaluation, chose 5 of them to put into production in the first wave of deployments:

• Flight full prediction helps SAS continuously optimize ticket pricing and revenue by predicting which flights will be full in specific time intervals.
• Sales forecasting with machine learning predicts sales as much as three months in advance with 98 percent accuracy, compared to limited one-month manual forecasts. This forecasting supports smart marketing and sales activities.
• Upgrade willingness helps determine with greater accuracy which flyers might be willing to pay for an upgrade to a higher service class.
• Onboard fresh food optimization helps ensure that precisely predicted quantities of fresh food get loaded onto flights, reducing food waste by as much as 45 percent.
• Loyalty fraud prevention helps SAS identify and mitigate fraud in its EuroBonus loyalty program using automated machine learning (AutoML) for automatic retraining and the InterpretML toolkit built into Azure Machine Learning for model interpretability. With interpretability, SAS data scientists can debug and verify model predictions. And they can produce explanations about model behavior that give stakeholders confidence in the machine learning models and assist with meeting regulatory requirements.

SAS has found that using Azure Machine Learning makes it easy to get all these projects up and running. “With Azure Machine Learning, we can quickly spin up new projects and promote collaboration between different teams from different organizations and even in different countries,” explains Engberg. “Each project team has its own set of resources, which is a scale we could never have achieved if we did this all on-premises.”

SAS developers also have the flexibility to work in whatever way they prefer—they aren’t limited to a single integrated development environment (IDE) or workflow. Azure Machine Learning provides a “sandbox” workspace where developers can do tests and create prototypes—accelerating developer velocity without disturbing other projects or affecting production data. For compute purposes, developers can use Azure Machine Learning compute, Azure Kubernetes Service (AKS), or Azure Databricks. Additionally, they can use the IDE of their choice, such as Microsoft Visual Studio Code or the Jupyter Notebook. For some projects, developers and business analysts started off in the no-code, drag-and-drop designer provided by Azure Machine Learning before moving the projects to other environments.

“Our developers have a lot of freedom within Azure Machine Learning, because it offers native integration with other Azure services, built-in access to datasets, data science frameworks, and support for R and Python programming,” says Engberg.

Combatting loyalty program fraud

The EuroBonus loyalty program is a valuable customer feature for SAS, and its popularity reflects the realities of air travel in Scandinavia. SAS research shows that more than 2 million Scandinavians make five or more air trips a year, and these frequent flyers represent 70 percent of all airline ticket sales in the region. Because of the exceptional service on SAS and the advantages of the EuroBonus program, 60 percent of these experienced travelers prefer to fly on SAS.

Unfortunately, the popularity of the EuroBonus program also makes it a target for fraud. “People see loyalty points in our program as a valuable asset, and scammers try to gain as many points as possible to redeem for their own travel or to purchase tickets and sell them to other people,” explains Engberg. “We consider it extremely important to fight fraud and ensure the integrity of our loyalty program.”

SAS has always had an active fraud detection program and a talented fraud detection team. But as scammers have grown more knowledgeable and gained access to more sophisticated technology, the fraud-prevention challenge has become more difficult. “We were always a few steps behind the scammers,” says Engberg. “So even though we eventually managed to shut individuals down, they may already have traveled with fraudulent tickets or stolen someone’s points.”

Adds Warren Edgren, Data Scientist at Scandinavian Airlines, “We made it our goal to proactively stop fraud and at the same time safeguard the EuroBonus experience for legitimate customers in a cost-effective way. We need to increase our detection capabilities without having to continually increase the size of our fraud detection staff.”

With Azure Machine Learning, SAS has valuable tools for getting ahead of the scammers and identifying them before they can exploit the system. SAS built a complex fraud detection model that looks for patterns across a wide variety of variables, including where people log in to SAS systems, the email addresses and credit cards they use, when and how they travel, whether they’re part of point-sharing pools, and other aspects of their profile and history.

SAS developers initially worked with Azure Databricks during a collaborative research and development phase. Then they used automated machine learning classification in Azure Machine Learning to train the model and used Azure Machine Learning compute resources for data preparation.

As a next step, they operationalized the release pipeline for the solution using the built-in interoperability of Azure Machine Learning with Azure DevOps and Azure Data Factory. The automation facilitates trigger-based and continuous model retraining, with deployment to AKS for online inferencing. If code or data changes, the model automatically retrains, making it smarter over time to outsmart the scammers.

Ensuring trust in machine learning models with interpretability

Accusing a EuroBonus customer of fraud is serious business, and SAS would not take that step lightly. For this reason, the business must have trust in any machine learning model used to identify potential sources of fraud. To provide that assurance, SAS developers turned to the model interpretability capabilities within Azure Machine Learning, which are based on techniques developed in InterpretML, an open-source Python package for training interpretable models and helping to explain black box AI systems.

“We take responsible AI very seriously,” says Gustavsson. “We need to trust that using the model won’t result in an accusation of fraud against an innocent customer. We still keep a human in the equation to make the final determination. But it’s important that the machine learning logic directing them to a potential source of fraud is based on explainable factors—not the output of an opaque black-box process.”

Model designers and evaluators can use the interpretability output for a model during the training phase to verify with stakeholders their hypotheses about the factors that identify fraud. This helps them build trust in the model. They can also use interpretability insights in Azure Machine Learning for debugging, for validating whether the model behavior matches their objectives, for identifying insignificant features, and as a way of uncovering potential compliance issues.

During model inferencing, the fraud detection team uses interpretability to understand how the model is working when deployed and how its decisions are affecting the actual people whose accounts come under scrutiny. The team can explore the results through the Azure Machine Learning interactive visualization user interface.

“With model interpretability in Azure Machine Learning, we have a high degree of confidence that our machine learning model is generating meaningful and fair results, and we have the ability to understand the particulars of any individual case,” says Engberg.

Achieving great results with Azure Machine Learning

By using Azure Machine Learning, SAS is accurately identifying fraud with proficiency that wasn’t possible through manual methods. In the case of retroactively registering a flight for EuroBonus miles—a common source of fraud—the new system predicts fraud with 99 percent accuracy. Over time, SAS has also been able to simplify its machine learning model while maintaining great results.

“We were able to decrease the number of features factored into the result from 54 to 31 without losing accuracy,” says Gustavsson. “This saves on preprocessing costs and increases productivity and the ease of use at inference time. With Azure Machine Learning, we also have the functionality we need to account for data drift, so results don’t degrade over time as EuroBonus member behavior patterns change. That’s an exciting capability.”

In addition, SAS now has better insight into levels of fraud that it previously struggled to unearth. “If someone tries to commit fraud on a large scale, you can find them relatively easily because they stick out,” explains Engberg. “Now we are much better equipped to identify fraud on a small scale. The individual transactions may be small, but we’ve been able to find 300 percent more of them, which produces significant end results.”

Working with Azure Machine Learning, SAS also takes advantage of its machine learning operations (MLOps) capabilities to automate and accelerate the machine learning life cycle. SAS has used these capabilities to improve efficiency of its machine learning pipeline through automatic retraining of models and the ability to do controlled rollouts. This has led to a continuous integration and continuous deployment (CI/CD) development model. 

“We see machine learning operations functionality in Azure Machine Learning as very important for us, particularly around automation,” says Engberg. “When we commit a code change to Git, it automatically triggers the process to create a model, train it, validate it, and publish it to AKS. That’s our time to value, and it is extremely quick. We also find the ability to automate everything really helpful for boosting our productivity and research utilization.”

SAS appreciates the cost control and monitoring capabilities it gained with its Azure Machine Learning environment. For example, the company uses batch processing to optimize compute usage to run for specific periods, rather than having it on all the time. SAS also has transparency into its cloud spend. “Our adoption of Azure provides us with the tools we need to monitor cloud resource usage down to the specific project and easily track the costs,” says Edgren. “We never had that in the past, and we consider it to be a great benefit of working with Azure Machine Learning.”

For SAS, working with machine learning in Azure means that the company hasn’t needed a huge team, new hardware, or lengthy development cycles to get its projects up and running—benefits it appreciates. “We’ve had a handful of data scientists working on these projects, and they’ve been able to come up with hypotheses and deliver on them in a matter of six or eight weeks without provisioning new hardware. From there, we can ensure the business value and scale the ideas into production,” says Engberg. “We use Azure Machine Learning to solve real business problems without worrying about building and managing infrastructure or creating new tools—we can focus directly on gaining value from the technology.” 

Find out more about SAS on Twitter, Facebook, YouTube, and LinkedIn.