Scandinavian Airlines speeds app development, lowers costs with Azure Database for PostgreSQL

Streamlining operations with a cloud-first approach

As Scandinavia’s leading airline, SAS has been streamlining operations since the governments of Sweden, Norway, and Denmark joined forces to create the airline in 1947. Today it flies to destinations in Europe, the United States, and Asia and aims to be the global leader in sustainable aviation.

In the spring of 2021, the airline was in the process of streamlining its application environment. The strategy was to move the company’s Java apps running in Apache Tomcat to Azure Kubernetes Service (AKS) and gain the benefits of microservices and continuous deployment. As experienced Azure users, SAS developers knew that using AKS would simplify app deployments and management while offloading the operational overhead to Azure.

According to SAS DevSecOps Engineer David Fuchslin, “We wanted to enable our web development teams to speed up their ability to deliver solutions for our travelers on a continuous basis—and without dependency on other IT functions.”

In the strict SecOps environment at SAS, developers used to hand their application code to an operational team to implement—a process that could take days. With their own AKS cluster, the application teams could instead deploy continuously without the extra step. “It’s about ownership and autonomy for the teams,” Fuchslin explains. 

The first app to move was the mission-critical booking app, and the team took a straightforward lift-and-shift approach, moving the app and its PostgreSQL database to Azure infrastructure as a service (IaaS). 

According to Fuchslin, “Our main focus was to enable continuous delivery, but we also saw opportunities to reduce costs.”

Remodeling an application stack—and a few surprises

Like home remodeling, migration projects usually pose unexpected surprises. You don’t really know what’s behind the walls until you remove them. In moving the high-priority booking app, the team planned to test, retest, and test again. New to AKS, team members weren’t sure how their failover code would work, and database downtime is not an option for the airline. 

In the initial lift-and-shift to Azure, Fuchslin recalls, “We created Azure virtual machine instances that our operations team set up, and then they installed open-source PostgreSQL.” The company has been using PostgreSQL for years, so continuing to use the same database in the same Linux environment made the move to Azure easier.

The preproduction environment ran PostgreSQL on a virtual machine (VM) as a single shared instance. However, the production environment on Azure paired a primary and secondary VM running the database in a high-availability configuration, as shown in the following “Before” diagram. The team maintained and patched the VMs manually, requiring periodic database downtime. To fix that, a separate failover solution was added to manage the process.

But that addition introduced an expensive problem—the failover licensing. Whenever a new app was added to the environment, the license had to be extended, upping the cost. Furthermore, the apps developers couldn’t test failover at all in the nonproduction setup, and that’s where they drew a line. How could the team members trust a new Kubernetes environment if they couldn’t test it?

“We had to rethink our strategy,” Fuchslin says. In moving to Azure, SAS was also moving to a cloud-first approach based on microservices for its development process. “We wanted all Azure, if possible.” And that’s how an AKS migration became the impetus for the team to move its open-source database to an Azure platform as a service (PaaS) replacement.

Designing an architecture that’s elegant in its simplicity

Accelerating the move to a managed database gave SAS a timely solution and an architecture that’s elegant in its simplicity, as shown in the following “After” diagram. Applications run in the Kubernetes cluster and communicate with Azure Database for PostgreSQL - Flexible Server, a service with full compatibility with community PostgreSQL. It provides zone-redundant high availability and burstable compute tiers to handle any spikes in traffic to the booking app, while also optimizing costs.

The simple design is replicated across the production and nonproduction environments, solving the testing problem for the app team. As Fuchslin puts it, “We just instantiate the database in Azure, and Microsoft takes care of the rest. The developers don't need to be experts at the low-level infrastructure.”

“We’re also becoming more agile in this move from IaaS to PaaS,” observes Olof Spångö, SAS DevOps Digital System Owner. “That was always our goal. Even though Flexible Server is in preview, the service does exactly what we need it to do.”

When SAS chose Azure Database for PostgreSQL, the Single Server deployment option was publicly available, but it ran on Windows Server. The developers at SAS wanted to keep running on Linux. That way, they reasoned, the file encoding would be exactly the same and there wouldn’t be any problems with the data in the database. The newest deployment option—Azure Database for PostgreSQL - Flexible Server—may have been in preview at the time, but it supported Linux.

“Using open-source operating systems is a plus for us,” Fuchslin notes, adding, “Microsoft was cool and gave us lots of support.”

In addition to the Linux support, Azure Database for PostgreSQL - Flexible Server features zone-redundant high availability, which does away with the need for primary and secondary VMs. Flexible Server runs on a single instance—a secondary isn’t needed. Even if the database unexpectedly goes down, the booking app is never offline for more than a minute or two. “We agreed that a couple of minutes is OK,” Fuchslin points out. The team has the option to enable a primary and secondary instance of Flexible Server, but for now, SAS doesn’t need it.

As a managed service, Flexible Server is always current, courtesy of Microsoft. SAS also gets to choose when database updates occurs—even restricting the maintenance window to a specific hour if needed.

“It really simplifies things for the app teams,” Spångö reports. “They can take full responsibility for their code without having to spend a lot of time patching their VMs. We have also reduced the number of internal service requests between teams, which saves a lot of time for more interesting work.”

Learning along the way

As anyone who has worked with prerelease software knows, there can be risks. For example, the product documentation for Azure Database for PostgreSQL - Flexible Server was under development, and there were a few gaps. When the team members tried to set up a private endpoint for the database—a requirement in their hub-spoke network topology—they couldn’t get it to work at first.

This type of network topology isolates workloads and can help reduce costs—big benefits for SAS. The hub acts as a central point of connectivity to all the spoke virtual networks (VNets). The locked-down environment also uses a custom domain name server (DNS) for added speed. When it was time to create the private endpoint for the database connection, however, it wasn’t clear to the team how to do it.

This could have been a showstopper. Instead, the Microsoft product team used the scenario to create an in-depth article to help SAS and other customers set up name resolution in a hub-spoke topology with a custom DNS server.

“Microsoft really listened to us and helped us,” recalls Fuchslin. “In some respects, we helped Microsoft, as well, so it was a nice collaboration.”

Microsoft and SAS also worked together on another potential roadblock. In preview software, certain planned features might not be available. In this case, the SAS booking app used PostgreSQL pg_cron jobs to schedule key tasks, but the feature hadn’t been added yet to Azure Database for PostgreSQL - Flexible Server. So the FastTrack for Azure (FTA) team quickly built a time-triggered function using Azure Functions to execute the jobs.

Or take the case of the unexpected CPU spike. At times, the database would eat up 80 percent of the CPU, something the team had never encountered before the migration. When the Microsoft team investigated, it uncovered a few quirky queries that needed to be optimized. How could SAS have known this, when the Query Performance Insight tool in the Azure portal didn’t yet support the preview database? Shortly afterward, the feature was added, so SAS could troubleshoot and optimize its queries.

Fuchslin admits that he and his teams learned a lot from the experience of working with Microsoft. Their advice for other teams? Move to infrastructure as code (IaC). The SAS developers use Terraform by HashiCorp and Azure Pipelines. “Using IaC to create your infrastructure makes it much easier to spin up, destroy, recreate it at will,” explains Daniel Engberg, Head of AI, Data, and Platforms at SAS. “You need that when you’re testing new things.”

Moving forward with the migration

An application migration to AKS turned into a win for the data tier when SAS switched to Azure Database for PostgreSQL - Flexible Server. The transformation is ongoing as SAS teams continue to move apps from the old environment to the new Azure platform.

“We're about two-thirds of the way through the migration,” Spångö reports. “We continue to look for ways to optimize our PostgreSQL instances—for example, by shutting down the development and test environments on the weekends when not in use.”

For now, the old environment coexists with the new. “We will definitely see cost savings after the old environment is fully decommissioned,” Engberg adds. In the meantime, the company is already seeing the impact on its web development teams as the number of successful customer deployments increases.

As Fuchslin puts it, “The AKS migration was the jump-start for us to move our databases and look at Azure Database for PostgreSQL - Flexible Server. It saved our app developers a lot of time.”