Trace Id is missing
October 02, 2020

Manulife future-proofs risk models on Azure using Azure SQL Database and saves a bundle

Technical Story

Manulife is in the business of predicting the future. The company analyzes terabytes of data to create complex models of risk for its financial services and insurance businesses. As part of a global digital initiative and a company-wide data migration effort, Manulife consolidated its data silos and moved its models to Azure, creating a single source of high-quality data in Azure SQL Database. Massive modeling calculations that used to take a week now run in a day or two on Azure, which has replaced a multitude of underutilized servers. In embracing Azure, Manulife changed its infrastructure-focused mindset to a cloud-first approach and, along the way, saved approximately 40 percent in hardware and labor costs in the past three years.

Manulife

“We came from an IaaS mindset. Last year, we migrated to Azure PaaS. Now everything is automated—we integrate servers with a few clicks of a button.”

Rohit Mistry, Infrastructure Analyst, Manulife

Too many servers in too many places

For Manulife, the future is riding on an accurate actuarial valuation. Actuarial formulas are complex calculations of risk, based on rigorous mathematical models of uncertainty. The company models what-if scenarios to determine life insurance coverage and forecasts outcomes that help it assess whether it has enough money to pay out claims as needed across plan members. Models also help the company to identify demographic trends and to address compliance requirements—data it uses to quickly shift service offerings.

After decades of growth and acquisitions, the Manulife back-office patchwork posed a serious challenge to systemwide change—the type of change the company knew was coming, in the form of new compliance requirements. Based in Canada, Manulife operates as John Hancock in the United States and as other identities around the world. A few years ago, these businesses were operating separate server infrastructures—sprawled across dozens of server closets, rooms, and farms—representing more than 5,000 cores. When the company took stock, it realized that data quality could become a problem with so many systems running their own assortments of applications and operating systems. It was time to modernize.

The actuarial models are the basis for the reports Manulife creates for compliance in its heavily regulated industry. The company maintained around 300 individually developed, overlapping valuation models for nearly two dozen different legal entities—each of which required its own reporting. Companies are audited quarterly and use the reports to prove they can pay claims against the policies they sell. However, regulations were changing, and the various teams were updating their models to handle more risk scenarios, which meant adding more granular data points in a manual process that took weeks. In turn, the larger models required more compute power and storage for the trillions of data points they were generating.

“As valuation models became more complicated, they were taking longer and longer to run,” explains Jon Bradbury, a Manulife Senior Vice President at the time. “The company was also doubling compute capacity every three years.” Manulife was provisioning its servers for peak load—the monthly or quarterly audits when the actuarial valuations take place. The rest of the time, the servers sat idle.

As the company began planning for its next stage of regulatory updates, it realized it was time to do something about its patchwork of increasingly costly IT systems.

A forecaster future-proofs its company

Manulife had to streamline its data intake systems and its reporting output. It wanted a new system capable of ingesting more data from a multitude of sources in a wide variety of formats—some from decades-old mainframes. The system needed to normalize the data to make it useable for the variety of applications that depended on it. The models themselves required more compute and storage capacity to run efficiently. And the system had to simplify reporting across the board for an array of global business units.

This was a great set of problems to hand off to cloud computing, with its endless scalability, on-demand provisioning, and built-in high availability and disaster recovery. “Azure offered the technological solution we were looking for,” says Bradbury. “We deal in long-range scenarios, and we trusted the commitment Microsoft was making to Azure.”

Bradbury led Valuation System Transformation (VST) program, which defined a new cloud-first strategy. The VST program was designed to create a strategic, enterprise-wide solution for the hundreds of actuaries across Manulife. The VST team wanted a modular solution that allowed components to be exchanged as needed and that supported flexible combinations of calculations—risk, liability, and assets, for example—to produce deeper insights.

Microsoft SQL Data Warehouse was at the center of the first iteration of the solution, providing a single source of truth for the terabytes of data used in the actuarial models. This data is used in AXIS, an actuarial analytic modeling tool from GGY and its parent company Moody’s Analytics. To run these big data workloads on Azure, the VST team took advantage of Azure infrastructure as a service (IaaS)—basically, really big virtual machines. How big was big enough, though? The VST team turned to Microsoft for help.

“We’re doing business with Microsoft because of its tremendous commitment to Azure and also its approach to open architectures and open-source capabilities. And Microsoft made itself extremely easy to do business with.”

Jon Bradbury, Senior Vice President, Manulife

An architecture for modeling at cloud speed and scale

The solution is a high-performance computing (HPC) architecture that ingests, processes, and analyzes huge volumes of data. It’s a classic extract, transform, and load (ETL) workload at big data scale on Azure.

Data is pulled from multiple data sources by an on-premises system, which sends it to a transaction data store on Azure. To filter, aggregate, and prepare the data for analysis, Manulife uses Informatica Intelligent Cloud Services, a cloud-based version of the popular data transformation and integration tool, PowerCenter. Informatica sends the cleansed data to SQL Server. By creating one source of data, Manulife simplifies analysis and improves the data quality for better reporting.

SQL Server is one example of the VST program’s intended flexibility. At first, the solution ran SQL Data Warehouse on virtual machines—an example of IaaS. Many companies move to the cloud this way, using IaaS resources to burst workloads. “With IaaS, there's the concept where the node performance is distributed across all the databases. People at the company were asking for that,” Mistry explains. But, as the engineering teams monitoring Azure grew more familiar with the platform’s capabilities, they saw an opportunity to get the compute power they needed more efficiently by using platform as a service (PaaS) in the form of Azure SQL Database.

PaaS refers to services hosted on and managed by Azure. Because the platform delivers key database management functions, such as upgrading, patching, backups, and disaster recovery, Manulife can focus on data quality. Azure SQL Database provides a highly available and high-performance data storage layer for the actuarial models, with built-in intelligence that provides performing-tuning recommendations and other automated monitoring insights. This setup enables line-of-business owners at Manulife to effectively build their own databases without impacting the performance of other teams using the data warehouse.

In addition, the various business owners can scale the compute and storage resources they need from Azure SQL Database, because the service is deployed in an unusual manner. Many enterprises run Azure SQL Database for their line-of-business applications—but after it’s deployed, they leave it alone. Manulife runs scripts developed in Informatica to dynamically increase and decrease the available CPU, memory, and input/output operations per second (IOPS), based on load. In Azure SQL Database, the virtual core (vCore) purchasing model lets customers choose the exact amount of compute and storage resources to provision for their workloads.

At Manulife, when teams run models that require a higher level of performance, they can request a boost to a higher service tier. For example, a team might scale up from a general-purpose S3 database with 250 GB of storage to a premium-tier P11 database with 4,096 GB of storage. After they run the modeling, the IT team runs the scripts to restore the S3 service tier. The ability to dynamically scale the vCore service tiers helps the company keep costs in check.

As for the actuarial models, they run in AXIS, which is deployed on compute-optimized virtual machines. As an HPC workload, AXIS places an extreme demand on the floating-point capabilities of the CPU. Manulife and GGY worked closely with Microsoft to optimize the environment that ran this key piece of software. The team chose virtual machines with huge capacity, including the Dv2-series with 16 CPU cores and 112 GB of memory.

Over time, the team found it could optimize AXIS performance and costs by scaling the workload out over more—and smaller—virtual machines. “We’re still learning,” Mistry notes. “It takes time to get to a cloud-first mindset.” The company even runs fewer models now to perform the same analysis with more consistent standards and data governance.

“One of the reasons why we went to the public cloud was the full encryption of data at rest and in transmission. What’s the benefit of having a private cloud if data encryption is fully available as a public service?”

Jon Bradbury, Senior Vice President, Manulife

Always good advice: Automate and monitor

A big component of the platform’s success comes from the automation processes that the engineering team set up using Azure Logic Apps. A type of serverless compute service, Logic Apps makes it easy to connect apps, data, and devices so teams can automate workflows and processes. “With Logic Apps, everything communicates without errors,” says Mistry. “It took me just a few hours to get the first proof of concept out. Now we use it to automate so many of our business processes.”

This automation has given line-of-business owners the ease of self-service provisioning on demand. In the past, business owners submitted a support ticket to request resources in a cumbersome approval process. That meant they needed to predict days in advance when to copy or scale up a database, and they didn’t know when the request would be approved. The long wait times was a sore spot for the company. 

Now they can fill out a simple web form to submit a ticket as part of a workflow solution that runs across the Azure infrastructure. The solution automates the approval process, while keeping everyone notified of the status. Turnaround times are significantly faster, and business owners are delighted.

The new ticketing system is commonly used by teams before they run a big model, giving them the power of scale-up with pushbutton ease. But it had an unexpected consequence at first—rising costs. Mistry advises any business getting into the cloud to pay attention to its infrastructure footprint. 

“Something we learned is to make sure we have real-time operational insights from day one,” he explains. “That lets you use the Azure services in their optimal capacity.”

The engineers found an easy way to provide better governance. They use Microsoft Power BI, a data visualization tool, to display real-time telemetry collected by Log Analytics, a feature of Azure Monitor. The engineering team now gets up-to-the-minute operational insights on a convenient Power BI dashboard that provides a consolidated view of Manulife’s Azure ecosystem.

“With minimal effort, we can automate so much using the support in Azure for open-source tools, like Terraform, with Azure Logic Apps and Azure Pipelines. That's a definite success story.”

Rohit Mistry, Infrastructure Analyst, Manulife

Next steps

When Manulife started its journey to the cloud, its disparate IT environment lacked even a global domain. As part of the cloud-first initiative, Manulife gradually collapsed more than 40 distinct Active Directory domains into a single global domain. Initially deployed in one Azure region, the solution now spans the globe in a dozen Azure regions. The original virtual private network (VPN) connection was replaced with several Azure ExpressRoute circuits for faster, more secure connections between the resources on Azure and those on-premises.

These numbers reflect a company that went all in when it moved to Azure. “Shifting our IaaS mindset was a challenge at first,” Mistry points out, “but now, it doesn’t make sense to do it any other way than on Azure.”

For more information, see Story collection: Explore how Manulife uses Microsoft cloud solutions for business innovation.

“Moving to Azure brought us a better, faster, cheaper, more secure infrastructure.”

Jon Bradbury, Senior Vice President, Manulife

Take the next step

Fuel innovation with Microsoft

Talk to an expert about custom solutions

Let us help you create customized solutions and achieve your unique business goals.

Drive results with proven solutions

Achieve more with the products and solutions that helped our customers reach their goals.

Follow Microsoft