Clearent scales its modern microservices architecture to handle 500 million payment transactions a year using the Azure SQL Database hyperscale service tier

Fast and affordable payment processing

Every time you make a purchase on your credit card, whether it’s for groceries, clothes, or concert tickets, that transaction must be processed. In total, details are transferred quickly and securely between issuing banks and merchants millions of times every day.

One of the companies providing this service is Clearent. Clearent has built its payment technology from the ground up to deliver fast processing speeds, transparency for merchants and their customers, and low overall costs.

But 500 million payment transactions a year come with massive data loads, high concurrency rates, and the need to scale quickly and elastically. To help save money, streamline operations, increase agility, and meet its brand promises of trust, confidence, and speed, Clearent chose the Microsoft Azure SQL Database hyperscale service tier to support its core offerings.

More payments, more data, more complexity

Payment processing can be a complex task. For example, Clearent must handle the daily spikes in demand that come with batch processing.

“Thousands of customers send us batches at the end of every day,” says Justin Rohde, Chief Transformation and Product Officer at Clearent. “So, we need to scale quickly to adequately service all those customers.”

As card transactions have played a bigger role in everyday life, Clearent has experienced rapid growth. It needed capacity to scale up and out organically, but with data volumes approaching 5 terabytes (TB), it became increasingly clear that the company’s existing data warehouse solution could no longer meet its needs.

If Clearent couldn’t scale quickly to handle growing demand, it wouldn’t be able to provide the vital insights its customers need to complete their reporting processes each morning—let alone the additional insights that can help customers improve their business models. More importantly, processing financial information leaves no room for failure. Every transaction must be handled perfectly, and any dip in fidelity could threaten the entire Clearent business model.

“We process billions of dollars a year in card payments,” says Andrew Wieck, Manager of Business Analytics at Clearent. “That’s hundreds of millions of transactions a year—and it’s a number that’s continuing to grow and placing a strain on our infrastructure.”

Clearent needed to replace its data warehouse with a powerful solution that could compile data, ensure it was available to the right people at the right time, and scale up, out, and back down again quickly, easily, and as necessary.

Taking data management to hyperscale

Clearent now uses the Azure SQL Database hyperscale service tier to support its workload of dozens of microservices, manage real-time data ingestion, and process millions of credit card transaction records every month. And if necessary, the company can effortlessly continue to scale up without the same limits on resources traditionally seen in cloud databases.

The company integrates data from dozens of different sources into an operational data store and enterprise data warehouse within Azure before it analyzes the information using Microsoft Power BI. To help customer data stay in the right hands, Clearent uses row-level security to control access in the hyperscale tier.

“The most important thing was to make sure we can ingest a huge amount of data from all our transactions and microservices and then rapidly serve that data up to our customers,” says Zach Fransen, Director of Data Management at Clearent. “We have an enormous amount of data that needs to be available to a large number of employees and customers, and the hyperscale service tier is perfect for hitting that sweet spot of concurrency and scale.”

The hyperscale tier suits this scenario well because of its easy, automated, and virtually unlimited scalability. Clearent can now ingest massive volumes of data with confidence and then seamlessly scale back after batch reporting. The solution automatically scales storage as needed—up to 100 TB. This means that as the company continues to grow, it can easily adapt without having to plan complex infrastructure changes. This seamless scalability also offers huge benefits to developers and architects in their day-to-day work on Clearent’s modern microservices architecture.

Clearent notes the hyperscale tier’s use of secondary SQL Database readable replicas as another big advantage. By replicating changes to the master instance in near real time and dedicating additional nodes to compute queries, Clearent can scale its CPU by up to 400 percent, helping to run the complex queries that lead to actionable insights.

“Because of the nature of the hyperscale tier architecture, the intelligent use of SQL features, and the fact that Microsoft adds so many read replicas, we can really throw as much data as we want in there without suffering any performance degradation,” says Fransen.

Real-time insights for everyone

Clearent can now quickly scale to meet growing demands, while it uses the flexibility and data integrity provided by the hyperscale service tier to help ensure that all transactions are handled safely and consistently. “Hyperscale made it easy for us to support our growing workload and the dozens of microservices that power our core ecosystem,” says Wieck.

It also provides thousands of merchants with concurrent access to a live database where they can observe what’s happening in near real time. This helps both Clearent and its customers gain vital insights without delay and with minimal IT management complexity.

“With the Azure SQL Database hyperscale service tier, we can look at anything from trends in the restaurant industry to how changes to Mastercard payments might affect our customers’ business models,” says Wieck. “Before we moved to the hyperscale tier, this work would have taken hours. But now we can present useful data within five minutes.”

Clearent applies its new insights to better anticipate industry trends, tailor marketing campaigns to the most relevant merchants, expand its offering, streamline development and upgrades, and set up new services with minimal development effort.

Most importantly, Clearent can give its customers what they really want.

“Our ability to aggregate data in real time helps us with everything from monitoring and mitigating risk to improving value for our customers,” says Fransen. “With our combination of SQL Database and Power BI, we can offer unique reporting capabilities. And that’s great, because it’s increasingly becoming what our customers expect.”

Find out more about Clearent on Twitter, Facebook, and LinkedIn.