Marks & Spencer speeds up insights with Azure Synapse Analytics

From penny bazaar to powerhouse brand

The year is 1884, and Michael Marks has just opened a market stall in the Kirkgate Market in Leeds, England. The air hums with commerce as Marks, a keen street-peddler-turned-salesman, combines a smart business idea with savvy advertising to stand out in the crowded marketplace —literally. Knowing that shoppers want fine goods at family prices, he coins the slogan, “Don’t ask the price, it’s a penny.” The rest is history. The penny bazaar is a hit, and the famed English retailer is born. 

More than 130 years later, Marks & Spencer (M&S) has outgrown the humble market stall, but not its commitment to selling high-quality items at a great value. The company keeps this promise alive through its branded M&S product lines, all of which are made in-house. “We don’t tend to sell too many third-party products. Instead, we concentrate our focus on our own label ranges across both food and clothing. We do all of our own product development, and it’s helped us create a really strong brand,” says Aaronpal Dhanda, Head of Data Technology at M&S Digital and Data. 

This lets M&S own every part of its supply chain and collect data across all touch points—from sourcing materials to manufacturing to logistics to a customer’s shopping cart. “We collect a lot of business and customer analytics. We want to know what they like, why they buy what they buy, why they abandon baskets, and so on. This lets us see how products perform and influence design as a result,” says James Ferguson, Product Manager at M&S.

The downside of an on-premises datacenter

M&S’s holistic approach allows the company to lead with innovation, but its on-premises datacenter struggled to keep pace with the increasing demands. “If you look at our food division, for example, they need to know where our product is in every store, every day, multiple times a day, and then model against that to forecast demand. The number of data points becomes massive, and our on-premises data cluster couldn’t get anywhere near that,” says Ferguson.

Scale was an additional limitation. Retailers’ data loads often would swing wildly from one day to the next, requiring the agility to spin clusters up and down as necessary. But M&S’s on-premises data warehouse didn’t offer the economy of scale to effectively respond to “peaky” ebbs and flows. And adding scale would mean layering additional hardware on top of its existing—and expensive—on-premises datacenter. Needing an agile solution that offered better scale, performance, and access to data, M&S turned to the cloud.

A singular partner in Microsoft

M&S evaluated cloud vendors over the next few months and ultimately decided to replace its on-premises data warehouse with a Microsoft Azure–based data platform built around Azure Synapse Analytics, based on its promises of analytics, storage, and scale. The company also wanted a single vendor that offered seamless integration. “There are very few cloud vendors that do the whole data science services or cloud services end to end. For us, this is important because we know we’ll get tighter and tighter integration between different products, giving us more options to how we design solutions,” says Dhanda.

Ferguson emphasizes the challenge of being a legacy enterprise with large, varied tool chains. “Being caught between two suppliers with a problem is a very hard place to be,” says Ferguson. “With Microsoft, we go to them for everything, even for products not native to them. It’s pretty seamless from a customer point of view.”

Democratizing data with Azure Synapse Analytics

While the implementation is currently in process, M&S migrated workloads from their customer growth team into Azure Synapse Analytics as a trial use case. The solution (Figure 1) pulls data stored as Parquet files from on-premises Cloudera HDFS clusters into Azure Data Lake Storage. Azure Databricks is used for transformation of the data, and data scientists can then push that data into Azure Synapse directly to run visualizations and create dashboards. 

According to Ferguson, this gives analyst teams new inroads to valuable data they didn’t have before. “We needed a place where they could use self-service to get responses quickly. They’re using Azure Synapse to push all that data out to Microsoft Power BI. Now, people who didn’t typically have access to data or had to wait for IT to create a dashboard, they’re able to figure it out themselves.”

Other teams are benefiting from self-service as well. “By democratizing access, we’re allowing more people to have ideas, and these ideas add incremental value to the business. Our retail support team is already reshaping how we report stock and stock loss to managers. Since they work closest to the retail side of the business, they know exactly how to consolidate and present information in ways that lead to real improvements,” explains Dhanda.

. M&S solution architecture; for a larger version, go to the Downloads section of the left-hand sidebar

Centralizing storage inside a lake

As a giant enterprise, every M&S department has different data requirements. The retail team stores data differently than the food supply team, which stores data differently than the financial team. With Azure Data Lake Storage (ADLS), M&S has a centralized place to store information that aligns with every team’s needs. “ADLS is a really simple way to centralize data,” says Dhanda. “This gives us a way of merging our use cases together in one place. We’re storing data once for those different types of use.”

ADLS also delivers a more scalable way to move tools between environments, which can be expensive due to overhead and reconciliation costs. But with Azure, it’s easy to quickly spin up new environments using a combination of technologies based on need. 

“You might have a requirement where you need a fast response time. Or you might want something that’s lower cost and scalable to the type of data you have. With Azure, you can choose between different technologies to form a whole solution,” explains Dhanda.

Building a scalable future

With these promising early use cases under its belt, M&S looks forward to taking advantage of the full scaling capabilities of Azure Synapse in the future. The company is particularly excited about the potential to scale and read data between ADLS and Azure Synapse. “That means we potentially won’t need to have one data warehouse environment. We can have several spinning up but still reading form the same data set. That opens a world of options around how we service different areas of the business in different ways,” says Dhanda. 

M&S may no longer sell goods for just single a penny. But through its new platform with Azure, M&S is continuing its tradition of turning customer insights into innovative, affordable products loved around the world.

Find out more about Marks & Spencer on Twitter, YouTube, and Facebook.