Established in 1971 as the world’s first electronic stock market, NASDAQ OMX owns and operates three clearing houses, five central securities depositories, and 26 markets (including the NASDAQ Stock Market) with a combined value that exceeds US$8 trillion. NASDAQ OMX is also a technology company. Its infrastructure includes custom applications and a trading engine that’s used internally and by 80 global marketplaces.
In terms of volume, the company processes more than 1 million messages per second. In addition, the company manages numerous databases that store petabytes of online information.
Financial traders need near-real-time insight into current and historical data to make decisions, in just seconds, that can involve millions of dollars. Analysts also need in-depth insight to understand trends and create regulatory reports. To meet these requirements in the past, IT staff had to constantly monitor and fine-tune databases so that they could provide the best possible performance, especially with uncompressed data growing by 500 TB each year. By 2013, NASDAQ OMX recognized that its existing database solutions couldn’t support projected long-term growth.
When NASDAQ OMX learned about SQL Server 2014 software and its in-memory columnstore, IT staff decided to migrate data warehouses to SQL Server 2014 Enterprise and the Windows Server 2008 R2 Enterprise operating system. By choosing SQL Server 2014 and taking advantage of in-memory columnstore—as well as other new capabilities—the company believed it could better manage its massive amounts of data, speed queries, and simplify database maintenance.
Commenting on the company’s continued use of the Microsoft platform, Stanley Orlowski, Director of Database Structures at NASDAQ OMX says, “The Microsoft technology stack gives us excellent total cost of ownership, and we have a high degree of confidence in the platform. We’ve also had SQL Server in place for well over a decade on mission-critical systems, and it has a positive track record for delivering reliable performance.”
In 2013, engineers initially migrated the Nordic OMX data warehouse—which includes 8 TB of page-compressed data—to SQL Server 2014. Engineers are currently upgrading the US Options and Equities archives to SQL Server 2014. They expect to complete the database upgrade by the middle of 2014 and finish converting tables to in-memory columnstore by the end of 2014. As a result, the company will save 500 TB of disk space this year by using new data compression and indexing capabilities. It is also saving time and money on data management, improving query performance, and helping to facilitate long-term goals. SQL Server 2014 is a game changer for us. Its in-memory columnstore, advanced statistics, and cardinality estimation are the tools we need to manage our very large databases for the long term,” says Orlowski. “By making data more readily available to our customers, we can also help them make better business decisions.”
- Compresses data to one-quarter its original size, saving 500 TB of space
- Speeds queries and increases accuracy
- Saves time on maintenance and cuts storage footprint in half
- Improves decision-making and helps facilitate long-term goals