Direct Edge, one of the largest equities exchanges in the world, wanted a better, faster business intelligence (BI) solution for its financial analysts to use to create reports. Direct Edge implemented a data warehouse and BI solution based on
Microsoft SQL Server 2008 R2 Parallel Data Warehouse, which has given the company more visibility into its data. The firm can also provide analytical reports in seconds rather than hours and can better drive business growth.
Direct Edge is the world’s fourth largest equities exchange. The organization offers its customers market data services and trading functionality across two distinct platforms: EDGA and EDGX.
As a fast-growing stock exchange, Direct Edge needed a better business intelligence solution to handle the increasing amounts of finance, compliance, and legal data collected through its trading operations. “We trade upwards of 10 percent of the total market
volume each day, so we generate a lot of important data,” says Richard Hochron, Chief Technology Officer, Direct Edge. However, although the firm used a Microsoft SQL Server 2008 database environment for reports and queries, the company’s financial analysts
lacked a proper interface to the data. “Our analysts relied on the IT department to create reports, but IT had to prioritize that task alongside maintaining the databases, so requests for reports weren’t handled fast enough,” he adds.
The problem became worse in late 2010, when Direct Edge converted from an electronic communications network to a registered securities exchange. “We doubled in size, and we had more users asking different questions,” says Roger Dahlstrom, Head of Back Office
and Reporting, Direct Edge. “Our data sets grew larger every time someone had a complex question, and it sometimes took between two and four hours to generate a response to the query, which was unacceptable.”
The IT team also relied on subsets of data to respond to queries. “We couldn’t get the complete data set, and that had the potential to affect business decisions.”
To address these issues, the firm decided to look for a BI and data warehouse solution.
||With SQL Server 2008 R2 Parallel Data Warehouse and the Microsoft BI tools we’re using, our analysts have a much deeper understanding of trading data.
| Richard Hochron
Chief Technology Officer
In June 2010, after investigating competitive offerings, the company decided to work with BI Voyage, a Microsoft partner with a Gold Business Intelligence competency, to implement a Microsoft data warehouse and BI solution. Specifically, Direct Edge was
interested in the Microsoft SQL Server 2008 R2 Parallel Data Warehouse, which provides high-speed parallel query processing, scalable data storage, and high-speed data transfer. “The solution is very powerful, and it is built on technologies we know and understand,”
In February 2011, BI Voyage and Microsoft executed a Parallel Data Warehouse proof of concept (POC) using 4 terabytes of Direct Edge sample data. This was then expanded to 68 terabytes. A trading data mart was designed and populated along with a collection
of reports. The POC clearly demonstrated the technology’s capabilities. “We saw load times of close to 1 terabyte per hour during testing, which easily surpassed our expectations,” says Hochron.
Satisfied with the results, in May 2011 Direct Edge purchased an HP Enterprise Data Warehouse Appliance optimized for Microsoft SQL Server 2008 R2 Parallel Data Warehouse. The solution, jointly engineered by HP and Microsoft, includes an enterprise data
warehouse based on HP ProLiant DL360 and ProLiant DL380 servers, and a BI environment based on HP ProLiant DL980 servers. The solution also features Microsoft Excel 2010 spreadsheet software and Microsoft SQL Server 2008 R2 PowerPivot for Excel, a data analysis
tool. Direct Edge also implemented Microsoft SQL Server 2008 R2 Analysis Services, used to create analytical cubes.
More than 30 employees use the solution, which incorporates the firm’s core trading data and additional data from its customer relationship and financial systems.
In early 2013, Direct Edge completed testing on Microsoft SQL Server 2012 Parallel Data Warehouse, the latest version of the technology. Tests showed an eight-fold increase in data compression, compared with a four-fold increase in SQL Server 2008 R2 Parallel
Data Warehouse. Direct Edge also tested query performance as well as integration with Apache Hadoop, an open-source software framework that analyzes large amounts of unstructured data in a distributed manner.
Direct Edge now has a more in-depth understanding of its data and can deliver analytical reports in seconds. The firm can also expand its business by focusing on analytical data.
Gives Analysts Deeper Understanding of Trading Data
With the new solution, Direct Edge has given its business analysts new capabilities. Using SQL Server Analysis Services and PowerPivot for Excel, the company’s power users can explore Parallel Data Warehouse information directly. “With SQL Server 2008 R2
Parallel Data Warehouse and the Microsoft BI tools we’re using, our analysts have a much deeper understanding of trading data. For example, they can better understand monthly fluctuations in trading fee revenue,” says Hochron. “And they can mine the data themselves,
so they no longer rely on IT to create reports.”
Runs Queries 163 Times Faster and Speeds Decision Making
During testing on SQL Server 2008 R2 Parallel Data Warehouse, Direct Edge ran complex queries and discovered they were processed 163 times faster than before. The company can now execute Big Data–based reports in 30 seconds or less, compared with several
hours on the previous system. “With the fast query processing provided by Parallel Data Warehouse, we can take in larger data sets,” says Hochron. “And with more data available in a shorter time frame, our analysts can make more-informed business decisions
faster than before.”
Direct Edge anticipates even faster performance once it implements SQL Server 2012 Parallel Data Warehouse, because tests showed a three-fold increase in performance compared with SQL Server 2008 R2 Parallel Data Warehouse. Additionally, SQL Server 2012
Parallel Data Warehouse tests demonstrated smooth integration with Hadoop, showing the potential for Direct Edge to use unstructured data for Big Data analysis.
Drives New Business Growth
Direct Edge is also using its new data warehouse and BI capabilities to drive new business. “This solution has opened up some new doors for us,” Hochron says. “It makes the whole company more data-driven. Structuring strategic decisions using this solution
will definitely help us build our business and grow in the future.”
This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.