DataPath: A Data-Centric Analytic Processing Engine

June 30, 2010
Alin Dobra | University of Florida

In this talk I will cover in detail DataPath, a database system designed from ground up to process analytical queries on 1-10TB loads on hardware in sub 100,000 range. DataPath has a number of unique features. First, it is a data-centric, as opposed to a compute centric, engine that focuses on data-flows rather than computation. The flow of data is of primary concern rather than processing. Throughout the memory hierarchy, once the data is accessed, as much processing as possible is performed. Second, DataPath relies heavily on fast linear scans. By accessing disks in parallel, DataPath is capable of scanning data at more than 2GB/s on 20,000 hardware. Third, DataPath relies on C++ code generation on the fly. By compiling SQL queries to low overhead C++ code, the execution engine can keep up with the fast I/O and process close to 100 million tuples/second. By consolidating the processing that multiple queries require around the data accesses, DataPath gives excellent performance on compute-intensive tasks like Q1 in TPCH. One of the main focus in the talk will be the interplay between the hardware, operating system and database code and how a careful end-to-end design can ensure good performance throughout. This is work in collaboration with Chris Jermaine at Rice University and students at both UF and Rice.

Speaker Details

Alin Dobra is an Associate Professor at University of Florida. He graduated with a Ph.D from Cornell University in 2003. His main specialty, until recently, is Approximate Query Processing with an emphasis on Sketching and Sampling. His work with Chris Jermaine on DBO System received the “Best Paper” award at SIGMOD 2007. For the last year, Alin changed focus to designing engines for Analytical Query Processing with an eye towards combining approximation techniques with high-performance engines like DataPath.