NoSQL on SQL

In last few years, we’ve seen a surge of NoSQL databases, systems that are built from scratch to tackle new data formats and application requirements commonly believed unfit to SQL databases. Building from scratch provides good opportunities to abandon historical burden and re-design systems with state-of-the-art wisdom. However, it also means that we have to re-build many canonical SQL technologies all over again, technologies that are products of years of research and development and are evolving at an unprecedented pace, such as columnar storage, vectorized query execution and lock free data structures, to name a few.

We launched a series of efforts to study a simple yet fundamental question: what is the fundamental gap between SQL and NoSQL, and how to bridge it? The answer to the question would have profound implications. First, if today’s SQL databases can be augmented to support NoSQL features without major kernel changes, we would be able to deliver NoSQL databases with only minor re-engineering. Second, even if new systems are to be built from scratch, taking new technological trends, designers and developers should still understand what technologies are unique to the very nature of NoSQL and what are applicable to both SQL and NoSQL.

Our efforts since then have produced two NoSQL systems, JSON Server and GraphView (opens in new tab), that take SQL databases and existing data platforms as the common core to support JSON and graphs respectively. JSON Server and GraphView are middleware that provide to applications a view of JSON and graph databases, while internally represent JSON documents/graphs in native data structures of the underlying systems and translate JSON/graph queries into queries executed by them. The key element is to resemble the physical data representation and runtime behavior of those NoSQL counterparts (such as MongoDB and Neo4j), so that computation complexity gap is bridged. Given that legacy databases enjoy a long history of development and optimizations, the JSON Server and GraphView offerings not only shine in features and functionality, but excel in performance.