Evolving the architecture of a DBMS for modern hardware

Abstract

The major commercial database systems were designed primarily for OLTP workloads and under the assumption that processors are slow, memory is scarce, and data lives on disk. These assumption are no longer valid: OLAP workloads are now as common as OLTP workloads, multi-core processors are the norm, large memories are affordable, and frequently accessed data lives mostly in the main memory buffer pool. So how can a vendor with a mature DBMS product exploit the opportunities offered by these changes? Rewriting from scratch is not realistic - it is way too expensive and risky. The only realistic option is to gradually evolve the architecture of the system. SQL Server has begun this journey by adding two features: column store indexes to speed up OLAP-type queries, and Hekaton, a new engine optimized for large memories and multicore processors. The talk will outline the design of these features, the main goals and constraints, and discuss the reasoning behind the design choices made.