TAMEX: A task-based query execution framework for mixed enterprise workloads on in-memory databases

Abstract

In-memory database management systems (DBMS) have been proposed to run transactional and analytical applications on a single database instance and to reduce the execution time of complex analytical queries to seconds. The two main reasons for this dramatic performance increase are massive intra-query parallelism on many-core CPUs and primary data storage in main memory. The benefits of these in-memory DBMS for enterprises are huge: analytical applications become largely independent of data staging delays, opening the way for real-time analytics. However, this promising approach will only be adopted, if DBMS can execute dynamically arriving transactional queries in a timely manner, even while complex analytical queries are executed. We believe that two system properties are key to achieve this objective: (1) splitting queries into fine granular atomic tasks and (2) efficiently assigning these tasks to a large number of processing units, thereby considering priorities of query classes. In this paper, we propose TAMEX, a framework for the execution of multiple query classes, designed for executing queries of heterogeneous workloads of enterprise applications on in-memory databases. The basic idea is to generate a task graph for each query during query compilation and assign these tasks to processing units by a user-level scheduler based on priorities. We evaluate the concept using a mix of transactional and join-heavy queries and focus on the impact of task sizes on load balancing and responsiveness of the system.