Integrating Cluster-Based Main-Memory Accelerators in Relational Data Warehouse Systems

Abstract

Today, data warehouse systems are faced with challenges for providing nearly realtime response times even for complex analytical queries on enormous data volumes. Highly scalable computing clusters in combination with parallel in-memory processing of compressed data are valuable techniques to address these challenges. In this paper, we give an overview on core techniques of the IBM Smart Analytics Optimizer—an accelerator engine for IBM’s mainframe database system DB2 for z/OS. We particularly discuss aspects of a seamless integration between the two worlds and describe techniques exploiting features of modern hardware such as parallel processing, cache utilization, and SIMD. We describe issues encountered during the development and evaluation of our system and outline current research activities for solving them.