Query Optimization in Heterogenous Event Processing Federations

Abstract

Continuous processing of event streams evolved to an important class of data management over the last years and will become even more important due to novel applications such as the Internet of Things. Because systems for data stream and event processing have been developed independent of each other, often in competition and without the existence of any standards, the Stream Processing System (SPS) landscape is extremely heterogeneous today. To overcome the problems caused by this heterogeneity, a novel event processing middleware, the Java Event Processing Connectivity (JEPC), has been presented recently. However, despite the fact that SPSs can be accessed uniformly using JEPC, their different performance profiles caused by different algorithms and implementations remain. This gives the opportunity to query optimization, because individual system strengths can be exploited. In this paper, we present a novel query optimizer that exploits the technical heterogeneity in a federation of different unified SPSs. Taking into account different performance profiles of SPSs, we address query plan partitioning, candidate selection, and reducing inter-system communication in order to improve the overall query performance. We suggest a heuristic that finds a good initial mapping of sub-plans to a set of heterogenous SPSs. An experimental evaluation clearly shows that heterogeneous federations outperform homogeneous federations, in general, and that our heuristic performs well in practice.