http://dl.gi.de/handle/20.500.12116/21526 2026-07-21T13:34:09Z 2026-07-21T13:34:09Z Bartnik, Adrian Del Monte, Bonaventura Rabl, Tilmann Markl, Volker http://dl.gi.de/handle/20.500.12116/21739 2019-04-17T00:56:56Z 2019-01-01T00:00:00Z

On-the-fly Reconfiguration of Query Plans for Stateful Stream Processing Engines Bartnik, Adrian; Del Monte, Bonaventura; Rabl, Tilmann; Markl, Volker Grust, Torsten; Naumann, Felix; Böhm, Alexander; Lehner, Wolfgang; Härder, Theo; Rahm, Erhard; Heuer, Andreas; Klettke, Meike; Meyer, Holger Stream Processing Engines (SPEs) must tolerate the dynamic nature of unbounded data streams and provide means to quickly adapt to fluctuations in the data rate. Many major SPEs however provide very little functionality to adjust the execution of a potentially infinite streaming query at runtime. Each modification requires a complete query restart, which involves an expensive redistribution of the state of a query and may require external systems in order to guarantee correct processing semantics. This results in significant downtime, which increase the operational cost of those SPEs. We present a modification protocol that enables modifying specific operators as well as the data flow of a running query while ensuring exactly-once processing semantics. We provide an implementation for Apache Flink, which enables stateful operator migration across machines, the introduction of new operators into a running query, and changes to a specific operator based on external triggers. Our results on two benchmarks show that migrating operators for queries with small state is as fast as using the savepoint mechanism of Flink. Migrating operators in the presence of large state even outperforms the savepoint mechanism by a factor of more than 2.3. Introducing and replacing operators at runtime is performed in less than 10 s. Our modification protocol demonstrates the general feasibility of runtime modifications and opens the door for many other modification use cases, such as online algorithm tweaking and up-or downscaling operator instances.

2019-01-01T00:00:00Z Klauck, Stefan Plauth, Max Knebel, Sven Strobl, Marius Santry, Douglas Eggert, Lars http://dl.gi.de/handle/20.500.12116/21737 2019-04-17T00:56:55Z 2019-01-01T00:00:00Z

Eliminating the Bandwidth Bottleneck of Central Query Dispatching Through TCP Connection Hand-Over Klauck, Stefan; Plauth, Max; Knebel, Sven; Strobl, Marius; Santry, Douglas; Eggert, Lars Grust, Torsten; Naumann, Felix; Böhm, Alexander; Lehner, Wolfgang; Härder, Theo; Rahm, Erhard; Heuer, Andreas; Klettke, Meike; Meyer, Holger In scale-out database architectures, client queries must be routed to individual backend database servers for processing. In dynamic database systems, where backend servers join and leave a cluster or data partitions move between servers, clients do not know which server to send queries to. Using a central dispatcher, all queries and responses are routed via a single node. In a system with many high-performance backends, such a central node can become the system bottleneck. This paper compares three different approaches for query dispatching in terms of scaling network throughput and processing flexibility. Off-the-shelf TCP/HTTP load-balancers cannot dispatch individual queries arriving over a single connection to different backend servers, unless they are extended to understand the database wire protocol. For small response sizes up to 4 KB, a purpose-built query dispatcher delivers the highest throughput. For larger responses (i.e., BLOBs or data sets for external analysis), a novel approach for network proxying that transparently maps TCP connections between backend servers performs best.We propose hybrid query dispatching that performs a TCP connection hand-over on demand when returning large database results.

2019-01-01T00:00:00Z Schüle, Maximilian Passing, Linnea Kemper, Alfons Neumann, Thomas http://dl.gi.de/handle/20.500.12116/21738 2019-04-17T00:56:57Z 2019-01-01T00:00:00Z

Ja-(zu-)SQL: Evaluation einer SQL-Skriptsprache für Hauptspeicherdatenbanksysteme Schüle, Maximilian; Passing, Linnea; Kemper, Alfons; Neumann, Thomas Grust, Torsten; Naumann, Felix; Böhm, Alexander; Lehner, Wolfgang; Härder, Theo; Rahm, Erhard; Heuer, Andreas; Klettke, Meike; Meyer, Holger Große Datenmengen in Wirtschaft und Wissenschaft werden klassischerweise in Daten-banksystemen verwaltet. Um die Daten für maschinelles Lernen sowie für die Datenanalyse zu nutzen, ist ein zeitintensiver zyklischer Transformations-und Verschiebeprozess nötig, da die Daten hierfür in anderen Formaten oder schlicht in anderen Plattformen vorliegen müssen. Forschungsarbeiten der letzten Jahre widmen sich der Aufgabe, Datenverwaltung und Datenanalyse in einem System zu integrieren, um teure Datentransfers zu vermeiden. Diese Arbeit untersucht die Leistungsfähigkeit gespeicherter Prozeduren (stored procedures) zur Implementierung von Data-Mining-Algorithmen in Datenbanksystemen. Grundlage hierfür bildet HyPerScript, die PL/SQL-ähnliche Skriptsprache des Hauptspeicherdatenbanksystems HyPer. Ins-besondere evaluieren wir die prototypische Implementierung von fünf Algorithmen, die ganz ohne separates Datenanalysesystem auskommt.

2019-01-01T00:00:00Z Radke, Bernhard Neumann, Thomas http://dl.gi.de/handle/20.500.12116/21735 2019-04-17T00:56:58Z 2019-01-01T00:00:00Z

LinDP++: Generalizing Linearized DP to Crossproducts and Non-Inner Joins Radke, Bernhard; Neumann, Thomas Grust, Torsten; Naumann, Felix; Böhm, Alexander; Lehner, Wolfgang; Härder, Theo; Rahm, Erhard; Heuer, Andreas; Klettke, Meike; Meyer, Holger Choosing the best join order is one of the main tasks of query optimization, as join ordering can easily affect query execution times by large factors. Finding the optimal join order is NP-hard in general, which means that the best known algorithms have exponential worst case complexity. As a consequence only relatively modest problems can be solved exactly, which is a problem for today’s large, machine generated queries. Two developments have improved the situation: If we disallow crossproducts, graph-based DP algorithms have pushed the boundary of solvable problems to a few dozen relations. Beyond that, the linearized DP strategy, where an optimal left-deep plan is used to linearize the search space of a subsequent DP, has proven to work very well up to a hundred relations or more. However, these strategies have limitations: Graph-based DP intentionally does not consider implicit crossproducts, which is almost always ok but sometimes undesirable, as in some cases such crossprod-ucts are beneficial. Even more severe, linearized DP can handle neither crossproducts nor non-inner joins, which is a serious limitation. Large queries with, e. g., outer joins are quite common and having to fall back on simple greedy heuristics in this case is highly undesirable. In this work we remove both limitations: First, we generalize the underlying linearization strategy to handle non-inner joins, which allows us to linearize the search space of arbitrary queries. And second, we explicitly recognize potential crossproduct opportunities, and expose them to the join ordering strategies by augmenting the query graph. This results in a very generic join ordering framework that can handle arbitrary queries and produces excellent results over the whole range of query sizes.

2019-01-01T00:00:00Z