Algorithm engineering for big data

Abstract

Perhaps the most fundamental challenge implied by advanced applications of big data sets is how to perform the vast amount of required computations sufficiently efficiently. Efficient algorithms are at the heart of this question. But how can we obtain innovative algorithmic solutions for demanding application problems with exploding input sizes using complex modern hardware and advanced algorithmic techniques? This tutorial gives examples how the methodology of algorithm engineering can be applied here. Examples include sorting, main memory based data bases, communication efficient algorithms, particle tracking at CERN LHC, 4D image processing, parallel graph algorithms, and full text indexing. Compared to a previous tutorial in Koblenz 2013 with the same title, this tutorial talks less about methodology and more about actual algorithms and applications. For further reading refer to [San13] and, for selected individual results to [DS03, KS07, SSP07, MS08, San09, RSS10, SS12, DS13].