An FPGA Avro Parser Generator for Accelerated Data Stream Processing

Abstract

Big Data applications frequently involve processing data streams encoded in semi-structured data formats such as JSON, Protobuf, or Avro.A major challenge in accelerating data stream processing on FPGAs is that the parsing of such data formats is usually highly complex.This is especially true for JSON parsing on FPGAs, which lies in the focus of related work.The parsing of the binary Avro format, on the other hand, is perfectly suited for being processed on FPGAs and can thus serve as an enabler for data stream processing on FPGAs.In this realm, we present a methodology for parsing, projection, and selection of Avro objects, which enforces an output format suitable for further processing on the FPGA.Moreover, we provide a generator to automatically create accelerators based on this methodology.The obtained accelerators can achieve significant speedups compared to CPU-based parsers, and at the same time require only very few FPGA resources.