Towards Large-Scale Meteorological Data Services: A Case Study

Abstract

Meteorological data contribute significantly to “Big Data”, handling multi-dimensional raster data cubes up to 5-D and with single cubes up to multi-Petabyte sizes. Due to the lack of support for raster data, traditionally file-based implementations have been used for serving such data to the community, rather than databases. Array databases overcome this by providing storage and query support.In this paper, we present a case study conducted by Deutscher Wetterdienst (DWD) where extraction and processing of gridded meteorological data sets has been investigated hands-on. Following a brief introduction of the rasdaman DBMS used, we present the database schema used and a series of array queries, selected according to their practical importance in weather forecast services. We discuss several issues that have come up, such as null values and time modeling, and how they have been addressed. To the best of our knowledge, this is the first non-academic deployment of an array database for up to 5-D data sets.