Fast and efficient operational time series storage: The missing link in dynamic software analysis

Abstract

Distributed applications, cloud systems, the Internet of Things, etc. are generating increasing amounts of operational data, such as CPU loads, thread states, memory consumptions, method runtimes, or logs. Many tools continuously collect and analyze such data that is best represented as time series. Typical analyses try to find and localize runtime incidents like outliers, leaks, or trend anomalies. However, these analyses need an efficient use of storage and a fast interactive query execution, that general purpose storage systems do not provide: neither storing operational time series data in general-purpose databases nor in conventional time series databases fulfills these requirements. We present Chronix, a novel time series storage that is optimized for operational time series and that improves the link between storage and analysis in a dynamic software analysis toolchain. With Chronix a toolchain not only stores data 4–33 times faster and it takes 5–171 times less storage space than with other time series databases, it also executes queries in 15–74% and analyses in 25–74% of the time.