Process Engine Benchmarking with Betsy in the Context of ISO/IEC Quality Standards

Abstract

Business process management and automation has been the focus of intense research for a long time. Today, a plethora of process languages for specifying and implementing process models have evolved. Examples for such languages are established international standards, such as BPEL 2.0 or, more recently, BPMN 2.0. Implementations of these standards which are able to execute models, so called process engines, differ in their quality of service, e.g., in performance or usability, but also in the degree to which they actually implement a given standard. Selecting the “best” engine for a particular use case is hard, as none of the existing process standards features an objective certification process to assess the quality of its implementations. To fill this gap, we present the current achievements in process engine benchmarking and point out future directions.