Performance measurement of heterogeneous workflow engines

Nowadays companies formalize their business processes to maximize their efficiency and effectiveness. Workflows are business processes with components that automate companies’ processes and the workflow executions are managed by the Workflow Management System (WfMS). In the last years the number of WfMSs, and in particular the WfMSs supporting the latest version of Business Process Model and Notation (BPMN 2.0), has increased. Even if a benchmark of WfMS BPMN 2.0 is recognized as necessary and similar tools for benchmarking WfMSs exist, a benchmark of WfMS BPMN 2.0 has still not been developed. In this context, the BenchFlow project has the goal to be the first BPMN 2.0 WfMS benchmark. The BenchFlow framework is formed by a set of services and its components are: the driver, the collector, the monitor, the data cleaner and reconciler and the performance meter and data aggregator. This thesis is a contribution to the BenchFlow project and it focuses on the design, implementation and evaluation of the data cleaner and reconciler and of the performance meter and data aggregator. Additionally, in this thesis, WfMS metrics are defined employing the Software Measure Definition Method; Processes Completion Time, Number of Completed Processes and Number of Uncompleted Processes metrics are implemented in the performance meter and data aggregator. Finally, to prove that the application of the two tools developed is feasible, as proof of concept, load tests are performed on two WfMSs and the implemented WfMS metrics are applied to compare their performance. Thus, the main contributions of this thesis to the research community are the creation of two tools, one that standardizes data from all the source databases of different WfMSs, and one that aggregates stored data to obtain information about WfMS BPMN 2.0 metrics, belonging to the BenchFlow framework, and their application on one trial experiment run as a proof of concept.