Distance-Based Sampling of Software Configuration Spaces

Abstract

Configurable software systems provide configuration options to adjust and optimize their functional and non-functional properties. However, to obtain accurate performance predictions, a representative sample set of configurations is required. Different sampling strategies have been proposed, which come with different advantages and disadvantages. In our experiments, we found that most sampling strategies do not achieve a good coverage of the configuration space with respect to covering relevant performance values. That is, they miss important configurations with distinct performance behavior. Based on this observation, we devise a new sampling strategy that is based on a distance metric and a probability distribution to spread the configurations of the sample set across the configuration space. To demonstrate the merits of distance-based sampling, we compare it to state-of-the-art sampling strategies on 10 real-world configurable software systems. Our results show that distance-based sampling leads to more accurate performance models for medium to large sample sets.