Applying Voting Methods in User Research

Abstract

In usability practice, it is often necessary to find a 'best' solution that is most suitable for most users. A common way to do this is to present multiple alternatives to a group of users and let them express their preferences. However, integrating many subjective ratings into one generally valid rating is non-trivial. For such data, voting methods from the field of *computational social choice* might prove useful. In order to investigate their suitability, we conducted an experiment where participants had to complete a pointing task using four different mouse acceleration values and rank the perceived performance of each option via Likert-type items and also by ranking them. For generating an overall ranking, we calculated arithmetic mean and median as well as Borda count and Schulze method for each of the response sets. All methods resulted in nearly the same overall ranking of options which suggests that they might be equally useful for generating consensus on a 'best' option.