Understanding Algorithms through Exploration: Supporting Knowledge Acquisition in Primary Tasks

Abstract

We investigate exploration as an alternative to explanation to improve user understanding of algorithms and algorithmic decision-making. Drawing on complex problem-solving as defined in cognitive science, we conducted a think-aloud study in the lab (N=10) as well as an MTurk online study (N=123) using a flight booking scenario to see if and how exploration supports \textit{knowledge acquisition} in two different tasks. One group was told to focus on booking the cheapest flight (knowledge acquisition as a secondary task), the other on understanding the system logic (knowledge acquisition as a primary task).

Our results indicate that exploration, even as a secondary task, may contribute to knowledge about the underlying algorithm. However, our study also suggests that the overall knowledge acquired through exploration is limited in the sense that it gives people an idea of how a system works, rather than teaching them concrete rules they can recall.

Overall, we conclude that exploration presents a design opportunity to interweave knowledge acquisition with users' primary task, and may thus contribute to (but not substitute) existing design solutions for supporting users in understanding algorithmic decision-making.