A Technical Framework for Investigating the Impact of Design Elements in the Wild: Using Instagram as an Example

Abstract

It is important to investigate effects of different interface designs not only for small sample systems but also for large real-world applications. However, it is not always possible for researchers to get direct access to these applications or relevant user data. Studies investigating possible negative effects such as keeping users occupied longer or making them spent more money might not get the consent of the application’s creator. We present a technical architecture and an open source implementation to conduct comparative studies on external apps by enabling researchers to turn specific interface elements on and off. The current implementation requires that the target app is available as a web app and that the study participants access it through a custom Android app. While the framework is not limited to a certain app, we use Instagram as an example and present results from a proof-of-concept study showcasing the framework.