Enabling Informational Autonomy through Explanation of Content Moderation: UI Design for Hate Speech Detection

Abstract

Content moderation using AI and in particular Hate Speech detection has been a research topic with a focus on natural language processing, classification algorithms and data benchmarks. Less attention has been dedicated to how the classification systems are later integrated into tools which support users in an application task. In this paper we review existing tools and prototypes. Furthermore, we design and implement an online user interface for explainability. The system is connected to a neural network classifier based on the HASOC benchmark. The interface allows users to enter messages, observe classification decisions and see similar messages for explanation. It provides support for users of social media who are interested in the performance of AI systems for content moderation and who want to observe the performance of hate speech detection tools. A qualitative evaluation with experts showed that our system can be helpful to bridge the gap between humans and AI.