http://dl.gi.de/handle/20.500.12116/37799 2026-07-21T13:37:42Z 2026-07-21T13:37:42Z Schiller, Benjamin Daxenberger, Johannes Gurevych, Iryna http://dl.gi.de/handle/20.500.12116/37819 2021-12-16T13:25:23Z 2021-01-01T00:00:00Z

Stance Detection Benchmark: How Robust is Your Stance Detection? Schiller, Benjamin; Daxenberger, Johannes; Gurevych, Iryna Stance detection (StD) aims to detect an author’s stance towards a certain topic and has become a key component in applications like fake news detection, claim validation, or argument search. However, while stance is easily detected by humans, machine learning (ML) models are clearly falling short of this task. Given the major differences in dataset sizes and framing of StD (e.g. number of classes and inputs), ML models trained on a single dataset usually generalize poorly to other domains. Hence, we introduce a StD benchmark that allows to compare ML models against a wide variety of heterogeneous StD datasets to evaluate them for generalizability and robustness. Moreover, the framework is designed for easy integration of new datasets and probing methods for robustness. Amongst several baseline models, we define a model that learns from all ten StD datasets of various domains in a multi-dataset learning (MDL) setting and present new state-of-the-art results on five of the datasets. Yet, the models still perform well below human capabilities and even simple perturbations of the original test samples (adversarial attacks) severely hurt the performance of MDL models. Deeper investigation suggests overfitting on dataset biases as the main reason for the decreased robustness. Our analysis emphasizes the need of focus on robustness and de-biasing strategies in multi-task learning approaches. To foster research on this important topic, we release the dataset splits, code, and fine-tuned weights.

2021-01-01T00:00:00Z Pustejovsky, James Krishnaswamy, Nikhil http://dl.gi.de/handle/20.500.12116/37818 2021-12-16T13:25:23Z 2021-01-01T00:00:00Z

Embodied Human Computer Interaction Pustejovsky, James; Krishnaswamy, Nikhil In this paper, we argue that embodiment can play an important role in the design and modeling of systems developed for Human Computer Interaction. To this end, we describe a simulation platform for building Embodied Human Computer Interactions (EHCI). This system, VoxWorld, enables multimodal dialogue systems that communicate through language, gesture, action, facial expressions, and gaze tracking, in the context of task-oriented interactions. A multimodal simulation is an embodied 3D virtual realization of both the situational environment and the co-situated agents, as well as the most salient content denoted by communicative acts in a discourse. It is built on the modeling language VoxML (Pustejovsky and Krishnaswamy in VoxML: a visualization modeling language, proceedings of LREC, 2016), which encodes objects with rich semantic typing and action affordances, and actions themselves as multimodal programs, enabling contextually salient inferences and decisions in the environment. VoxWorld enables an embodied HCI by situating both human and artificial agents within the same virtual simulation environment, where they share perceptual and epistemic common ground. We discuss the formal and computational underpinnings of embodiment and common ground, how they interact and specify parameters of the interaction between humans and artificial agents, and demonstrate behaviors and types of interactions on different classes of artificial agents.

2021-01-01T00:00:00Z Shwartz, Vered http://dl.gi.de/handle/20.500.12116/37807 2021-12-16T13:25:23Z 2021-01-01T00:00:00Z

Dissertation Abstract:Learning High Precision Lexical Inferences Shwartz, Vered The fundamental goal of natural language processing is to build models capable of human-level understanding of natural language. One of the obstacles to building such models is lexical variability , i.e. the ability to express the same meaning in various ways. Existing text representations excel at capturing relatedness (e.g. blue / red ), but they lack the fine-grained distinction of the specific semantic relation between a pair of words. This article is a summary of a Ph.D. dissertation submitted to Bar-Ilan University in 2019, under the supervision of Professor Ido Dagan of the Computer Science Department. The dissertation explored methods for recognizing and extracting semantic relationships between concepts ( cat is a type of animal ), the constituents of noun compounds (baby oil is oil for babies), and verbal phrases (‘X died at Y’ means the same as ‘X lived until Y’ in certain contexts). The proposed models outperform highly competitive baselines and improve the state-of-the-art in several benchmarks. The dissertation concludes in discussing two challenges in the way of human-level language understanding: developing more accurate text representations and learning to read between the lines.

2021-01-01T00:00:00Z Van Gysel, Jens E. L. Vigus, Meagan Chun, Jayeol Lai, Kenneth Moeller, Sarah Yao, Jiarui O’Gorman, Tim Cowell, Andrew Croft, William Huang, Chu-Ren Hajič, Jan Martin, James H. Oepen, Stephan Palmer, Martha Pustejovsky, James Vallejos, Rosa Xue, Nianwen http://dl.gi.de/handle/20.500.12116/37809 2021-12-16T13:25:23Z 2021-01-01T00:00:00Z

Designing a Uniform Meaning Representation for Natural Language Processing Van Gysel, Jens E. L.; Vigus, Meagan; Chun, Jayeol; Lai, Kenneth; Moeller, Sarah; Yao, Jiarui; O’Gorman, Tim; Cowell, Andrew; Croft, William; Huang, Chu-Ren; Hajič, Jan; Martin, James H.; Oepen, Stephan; Palmer, Martha; Pustejovsky, James; Vallejos, Rosa; Xue, Nianwen In this paper we present Uniform Meaning Representation (UMR), a meaning representation designed to annotate the semantic content of a text. UMR is primarily based on Abstract Meaning Representation (AMR), an annotation framework initially designed for English, but also draws from other meaning representations. UMR extends AMR to other languages, particularly morphologically complex, low-resource languages. UMR also adds features to AMR that are critical to semantic interpretation and enhances AMR by proposing a companion document-level representation that captures linguistic phenomena such as coreference as well as temporal and modal dependencies that potentially go beyond sentence boundaries.

2021-01-01T00:00:00Z