Filtering relevant text passages based on lexical cohesion

Abstract

Monitoring news and blogs has become a promising application for global operating groups, who are interested in recognizing topic developments in a fragmented topic landscape. News articles especially long ones may consist of several topics or different aspects of the same topic. In terms of Topic Detection and Tracking (TDT) it is hard to figure out the topic development in a stream of news or blog articles with the scope of a certain information need since articles often contain only a limited amount of the relevant information. In this paper we address the problem of filtering relevant portions of text, commonly known as passage retrieval, by using linear text segmentation methods based on lexical cohesion. We present two strategies for passage retrieval and compare their performance with cohesion based approaches – TextTiling (cf. [Hea97]) and TSF (cf. [KG09]) – developed in the context of linear text segmentation.