2004 Reports

Machine Learning and Text Segmentation in Novelty Detection

Schiffman, Barry; McKeown, Kathleen

This paper explores a combination of machine learning, approximate text segmentation and a vector-space model to distinguish novel information from repeated information. In experiments with the data from the Novelty Track at the Text Retrieval Conference, we show improvements over a variety of approaches, in particular in raising precision scores on this data, while maintaining a reasonable amount of recall.