Selecting and Categorizing Textual Descriptions of Images in the Context of an Image Indexer's Toolkit

Rebecca J. Passonneau; Tae Yano; Judith L. Klavans; Rachael Bradley; Carolyn Sheffield; Eileen Abels; Laura Jenemann

Selecting and Categorizing Textual Descriptions of Images in the Context of an Image Indexer's Toolkit
Passonneau, Rebecca J.
Yano, Tae
Klavans, Judith L.
Bradley, Rachael
Sheffield, Carolyn
Abels, Eileen
Jenemann, Laura
Technical reports
Center for Computational Learning Systems
Permanent URL:
CCLS Technical Report
Part Number:
Center for Computational Learning Systems, Columbia University
Publisher Location:
New York
We describe a series of studies aimed at identifying specifications for a text extraction module of an image indexer's toolkit. The materials used in the studies consist of images paired with paragraph sequences that describe the images. We administered a pilot survey to visual resource center professionals at three universities to determine what types of paragraphs would be preferred for metadata selection. Respondents generally showed a strong preference for one of two paragraphs they were presented with, indicating that not all paragraphs that describe images are seen as good sources of metadata. We developed a set of semantic category labels to assign to spans of text in order to distinguish between different types of information about the images, thus to classify metadata contexts. Human agreement on metadata is notoriously variable. In order to maximize agreement, we conducted four human labeling experiments using the seven semantic category labels we developed. A subset of our labelers had much higher inter-annotator reliability, and highest reliability occurs when labelers can pick two labels per text unit.
Computer science
Artificial intelligence
Item views:
text | xml

In Partnership with the Center for Digital Research and Scholarship at Columbia University Libraries/Information Services | Terms of Use