Evaluating Content Selection in Human- or Machine-Generated Summaries: The Pyramid Scoring Method
Rebecca J. Passonneau; Ani Nenkova
- Evaluating Content Selection in Human- or Machine-Generated Summaries: The Pyramid Scoring Method
Passonneau, Rebecca J.
- Technical reports
- Computer Science
- Permanent URL:
- Columbia University Computer Science Technical Reports
- Part Number:
- Department of Computer Science, Columbia University
- Publisher Location:
- New York
- From the outset of automated generation of summaries, the difficulty of evaluation has been widely discussed. Despite many promising attempts, we believe it remains an unsolved problem. Here we present a method for scoring the content of summaries of any length against a weighted inventory of content units, which we refer to as a pyramid. Our method is derived from empirical analysis of human-generated summaries, and provides an informative metric for human or machine-generated summaries.
- Computer science
- Item views: