HomeHome

Applying the Pyramid Method in the 2006 Document Understanding Conference

Rebecca Passonneau; Kathleen McKeown; Sergey Sigelman; Adam Goodkind

Title:
Applying the Pyramid Method in the 2006 Document Understanding Conference
Author(s):
Passonneau, Rebecca
McKeown, Kathleen
Sigelman, Sergey
Goodkind, Adam
Date:
Type:
Presentations (Communicative Events)
Department(s):
Computer Science
Persistent URL:
Publisher:
Proceedings of the 2006 Workshop of the Document Understanding Conference (DUC)
Abstract:
The pyramid evaluation effort for the 2006 Document Understanding Conference involved twenty-two sites on twenty document sets. Each pyramid content model (one per document set) was constructed from four human summaries. Peer systems were scored using the modified pyramid score introduced in DUC 2005. ANOVAs with score as the independent variable and nine factors yielded three significant factors: document set, peer, and content responsiveness. There were many more significant differences among peer systems in 2006 than for DUC 2005. We speculate this is due to a combination of improved systems and improvements in our evaluation procedures.
Subject(s):
Information technology
Computer science
Item views
182
Metadata:
text | xml
Suggested Citation:
Rebecca Passonneau, Kathleen McKeown, Sergey Sigelman, Adam Goodkind, , Applying the Pyramid Method in the 2006 Document Understanding Conference, Columbia University Academic Commons, .

Columbia University Libraries | Policies | FAQ