Academic Commons

Presentations (Communicative Events)

Applying the Pyramid Method in the 2006 Document Understanding Conference

Passonneau, Rebecca; McKeown, Kathleen; Sigelman, Sergey; Goodkind, Adam

The pyramid evaluation effort for the 2006 Document Understanding Conference involved twenty-two sites on twenty document sets. Each pyramid content model (one per document set) was constructed from four human summaries. Peer systems were scored using the modified pyramid score introduced in DUC 2005. ANOVAs with score as the independent variable and nine factors yielded three significant factors: document set, peer, and content responsiveness. There were many more significant differences among peer systems in 2006 than for DUC 2005. We speculate this is due to a combination of improved systems and improvements in our evaluation procedures.

Files

More About This Work

Academic Units
Computer Science
Publisher
Proceedings of the 2006 Workshop of the Document Understanding Conference (DUC)
Published Here
July 5, 2013
Academic Commons provides global access to research and scholarship produced at Columbia University, Barnard College, Teachers College, Union Theological Seminary and Jewish Theological Seminary. Academic Commons is managed by the Columbia University Libraries.