Applying the Pyramid Method in the 2006 Document Understanding Conference
- Applying the Pyramid Method in the 2006 Document Understanding Conference
- Passonneau, Rebecca
- Computer Science
Center for Computational Learning Systems
- Persistent URL:
- Proceedings of the 2006 Workshop of the Document Understanding Conference (DUC)
- The pyramid evaluation effort for the 2006 Document Understanding Conference involved twenty-two sites on twenty document sets. Each pyramid content model (one per document set) was constructed from four human summaries. Peer systems were scored using the modified pyramid score introduced in DUC 2005. ANOVAs with score as the independent variable and nine factors yielded three significant factors: document set, peer, and content responsiveness. There were many more significant differences among peer systems in 2006 than for DUC 2005. We speculate this is due to a combination of improved systems and improvements in our evaluation procedures.
- Information technology
- Item views
text | xml
- Suggested Citation:
- Rebecca Passonneau, Kathleen McKeown, Sergey Sigelman, Adam Goodkind, 2006, Applying the Pyramid Method in the 2006 Document Understanding Conference, Columbia University Academic Commons, http://hdl.handle.net/10022/AC:P:20934.