2006 Presentations (Communicative Events)
Applying the Pyramid Method in the 2006 Document Understanding Conference
The pyramid evaluation effort for the 2006 Document Understanding Conference involved twenty-two sites on twenty document sets. Each pyramid content model (one per document set) was constructed from four human summaries. Peer systems were scored using the modified pyramid score introduced in DUC 2005. ANOVAs with score as the independent variable and nine factors yielded three significant factors: document set, peer, and content responsiveness. There were many more significant differences among peer systems in 2006 than for DUC 2005. We speculate this is due to a combination of improved systems and improvements in our evaluation procedures.
- passonneau_al_06c.pdf application/pdf 72.5 KB Download File
- Proceedings of the 2006 Workshop of the Document Understanding Conference (DUC)
- Academic Units
- Computer Science