Elamur Research⌕
☰
home / papers / 2009 / A Pairwise Event Coreference Model, Feature Impact and Evaluation for Event Coreference Resolution

A Pairwise Event Coreference Model, Feature Impact and Evaluation for Event Coreference Resolution

Summary

Event coreference resolution, which groups event mentions in a document into clusters that each denote a single real-world event, has received far less attention than entity coreference. Working within the ACE framework, the authors give a formal definition of the task. They solve it with a left-to-right agglomerative clustering procedure: each new mention is attached to the highest-scoring earlier event if that score passes a threshold δ, and otherwise it starts a new event. A Maximum-entropy pairwise model computes the scores from four feature groups: base trigger features, distance, argument compatibility, and event attributes. NYU's 2005 ACE system only produces triggers, arguments, and type/subtype, so the authors built separate classifiers for polarity, modality, genericity, and tense. They evaluate with MUC, B-Cubed, and ECM F-Measures. On the omitted gold-mention results, the summary reports that attribute features gave the largest gain. With automatically extracted mentions, adding features helps little. Error analysis shows that most mistakes come from faulty mention extraction rather than from the coreference model, so better event extraction is the main route to progress.

Key points

  • The clustering algorithm runs in O(N²). Comparing only mentions that share the same event type/subtype reduces the cost.
  • Features include trigger exact/stem match, a WordNet-based trigger similarity quantized to 0–5, and token/sentence/event distances. They also cover argument overlap with Time-Within/Place conflicts, and conflicts between the four attributes.
  • In the feature-impact study, the attribute feature set brought the largest improvement, 6.08% absolute.
  • With system-generated mentions, the full feature set reaches MUC F 0.28, B-Cubed F 0.554, and ECM F 0.492. The base set alone scores 0.265 / 0.558 / 0.489.
  • Spurious links caused by extraction errors (type I) make up 90–99.8% of errors. Misses caused by missing mentions account for 77.5–82.3% of all misses.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

30 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 19 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{chen2009pairwise,
  title = {{A Pairwise Event Coreference Model, Feature Impact and Evaluation for Event Coreference Resolution}},
  author = {Chen, Zheng and Ji, Heng and Haralick, Robert},
  booktitle = {Proceedings of the Workshop on Events in Emerging Text Types},
  year = {2009},
  url = {https://aclanthology.org/W09-4303/}
}

References

14 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. NIST. 2005. The ACE 2005 Evaluation Plan. http://www.itl.nist.gov/iad/mig/tests/ace/ace05/doc/ace 05-evalplan.v3.pdf. ↗
  2. R. Grishman, D. Westbrook, and A. Meyers. 2005. NYU’s English ACE 2005 System Description. In ACE 05 Evaluation Workshop, Gaithersburg, MD. find
  3. R. Saurí, M. Verhagen and J. Pustejovsky. 2006. Annotating and Recognizing Event Modality in Text. In Proceedings of the 19 th International FLAIRS Conference, FLAIRS 2006. Melbourne Beach, Florida. May 11-13, 2006. find
  4. M. Vilain, J. Burger, J. Aberdeen, D. Connolly and L. Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the Sixth Message Understanding Conference (MUC-6). find our card
  5. A. Bagga and B. Baldwin. 1998. Algorithms for scoring coreference chains. Proc. The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference. find our card
  6. X. Luo. 2005. On coreference resolution performance metrics. Proc. of HLT-EMNLP. find our card
  7. A. Bagga and B. Baldwin. 1999. Cross-document event coreference: Annotations, experiments, and observations. In Proc. ACL-99 Workshop on Coreference and Its Applications. find our card
  8. K. Humphreys, R. Gaizauskas, S. Azzam. 1997. Event coreference for information extraction. In Proceedings of the ACL Workshop on Operational Factors in Practical Robust Anaphora Resolution for Unrestricted Texts. find our card
  9. D. Ahn. 2006. The stages of event extraction. Proc. COLING/ACL 2006 Workshop on Annotating and Reasoning about Time and Events. Sydney, Australia. find
  10. Z. Chen and H. Ji. 2009. Graph-based Event Coreference Resolution. Proc. ACL-IJCNLP 2009 workshop on TextGraphs-4: Graph-based Methods for Natural Language Processing. find our card
  11. R. Grishman, D. Westbrook and A. Meyers. 2005. NYU’s English ACE 2005 System Description. Proc. ACE 2005 Evaluation Workshop. Washington, US. find
  12. H. Ji and R. Grishman. 2008. Refining Event Extraction Through Cross-document Inference. Proc. ACL 2008. Ohio, USA. find
  13. Z. Chen and H. Ji. 2009. Language Specific Issue and Feature Exploration in Chinese Event Extraction. Proc. HLT-NAACL 2009. Boulder, Co. find
  14. Z. Chen and H. Ji. 2009. Can One Language Bootstrap the Other: A Case Study on Event Extraction. Proc. HLT-NAACL Workshop on Semi-supervised Learning for Natural Language Processing. Boulder, Co. find

↑ back to top

This page in Markdown →