Elamur Research⌕
☰
home / papers / 2020 / New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach

New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach

Summary

Cross-Document Event Coreference (CDEC) links mentions of the same real-world event across different documents. It is usually evaluated on the ECB+ corpus. The authors reviewed prior systems and found them hard to compare for three reasons: some use gold labels and others computed ones, document clustering is handled differently, and test sets are chosen arbitrarily. They then built a deliberately simple and explainable pipeline modelled on the Bag of Events system by Vossen and Cybulska (2018). It uses only event trigger annotations, a compact pairwise feature set, document clustering that does not rely on gold annotations, and a new event clustering step. Every pipeline stage was evaluated separately. The system matches or slightly surpasses the best trigger-only results on ECB+ despite being much simpler. The work also offers concrete evaluation practices, so future CDEC results can be compared more fairly.

Key points

  • The pairwise event coreference classifier uses 4 features. Bag of Events uses 19, of which 5 come from triggers and 14 from entities.
  • Experiments use the verified ECB+ subset, which has 1,840 sentences, 5,726 events and 897 coreference chains with a mean length of 5.5. The full corpus covers 982 documents across 43 topics.
  • This is the first cross-validated CDEC evaluation on ECB+, using 5-fold CV. It is also the first explicit evaluation of the pairwise classification step.
  • Better document clustering can raise CDEC scores by at most about 3 points. This may reflect how easy it is to cluster ECB+ documents.
  • The authors recommend reporting results both with and without gold document clusters. They also note that tuning the pairwise classifier for F0.8 rather than F1 gives the best downstream CDEC performance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

5 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 1 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.nuse-1.1

@inproceedings{cremisini2020new,
  title = {{New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach}},
  author = {Cremisini, Andres and Finlayson, Mark},
  booktitle = {Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events},
  year = {2020},
  doi = {10.18653/v1/2020.nuse-1.1},
  url = {https://doi.org/10.18653/v1/2020.nuse-1.1}
}

References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, and Ido Dagan. 2019. Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution. arXiv, 2. find our card
  2. Jing Lu and Vincent Ng. 2017. Joint Learning for Event Coreference Resolution. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 90–101. find
  3. Cosmin Bejan and Sanda Harabagiu. 2010. Unsupervised Event Coreference Resolution with Rich Linguistic Features. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1412–1422. find our card
  4. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25–32, Vancouver, Canada. find our card
  5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606. find
  6. Nathanael Chambers, Taylor Cassidy, Bill McDowell, and Steven Bethard. 2014. Dense Event Ordering with a Multi-Pass Architecture. Transactions of the Association for Computational Linguistics, 2:273– 284. find
  7. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J Bethard, and David Mc-Closky. 2014. The {Stanford} {CoreNLP} Natural Language Processing Toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. find our card
  8. Zheng Chen and H Ji. 2009. Graph-based event coreference resolution. Proceedings of the 2009 Workshop on Graph-based Methods for Natural Language Processing, pages 54–57. find our card
  9. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. CoRR, abs/1802.05365. find
  10. Prafulla Kumar Choubey and Ruihong Huang. 2017. Event Coreference Resolution by Iteratively Unfolding Inter-dependencies among Events. arXiv. find our card
  11. Scikit-Learn. 2019. Scikit-learn probability calibration. https://scikit-learn.org/stable/ modules/calibration.html . Accessed: 2019-11-23. ↗
  12. Agata Cybulska and Piek Vossen. 2014. Using a sledgehammer to crack a nut? Lexical diversity and event coreference resolution. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4545– 4552. find our card
  13. Emma Strubell, Pat Verga, David Belanger, and Andrew McCallum. 2017. Fast and accurate entity recognition with iterated dilated convolutions. In EMNLP. find
  14. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ace) program-tasks, data, and evaluation. Proceedings of LREC, 2. find our card
  15. Piek Vossen and Agata Cybulska. 2018. Identity and Granularity of Events in Text. In Lecture Notes in Computer Science, volume 9624 LNCS, pages 501– 522. Springer-Verlag. find
  16. Brendan J. Frey and Delbert Dueck. 2007. Clustering by passing messages between data points. Science, 315(5814):972–976. find
  17. Bishan Yang, Claire Cardie, and Peter Frazier. 2015. A Hierarchical Distance-dependent Bayesian Model for Event Coreference Resolution. arXiv. find
  18. Sen Yang, Dawei Feng, Linbo Qiao, Zhigang Kan, and Dongsheng Li. 2019. Exploring Pre-trained Language Models for Event Extraction and Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5284–5294, Florence, Italy. Association for Computationa… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cremisini-2020-new-insights-into-cross-document/