Elamur Research⌕
☰
home / papers / 2021 / Cross-document Coreference Resolution over Predicted Mentions

Cross-document Coreference Resolution over Predicted Mentions

Summary

Most coreference research targets single documents. The cross-document (CD) version, which links mentions of the same event or entity across many texts, has seen far less work. Earlier CD systems were tested only on gold mentions and depended on extra tools such as SRL or paraphrase resources. Cattan et al. present the first end-to-end CD coreference system that works directly on raw text. It adapts the e2e-coref within-document model of Lee et al. (2017) and uses frozen RoBERTa LARGE as the encoder. Documents have no natural order, so the antecedent-linking step is replaced. Instead, a pairwise scorer compares spans across all documents, and agglomerative clustering with average linkage builds the clusters. Because ECB+ is small, the mention scorer is first pre-trained on gold spans. On gold mentions the model is close to the state of the art. On predicted mentions it sets the first baseline. It is also much simpler and faster than earlier systems and uses no external resources. This makes realistic CD coreference research more practical.

Key points

  • Results on ECB+ (CoNLL F1):
    • Gold mentions: 81.0 for events, 73.1 for entities, and 76.7 for the joint "ALL" setting, which mixes event and entity mentions.
    • Predicted mentions: 54.4 for events, 35.7 for entities, and 43.4 for ALL.
  • Why scores drop on predicted mentions: Events lose 26.6 F1 when moving from gold to predicted mentions. About 10 points of that gap come from how singletons are treated in evaluation. Using two separate event and entity models beats the single ALL model by only 0.6 F1.
  • Ablations (development set, event coreference): Removing pre-training of the mention scorer costs 3.2 F1. Freezing candidate spans instead of dynamic pruning costs 4.0. Training on all negative pairs instead of sampling 20× negatives costs 1.4 and slows training.
  • Efficiency: The system runs on a single 12GB GTX 1080 Ti. Training takes about 2.5 hours in the hardest setting. Inference takes about 2 minutes, compared with 37 minutes for Barhom et al. (2019).
  • Error analysis: The main errors come from over-reliance on lexical similarity and from weak temporal reasoning. Code is available at github.com/ariecattan/coref.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

33 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 21 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-acl.453 · arXiv 2106.01210

@inproceedings{cattan2021cross,
  title = {{Cross-document Coreference Resolution over Predicted Mentions}},
  author = {Cattan, Arie and Eirew, Alon and Stanovsky, Gabriel and Joshi, Mandar and Dagan, Ido},
  booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.453},
  eprint = {2106.01210},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.findings-acl.453}
}

References

25 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, and Ido Dagan. 2019. Revisiting joint modeling of cross-document entity and event coreference resolution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4179–4189, Florence, Ital… find our card
  2. Cosmin Bejan and Sanda Harabagiu. 2010. Unsupervised event coreference resolution with rich linguistic features. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1412–1422, Uppsala, Sweden. Association for Computational Linguistics. find our card
  3. Michael Bugert, N. Reimers, and Iryna Gurevych. 2020. Cross-document event coreference resolution beyond corpus-tailored systems. ArXiv, abs/2011.12249. find our card
  4. Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, and Ido Dagan. 2021. Cross-document language modeling. ArXiv, abs/2101.00406. — [arXiv:2101.00406](https://arxiv.org/abs/2101.00406) find
  5. Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, and Ido Dagan. 2021. Realistic evaluation principles for cross-document coreference resolution. In Proceedings of the Joint Conference on Lexical and Computational Semantics. Association for Computational Linguistics. find our card
  6. Prafulla Kumar Choubey and Ruihong Huang. 2017. Event coreference resolution by iteratively unfolding inter-dependencies among events. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2124–2133, Copenhagen, Denmark. Association for Computational Lingui… find our card
  7. Agata Cybulska and Piek Vossen. 2014. Using a sledgehammer to crack a nut? lexical diversity and event coreference resolution. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4545– 4552, Reykjavik, Iceland. European Language Resources Associ… find our card
  8. Agata Cybulska and Piek Vossen. 2015. Translating granularity of event slots into features for event coreference resolution. In Proceedings of the The Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth in… find our card
  9. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. find our card
  10. Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019. BERT for coreference resolution: Baselines and analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP… find our card
  11. Kian Kenyon-Dean, Jackie Chi Kit Cheung, and Doina Precup. 2018. Resolving event coreference with supervised representation learning and clustering-oriented regularization. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 1–10, New Orleans, Louisiana. Asso… find our card
  12. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  13. Heeyoung Lee, Marta Recasens, Angel Chang, Mihai Surdeanu, and Dan Jurafsky. 2012. Joint entity and event coreference resolution across documents. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 489–50… find our card
  14. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  16. Yehudit Meged, Avi Caciularu, Vered Shwartz, and Ido Dagan. 2020. Paraphrasing vs coreferring: Two sides of the same coin. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4897–4907, Online. Association for Computational Linguistics. find our card
  17. Nicholas Monath, Kumar Avinava Dubey, Guru Gu-ruganesh, M. Zaheer, Amr Ahmed, A. McCallum, Gokhan Mergen, Marc Najork, Mert Terzihan, B. Tjanaka, Yuan Wang, and Yuchen Wu. 2021. Scalable Bottom-up Hierarchical Clustering. Proceedings of the 27th ACM SIGKDD International Conference on Knowledge Disco… find
  18. Nicholas Monath, A. Kobren, A. Krishnamurthy, Michael R. Glass, and A. McCallum. 2019. Scalable Hierarchical Clustering with Tree Grafting. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining. find
  19. Bishan Yang, Claire Cardie, and Peter Frazier. 2015. A hierarchical distance-dependent Bayesian model for event coreference resolution. Transactions of the Association for Computational Linguistics, 3:517–528. find
  20. Yutao Zeng, Xiaolong Jin, Saiping Guan, Jiafeng Guo, and Xueqi Cheng. 2020. Event coreference resolution with their paraphrases and argument-aware embeddings. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3084–3094, Barcelona, Spain (Online). International C… find our card
  21. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and… find our card
  22. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task, pages 1–40, Jeju Island, Korea. Association for Computational Linguisti… find our card
  23. Vered Shwartz, Gabriel Stanovsky, and Ido Dagan. 2017. Acquiring predicate paraphrases from news tweets. In Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 155–160, Vancouver, Canada. Association for Computational Linguistics. find
  24. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find our card
  25. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. CorefQA: Coreference resolution as query-based span prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6953–6963, Online. Association for Computational Linguistics. find our card

↑ back to top