Elamur Research⌕
☰
home / papers / 2021 / Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference

Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference

Summary

Cross-document coreference resolution (CDCR) for events and entities faces a quadratic explosion of candidate mention pairs. Most systems avoid this by first grouping documents into topic clusters, which misses coreference that crosses clusters and transfers poorly beyond ECB+. Held, Iter and Jurafsky borrow the idea of attentional state from discourse coherence theory: a reader only considers the few entities or events currently in focus.

They model this focus as a mention's K nearest neighbours in a learned embedding space. A RoBERTa-large bi-encoder is trained so that each mention lands close to the centroid of its gold coreference cluster. These neighbourhoods then serve two purposes. They limit which pairs are compared at inference. They also supply hard negatives for training a cross-encoder pairwise classifier that sees the surrounding sentences as context. A variant of agglomerative clustering produces the final clusters.

The method sets new state-of-the-art results on ECB+, GVC, FCC and CD2CR. Training on several corpora at once yields a far more robust model for downstream applications where the distribution of coreference links is unknown.

Key points

  • The pipeline has three stages: bi-encoder retrieval (span-boundary mention representations, centroid-based classification loss), a cross-encoder classifier trained on hard negatives, and agglomerative clustering adapted from Barhom et al. (2019).
  • B³ F1 gains over prior best:
    • ECB+ events: 86.6 (+0.2)
    • ECB+ entities: 82.8 (+0.7)
    • Gun Violence Corpus: 83.0 (+11.3)
    • Football Coreference Corpus: 63.5 (+13.3)
    • CD2CR: 78.5 (+34.5)
  • Training jointly on all event CDCR corpora raises average performance across datasets by 17.2 F1 points.
  • The pruning step is close to lossless. With only 5 neighbours per mention and an oracle classifier, recall reaches 96.3 and the upper-bound F1 is 98.1.
  • Local discourse context is important:
    • Widening the context window from 0 to 3 sentences raises event F1 from 82.9 to 86.2.
    • Masking within-document coreference tokens in the context hurts entity F1 by 2.9.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Coreference resolution

Cited by

23 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 14 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.emnlp-main.106 · arXiv 2110.05362

@inproceedings{held2021focus,
  title = {{Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference}},
  author = {Held, William and Iter, Dan and Jurafsky, Dan},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.106},
  eprint = {2110.05362},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.emnlp-main.106}
}

References

36 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, and Ido Dagan. 2019. Revisiting joint modeling of cross-document entity and event coreference resolution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4179–4189, Florence, Ital… find our card
  2. Cosmin Bejan and Sanda Harabagiu. 2010. Unsupervised event coreference resolution with rich linguistic features. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1412–1422, Uppsala, Sweden. Association for Computational Linguistics. find our card
  3. Michael Bugert, N. Reimers, and Iryna Gurevych. 2020a. Cross-document event coreference resolution beyond corpus-tailored systems. ArXiv, abs/2011.12249. find our card
  4. Michael Bugert, Nils Reimers, Shany Barhom, I. Dagan, and Iryna Gurevych. 2020b. Breaking the subtopic barrier in cross-document event coreference resolution. In Text2Story@ECIR. find
  5. Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, and Ido Dagan. 2021. Cross-document language modeling. ArXiv, abs/2101.00406. — [arXiv:2101.00406](https://arxiv.org/abs/2101.00406) find
  6. Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, and Ido Dagan. 2020. Streamlining cross-document coreference resolution: Evaluation and modeling. ArXiv, abs/2009.11032. find
  7. Jifan Chen, Shih-Ting Lin, and Greg Durrett. 2019. Multi-hop question answering via reasoning chains. ArXiv, abs/1910.02610. — [arXiv:1910.02610](https://arxiv.org/abs/1910.02610) find
  8. Prafulla Kumar Choubey and Ruihong Huang. 2017. Event coreference resolution by iteratively unfolding inter-dependencies among events. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2124–2133, Copenhagen, Denmark. Association for Computational Lingui… find our card
  9. Andres Cremisini and Mark Finlayson. 2020. New insights into cross-document event coreference: Systematic comparison and a simplified approach. In Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events, pages 1– 10, Online. Association for Computational Linguistic… find our card
  10. A. Cybulska and Piek T. J. M. Vossen. 2015. ”bag of events” approach to event coreference resolution. supervised classification of event templates. Int. J. Comput. Linguistics Appl., 6:11–27. find
  11. Agata Cybulska and Piek Vossen. 2014. Using a sledgehammer to crack a nut? lexical diversity and event coreference resolution. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4545– 4552, Reykjavik, Iceland. European Language Resources Associ… find our card
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  13. Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2018. Neural models for reasoning over multiple mentions using coreference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie… find our card
  14. Tobias Falke, Christian M. Meyer, and Iryna Gurevych. 2017. Concept-map-based multi-document summarization using concept coreference resolution and global importance optimization. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), page… find our card
  15. Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego Garcia-Olano. 2019. Learning dense representations for entity retrieval. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 528–537, Hong Kong, China.… find
  16. Barbara J. Grosz. 1977. The Representation and Use of Focus in Dialogue Understanding. Ph.D. thesis. AAI7731381. find
  17. Barbara J. Grosz. 1978. Focusing in dialog. In Theoretical Issues in Natural Language Processing-2. find
  18. Barbara J. Grosz and Candace L. Sidner. 1986. Attention, intentions, and the structure of discourse. Computational Linguistics, 12(3):175–204. find our card
  19. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. find
  20. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale similarity search with gpus. arXiv preprint arXiv:1702.08734. find our card
  21. Kian Kenyon-Dean, Jackie Chi Kit Cheung, and Doina Precup. 2018. Resolving event coreference with supervised representation learning and clustering-oriented regularization. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 1–10, New Orleans, Louisiana. Asso… find our card
  22. Diederik P. Kingma and Jimmy Ba. 2015. A method for stochastic optimization. abs/1412.6980. find
  23. Adam: CoRR, Heeyoung Lee, Marta Recasens, Angel Chang, Mihai Surdeanu, and Dan Jurafsky. 2012. Joint entity and event coreference resolution across documents. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,… find our card
  24. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  25. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–… find our card
  26. Y. Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692. find
  27. Yehudit Meged, Avi Caciularu, Vered Shwartz, and Ido Dagan. 2020. Paraphrasing vs coreferring: Two sides of the same coin. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4897–4907, Online. Association for Computational Linguistics. find our card
  28. Nafise Sadat Moosavi and Michael Strube. 2016. Which coreference evaluation metric do you trust? a proposal for a link-based entity aware metric. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 632–642, Berlin, Germany. Associ… find our card
  29. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and… find our card
  30. James Ravenscroft, Amanda Clare, Arie Cattan, Ido Dagan, and Maria Liakata. 2021. CDˆ2CR: Coreference resolution across documents and domains. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 270–280, Online. Associati… find
  31. P. Vossen and Agata Cybulska. 2016. Identity and granularity of events in text. In CICLing. find
  32. Piek Vossen, Filip Ilievski, Marten Postma, and Roxane Segers. 2018. Don’t annotate, but validate: a data-to-text method for capturing event data. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources As… find
  33. Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. Scalable zero-shot entity linking with dense entity retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6397–6407, Online. Association for Computa… find our card
  34. Bishan Yang, Claire Cardie, and Peter Frazier. 2015. A hierarchical distance-dependent Bayesian model for event coreference resolution. Transactions of the Association for Computational Linguistics, 3:517–528. find
  35. Xiaodong Yu, Wenpeng Yin, and D. Roth. 2020. Paired representation learning for event and entity coreference. ArXiv, abs/2010.12808. — [arXiv:2010.12808](https://arxiv.org/abs/2010.12808) find
  36. Yutao Zeng, Xiaolong Jin, Saiping Guan, Jiafeng Guo, and Xueqi Cheng. 2020. Event coreference resolution with their paraphrases and argument-aware embeddings. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3084–3094, Barcelona, Spain (Online). International C… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/held-2021-focus-what-matters-applying-discourse/