Elamur Research⌕
☰
home / papers / 2019 / Coreference Resolution with Entity Equalization

Coreference Resolution with Entity Equalization

Summary

Coreference resolution groups textual mentions into the entities they refer to. Most neural systems score mentions in pairs, so they miss information spread across a whole cluster. Kantor and Globerson propose Entity Equalization (EE), which represents every mention by the summed representations of all mentions in its predicted cluster. Clustering is normally a discrete, non-differentiable step. To keep the model trainable end-to-end, they use the soft mention-to-entity probabilities of Le and Titov (2017), computed recursively from the antecedent distribution, and build running entity vectors from them. EE replaces the antecedent-averaging refinement in the span-ranking model of Lee et al. (2018). The authors also swap ELMo for BERT, the first use of BERT for coreference. BERT has a fixed input length, so they run it as a sliding window around each token. On the English CoNLL-2012 test set, the combined system set a new state of the art. This shows that entity-level information and stronger contextual encoders both help with coreference.

Key points

  • Data: the English portion of the CoNLL-2012 shared task, scored with the official scripts. The main metric is the average F1 of MUC, B³ and CEAFφ4.
  • Result: average F1 went from 73.0 (Lee et al., 2018 with ELMo) to 76.61 with BERT + EE, a gain of about 3.6 points. EE had the best F1 on all three metrics.
  • Ablations: replacing ELMo with BERT gives 76.25. Dropping second-order refinement gives 76.37. Using EE instead of second-order refinement gives 76.61.
  • BERT setup: each token gets a context window of D = 64 WordPieces on each side (well under the 512-token limit). A learned weighted average of the last four layers is used, and only the first sub-word of each word is kept.
  • Code is released at github.com/kkjawz/coref-ee. The paper appeared at ACL 2019, pp. 673–677.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

69 papers of our library cite it (of the 140673 whose bibliographies we read); 32 of them have a page here, 37 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1066

@inproceedings{kantor2019coreference,
  title = {{Coreference Resolution with Entity Equalization}},
  author = {Kantor, Ben and Globerson, Amir},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1066},
  url = {https://doi.org/10.18653/v1/p19-1066}
}

References

18 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sebastian Martschat and Michael Strube. 2015. Latent structures for coreference resolution. Transactions of the Association for Computational Linguistics, 3:405–418. find our card
  2. Kai-Wei Chang, Rajhans Samdani, and Dan Roth. 2013. A constrained latent variable model for coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 601–612. find our card
  3. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In Joint Conference on EMNLP and CoNLL-Shared Task, pages 1– 40. Association for Computational Linguistics. find our card
  4. Kevin Clark and Christopher D Manning. 2015. Entity-centric coreference resolution with model stacking. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume… find our card
  5. Altaf Rahman and Vincent Ng. 2009. Supervised models for coreference resolution. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2-Volume 2, pages 968–977. Association for Computational Linguistics. find our card
  6. Kevin Clark and Christopher D Manning. 2016. Deep reinforcement learning for mention-ranking coreference models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2256–2262. find our card
  7. Sameer Singh, Amarnag Subramanya, Fernando Pereira, and Andrew McCallum. 2011. Large-scale cross-document coreference using distributed inference and hierarchical models. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1,… find our card
  8. Aron Culotta, Michael Wick, and Andrew McCallum. 2007. First-order probabilistic models for coreference resolution. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 81–88. find our card
  9. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008. find
  10. Pascal Denis and Jason Baldridge. 2008. Specialized models and ranking for coreference resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 660– 669. Association for Computational Linguistics. find our card
  11. Michael Wick, Aron Culotta, Khashayar Rohani-manesh, and Andrew McCallum. 2009. An entity based model for coreference resolution. In Proceedings of the 2009 SIAM International Conference on Data Mining, pages 365–376. SIAM. find
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  13. Sam Wiseman, Alexander M Rush, and Stuart M Shieber. 2016. Learning global features for coreference resolution. arXiv preprint arXiv:1604.03035. find our card
  14. Greg Durrett, David Hall, and Dan Klein. 2013. Decentralized entity-level modeling for coreference resolution. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 114–124. find our card
  15. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. find
  16. Phong Le and Ivan Titov. 2017. Optimizing differentiable relaxations of coreference evaluation metrics. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 390–399. find
  17. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197. find our card
  18. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), volume 2,… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kantor-2019-coreference-resolution-entity-equalization/