Elamur Research
☰
home / papers / 2021 / Coreference Resolution without Span Representations

Coreference Resolution without Span Representations

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2021.acl-short.3.pdf

Summary

Coreference resolution has resisted a broad NLP trend: most tasks now need only a thin layer on top of a pretrained transformer, but coreference still uses a heavy task-specific model. That model, coarse-to-fine (c2f), builds explicit span and span-pair vectors and relies on handcrafted features and pruning heuristics. It uses so much GPU memory that documents are often split into windows and each batch holds a single example.

Kirstain, Ram and Levy propose start-to-end (s2e), which never builds span vectors. Each token gets separate start and end projections. A mention score comes from a biaffine combination of a span's boundary tokens. An antecedent score sums four bilinear interactions between the endpoints of the candidate and query spans. Speaker metadata is inserted into the text instead of being encoded as a feature.

The model uses Longformer-Large and is trained without hyperparameter tuning on OntoNotes 5.0 and GAP. It matches the strongest c2f baseline while needing less than a third of the memory. This suggests pretrained transformers can learn much of the structure that complex coreference architectures hand-engineer.

Key points

  • On the English OntoNotes 5.0 test set, s2e + Longformer-Large reaches 80.3 average F1 (MUC/B³/CEAF φ4). The best c2f baseline (SpanBERT-Large) reaches 80.2, and c2f with Longformer-Base and Longformer-Large scores 79.2 and 78.1.
  • On GAP, s2e scores 88.3 overall F1 versus 88.4 for c2f + SpanBERT-Large, with the same 0.95 gender bias ratio.
  • Peak inference memory on OntoNotes is 4.3 GB for s2e, compared with 12.0–16.2 GB for the c2f variants.
  • Space complexity drops to that of a transformer layer, O(n² + nd), instead of O(n²d) for explicit span representations. The model has no handcrafted features and no antecedent pruning, and it filters mentions only by their mention score and a maximum span length.
  • With dynamic batching of 5,000 tokens, s2e fits about 5–6 full documents per batch on a 32GB GPU without sliding windows. Code is available at github.com/yuvalkirstain/s2e-coref.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

50 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 46 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.acl-short.3 · arXiv 2101.00434

@inproceedings{kirstain2021coreference,
  title = {{Coreference Resolution without Span Representations}},
  author = {Kirstain, Yuval and Ram, Ori and Levy, Omer},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2021},
  doi = {10.18653/v1/2021.acl-short.3},
  eprint = {2101.00434},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.acl-short.3}
}

References

10 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. ↗
  2. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. ArXiv, abs/1611.01603. ↗ our card
  3. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced LSTM for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1657–1668, Vancouver, Canada. Association for Computational Linguistics. find
  4. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In ICLR 2017. find our card
  5. Shubham Toshniwal, Sam Wiseman, Allyson Ettinger, Karen Livescu, and Kevin Gimpel. 2020. Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8519–8526, Online. Association for Computational Linguistics. ↗ our card
  6. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. find our card
  7. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A balanced corpus of gendered ambiguous pronouns. Transactions of the Association for Computational Linguistics, 6:605–617. ↗ our card
  8. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. CorefQA: Coreference resolution as query-based span prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6953–6963, Online. Association for Computational Linguistics. ↗ our card
  9. Patrick Xia, João Sedoc, and Benjamin Van Durme. 2020. Incremental neural coreference resolution in constant memory. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8617–8624, Online. Association for Computational Linguistics. ↗ our card
  10. Liyan Xu and Jinho D. Choi. 2020. Revealing the myth of higher-order inference in coreference resolution. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8527–8533, Online. Association for Computational Linguistics. ↗ our card