Elamur Research⌕
☰
home / papers / 2023 / Dual Cache for Long Document Neural Coreference Resolution

Dual Cache for Long Document Neural Coreference Resolution

Summary

Cache-based coreference models handle long documents by reading mentions left to right and keeping only a bounded set of entities in memory. This avoids the quadratic cost of pairwise models. However, the usual LRU eviction drops important entities when the narrative shifts topics and later returns to them, which causes cache misses and wrong links. The authors found that misses follow a Pareto-like pattern: a small number of frequent, document-wide entities cause most of them. They therefore propose a Dual cache that combines two policies. An L-cache uses LRU to track locally active entities, and a G-cache uses LFU to keep high-frequency global ones. It is built on a Longformer-based incremental model from Toshniwal et al. (2021). The approach was tested on OntoNotes, LongtoNotes, LitBank and WikiCoref, plus a newly annotated book. It cuts cache misses by up to half, raises F1 and sets a new state of the art on LitBank. It also runs much faster than non-cache models, making long and book-length coreference more practical.

Key points

  • With plain LRU, 11.1% of mentions hit a cache miss. In LitBank, about 6% of entities produce 97% of all misses.
  • The Dual cache improves coreference F1 by 0.7–5.7 points, with the largest gains for small caches. It reaches 79.5 F1 on LitBank, a new state of the art.
  • It saves up to 83% of inference time compared with non-cache-based models.
  • The authors annotated a new book-scale test set, Orwell's Animal Farm: about 30,000 words, 20 characters and 1,722 mentions. On it, the Dual cache scores 36.3 F1 in 375 s and 4.5 GB, against 25.8 F1, 1284 s and 5.2 GB for the Toshniwal et al. (2021) baseline.
  • The G-cache matters most when the total cache size is under about 20. Whether the G-cache or L-cache gets priority changes results only by about 0.1%. Data and code are released at github.com/QipengGuo/dual-cache-coref.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.acl-long.851

@inproceedings{guo2023dual,
  title = {{Dual Cache for Long Document Neural Coreference Resolution}},
  author = {Guo, Qipeng and Hu, Xiangkun and Zhang, Yue and Qiu, Xipeng and Zhang, Zheng},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.851},
  url = {https://doi.org/10.18653/v1/2023.acl-long.851}
}

References

32 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saliha Azzam, Kevin Humphreys, and Robert J. Gaizauskas. 1999. Using coreference chains for text summarization. In COREF@ACL. Association for Computational Linguistics. find our card
  2. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In The first international conference on language resources and evaluation workshop on linguistics coreference, volume 1, pages 563–566. Citeseer. find our card
  3. Alexandra Balahur, Jesús M. Hermida, and Andrés Mon-toyo. 2011. Detecting implicit expressions of sentiment in text based on commonsense knowledge. In WASSA@ACL, pages 53–60. Association for Computational Linguistics. find
  4. David Bamman, Sejal Popat, and Sheng Shen. 2019. An annotated dataset of literary entities. In NAACL-HLT (1), pages 2138–2144. Association for Computational Linguistics. find
  5. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. CoRR, abs/2004.05150. find
  6. Pradeep Dasigi, Nelson F. Liu, Ana Marasovic, Noah A. Smith, and Matt Gardner. 2019. Quoref: A reading comprehension dataset with questions requiring coreferential reasoning. In EMNLP/IJCNLP (1), pages 5924–5931. Association for Computational Linguistics. find our card
  7. Abbas Ghaddar and Philippe Langlais. 2016. Wikicoref: An english coreference-annotated corpus of wikipedia articles. In LREC. European Language Resources Association (ELRA). find our card
  8. Heng Ji and Ralph Grishman. 2008. Refining event extraction through cross-document inference. In ACL, pages 254–262. The Association for Computer Linguistics. find
  9. Fan Jiang and Trevor Cohn. 2021. Incorporating syntax and semantics in coreference resolution with heterogeneous graph attention network. In NAACL-HLT, pages 1584–1591. Association for Computational Linguistics. find our card
  10. Kumar Shridhar, Nicholas Monath, Raghuveer Thirukovalluru, Alessandro Stolfo, Manzil Zaheer, Andrew McCallum, and Mrinmaya Sachan. 2022. Longtonotes: Ontonotes with longer coreference chains. CoRR, abs/2210.03650. find
  11. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. Trans. Assoc. Comput. Linguistics, 8:64– 77. find our card
  12. Shane Storks, Qiaozi Gao, and Joyce Y. Chai. 2019. Commonsense reasoning for natural language understanding: A survey of benchmarks, resources, and approaches. CoRR, abs/1904.01172. find our card
  13. Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel S. Weld. 2019. BERT for coreference resolution: Baselines and analysis. In EMNLP/IJCNLP (1), pages 5802–5807. Association for Computational Linguistics. find our card
  14. Richard Koch. 2011. The 80/20 Principle: The Secret of Achieving More with Less: Updated 20th anniversary edition of the productivity and business classic. Hachette UK. find
  15. Hector J. Levesque, Ernest Davis, and Leora Morgenstern. 2012. The winograd schema challenge. In KR. AAAI Press. find
  16. Chunhua Liu, Trevor Cohn, and Lea Frermann. 2021a. Commonsense knowledge in word associations and conceptnet. In CoNLL, pages 481–495. Association for Computational Linguistics. find
  17. Zhengyuan Liu, Ke Shi, and Nancy F. Chen. 2021b. Coreference-aware dialogue summarization. In SIG-DIAL, pages 509–519. Association for Computational Linguistics. find our card
  18. Jing Lu and Vincent Ng. 2018. Event coreference resolution: A survey of two decades of research. In IJCAI, pages 5479–5486. ijcai.org. find
  19. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In HLT/EMNLP, pages 25–32. The Association for Computational Linguistics. find our card
  20. Raghuveer Thirukovalluru, Nicholas Monath, Kumar Shridhar, Manzil Zaheer, Mrinmaya Sachan, and Andrew McCallum. 2021. Scaling within document coreference to long texts. In ACL/IJCNLP (Findings), volume ACL/IJCNLP 2021 of Findings of ACL, pages 3921–3931. Association for Computational Linguistics. find our card
  21. Shubham Toshniwal, Sam Wiseman, Allyson Ettinger, Karen Livescu, and Kevin Gimpel. 2020. Learning to ignore: Long document coreference with bounded memory neural networks. In EMNLP (1), pages 8519–8526. Association for Computational Linguistics. find our card
  22. Shubham Toshniwal, Patrick Xia, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2021. On generalization in coreference resolution. CoRR, abs/2109.09667. find our card
  23. Marc B. Vilain, John D. Burger, John S. Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In MUC, pages 45–52. ACL. find our card
  24. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. Corefqa: Coreference resolution as query-based span prediction. In ACL, pages 6953–6963. Association for Computational Linguistics. find our card
  25. Patrick Xia, João Sedoc, and Benjamin Van Durme. 2020. Incremental neural coreference resolution in constant memory. In EMNLP (1), pages 8617–8624. Association for Computational Linguistics. find our card
  26. George Orwell. 2021. Animal farm. Oxford University Press. find
  27. Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Discourse-aware neural extractive text summarization. In ACL, pages 5021–5031. Association for Computational Linguistics. find our card
  28. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In EMNLP-CoNLL Shared Task, pages 1–40. ACL. find our card
  29. Liyan Xu and Jinho D. Choi. 2020. Revealing the myth of higher-order inference in coreference resolution. In EMNLP (1), pages 8527–8533. Association for Computational Linguistics. find our card
  30. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: an adversarial winograd schema challenge at scale. Commun. ACM, 64(9):99–106. find
  31. Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, and Maosong Sun. 2019. Docred: A large-scale document-level relation extraction dataset. In ACL (1), pages 764–777. Association for Computational Linguistics. find
  32. Ulrich Schäfer, Christian Spurk, and Jörg Steffen. 2012. A fully coreference-annotated corpus of scholarly papers from the ACL anthology. In COLING (Posters), pages 1059–1070. Indian Institute of Technology Bombay. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/guo-2023-dual-cache-long-document-neural/