Elamur Research⌕
☰
home / papers / 2024 / Major Entity Identification: A Generalizable Alternative to Coreference Resolution

Major Entity Identification: A Generalizable Alternative to Coreference Resolution

Summary

Coreference resolution (CR) systems transfer poorly to new domains. A major cause is that benchmarks disagree about what counts as a mention. The authors avoid annotating more target-domain data and instead reframe the problem as Major Entity Identification (MEI). The user supplies a short designative phrase for each frequent entity in a document. The system must then find every mention of those entities and assign all other spans to a null class.

They turn two literary CR benchmarks, LitBank and FantasyCoref, into MEI data by treating frequently mentioned entities as major. For supervised learning they build MEIRa, a Longformer-Large entity-ranking model. For LLMs they design a two-stage few-shot prompt that tags words first and recovers full spans afterwards.

The gap between in-domain and out-of-domain performance is much smaller for MEI than for CR. Prompted GPT-4 is only slightly behind supervised models. Because MEI is a classification problem, it supports simpler, more interpretable metrics. It also fits applications that need the mentions of specific, user-chosen entities.

Key points

  • Major entities dominate narratives. In FantasyCoref, 942 of 5,829 entities (about 16%) produce 35,938 of 56,968 mentions (about 63%). Average cluster size grows from 3.67 to 34.66 (LitBank) and from 9.77 to 38.15 (FantasyCoref) when moving from CR to MEI.
  • MEIRa has three components:
    • a mention proposal network that detects all mentions, not only those of major entities;
    • an MLP that scores each mention against an entity memory, using log-distance metadata;
    • a threshold τ = 0 below which mentions go to the null entity.
  • MEIRa comes in two variants:
    • MEIRa-Static keeps the entity memory fixed, which allows full parallelization. It runs about 25x faster than a comparable CR model.
    • MEIRa-Hybrid averages scores from the updated memory and the initial phrase representations.
  • The LLM pipeline runs end-to-end without oracle mentions, unlike Le and Ritter (2023). Word-level tagging uses entity IDs, followed by a span-retrieval prompt. Most GPT-4 errors fall into the Missing Major and Extra-Major categories.
  • Code is released at github.com/KawshikManikantan/MEI.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.emnlp-main.652

@inproceedings{manikantan2024major,
  title = {{Major Entity Identification: A Generalizable Alternative to Coreference Resolution}},
  author = {Manikantan, Kawshik and Toshniwal, Shubham and Tapaswi, Makarand and Gandhi, Vineet},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.652},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.652}
}

References

49 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag. 2022. Large Language Models are Few-Shot Clinical Information Extractors. In EMNLP. find
  2. Maria Antoniak, Anjalie Field, Jimin Mun, Melanie Walsh, Lauren Klein, and Maarten Sap. 2023. Riveter: Measuring Power and Social Dynamics Between Entities. In ACL (Volume 3: System Demonstrations). find
  3. Rahul Aralikatte, Heather Lent, Ana Valeria Gonzalez, Daniel Herschcovich, Chen Qiu, Anders Sandholm, Michael Ringaard, and Anders Søgaard. 2019. Rewarding Coreference Resolvers for Being Consistent with World Knowledge. In EMNLP-IJCNLP. find our card
  4. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An Annotated Dataset of Coreference in English Literature. In LREC. find our card
  5. Sabyasachee Baruah and Shrikanth Narayanan. 2023. Character Coreference Resolution in Movie Screenplays. In Findings of ACL. find our card
  6. Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The Long-Document Transformer. arXiv preprint arXiv:2004.05150. find
  7. Bernd Bohnet, Chris Alberti, and Michael Collins. 2023. Coreference Resolution through a seq2seq Transition-Based System. TACL. find our card
  8. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  9. Hong Chen, Zhenhua Fan, Hao Lu, Alan Yuille, and Shu Rong. 2018. PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution. In EMNLP. find our card
  10. Yu-Hsin Chen and Jinho D. Choi. 2016. Character Identification on Multiparty Conversation: Identifying Mentions of Characters in TV Shows. In Annual Meeting of the Special Interest Group on Discourse and Dialogue. find
  11. Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2018. Neural Models for Reasoning over Multiple Mentions Using Coreference. In NAACL-HLT. find our card
  12. Vladimir Dobrovolskii. 2021. Word-Level Coreference Resolution. In EMNLP. find our card
  13. Nupoor Gandhi, Anjalie Field, and Emma Strubell. 2023. Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution. In ACL. find our card
  14. Sooyoun Han, Sumin Seo, Minji Kang, Jongin Kim, Nayoung Choi, Min Song, and Jinho D. Choi. 2021. FantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer’s Point of View. In Fourth Workshop on Computational Models of Reference, Anaphora and Coreference. find our card
  15. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. find
  16. Naoya Inoue, Charuta Pethe, Allen Kim, and Steven Skiena. 2022. Learning and Evaluating Character Representations in Novels. In Findings of ACL. find
  17. Heng Ji, Joel Nothman, Ben Hachey, and Radu Florian. 2015. Overview of tac-kbp2015 tri-lingual entity discovery and linking. Theory and Applications of Categories. find
  18. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825. find
  19. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. Span-BERT: Improving Pre-training by Representing and Predicting Spans. TACL, 8. find our card
  20. Ben Kantor and Amir Globerson. 2019. Coreference Resolution with Entity Equalization. In ACL. find our card
  21. Najoung Kim, Sebastian Schuster, and Shubham Toshniwal. 2024. Code Pretraining Improves Entity Tracking Abilities of Language Models. find
  22. Rik Koncel-Kedziorski, Dhanush Bekal, Yi Luan, Mirella Lapata, and Hannaneh Hajishirzi. 2019. Text Generation from Knowledge Graphs with Graph Transformers. In NAACL-HLT. find our card
  23. Jonathan K. Kummerfeld and Dan Klein. 2013. Error-Driven Analysis of Challenges in Coreference Resolution. In EMNLP. find our card
  24. Nghia T. Le and Alan Ritter. 2023. Are Large Language Models Robust Coreference Resolvers? find our card
  25. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end Neural Coreference Resolution. In EMNLP. find our card
  26. Jessica Lin and Amir Zeldes. 2024. GUMsley: Evaluating entity salience in summarization for 12 English genres. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2575–2588, St. Julian’s, Malta. Association for… find our card
  27. Jing Lu and Vincent Ng. 2020. Conundrums in Entity Coreference Resolution: Making Sense of the State of the Art. In EMNLP. find our card
  28. Nafise Sadat Moosavi and Michael Strube. 2016. Which Coreference Evaluation Metric Do You Trust? A Proposal for a Link-based Entity Aware Metric. In ACL. find our card
  29. Nafise Sadat Moosavi and Michael Strube. 2017. Lexical features in coreference resolution: To be used with caution. In ACL. find our card
  30. OpenAI. 2024. GPT-4 Technical Report. find our card
  31. Shon Otmazgin, Arie Cattan, and Yoav Goldberg. 2023. LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolution. In EACL. find our card
  32. Ian Porada, Alexandra Olteanu, Kaheer Suleman, Adam Trischler, and Jackie Chi Kit Cheung. 2023. Investigating Failures to Generalize for Coreference Resolution Models. find
  33. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards Robust Linguistic Analysis using OntoNotes. In CONLL. find
  34. M. Recasens and E. Hovy. 2011. Blanc: Implementing the rand index for coreference evaluation. Nat. Lang. Eng., 17(4):485–510. find
  35. Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, J… find our card
  36. Eva Sharma, Luyang Huang, Zhe Hu, and Lu Wang. 2019. An Entity-Driven Framework for Abstractive Summarization. In EMNLP-IJCNLP. find
  37. Shubham Toshniwal, Sam Wiseman, Allyson Ettinger, Karen Livescu, and Kevin Gimpel. 2020. Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks. In EMNLP. find our card
  38. Shubham Toshniwal, Patrick Xia, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2021. On Generalization in Coreference Resolution. In Workshop on Computational Models of Reference, Anaphora and Coreference. find our card
  39. Brandon T Willard and Rémi Louf. 2023. Efficient Guided Generation for LLMs. arXiv preprint arXiv:2307.09702. find
  40. Congying Xia, Wenpeng Yin, Yihao Feng, and Philip Yu. 2021. Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System. In NAACL-HLT. find
  41. James Munkres. 1957. Algorithms for the assignment and transportation problems. Journal of the society for industrial and applied mathematics. find
  42. Patrick Xia and Benjamin Van Durme. 2021. Moving on from OntoNotes: Coreference Resolution Model Transfer. In EMNLP. find
  43. Saul B Needleman and Christian D Wunsch. 1970. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of molecular biology. find
  44. Xiaohan Yang, Eduardo Peynetti, Vasco Meerman, and Chris Tanner. 2022. What GPT Knows About Who is Who. In Workshop on Insights from Negative Results in NLP. find
  45. Michelle Yuan, Patrick Xia, Chandler May, Benjamin Van Durme, and Jordan Boyd-Graber. 2022. Adapting Coreference Resolution Models through Active Learning. In ACL. find our card
  46. Sayyed M. Zahiri and Jinho D. Choi. 2017. Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks. find
  47. Wenzheng Zhang, Sam Wiseman, and Karl Stratos. 2023. Seq2seq is All You Need for Coreference Resolution. In EMNLP. find our card
  48. Shanheng Zhao and Hwee Tou Ng. 2014. Domain Adaptation with Active Learning for Coreference Resolution. In Workshop on Health Text Mining and Information Analysis (Louhi). find our card
  49. Yilun Zhu, Sameer Pradhan, and Amir Zeldes. 2021. OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres. In ACL-IJCNLP. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/manikantan-2024-major-entity-identification-generalizable-alternative/