Elamur Research⌕
☰
home / papers / 2026 / Do Lexical and Contextual Coreference Resolution Systems Degrade Differently under Mention Noise? An Empirical Study on Scientific Software Mentions

Do Lexical and Contextual Coreference Resolution Systems Degrade Differently under Mention Noise? An Empirical Study on Scientific Software Mentions

Summary

The paper describes the authors' participation in the SOMD 2026 shared task. The task is cross-document coreference resolution of scientific software mentions: mentions from a collection of papers must be grouped so that each cluster corresponds to one software tool. The authors compare two approaches that require no fine-tuning. Lexical Fuzzy Matching (FM) compares strings by similarity. Context Aware Representations (CAR) clusters mentions using a combination of embeddings of the mention itself and of the whole document. Both systems ranked second in all three subtasks and reached CoNLL F1 of 0.94–0.96. A simple method performs this well because software names vary little in form. Separately, the authors ran a controlled experiment in which they deliberately added noise to the input mentions. It showed that the two systems break down in different ways: CAR is more robust to mention boundary errors, while FM better withstands mention substitution. In addition, FM's runtime grows faster than linearly, whereas CAR scales roughly linearly. The authors conclude that the choice of system should depend on the type of noise produced by the mention detector and on the size of the corpus.

Key points

  • SOMD 2026 has three subtasks. Subtask 1 uses gold mentions, Subtasks 2 and 3 use automatically predicted ones, and Subtask 3 additionally requires processing a large corpus. SOMD 2026 is the first standardized benchmark for software mention coreference.
  • The Subtask 1 training corpus contains 973 documents, 2,974 mentions, 837 unique surface forms and 733 clusters. Singletons make up 51.7%, and cross-document clusters 20.6%.
  • On the official test set, CAR consistently outperforms FM by about 1 CoNLL F1 point.
  • When mention boundaries are corrupted, going from clean to fully corrupted input, CAR loses only 0.07 F1 and FM loses 0.20. With mention substitution the picture is reversed: FM degrades more gracefully (0.52 vs. 0.63 for CAR).
  • The code is available at github.com/adsabs/SOMD-2026.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2f5gzhatrnue · arXiv 2604.02171

@inproceedings{alkan2026do,
  title = {{Do Lexical and Contextual Coreference Resolution Systems Degrade Differently under Mention Noise? An Empirical Study on Scientific Software Mentions}},
  author = {Alkan, Atilla Kaan and Grezes, Felix and Bartlett, Jennifer Lynn and Kelbert, Anna and Lockhart, Kelly and Accomazzi, Alberto},
  booktitle = {Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/2f5gzhatrnue},
  eprint = {2604.02171},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.63317/2f5gzhatrnue}
}

References

50 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Atilla Kaan Alkan, Felix Grezes, Cyril Grouin, Fabian Schussler, and Pierre Zweigenbaum. 2024. Enriching a time-domain astrophysics corpus with named entity, coreference and astro-physical relationship annotations. In Proceedings of the 2024 Joint International Conference on Computational Linguistic… find
  2. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. find our card
  3. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, and Ido Dagan. 2019. Revisiting joint modeling of cross-document entity and event coreference resolution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4179–4189, Florence, Ital… find our card
  4. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciB-ERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615–3620,… find
  5. Terra Blevins, Hila Gonen, and Luke Zettlemoyer. 2023. Prompting language models for linguistic structure. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6649– 6663, Toronto, Canada. Association for Computational Linguistics. find
  6. Arthur Brack, Daniel Uwe Müller, Anett Hoppe, and Ralph Ewerth. 2021. Coreference Resolution in Research Papers from Multiple Domains. arXiv e-prints, page arXiv:2101.00884. — [arXiv:2101.00884](https://arxiv.org/abs/2101.00884) find
  7. Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, and Ido Dagan. 2021a. Cross-document coreference resolution over predicted mentions. CoRR, abs/2106.01210. find our card
  8. Arie Cattan, Sophie Johnson, Daniel S. Weld, Ido Dagan, Iz Beltagy, Doug Downey, and Tom Hope. 2021b. Scico: Hierarchical cross-document coreference for scientific concepts. ArXiv, abs/2104.08809. — [arXiv:2104.08809](https://arxiv.org/abs/2104.08809) find
  9. Haixia Chai, Nafise Sadat Moosavi, Iryna Gurevych, and Michael Strube. 2022. Evaluating coreference resolvers on community-based question answering: From rule-based to state of the art. In Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 61–73,… find
  10. Panot Chaimongkol, Akiko Aizawa, and Yuka Tateisi. 2014. Corpus for coreference resolution on scientific papers. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3187– 3190, Reykjavik, Iceland. European Language Resources Association (ELRA). find
  11. Kevin Clark and Christopher D. Manning. 2016. Improving coreference resolution by learning entity-level distributed representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 643–653, Berlin, Germany. Association for Com… find our card
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  13. Zhang Dong, Mingbang Wang, Songhang deng, Le Dai, Jiyuan Li, Xingzu Liu, and Ruilin Nong. 2025. Cross-Document Contextual Coreference Resolution in Knowledge Graphs. arXiv e-prints, page arXiv:2504.05767. — [arXiv:2504.05767](https://arxiv.org/abs/2504.05767) find
  14. Caifan Du, Johanna Cohoon, Patrice Lopez, and James Howison. 2021. Softcite dataset: A dataset of software mentions in biomedical and economic research publications. J. Assoc. Inf. Sci. Technol., 72(7):870–884. find
  15. Lior Forer and Tom Hope. 2024. Inferring scientific cross-document coreference and hierarchy with definition-augmented relational reasoning. ArXiv, abs/2409.15113. find
  16. Quentin Gliosca and Pascal Amsili. 2019. Résolution des coréférences neuronale : une approche basée sur les têtes (neural coreference resolution : a head-based approach). In Actes de la Conférence sur le Traitement Automatique des Langues Naturelles (TALN) PFIA 2019. Volume II : Articles courts, pag… find
  17. Felix Grezes, Sergi Blanco-Cuaresma, Thomas Allen, and Tirthankar Ghosal. 2022. Overview of the first shared task on detecting entities in the astrophysics literature (DEAL). In Proceedings of the First Workshop on Information Extraction from Scientific Publications, pages 1–7, Online. Association f… find
  18. Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995. Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203–225. find our card
  19. Barbara J. Grosz and Candace L. Sidner. 1986. Attention, intentions, and the structure of discourse. Computational Linguistics, 12(3):175–204. find our card
  20. Aria Haghighi and Dan Klein. 2010. Coreference resolution in a modular, entity-centered model. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 385–393, Los Angeles, California. Association for Computatio… find our card
  21. Jerry R. Hobbs. 1978. Resolving pronoun references. Lingua, 44(4):311–338. find our card
  22. Yin Jou Huang and Sadao Kurohashi. 2021. Extractive summarization considering discourse and coreference relations based on heterogeneous graph. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3046–3052, Online. Associ… find our card
  23. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64– 77. find our card
  24. Daniel Jurafsky and James H. Martin. 2026. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, with Language Models, 3rd edition. Online manuscript released January 6, 2026. find
  25. Nghia T. Le, Fan Bai, and Alan Ritter. 2022. Few-shot anaphora resolution in scientific protocols via mixtures of in-context experts. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2693–2706, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find our card
  26. Nghia T. Le and Alan Ritter. 2023. Are large language models robust coreference resolvers? find our card
  27. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  28. Lishuang Li, Liuke Jin, Zhenchao Jiang, Jing Zhang, and Degen Huang. 2014. Coreference resolution in biomedical texts. In 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 12–14. find
  29. Pengcheng Lu and Massimo Poesio. 2021. Coreference resolution for the biomedical domain: A survey. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 12–23, Punta Cana, Dominican Republic. Association for Computational Linguistics. find our card
  30. Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3219– 3232, Brussels, Bel… find our card
  31. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25–32, Vancouver, British Columbia, Canada. Association for Computational Linguistics. find our card
  32. Stasa Mandic, Georg Niess, and Roman Kern. 2025. From in-distribution to out-of-distribution: Joint loss for improving generalization in software mention and relation extraction. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 146–153, Vienna, Austria. Associa… find
  33. Vaghawan Ojha, Projan Shakya, Kristina Ghimire, Kashish Bataju, Ashwini Mandal, Sadikshya Gyawali, Manish Dahal, Manish Awale, Shital Adhikari, and Sanjay Rijal. 2025. find
  34. SOMD 2025: Fine-tuning ModernBERT for in- and out-of-distribution NER and relation extraction of software mentions in scientific texts. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 154–163, Vienna, Austria. Association for Computational Linguistics. find
  35. Shon Otmazgin, Arie Cattan, and Yoav Goldberg. 2022. F-coref: Fast, accurate and easy to use coreference resolution. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Proce… find our card
  36. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring coreference partitions of predicted mentions: A reference implementation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pag… find our card
  37. Pranshu Rastogi and Rajneesh Tiwari. 2025. Extracting software mentions and relations using transformers and LLM-generated synthetic data at SOMD 2025. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 173–181, Vienna, Austria. Association for Computational Ling… find
  38. John W. Ratcliff and John A. Obershelp. 1988. JUL88: Pattern Matching: The Gestalt Approach. Dr. Dobb’s Journal, 13:46–71. find
  39. D. Schindler, F. Bensmann, S. Dietze, and F. Krüger. 2022. The role of software in science: a knowledge graph-based analysis of software mentions in PubMed Central. PeerJ Computer Science, 8:e835. find
  40. David Schindler, Felix Bensmann, Stefan Dietze, and Frank Krüger. 2021. Somesci- a 5 star open data gold standard knowledge graph of software mentions in scientific articles. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM ’21, page 4574–4583, New… find
  41. Isabel Segura-Bedmar, Mario Crespo, Cesar de Pablo, and Paloma Martínez. 2009. Drugnerar: linguistic rule-based anaphora resolver for drug-drug interaction extraction in pharmacological documents. In Proceedings of the Third International Workshop on Data and Text Mining in Bioinformatics, DTMBIO ’0… find
  42. Gabriel Silva, Mário Rodriges, António Teixeira, and Marlene Amorim. 2025. Inductive learning on heterogeneous graphs enhanced by LLMs for software mention detection. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 164–172, Vienna, Austria. Association for Com… find
  43. Alessandro Stolfo, Chris Tanner, Vikram Gupta, and Mrinmaya Sachan. 2022. A simple unsupervised approach for coreference resolution using rule-based weak supervision. In Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pages 79–88, Seattle, Washington. Association for… find
  44. Sharmila Upadhyaya, Wolfgang Otto, Frank Krüger, and Stefan Dietze. 2025. SOMD2025: A challenging shared tasks for software related information extraction. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 137–145, Vienna, Austria. Association for Computational… find
  45. Marc B. Vilain, John D. Burger, John S. Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Message Understanding Conference. find our card
  46. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. Minilm: deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. Cu… find
  47. Sam Wiseman, Alexander M. Rush, Stuart M. Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Annual Meeting of the Association for Computational Linguistics. find our card
  48. Dmitry Zelenko, Chinatsu Aone, and Jason Tib-betts. 2004. Coreference resolution for information extraction. In Proceedings of the Conference on Reference Resolution and Its Applications, pages 24–31, Barcelona, Spain. Association for Computational Linguistics. find our card
  49. Pierre Zweigenbaum, Guillaume Wisniewski, Marco Dinarelli, Cyril Grouin, and Sophie Rosset. 2012. Résolution des coréférences dans des comptes rendus cliniques. Une expérimentation issue du défi i2b2/VA 2011. In Actes de la conférence RFIA 2012, pages 978–2–9539515–2–3, Lyon, France. Session ”Articl… find
  50. Julien Tourille, Olivier Ferret, Aurélie Névéol, and Xavier Tannier. 2020. Modèle neuronal pour la résolution de la coréférence dans les dossiers médicaux électroniques (neural approach for coreference resolution in electronic find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/alkan-2026-do-lexical-contextual-coreference-resolution/