Elamur Research⌕
☰
home / papers / 2023 / Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution

Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution

Summary

Coreference systems trained on benchmarks such as OntoNotes transfer poorly to specialized domains. These domains have unfamiliar vocabulary, noisy text and different annotation conventions, such as whether singleton mentions are marked. Adaptation normally needs full coreference-chain labels in the target domain. Those labels are slow to produce, especially where data is private and experts have little time.

The authors first freeze the antecedent linker during continued training. This shows that most of the improvement comes from updating only the encoder and mention detector. Timed annotation studies on i2b2/VA medical notes then show that marking mentions alone takes roughly half the effort of building full chains.

Building on this, they adapt a SpanBERT-based coarse-to-fine (c2f) model using only target-domain mention labels. They add an auxiliary objective that rewards precise mention detection, which shrinks the pool of antecedent candidates.

The method was tested on CoNLL-2012, i2b2/VA and a new child welfare corpus. With equal annotator time, it beats continued training on coreference annotations and raises average F1 by 7–14%. This makes coreference adaptation more feasible in low-resource, high-stakes settings.

Key points

  • Preliminary experiment (OntoNotes → i2b2/VA, SpanBERT + c2f): tuning only the encoder and mention detector lifts F1 from 39.10 to 56.42. Also tuning the antecedent linker adds little (56.71).
  • Timing study: 7 annotators with NLP background used INCEpTION after a 45-minute training session and annotated 28 i2b2/VA documents in total. Full coreference took 1.85× longer than mention-only annotation overall and about 2× longer for long (~800-word) documents.
  • Method: continued training with target-domain mentions only, plus a high-precision mention detection objective. Fewer antecedent candidates led to a stronger linker.
  • Evaluation domains: CoNLL-2012/OntoNotes (news/conversation), i2b2/VA (medical notes) and an unreleased corpus of child welfare notes from a county Department of Human Services. The authors describe this as the first coreference study in child protective services.
  • Limitations: the work covers only the Lee et al. (2018) c2f architecture with SpanBERT. Timing results may depend on the interface, annotator expertise and document length. The authors say the child welfare system is not ready for deployment without a fairness audit and stakeholder oversight.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

7 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.acl-long.588

@inproceedings{gandhi2023annotating,
  title = {{Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution}},
  author = {Gandhi, Nupoor and Field, Anjalie and Strubell, Emma},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.588},
  url = {https://doi.org/10.18653/v1/2023.acl-long.588}
}

References

35 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rahul Aralikatte and Anders Søgaard. 2020. Model-based annotation of coreference. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 74–79, Marseille, France. European Language Resources Association. pages 5–9. Association for Computational Linguistics. Event Title: The 27th International Conference on Computational Linguistics (COLING 2018). ↗ our card
  2. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An annotated dataset of coreference in English literature. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 44–54, Marseille, France. European Language Resources Association. ↗ our card
  3. Sandra Kübler and Desislava Zhekova. 2011. Singletons and coreference resolution evaluation. In Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pages 261–267, Hissar, Bulgaria. Association for Computational Linguistics. find our card
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  5. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. ↗ our card
  6. Vladimir Dobrovolskii. 2021. Word-level coreference resolution. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7670–7675, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗ our card
  7. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–692, New Orleans, Louisiana. Association for Computational Linguistics. ↗ our card
  8. Greg Durrett and Dan Klein. 2013. Easy victories and uphill battles in coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1971–1982, Seattle, Washington, USA. Association for Computational Linguistics. ↗ our card
  9. Maolin Li, Hiroya Takamura, and Sophia Ananiadou. 2020a. A neural model for aggregating coreference annotation in crowdsourcing. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5760–5773, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  10. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. Association for Computational Linguistics. ↗
  11. Pengshuai Li, Xinsong Zhang, Weijia Jia, and Wei Zhao. 2020b. Active testing: An unbiased evaluation method for distantly supervised relation extraction. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 204–211, Online. Association for Computational Linguistics. find
  12. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. ↗ our card
  13. Yuval Kirstain, Ori Ram, and Omer Levy. 2021. Coreference resolution without span representations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 14–19, Online. Association for Computational Linguistics. ↗ our card
  14. Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard Eckart de Castilho, and Iryna Gurevych. 2018. The inception platform: Machine-assisted and knowledge-oriented interactive annotation. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, Jing Lu and Vincent Ng. 2020. Conundrums in entity coreference resolution: Making sense of the state of the art. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6620–6631, Online. Association for Computational Linguistics. find our card
  15. Timothy Miller, Dmitriy Dligach, and Guergana Savova. 2012. Active learning for coreference resolution. In BioNLP: Proceedings of the 2012 Workshop on Biomedical Natural Language Processing, pages 73–81, Montréal, Canada. Association for Computational Linguistics. find our card
  16. Nafise Sadat Moosavi and Michael Strube. 2016a. Search space pruning: A simple solution for better coreference resolvers. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1005–1011, San Diego, California. Association for Computational Linguistics. find our card
  17. Nafise Sadat Moosavi and Michael Strube. 2016b. Which coreference evaluation metric do you trust? a proposal for a link-based entity aware metric. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 632–642, Berlin, Germany. Association for Computational Linguistics. find our card
  18. Silviu Paun, Juntao Yu, Nafise Sadat Moosavi, and Massimo Poesio. 2022. Scoring Coreference Chains with Split-Antecedent Anaphors. ↗ our card
  19. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task, pages 1–40, Jeju Island, Korea. Association for Computational Linguistics. find our card
  20. Marta Recasens, Marie-Catherine de Marneffe, and Christopher Potts. 2013. The life and death of discourse entities: Identifying singleton mentions. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 627–633, Atlanta, Georgia. Association for Computational Linguistics. find
  21. Mrinmaya Sachan, Eduard Hovy, and Eric P Xing. 2015. An active learning approach to coreference resolution. In Twenty-Fourth International Joint Conference on Artificial Intelligence. find
  22. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A human-centered review of algorithms used within the us child welfare system. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–15. find
  23. Hariprasad Timmapathini, Anmol Nayak, Sarathchan-dra Mandadi, Siva Sangada, Vaibhav Kesri, Karthikeyan Ponnalagu, and Vijendran Gopalan Venkoparao. 2021. Probing the spanbert architecture to interpret scientific domain adaptation challenges for coreference resolution. In Proceedings of the Workshop on Scientific Document Understanding co-located with 35th AAAI Conference on Artificial Inteligence. find
  24. Shubham Toshniwal, Patrick Xia, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2021. On generalization in coreference resolution. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 111– 120, Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗ our card
  25. Ankith Uppunda, Susan Cochran, Jacob Foster, Alina Arseniev-Koehler, Vickie Mays, and Kai-Wei Chang. 2021. Adapting coreference resolution for processing violent death narratives. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4553–4559, Online. Association for Computational Linguistics. ↗
  26. Ozlem Uzuner, Andreea Bodnari, Shuying Shen, Tyler Forbush, John Pestian, and Brett R South. 2012. Evaluating the state of the art in coreference resolution for electronic medical records. Journal of the American Medical Informatics Association, 19(5):786–791. ↗
  27. Yanshan Wang, Liwei Wang, Majid Rastegar-Mojarad, Sungrim Moon, Feichen Shen, Naveed Afzal, Sijia Liu, Yuqun Zeng, Saeed Mehrabi, Sunghwan Sohn, et al. 2018. Clinical information extraction applications: a literature review. Journal of biomedical informatics, 77:34–49. ↗
  28. Zhaofeng Wu and Matt Gardner. 2021. Understanding mention detector-linker interaction in neural coreference resolution. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 150–157, Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗ our card
  29. Patrick Xia and Benjamin Van Durme. 2021. Moving on from OntoNotes: Coreference resolution model transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5241–5256, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗
  30. Juntao Yu, Nafise Sadat Moosavi, Silviu Paun, and Massimo Poesio. 2021. Stay together: A system for single and split-antecedent anaphora resolution. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4174–4184, Online. Association for Computational Linguistics. find our card
  31. Michelle Yuan, Patrick Xia, Chandler May, Benjamin Van Durme, and Jordan Boyd-Graber. 2022. Adapting coreference resolution models through active learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7533–7549, Dublin, Ireland. Association for Computational Linguistics. ↗ our card
  32. Amir Zeldes. 2022. Opinion Piece: Can we Fix the Scope for Coreference? Problems and Solutions for Benchmarks beyond OntoNotes. Dialogue & Discourse, 13(1):41–62. ↗ our card
  33. Amir Zeldes and Shuo Zhang. 2016. When annotation schemes change rules help: A configurable approach to coreference resolution beyond OntoNotes. In Proceedings of the Workshop on Coreference Resolution Beyond OntoNotes (CORBON 2016), pages 92– 101, San Diego, California. Association for Computational Linguistics. ↗ our card
  34. Shanheng Zhao and Hwee Tou Ng. 2014. Domain adaptation with active learning for coreference resolution. In Proceedings of the 5th International Workshop on Health Text Mining and Information Analysis (Louhi), pages 21–29, Gothenburg, Sweden. Association for Computational Linguistics. ↗ our card
  35. Yilun Zhu, Sameer Pradhan, and Amir Zeldes. 2021. Anatomy of OntoGUM—Adapting GUM to the OntoNotes scheme to evaluate robustness of SOTA coreference algorithms. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 141–149, Punta Cana, Dominican Republic. Association for Computational Linguistics. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gandhi-2023-annotating-mentions-alone-enables-efficient/