Elamur Research⌕
☰
home / papers / undated / Neural End-to-end Coreference Resolution for German in Different Domains

Neural End-to-end Coreference Resolution for German in Different Domains

Summary

The paper addresses German coreference resolution, where neural methods had seen little use and no neural results existed for German news corpora. The authors adapt the English end-to-end architecture of Lee et al. (2017, 2018), as extended by Joshi et al. (2019) and reimplemented in PyTorch by Xu and Choi (2020). Token representations come from contextual ELECTRA embeddings, trained jointly with mention and entity scoring. They compare two variants that share a mention-proposal step. One is a coarse-to-fine model that links mentions to antecedents. The other is an incremental model that matches mentions against entity representations, so memory grows with the number of entities instead of quadratically with mentions. They evaluate on news data (SemEval-2010, TüBa-D/Z 10.0) and literary data (DROC), and also study language models, pre-training and compute limits. Both variants beat earlier systems by 10–30 CoNLL-F1 points on all three datasets. Error analysis reveals recency bias and inconsistency across long texts. The models and code are released, giving German NLP a strong baseline for news and literary texts.

Key points

  • Datasets: TüBa-D/Z 10.0 (no singletons, using Roesiger and Kuhn's splits), SemEval-2010 German (singletons annotated) and DROC.
  • DROC details: 90 literary chapters averaging about 4,369 tokens each, split 58/14/18. Only character references are annotated, and only mention heads are marked.
  • DROC result: With singletons, the incremental model reaches 64.72 F1, compared with 53.17 for Krug's (2020) best system (+11.6). The coarse-to-fine model scores 65.50 without singletons. Human agreement is about 82.54 F1.
  • Scaling to long texts: The incremental model loses more accuracy on long documents than coarse-to-fine. For example, when run on the German-language version of The Hound of the Baskervilles, the model scattered mentions of "Holmes" over 31 clusters. Removing the token-distance features reduced recency bias.
  • Availability: Code and models are at github.com/uhh-lt/neural-coref (branch konvens).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 8 not yet.

References

38 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An annotated dataset of coreference in English literature. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 44–54, Marseille, France. European Language Resources Association. ↗ our card
  2. David Bamman, Ted Underwood, and Noah A. Smith. 2014. A Bayesian mixed effects model of literary character. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 370–379, Baltimore, Maryland. Association for Computational Linguistics. ↗ our card
  3. Anders Björkelund and Jonas Kuhn. 2014. Learning Structured Perceptrons for Coreference Resolution with Latent Antecedents and Non-local Features. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 47–57, Baltimore, Maryland. Association for Computational Linguistics. find our card
  4. Branden Chan, Stefan Schweter, and Timo Möller. 2020. German’s next language model. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6788–6796, Barcelona, Spain (Online). International Committee on Computational Linguistics. find our card
  5. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: Pre-training text encoders as discriminators rather than generators. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. OpenReview.net. find
  6. Kevin Clark and Christopher D. Manning. 2016a. Deep reinforcement learning for mention-ranking coreference models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2256–2262, Austin, Texas. Association for Computational Linguistics. find our card
  7. Kevin Clark and Christopher D. Manning. 2016b. Improving coreference resolution by learning entity-level distributed representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 643–653, Berlin, Germany. Association for Computational Linguistics. find our card
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  9. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. ↗
  10. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. ↗ our card
  11. Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019. BERT for coreference resolution: Baselines and analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5803–5808, Hong Kong, China. Association for Computational Linguistics. ↗ our card
  12. Ben Kantor and Amir Globerson. 2019. Coreference resolution with entity equalization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 673–677, Florence, Italy. Association for Computational Linguistics. ↗ our card
  13. Manfred Klenner and Don Tuggener. 2011. An Incremental Entity-Mention Model for Coreference Resolution with Restrictive Antecedent Accessibility. In Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pages 178–185, Hissar, Bulgaria. Association for Computational Linguistics. find our card
  14. Markus Krug. 2020. Techniques for the Automatic Extraction of Character Networks in German Historic Novels. Ph.D. thesis, Universität Würzburg. ↗
  15. Markus Krug, Lukas Weimer, Isabella Reger, Luisa Macharowsky, Stephan Feldhaus, Frank Puppe, and Fotis Jannidis. 2018. Description of a corpus of character references in German novels-DROC [Deutsches ROman Corpus]. DARIAH-DE Working Papers, 27. find
  16. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 shared task. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 28–34, Portland, Oregon. Association for Computational Linguistics. find our card
  17. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. ↗ our card
  18. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–692, New Orleans, Louisiana. Association for Computational Linguistics. ↗ our card
  19. Thomas S. Morton. 1999. Using coreference for question answering. In Coreference and Its Applications, pages 85–89, College Park, Maryland. Association for Computational Linguistics. find our card
  20. Karin Naumann and Vera Möller. 2006. Manual for the annotation of in-document referential relations. Technical report, Universität Tübingen. find
  21. Janis Pagel and Nils Reiter. 2020. GerDraCor-coref: A coreference corpus for dramatic texts in German. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 55–64, Marseille, France. European Language Resources Association. ↗ our card
  22. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc. ↗ our card
  23. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics. find
  24. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task, pages 1–40, Jeju Island, Korea. Association for Computational Linguistics. find our card
  25. Marta Recasens, Lluı́s Màrquez, Emili Sapena, M. Antònia Martı́, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. SemEval-2010 task 1: Coreference resolution in multiple languages. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 1–8, Uppsala, Sweden. Association for Computational Linguistics. find our card
  26. Ina Roesiger and Jonas Kuhn. 2016. IMS HotCoref DE: A Data-driven Co-reference Resolver for German. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 155–160, Portorož, Slovenia. European Language Resources Association (ELRA). find our card
  27. Ina Roesiger, Sarah Schulz, and Nils Reiter. 2018. Towards Coreference for Literary Text: Analyzing Domain-Specific Phenomena. In Proceedings of the Second Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 129–138, Santa Fe, New Mexico. Association for Computational Linguistics. find our card
  28. Josef Steinberger, Massimo Poesio, Mijail A. Kabadjov, and Karel Ježek. 2007. Two uses of anaphora resolution in summarization. Information Processing & Management, 43(6):1663–1680. ↗
  29. Heike Telljohann, Erhard W. Hinrichs, Sandra Kübler, Heike Zinsmeister, and Kathrin Beck. 2017. Stylebook for the tübingen treebank of written german (TüBa-D/Z). In Seminar für Sprachwissenschaft, Universität Tübingen, Germany. find
  30. Shubham Toshniwal, Sam Wiseman, Allyson Ettinger, Karen Livescu, and Kevin Gimpel. 2020. Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8519–8526, Online. Association for Computational Linguistics. ↗ our card
  31. Don Tuggener and Manfred Klenner. 2014. A Hybrid Entity-Mention Pronoun Resolution Model for German Using Markov Logic Networks. In Proceedings of the 12th Edition of the Konvens Conference, pages 21–29, Hildesheim, Germany. Universitätsbibliothek Hildesheim. ↗
  32. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. 2013. OntoNotes release 5.0 ldc2013t19. Linguistic Data Consortium, Philadelphia, Pennsylvania, 23. find
  33. Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1416–1426, Beijing, China. Association for Computational Linguistics. find our card
  34. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 994–1004, San Diego, California. Association for Computational Linguistics. find our card
  35. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. CorefQA: Coreference resolution as query-based span prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6953–6963, Online. Association for Computational Linguistics. ↗ our card
  36. Patrick Xia and Benjamin Van Durme. 2021. Moving on from OntoNotes: Coreference resolution model transfer. CoRR, abs/2104.08457. ↗
  37. Patrick Xia, João Sedoc, and Benjamin Van Durme. 2020. Incremental neural coreference resolution in constant memory. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8617–8624, Online. Association for Computational Linguistics. ↗ our card
  38. Liyan Xu and Jinho D. Choi. 2020. Revealing the myth of higher-order inference in coreference resolution. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8527–8533, Online. Association for Computational Linguistics. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/coref--schroder-neural-end-end-coreference-resolution/