Elamur Research⌕
☰
home / papers / 2019 / Deep Cross-Lingual Coreference Resolution for Less-Resourced Languages: The Case of Basque

Deep Cross-Lingual Coreference Resolution for Less-Resourced Languages: The Case of Basque

Summary

The paper addresses coreference resolution for Basque, a less-resourced language whose only annotated corpus is too small to train neural models well. The authors build the first neural coreference system for Basque. It is a mention-pair feedforward network adapted from a Polish system (Nitoń et al., 2018), chosen because Polish shares Basque's agglutinative morphology and free word order. The network uses averaged FastText embeddings plus simple pair features such as distance, string match and lemma match. They train it in two ways: monolingually on EPEC-KORREF, and cross-lingually on English OntoNotes news data. In the cross-lingual setup, embeddings are aligned with VecMap and no Basque training data is used. The English-trained model slightly outperforms the Basque-trained one. Both neural variants still fall well short of earlier rule-based and ML-based Basque systems. Pronouns cause many errors, partly because Basque pronouns carry no gender and are often dropped. The work suggests that transfer from a high-resource language can replace scarce target-language annotation, and it motivates future multilingual coreference systems.

Key points

  • Data: EPEC-KORREF (Basque news) has about 46K words and 12.8K mentions, with singletons annotated. The English training set uses the OntoNotes newswire and broadcast news portions, about 825K words and 100K mentions.
  • Model: The network has three ReLU hidden layers (500/300/100 units), a sigmoid output and 1,226-dimensional inputs, and is implemented in KERAS. Clustering thresholds were tuned on the dev set: 0.5 for the monolingual model and 0.9 for the cross-lingual one.
  • CoNLL F1 with automatic mentions: 39.12 monolingual vs 40.93 cross-lingual. With gold mentions the scores are 53.19 vs 54.46, and mention detection reaches 73.79 F1.
  • Comparison with earlier systems: The prior rule-based system scores 55.98 (auto) and 76.51 (gold), and the ML-based system scores 54.21 (auto) and 73.94 (gold), so both clearly beat the neural approaches.
  • Future work: The authors plan to apply the cross-lingual approach to the end-to-end architecture of K. Lee et al. (2018) and to test other language pairs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

8 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-2806

@inproceedings{urbizu2019deep,
  title = {{Deep Cross-Lingual Coreference Resolution for Less-Resourced Languages: The Case of Basque}},
  author = {Urbizu, Gorka and Soraluze, Ander and Arregi, Olatz},
  booktitle = {Proceedings of the Second Workshop on Computational Models of Reference, Anaphora and Coreference},
  year = {2019},
  doi = {10.18653/v1/w19-2806},
  url = {https://doi.org/10.18653/v1/w19-2806}
}

References

32 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018. A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 789–798. find our card
  2. Amit Bagga and Breck Baldwin. 1998. Entity-based cross-document coreferencing using the vector space model. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics-Volume 1, pages 79–85. Association for… find
  3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. find
  4. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 25–32. Association for Computational Linguistics. find our card
  5. Klara Ceberio, Itziar Aduriz, Arantza Dı́az de Ilarraza, and Ines Garcia-Azkoaga. 2018. Coreferential relations in Basque: the annotation process. Journal of psycholinguistic research, 47(2):325–342. find
  6. Kevin Clark and Christopher D Manning. Improving Coreference Resolution by Learning Entity-Level Distributed Representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Nafise Sadat Moosavi and Michael Strube. Which coreference… find our card
  7. Nicolas Nicolov, Franco Salvetti, and Steliana Ivanova. 2008. Sentiment Analysis: Does Coreference Matter? In AISB 2008 Convention Communication, Interaction and Social Intelligence, pages 37–40. find
  8. Bartłomiej Nitoń, Paweł Morawiecki, and Maciej Ogrodniczuk. 2018. Deep neural networks for coreference resolution for Polish. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018), pages 395–400. find our card
  9. Pascal Denis and Jason Baldridge. 2009. Global joint models for coreference resolution and named entity classification. Procesamiento del Lenguaje Natural, 42:87–96. find
  10. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. Ontonotes: The 90% solution. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers. find
  11. Cheoneum Park, KyoungHo Choi, Changki Lee, and Soojong Lim. 2016. Korean Coreference Resolution with Guided Mention Pair Model Using Deep Learning. ETRI Journal, 38(6):1207–1217. find
  12. Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, pages 448–456. find
  13. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pag… find our card
  14. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. International Conference on Learning Representations. find
  15. Marta Recasens and Eduard Hovy. 2011. BLANC: Implementing the Rand index for coreference evaluation. Natural Language Engineering, 17(4):485– 510. find
  16. Gourab Kundu, Avi Sil, Radu Florian, and Wael Hamza. Neural cross-lingual coreference resolution and its application to entity linking. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). find our card
  17. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational linguistics, 27(4):521–544. find our card
  18. Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291. find
  19. Ander Soraluze, Olatz Arregi, Xabier Arregi, and Arantza Diaz de Ilarraza. 2015. Coreference Resolution for Morphologically Rich Languages. Adaptation of the Stanford System to Basque. In Procesamiento del Lenguaje Natural, volume 55, pages 23–30. find
  20. Heeyoung Lee, Angel Chang, Yves Peirsman, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2013. Deterministic coreference resolution based on entity-centric, precision-ranked rules. Computational Linguistics, 39(4):885–916. find our card
  21. Ander Soraluze, Olatz Arregi, Xabier Arregi, and Arantza Dı́az de Ilarraza. 2017. Improving mention detection for Basque based on a deep error analysis. Natural Language Engineering, 23(3):351–384. find
  22. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end Neural Coreference Resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197. find our card
  23. Ander Soraluze, Olatz Arregi, Xabier Arregi, Arantza Diaz de Ilarraza, Mijail Kabadjov, and Massimo Poesio. 2016. Coreference Resolution for the Basque Language with BART. In Proceedings of the Workshop on Coreference Resolution Beyond OntoNotes (CORBON 2016), pages 67–73. find
  24. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In NAACL-HLT (2), pages 687–692. Association for Computational Linguistics. find our card
  25. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958. find
  26. Josef Steinberger, Massimo Poesio, Mijail A. Kabadjov, and Karel Jeek. 2007. Two Uses of Anaphora Resolution in Summarization. Information Processing and Management, 43(6):1663–1680. find
  27. Yannick Versley, Simone Paolo Ponzetto, Massimo Poesio, Vladimir Eidelman, Alan Jern, Jason Smith, Xiaofeng Yang, and Alessandro Moschitti. 2008. BART: A modular toolkit for coreference resolution. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human La… find our card
  28. Jose Vicedo and Antonio Ferrández. 2006. Coreference In Q&A. In Advances in Open Domain Question Answering, volume 32 of Text, Speech and Language Technology, pages 71–96. Springer. find
  29. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th Conference on Message Understanding, pages 45–52. Association for Computational Linguistics. find our card
  30. Lesly Miculicich Werlen and Andrei Popescu-Belis. 2017. Using coreference links to improve Spanish-to-English machine translation. In Proceedings of the 2nd Workshop on Coreference Resolution Beyond OntoNotes (CORBON 2017), pages 30–40. find our card
  31. Sam Wiseman, Alexander M Rush, Stuart Shieber, and Jason Weston. Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language… find our card
  32. Sam Wiseman, Alexander M Rush, and Stuart M Shieber. 2016. Learning Global Features for Coreference Resolution. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 994–1004. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/urbizu-2019-deep-cross-lingual-coreference-resolution/