Elamur Research⌕
☰
home / papers / 2020 / Entity Linking in 100 Languages

Entity Linking in 100 Languages

Summary

Most entity linking (EL) research beyond monolingual settings is cross-lingual. It assumes one KB language, usually English Wikipedia, and so ignores the many entities described only in other languages. Botha, Shan and Gillick instead propose Multilingual Entity Linking (MEL). In MEL, mentions in any language are resolved against WikiData, a language-agnostic KB, and each entity is represented by Wikipedia descriptions in whichever languages exist for it. They build training data from Wikipedia hyperlinks, using anchor texts as mentions. On this data they train a dual encoder retrieval model with improved encoders, better-targeted negative mining and an auxiliary entity-pairing task. A single model handles 104 languages and about 20 million entities. It beats earlier results on the narrower cross-lingual task. The authors also release Mewsli-9, a WikiNews-based test set, and analyse results by entity frequency. Overall accuracy is high, but rare and unseen entities remain difficult. The work matters because it moves EL toward fairer, large-scale multilingual coverage, including low-resource languages and non-English entities.

Key points

  • Scale of the setup: the KB contains 19,666,787 WikiData items that have a Wikipedia page in at least one language. Training supervision consists of 684 million hyperlink mentions in 104 languages.
  • Mewsli-9 dataset:
    • 289,087 mentions from 58,717 WikiNews articles, covering 9 languages: Japanese, German, Spanish, Arabic, Serbian, Turkish, Persian, Tamil and English.
    • The languages span 5 families and 6 scripts.
    • Of the 82,162 distinct entities, 11% have no English Wikipedia page.
  • Model F+ architecture: a dual encoder built from 4-layer BERT Transformers. The mention side encodes the document title, left and right context, and the mention marked with [E]/[/E]. The entity side encodes the entity description.
  • Results on Mewsli-9:
    • Micro-averaged R@1 is 0.89 and R@10 is 0.96. The macro-average R@1 across frequency bins is 0.70.
    • For entities never seen in training, R@1 drops to just 0.08.
  • Reranking: a 12-layer cross-attention reranker initialized from multilingual BERT re-scores the top-5 candidates. It raises micro R@1 to 0.91 and helps most frequency buckets.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Named entity recognition

Cited by

46 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 44 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.630 · arXiv 2011.02690

@inproceedings{botha2020entity,
  title = {{Entity Linking in 100 Languages}},
  author = {Botha, Jan A. and Shan, Zifei and Gillick, Daniel},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.630},
  eprint = {2011.02690},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.630}
}

References

40 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pag… find
  2. Oshin Agarwal and Daniel M Bikel. 2020. Entity linking via dual and cross-attention encoders. arXiv preprint arXiv:2004.03555. — [arXiv:2004.03555](https://arxiv.org/abs/2004.03555) find
  3. Razvan Bunescu and Marius Paşca. 2006. Using encyclopedic knowledge for named entity disambiguation. In 11th Conference of the European Chapter of the Association for Computational Linguistics, Trento, Italy. Association for Computational Linguistics. find
  4. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  6. Thibault Févry, Nicholas FitzGerald, Livio Baldini Soares, and Tom Kwiatkowski. 2020a. Empirical evaluation of pretraining strategies for supervised entity linking. In Automated Knowledge Base Construction. find
  7. Thibault Févry, Livio Baldini Soares, Nicholas FitzGerald, Eunsol Choi, and Tom Kwiatkowski. 2020b. Entities as experts: Sparse memory access with entity supervision. arXiv preprint arXiv:2004.07202. find
  8. Matthew Francis-Landau, Greg Durrett, and Dan Klein. 2016. Capturing semantic similarity for entity linking with convolutional neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 125… find
  9. Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego Garcia-Olano. 2019. Learning dense representations for entity retrieval. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 528–537, Hong Kong, China.… find
  10. Daniel Gillick, Alessandro Presta, and Gaurav Singh Tomar. 2018. End-to-end retrieval in continuous space. arXiv preprint arXiv:1811.08008. — [arXiv:1811.08008](https://arxiv.org/abs/1811.08008) find
  11. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrieval-augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria. PMLR. find
  12. Filip Ilievski, Piek Vossen, and Stefan Schlobach. 2018. Systematic study of long tail phenomena in entity linking. In Proceedings of the 27th International Conference on Computational Linguistics, pages 664–674, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  13. Heng Ji, Ralph Grishman, Hoa Trang Dang, Kira Griffitt, and Joe Ellis. 2010. Overview of the TAC 2010 knowledge base population track. In Proceedings of the Text Analytics Conference. find
  14. Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Ass… find
  15. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations, San Diego, CA. find
  16. Xiao Ling, Sameer Singh, and Daniel S. Weld. 2015. Design challenges for entity linking. Transactions of the Association for Computational Linguistics, 3:315–328. find
  17. Robert Logan, Nelson F. Liu, Matthew E. Peters, Matt Gardner, and Sameer Singh. 2019. Barack’s wife Hillary: Using knowledge graphs for fact-aware language modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5962–5971, Florence, Italy. Associat… find
  18. Lajanugen Logeswaran, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, Jacob Devlin, and Honglak Lee. 2019. Zero-shot entity linking by reading entity descriptions. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3449–3460. Association for Computation… find
  19. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  20. Paul McNamee, James Mayfield, Dawn Lawrie, Douglas Oard, and David Doermann. 2011. Cross-language entity linking. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 255–263, Chiang Mai, Thailand. Asian Federation of Natural Language Processing. find
  21. Anne-Lyse Minard, Manuela Speranza, Ruben Urizar, Begoña Altuna, Marieke van Erp, Anneleen Schoen, and Chantal van Son. 2016. MEANTIME, the NewsReader multilingual event and time corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation, pages 4417–4422, Porto… find
  22. Sameer Singh, Amarnag Subramanya, Fernando Pereira, and Andrew McCallum. 2012. Wikilinks: A large-scale cross-document coreference corpus labeled via links to Wikipedia. Technical Report UM-CS-2012-015. find
  23. Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017. Cross-lingual name tagging and linking for 282 languages. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1946–1958, Vancouver, Canada. A… find
  24. Chen-Tse Tsai and Dan Roth. 2016. Cross-lingual wikification using multilingual embeddings. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 589–598, San Diego, California. Association for Computa… find
  25. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996– 5001, Florence, Italy. Association for Computational Linguistics. find our card
  26. Shyam Upadhyay, Nitish Gupta, and Dan Roth. 2018. Joint multilingual supervision for cross-lingual entity linking. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2486–2495, Brussels, Belgium. Association for Computational Linguistics. find
  27. Ratish Puduppully, Li Dong, and Mirella Lapata. 2019. Data-to-text generation with entity modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2023– 2035. Association for Computational Linguistics. find
  28. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008. find
  29. Shruti Rijhwani, Jiateng Xie, Graham Neubig, and Jaime Carbonell. 2019. Zero-shot neural transfer for cross-lingual entity linking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6924–6931. find
  30. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? arXiv preprint arXiv:2002.08910. find
  31. Henry Rosales-Méndez, Aidan Hogan, and Barbara Poblete. 2018. VoxEL: a benchmark dataset for multilingual entity linking. In International Semantic Web Conference, pages 170–186. Springer. find
  32. Peter Shaw, Philip Massey, Angelica Chen, Francesco Piccinno, and Yasemin Altun. 2019. Generating logical forms from graph representations of text and entities. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 95–106. Association for Computational Lin… find
  33. Wei Shen, Jianyong Wang, and Jiawei Han. 2014. Entity linking with a knowledge base: Issues, techniques, and solutions. IEEE Transactions on Knowledge and Data Engineering, 27(2):443–460. find
  34. Avirup Sil and Radu Florian. 2016. One for all: Towards language independent named entity linking. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2255–2264, Berlin, Germany. Association for Computational Linguistics. find
  35. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78–85. find
  36. Jörg Waitelonis, Henrik Jürges, and Harald Sack. 2016. Don’t compare apples to oranges: Extending gerbil for a fine grained nel evaluation. In Proceedings of the 12th International Conference on Semantic Systems, SEMANTiCS 2016, page 65–72, New York, NY, USA. Association for Computing Machinery. find
  37. Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2019. Zero-shot entity linking with dense entity retrieval. arXiv preprint arXiv:1911.03814. find our card
  38. Shuyan Zhou, Shruti Rijhwani, and Graham Neubig. 2019. Towards zero-resource cross-lingual entity linking. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 243–252. Association for Computational Linguistics. find
  39. Shuyan Zhou, Shruti Rijhwani, John Wieting, Jaime Carbonell, and Graham Neubig. 2020. Improving candidate generation for low-resource cross-lingual entity linking. Transactions of the Association for Computational Linguistics, 8:109–124. find
  40. Avirup Sil, Gourab Kundu, Radu Florian, and Wael Hamza. 2018. Neural cross-lingual entity linking. In Thirty-Second AAAI Conference on Artificial Intelligence. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/botha-2020-entity-linking-100-languages/