Elamur Research⌕
☰
home / papers / 2016 / Cross-lingual Wikification Using Multilingual Embeddings

Cross-lingual Wikification Using Multilingual Embeddings

Paper · 2016 · Chen-Tse Tsai and Dan Roth

Summary

Tsai and Roth address cross-lingual Wikification: linking entity mentions in non-English documents to the matching English Wikipedia titles. The method does not depend on machine translation or parallel corpora, so it can be applied to any language that has a Wikipedia edition. Their system, WikiME, works in three steps:

  1. It learns monolingual skip-gram embeddings for each language. Before training, hyperlinked anchor text is replaced with the target title, so words and titles share one vector space.
  2. It projects foreign and English embeddings into a common space with CCA. Inter-language title links serve as the bilingual dictionary instead of parallel text.
  3. A ranker trained on Wikipedia scores English title candidates using cross-lingual similarity features.

The authors built a new, difficult evaluation set from Wikipedia covering 12 languages, on which WikiME beats several baselines. On TAC KBP2015 Entity Linking it is competitive with the top Spanish and Chinese systems, even though those systems relied on translation. This makes the approach promising for low-resource languages.

Key points

  • Monolingual embeddings follow the "Alignment by Wikipedia Anchors" model of Wang et al. (2014), trained with skip-gram.
  • The cross-lingual projection follows the CCA approach of Faruqui and Dyer (2014). The usual translation dictionary is replaced by Wikipedia title alignments, so only Wikipedia articles and inter-language links are needed.
  • Ranking features include:
    • the link statistics Pr(c|m) and Pr(m|c);
    • cosine similarity between the candidate title embedding and local context windows of 30, 100 and 200 words;
    • similarity to other mentions in the document;
    • similarity to previously linked titles.
  • Candidate generation looks only at the mention's surface form. Titles that exist solely in English Wikipedia are out of scope; the authors name transliteration-based candidate generation as future work.
  • Evaluation uses a new 12-language Wikipedia dataset and TAC KBP2015 (Spanish, Chinese), where WikiME reaches state-of-the-art level.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Illinois Urbana-ChampaignUrbana · United States108 works · 67,432 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputer Vision and Pattern Recognition · 1,282 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 467 works

Cite

Cite as Chen-Tse Tsai and Dan Roth. 2016. Cross-lingual Wikification Using Multilingual Embeddings. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1072

@inproceedings{tsai2016cross,
  title = {{Cross-lingual Wikification Using Multilingual Embeddings}},
  author = {Chen-Tse Tsai and Dan Roth},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1072},
  url = {https://doi.org/10.18653/v1/n16-1072}
}

References

28 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. R. Bunescu and M. Pasca. 2006. Using encyclopedic knowledge for named entity disambiguation. In Proceedings of the European Chapter of the ACL (EACL).
  2. P.-C. Chang, M. Galley, and C. D. Manning. 2008. Optimizing chinese word segmentation for machine translation performance. In Proceedings of the third workshop on statistical machine translation, Association for Computational Linguistics.
  3. X. Cheng and D. Roth. 2013. Relational inference for wikification. In Proceedings of the Conference on Empirical Methods for Natural Language Processing (EMNLP).
  4. J. Coulmance, J.-M. Marty, G. Wenzek, and A. Benhal-loum. 2015. Trans-gram, fast cross-lingual word-embeddings. In Proceedings of EMNLP. find
  5. S. Cucerzan. 2007. Large-scale named entity disambiguation based on Wikipedia data. In Proceedings of the 2007 Joint Conference of EMNLP-CoNLL, pages 708–716. find
  6. M. Faruqui and C. Dyer. 2014. Improving vector space word representations using multilingual correlation. Association for Computational Linguistics.
  7. P. Ferragina and U. Scaiella. 2010. Tagme: on-the-fly annotation of short text fragments (by wikipedia entities). In Proceedings of the 19th ACM international conference on Information and knowledge management, pages 1625–1628.
  8. ACM. S. Gouws, Y. Bengio, and G. Corrado. 2014. Bilbowa: Fast bilingual distributed representations without word alignments. In Deep Learning Workshop, NIPS. find
  9. X. Han and J. Zhao. 2009. Named entity disambiguation by leveraging wikipedia semantic knowledge. In Proceedings of the 18th ACM conference on Information and knowledge management, pages 215–224. find
  10. ACM. K. M. Hermann and P. Blunsom. 2014. Multilingual distributed representations without word alignment. In Proceedings of ICLR. find
  11. H. Hotelling. 1936. Relations between two sets of variates. Biometrika, pages 321–377. find
  12. H. Ji, R. Grishman, and H. T. Dang. 2012. Overview of the tac2011 knowledge base population track. In Text Analysis Conference (TAC2011). find
  13. H. Ji, J. Nothman, and B. Hachey. 2015. Overview of tac-kbp2014 entity discovery and linking tasks. In Text Analysis Conference (TAC2014). find
  14. H. Ji, J. Nothman, B. Hachey, and R. Florian. 2016. Overview of tac-kbp2015 tri-lingual entity discovery and linking. In Text Analysis Conference (TAC2015). find
  15. C.-P. Lee and C.-J. Lin. 2014. Large-scale linear RankSVM. Neural computation, 26(4):781–817. find
  16. T. Luong, H. Pham, and C. D. Manning. 2015. Bilingual word representations with monolingual quality in mind. In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing.
  17. P. McNamee, J. Mayfield, D. Lawrie, D. W. Oard, and D. S. Doermann. 2011. Cross-language entity linking. In Proceedings of IJCNLP, pages 255–263. find
  18. R. Mihalcea and A. Csomai. 2007. Wikify!: linking documents to encyclopedic knowledge. In Proceedings of ACM Conference on Information and Knowledge Management (CIKM), pages 233–242. find
  19. T. Mikolov, K. Chen, G. Corrado, and J. Dean. 2013a. Efficient estimation of word representations in vector space. In Proceedings of Workshop at ICLR.
  20. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111– 3119.
  21. D. Milne and I. H. Witten. 2008. Learning to link with wikipedia. In Proceedings of ACM Conference on Information and Knowledge Management (CIKM), pages 509–518. find
  22. A. Moro, A. Raganato, and R. Navigli. 2014. Entity linking meets word sense disambiguation: a unified approach. In Transactions of the Association for Computational Linguistics, volume 2, pages 231–244.
  23. L. Ratinov, D. Downey, M. Anderson, and D. Roth. 2011. Local and global algorithms for disambiguation to wikipedia. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  24. T. Shi, Z. Liu, Y. Liu, and M. Sun. 2015. Learning crosslingual word embeddings via matrix co-factorization. In Proceedings of ACL. find
  25. H. Soyer, P. Stenetorp, and A. Aizawa. 2015. Leveraging monolingual data for crosslingual compositional word representations. In Proceedings of ICLR. I. Vulić and M.-F. find
  26. Moens. 2015. Bilingual word embeddings from non-parallel document-aligned data applied to bilingual lexicon induction. In Proceedings of ACL. find
  27. Z. Wang, J. Zhang, J. Feng, and Z. Chen. 2014. Knowledge graph and text jointly embedding. In Proceedings of EMNLP.
  28. H. Wang, J. G. Zheng, X. Ma, P. Fox, and H. Ji. 2015. Language and domain independent entity linking with quantified collective validation. In Proceedings of EMNLP. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tsai-2016-cross-lingual-wikification-multilingual-embeddings/