Elamur Research⌕
☰
home / papers / 2017 / Cross-lingual Name Tagging and Linking for 282 Languages

Cross-lingual Name Tagging and Linking for 282 Languages

Summary

The paper addresses name tagging and entity linking for languages with almost no annotated data. It aims to cover all 282 languages present in Wikipedia and to link mentions to an English KB (English Wikipedia). Instead of relying on manual labels, the authors build "silver-standard" training data automatically:

  • Typing English pages. English Wikipedia pages are assigned types by Maximum Entropy classifiers. These use DBpedia properties as features and are trained on mentions from the AMR corpus.
  • Spreading labels. The types are carried to other languages through cross-lingual and redirect links, then projected onto anchor links.
  • Refining labels. Self-training with NPMI-filtered seeds fills in missing labels. For large languages, sentences are filtered by entity commonness and topic relatedness.
  • Morphology. Wikipedia bracket markup provides affix lists for morphologically rich languages.
  • Linking. Aligned cross-lingual titles yield word translations that help generate linking candidates.

A BiLSTM-CRF tagger is trained separately for each language. Results are promising on Wikipedia text and on external data such as the LORELEI sets, without any native-speaker input. All annotations, morphology resources and systems are released as a public benchmark (WikiANN), a common starting point for low-resource NER, including Ukrainian.

Key points

  • Typing schemas. Mentions get types at three granularities:
    • 4 coarse types (PER, ORG, GPE, LOC);
    • 139 AMR types;
    • 9,154 YAGO types, matched to AMR types via PMI.
  • English page classifier. It uses 60,231 DBpedia properties as features and labels about 10 million English pages as entities. On the 4,853 pages hand-annotated by Nothman et al. (2013), it reaches 96.0% F-score, versus 94.6% in that earlier work.
  • Tagger and features. The tagger is a BiLSTM-CRF (Lample et al., 2016) with these features:
    • word form, case and syllable features;
    • stems and affixes;
    • cross-lingual gazetteers;
    • character and word embeddings (character-only for languages without word segmentation).
  • Ablation results on LORELEI data.
    • Markup-derived morphology features add 11.1 F-score points for Turkish and 7.1 for Uzbek.
    • Topic-based data selection raises Turkish from 49.7% to 51.5% while also shrinking the training set.
  • Linking approach. Mined word translations let names be translated into English and then linked with an unsupervised English linker (Pan et al., 2015).

Interesting ideas

  • I1. Typing entities by their KB properties. Properties such as a birth date or a population count serve as language-independent typing features. This avoids POS taggers or other per-language tools, so it could type entities in any KB-backed project.
  • I2. Free morphology from link markup. In [[stem]]suffix patterns, the text inside the brackets is the canonical form and whatever follows is an affix. This gives cheap stemming for agglutinative languages, and similar link markup in wikis or annotated corpora could be mined the same way.
  • I3. Confidence-gated self-training. Seed sentences are kept only if every tag–token pair passes an NPMI threshold. Unlabeled text is then grouped into clusters by cross-entropy and absorbed one cluster at a time. This is a useful recipe whenever distant supervision leaves labels sparse, such as linking only the first mention of a name.
  • I4. Curating training data for the target domain. Sentences are ranked by entity commonness and topic relatedness to the target domain. This made training faster and also improved quality, which is worth remembering when large silver corpora are noisy and time is limited.
  • I5. Dictionaries from title alignment. Wikipedia titles follow consistent styles within each language, so aligning cross-lingual title pairs yields word-level translation pairs. Such a dictionary covers names that never appear as whole title pairs, which is helpful for cross-lingual search or gazetteer building.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Part-of-speech tagging

Cited by

356 papers of our library cite it (of the 140884 whose bibliographies we read); 29 of them have a page here, 327 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p17-1178

@inproceedings{pan2017cross,
  title = {{Cross-lingual Name Tagging and Linking for 282 Languages}},
  author = {Pan, Xiaoman and Zhang, Boliang and May, Jonathan and Nothman, Joel and Knight, Kevin and Ji, Heng},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-1178},
  url = {https://doi.org/10.18653/v1/p17-1178}
}

References

60 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Malin Ahlberg, Markus Forsberg, and Mans Hulden. 2015. Paradigm classification in supervised learning of morphology. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Lingui… find
  2. Fahd Alotaibi and Mark Lee. 2012. Mapping arabic wikipedia into the named entities taxonomy. In Proceedings of COLING 2012: Posters. find
  3. The COL-ING 2012 Organizing Committee, pages 43–52. http://aclweb.org/anthology/C12-2005. ↗
  4. Maha Althobaiti, Udo Kruschwitz, and Massimo Poesio. 2013. A semi-supervised learning approach to arabic named entity recognition. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013. IN-COMA Ltd. Shoumen, BULGARIA, pages 32–40. http://aclweb.org/… ↗
  5. Maha Althobaiti, Udo Kruschwitz, and Massimo Poesio. 2014. Automatic creation of arabic named entity annotated corpus using wikipedia. In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics. Association for Comp… find
  6. Joohui An, Seungwoo Lee, and Gary Geunbae Lee. 2003. Automatic acquisition of named entity tagged corpus from world wide web. In The Companion Volume to the Proceedings of 41st Annual Meeting of the Association for Computational Linguistics. http://aclweb.org/anthology/P03-2031. ↗
  7. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discou… find our card
  8. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, USA, SIGMOD ’08, pages 1247–… find
  9. Gerlof Bouma. 2009. Normalized (pointwise) mutual information in collocation extraction. In Proceedings of the Biennial GSCL Conference 2009. find
  10. Seng Yee Chan and Dan Roth. 2010. Exploiting background knowledge for relation extraction. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010). find
  11. Coling 2010 Organizing Committee, pages 152–160. http://aclweb.org/anthology/C10-1018. ↗
  12. Wanxiang Che, Mengqiu Wang, D. Christopher Manning, and Ting Liu. 2013. Named entity recognition with bilingual constraints. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computationa… find
  13. Wisam Dakka and Silviu Cucerzan. 2008. Augmenting wikipedia with named entity tags. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I. http://aclweb.org/anthology/I08-1071. ↗
  14. Luciano Del Corro, Abdalghani Abujabal, Rainer Gemulla, and Gerhard Weikum. 2015. Finet: Context-aware fine-grained named entity typing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 868–878. https://doi.or… ↗
  15. Asif Ekbal, Eva Sourjikova, Anette Frank, and Simone Paolo Ponzetto. 2010. Assessing the challenge of fine-grained named entity recognition and classification. In Proceedings of the 2010 Named Entities Workshop. Association for Computational Linguistics, pages 93–101. http://aclweb.org/anthology/W10… ↗
  16. Michael Fleischman and Eduard Hovy. 2002. Fine grained classification of named entities. In COLING 2002: The 19th International Conference on Computational Linguistics. http://aclweb.org/anthology/C02-1130. ↗
  17. Dan Gillick, Nevena Lazic, Kuzman Ganchev, Jesse Kirchner, and David Huynh. 2014. Context-dependent fine-grained entity type tagging. CoRR abs/1412.1820. http://arxiv.org/abs/1412.1820. ↗
  18. Claudio Giuliano. 2009. Fine-grained classification of named entities exploiting latent semantic kernels. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009). Association for Computational Linguistics, pages 201–209. http://aclweb.org/anthology/W09-112… ↗
  19. Stig-Arne Grönroos, Sami Virpioja, Peter Smit, and Mikko Kurimo. 2014. Morfessor flatcat: An hmm-based method for unsupervised and semi-supervised learning of morphology. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. Dublin City Uni… find
  20. Jiang Guo, Wanxiang Che, Haifeng Wang, and Ting Liu. 2014. Revisiting embedding features for simple semi-supervised learning. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, pages 110–120. https://doi.org/1… find
  21. Qi Li, Haibo Li, Heng Ji, Wen Wang, Jing Zheng, and Fei Huang. 2012. Joint bilingual name tagging for parallel corpora. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management. ACM, New York, NY, USA, CIKM ’12, pages 1727–1731. https://doi.org/10.1145/2396761.… — [doi:10.1145/2396761.2398506](https://doi.org/10.1145/2396761.2398506) ↗
  22. Alexander Hogue, Joel Nothman, and James R. Curran. 2014. Unsupervised biographical event extraction using wikipedia traffic. In Proceedings of the Australasian Language Technology Association Workshop 2014. pages 41–49. http://aclweb.org/anthology/U14-1006. ↗
  23. Xiao Ling and Daniel S. Weld. 2012. Fine-grained entity recognition. In Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence. AAAI Press, AAAI’12, pages 94–100. find
  24. Heng Ji and Ralph Grishman. 2006. Analysis and repair of name tagger errors. In Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions. Association for Computational Linguistics, pages 420–427. http://aclweb.org/anthology/P06-2055. ↗
  25. Heng Ji, Joel Nothman, and Hoa Trang Dang. 2016. Overview of tac-kbp2016 tri-lingual edl and its impact on end-to-end kbp. In Proceedings of the Text Analysis Conference. find
  26. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, Volume 29, Number 1, March 2003 http://aclweb.org/anthology/J03-1002. ↗
  27. Jun’ichi Kazama and Kentaro Torisawa. 2007. Exploiting wikipedia as external knowledge for named entity recognition. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). http://aclweb.org/anthology/… ↗
  28. Sungchul Kim, Kristina Toutanova, and Hwanjo Yu. 2012. Multilingual named entity recognition using parallel data and metadata from wikipedia. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics,… find
  29. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Asso… find
  30. Hao Li, Heng Ji, Hongbo Deng, and Jiawei Han. 2011. Exploiting background information networks to enhance bilingual event extraction through topic modeling. In Proceedings of International Conference on Advances in Information Mining and Management (IMMM2011). find
  31. Patrick Littell, Kartik Goyal, R. David Mortensen, Alexa Little, Chris Dyer, and Lori Levin. 2016. Named entity recognition for linguistic rapid response in low-resource languages: Sorani kurdish and tajik. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics… find
  32. The COL-ING 2016 Organizing Committee, pages 998–1006. http://aclweb.org/anthology/C16-1095. ↗
  33. Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. 2015. Yago3: A knowledge base from multilingual wikipedias. In Proceedings of the Conference on Innovative Data Systems Research. find our card
  34. Alireza Mahmoudi, Mohsen Arabsorkhi, and Heshaam Faili. 2013. Supervised morphology generation using parallel corpus. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013. IN-COMA Ltd. Shoumen, BULGARIA, pages 408– 414. http://aclweb.org/anthology/… ↗
  35. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David Mc-Closky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computati… find our card
  36. James Mayfield, Dawn Lawrie, Paul McNamee, and Douglas W. Oard. 2011. Building a cross-language entity linking collection in twenty-one languages. In Multilingual and Multimodal Information Access Evaluation: Second International Conference of the Cross-Language Evaluation Forum. find
  37. Paul McNamee, James Mayfield, Dawn Lawrie, Douglas Oard, and David Doermann. 2011. Cross-language entity linking. In Proceedings of 5th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, pages 255–263. http://aclweb.org/anthology/I11-1029. ↗
  38. Peter Mika, Massimiliano Ciaramita, Hugo Zaragoza, and Jordi Atserias. 2008. Learning to tag and tagging to learn: A case study on wikipedia. IEEE Intelligent Systems . find
  39. Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Assoc… find
  40. Ndapandula Nakashole, Tomasz Tylenda, and Gerhard Weikum. 2013. Fine-grained semantic typing of emerging entities. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, pages 1488–1497. http://ac… find
  41. Joel Nothman, R. James Curran, and Tara Murphy. 2008. Transforming wikipedia into named entity training data. In Proceedings of the Australasian Language Technology Association Workshop 2008. pages 124–132. http://aclweb.org/anthology/U08-1016. ↗
  42. Joel Nothman, Nicky Ringland, Will Radford, Tara Murphy, and James R. Curran. 2013. Learning multilingual named entity recognition from wikipedia. Artificial Intelligence 194:151–175. https://doi.org/10.1016/j.artint.2012.03.006. ↗
  43. Xiaoman Pan, Taylor Cassidy, Ulf Hermjakob, Heng Ji, and Kevin Knight. 2015. Unsupervised entity linking with abstract meaning representation. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Associatio… find
  44. Simone Paolo Ponzetto and Michael Strube. 2006. Exploiting semantic role labeling, wordnet and wikipedia for coreference resolution. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference. http://aclweb.org/anthology/N06-1025. ↗ our card
  45. Xiang Ren, Ahmed El-Kishky, Chi Wang, Fangbo Tao, Clare R. Voss, and Jiawei Han. 2015. Clustype: Effective entity recognition and typing by relation phrase-based clustering. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA,… — [doi:10.1145/2783258.2783362](https://doi.org/10.1145/2783258.2783362) find
  46. E. Alexander Richman and Patrick Schone. 2008. Mining wiki resources for multilingual named entity recognition. In Proceedings of ACL-08: HLT. Association for Computational Linguistics, pages 1–9. http://aclweb.org/anthology/P08-1001. ↗
  47. Nicky Ringland, Joel Nothman, Tara Murphy, and R. James Curran. 2009. Classifying articles in english and german wikipedia. In Proceedings of the Australasian Language Technology Association Workshop 2009. pages 20–28. http://aclweb.org/anthology/U09-1004. ↗
  48. Ryan Roth, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic morphological tagging, diacritization, and lemmatization using lexeme models and feature ranking. In Proceedings of ACL-08: HLT, Short Papers. Association for Computational Linguistics, pages 117–120. http://aclweb.org/… ↗ our card
  49. Teemu Ruokolainen, Oskar Kohonen, Kairit Sirts, Stig-Arne Grönroos, Mikko Kurimo, and Sami Virpioja. 2016. A comparative study of minimally supervised morphological segmentation. Computational Linguistics . find
  50. Avirup Sil and Radu Florian. 2016. One for all: Towards language independent named entity linking. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, pages 2255–2264. https://doi.org/10.18653/… find
  51. Chen-Tse Tsai, Stephen Mayhew, and Dan Roth. 2016. Cross-lingual named entity recognition via wikification. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning. Association for Computational Linguistics, pages 219–228. https://doi.org/10.18653/v1/K16-1022. ↗
  52. Mengqiu Wang, Wanxiang Che, and D. Christopher Manning. 2013. Joint word alignment and bilingual named entity recognition using dual decomposition. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Lingui… find
  53. Mengqiu Wang and D. Christopher Manning. 2014. Cross-lingual projected expectation regularization for weakly supervised learning. Transactions of the Association of Computational Linguistics 2:55–66. http://aclweb.org/anthology/Q14-1005. ↗
  54. Kenneth Ward Church and Patrick Hanks. 1990. Word association norms mutual information, and lexicography. Computational Linguistics, Volume 16, Number 1, March 1990 http://aclweb.org/anthology/J90-1003. ↗ our card
  55. Dani Yogatama, Daniel Gillick, and Nevena Lazic. 2015. Embedding methods for fine grained entity type classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short… find
  56. Amir Mohamed Yosef, Sandro Bauer, Johannes Hoffart, Marc Spaniol, and Gerhard Weikum. 2012. Hyena: Hierarchical type classification for entity names. In Proceedings of COLING 2012: Posters. find
  57. The COLING 2012 Organizing Committee, pages 1361–1370. http://aclweb.org/anthology/C12-2133. ↗
  58. Boliang Zhang, Xiaoman Pan, Tianlu Wang, Ashish Vaswani, Heng Ji, Kevin Knight, and Daniel Marcu. 2016a. Name tagging for low-resource incident languages based on expectation-driven learning. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Lin… find
  59. Dongxu Zhang, Boliang Zhang, Xiaoman Pan, Xiaocheng Feng, Heng Ji, and Weiran XU. 2016b. Bitext name tagging for cross-lingual entity annotation projection. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. find
  60. The COL-ING 2016 Organizing Committee, pages 461–470. http://aclweb.org/anthology/C16-1045. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pan-2017-cross-lingual-name-tagging-linking/