Elamur Research⌕
☰
home / papers / 2017 / Cheap Translation for Cross-Lingual Named Entity Recognition

Cheap Translation for Cross-Lingual Named Entity Recognition

Paper · 2017 · Stephen Mayhew, Chen-Tse Tsai and Dan Roth

Summary

Most cross-lingual NER methods rely on resources that the majority of languages lack, such as large parallel corpora or a rich Wikipedia. This paper proposes "cheap translation": NER-annotated text in a high-resource language is converted into the target language word by word and phrase by phrase, using only a bilingual lexicon. An ordinary monolingual NER model is then trained on the resulting text. When a lexicon entry has several possible translations, the choice is made greedily. Each candidate is scored by a target-language language model (SRILM trained on Wikipedia) times a "prominence" score, which is derived from how often the word pair co-occurs inside the lexicon. Gold BIO labels move from source phrases to target phrases directly, so no alignment step is needed. Combined with Wikipedia-based wikifier features, the method sets a new state of the art on 7 languages, gaining 5.5 F1 on average. A Uyghur case study shows it still works with almost no target-language resources. This makes the approach highly portable to languages with little more than a dictionary.

Key points

  • On German, Spanish and Dutch, the average F1 is 62.65 for cheap translation, 56.74 for the previous state of the art (Tsai et al., 2016) and 37.27 for a direct transfer baseline. These are gains of about 5.9 and 25.4 points.
  • The lexicons come from Rolston and Kirchhoff (2016) and are built from PanLex, Wiktionary and other sources. There are 103 English–target lexicons, ranging from 56K to 1.36M entries.
  • The decoder does no reordering beyond what multi-word lexicon entries already contain. The output text is often ungrammatical, but the context around entities survives reasonably well.
  • For Uyghur, the 5K-entry lexicon was enlarged to 116K entries with the LORELEI dictionary. Other additions were a 45-suffix stemmer and replacement of untranslatable names with names from Uyghur Wikipedia gazetteers.
  • Plain translation from English, Turkish or Uzbek gave 27–33 F1 on Uyghur, while combining all language-specific methods reached 51.32. For comparison, a monolingual model scored 69.92 on a 41-document test set. The official LoReHLT16 submission scored 55.6, versus 51.2 for Bharadwaj et al. (2016).

Interesting ideas

  • I1. Lexicon-internal prominence. You can rank translation candidates by how often a word pair appears together across a dictionary's own entries. This yields translation probabilities without any parallel corpus and could help any dictionary-driven pipeline, for example bootstrapping lexicons for Ukrainian dialects or minority languages.
  • I2. Label transfer by construction. When translation happens phrase by phrase, each source span's labels can be copied directly onto its target span. This avoids the noise of word alignment and could apply to other span-labeling tasks such as POS tagging, slot filling or chunking.
  • I3. Fluency is optional for NER. The training text does not have to be good translation, as long as entity names and the cue words around them survive. This suggests that cheap, rough synthetic data can be sufficient for sequence labeling.
  • I4. Name substitution from gazetteers. Untranslatable entities can be replaced with random same-type names from target-language lists. The sentences become semantically odd, but the entity strings look native, which is a simple augmentation trick for any low-resource NER setup.
  • I5. Exploit related languages and combine data sources. Uyghur–Uzbek word pairs were mined using edit distance with zero-cost substitutions and CCA-aligned word vectors. Each source helped little alone but added nearly 10 points when combined, so pooling diverse weak training sets is worth trying.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Illinois Urbana-ChampaignUrbana · United States108 works · 67,432 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputer Vision and Pattern Recognition · 1,282 works

🧭 Topics 2Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 467 worksNamed entity recognition · 101 works

Cite

Cite as Stephen Mayhew, Chen-Tse Tsai and Dan Roth. 2017. Cheap Translation for Cross-Lingual Named Entity Recognition. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1269

@inproceedings{mayhew2017cheap,
  title = {{Cheap Translation for Cross-Lingual Named Entity Recognition}},
  author = {Stephen Mayhew, Chen-Tse Tsai and Dan Roth},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1269},
  url = {https://doi.org/10.18653/v1/d17-1269}
}

References

30 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akash Bharadwaj, David R. Mortensen, Chris Dyer, and Jaime G. Carbonell. 2016. Phonologically aware neural model for named entity recognition in low resource transfer settings. In EMNLP. find
  2. Xavier Carreras, Lluı́s Màrquez, and Lluı́s Padró. 2003. Named entity recognition for catalan using spanish resources. In Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics-Volume 1, pages 43–50. Association for Computational Linguistics. find
  3. Dipanjan Das and Slav Petrov. 2011. Unsupervised part-of-speech tagging with bilingual graph-based projections. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Portland, OR. Association for Computational Linguistics.
  4. Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. find
  5. Long Duong, Trevor Cohn, Karin Verspoor, Steven Bird, and Paul Cook. 2014. What can we get from 1000 tokens? A case study of multilingual POS tagging for resource-poor languages. In EMNLP, pages 886–897. Citeseer. find
  6. Chris Dyer, Victor Chahuneau, and Noah A Smith. 2013. A simple, fast, and effective reparameterization of ibm model 2. Association for Computational Linguistics. find
  7. Maud Ehrmann, Marco Turchi, and Ralf Steinberger. 2011. Building a multilingual named entity-annotated corpus using annotation projection. In RANLP. find
  8. M. Faruqui and C. Dyer. 2014. Improving vector space word representations using multilingual correlation. Association for Computational Linguistics.
  9. Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara I. Cabezas, and Okan Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel texts. Natural Language Engineering, 11:311–325. find
  10. Ann Irvine and Chris Callison-Burch. 2016. End-to-end statistical machine translation with zero or small parallel texts. Natural Language Engineering, 22:517–548. find
  11. David Kamholz, Jonathan Pool, and Susan M Colowick. 2014. Panlex: Building a resource for panlingual lexical translation. In LREC, pages 3145–3150. find
  12. Sungchul Kim, Kristina Toutanova, and Hwanjo Yu. 2012. Multilingual named entity recognition using parallel data and metadata from Wikipedia. In ACL. find
  13. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting of the ACL on interacti… find
  14. Philipp Koehn and Kevin Knight. 2001. Knowledge sources for word-level translation models. find
  15. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proceedings of CoNLL-2003, pages 142–147. Edmonton, Canada.
  16. Chen-Tse Tsai, Stephen Mayhew, and Dan Roth. 2016. Cross-lingual named entity recognition via wikification. find
  17. Stephen Mausam, Soderland, Oren Etzioni, Daniel S Weld, Kobi Reiter, Michael Skinner, Marcus Sammer, Jeff Bilmes, et al. 2010. Panlingual lexical translation via probabilistic inference. Artificial Intelligence, 174(9-10):619–637. find
  18. Chen-Tse Tsai and Dan Roth. 2016. Cross-lingual wikification using multilingual embeddings. In NAACL.
  19. Ryan McDonald, Slav Petrov, and Keith Hall. 2011. Multi-source transfer of delexicalized dependency parsers. In Proceedings of the Conference on Empirical Methods for Natural Language Processing (EMNLP), pages 62–72, Edinburgh, Scotland, UK. Association for Computational Linguistics.
  20. David Yarowsky, Grace Ngai, and Richard Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the first international conference on Human language technology research, pages 1– 8. Association for Computational Linguistics.
  21. Jian Ni and Radu Florian. 2016. Improving multilingual named entity recognition with wikipedia entity type mapping. find
  22. Imed Zitouni and Radu Florian. 2008. Mention detection crossing the language barrier. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 600–609. Association for Computational Linguistics. find
  23. L. Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proc. of the Annual Conference on Computational Natural Language Learning (CoNLL).
  24. Mengqiu Wang and Christopher D Manning. 2014. Cross-lingual projected expectation regularization for weakly supervised learning. In TACL. find
  25. Leanne Rolston and Katrin Kirchhoff. 2016. Collection of bilingual data for lexicon transfer learning. Technical report, University of Washington. find
  26. Heather Simpson, Christopher Cieri, Kazuaki Maeda, Kathryn Baker, and Boyan Onyshkevych. 2008. Human language technology resources for less commonly taught languages: Lessons learned toward creation of basic language resources. Collaboration: interoperability between people in the creation of langua… find
  27. Andreas Stolcke et al. 2002. Srilm-an extensible language modeling toolkit. In Interspeech, volume 2002, page 2002. find
  28. Oscar Täckström. 2012. Nudging the envelope of direct transfer methods for multilingual named entity recognition. In Proceedings of the NAACL-HLT Workshop on the Induction of Linguistic Structure, pages 55–63. Association for Computational Linguistics. find
  29. Oscar Täckström, Ryan T. McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In NAACL.
  30. Erik F. Tjong Kim Sang. 2002. Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition. In Proceedings of CoNLL-2002, pages 155–158. Taipei, Taiwan.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mayhew-2017-cheap-translation-cross-lingual-named/