Elamur Research
☰
home / papers / 2017 / URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors

URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/E17-2002.pdf

Summary

Multilingual ("polyglot") NLP models can benefit from knowing what each language is like. However, linguistic knowledge is scattered across databases that differ in language coverage, identifiers, feature names and formats, which range from binary values to trees to prose. The authors present URIEL, a knowledge base aimed at massively multilingual and low-resource NLP, and lang2vec, a utility built on it. Given a list of ISO 639-3 codes, lang2vec returns vectors whose values lie in [0.0, 1.0].

The vector types are:

  • Typological: syntax, phonology and phonetic inventory, drawn from WALS, SSWL, Ethnologue and PHOIBLE.
  • Phylogenetic: family membership based on Glottolog.
  • Geographical: distances to fixed reference points on the globe.
  • Identity: one-hot vectors.

Features within a category share the same dimensionality, so sources can be merged or compared with simple vector operations. Gaps in typological data are filled using a k-nearest-neighbors predictor. Work by Tsvetkov et al. (2016) showed that such language vectors lower perplexity in polyglot language models compared with one-hot language identifiers. The resource therefore makes these experiments easier to run and reproduce, and it also helps expose gaps in documentation.

Key points

  • Imputation accuracy: missing typological values are predicted by weighted 10-nearest-neighbors. The method averages genetic, geographic and feature-based distances and reaches 92.93% accuracy in 10-fold cross-validation. The resulting knn vectors have 100% coverage.
  • Coverage varies by source: syntax_wals spans 1808 languages and 98 features but fills only 44% of cells. PHOIBLE-based inventory vectors have 158 features, built from natural classes derived with PanPhon. PHOIBLE itself merges seven phonological databases.
  • Perplexity gains: in Tsvetkov et al. (2016), adding phonology and inventory vectors cut perplexity by roughly 19–26% relative to one-hot id vectors. The polyglot models even beat the monolingual baselines (Italian 4.24 vs 4.36; Hindi 3.35 vs 3.70).
  • Geographic encoding: geo vectors give great-circle distances to points on a spherical Fibonacci lattice, scaled by the antipodal distance. About 2% of language codes had no location data and were filled in manually where possible.
  • Format details: features are not mutually exclusive, so a language can have both S_SVO and S_SOV set to 1. Missing values are marked "--".

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

212 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 208 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{littell2017uriel,
  title = {{URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors}},
  author = {Littell, Patrick and Mortensen, David R. and Lin, Ke and Kairis, Katherine and Turner, Carlisle and Levin, Lori},
  booktitle = {Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers},
  year = {2017},
  url = {https://aclanthology.org/E17-2002/}
}

References

30 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Manaal Faruqui and Chris Dyer. 2014. Improving vector space word representations using multilingual correlation. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 462–471, Gothenburg, Sweden, April. Association for Computational Lin… find
  2. Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah Smith. 2016. Many languages, one parser. Transactions of the Association for Computational Linguistics, 4:431–444. find our card
  3. Akash Bharadwaj, David Mortensen, Chris Dyer, and Jaime Carbonell. 2016. Phonologically aware neural model for named entity recognition in low resource transfer settings. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1462–1472, Austin, Texas, Novemb… find
  4. Álvaro González. 2009. Measurement of areas on a sphere using Fibonacci and latitude–longitude lattices. Mathematical Geosciences, 42(1):49–64. find
  5. Harald Hammarström, Robert Forkel, Martin Haspelmath, and Sebastian Bank. 2015. Glottolog 2.6. Max Planck Institute for the Science of Human History, Jena. find
  6. Christian Chanard. 2006. Systmes Alphabtiques Des Langues Africaines. UNESCO-SIL. find
  7. Rhonda L. Hartell. 1993. Alphabets des langues africaines. UNESCO and Socit Internationale de Linguistique. find
  8. Reed Coke, Ben King, and Dragomir R. Radev. 2016. Classifying syntactic regularities for hundreds of languages. Computing Research Repository, abs/1603.08016. — [arXiv:1603.08016](https://arxiv.org/abs/1603.08016) find
  9. Karl Moritz Hermann and Phil Blunsom. 2014. Multilingual models for compositional distributed semantics. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 58–68, Baltimore, Maryland, June. Association for Computational Linguisti… find
  10. Chris Collins and Richard Kayne. 2011. Syntactic Structures of the World’s Languages. New York University, New York. find
  11. John H. Crothers, James P. Lorentz, Donald A. Sherman, and Marilyn M. Vihman. 1979. Handbook of Phonological Data From a Sample of the World’s Languages: A Report of the Stanford Phonology Archive. find
  12. Benjamin Keinert, Matthias Innmann, Michael Sänger, and Marc Stamminger. 2015. Spherical Fibonacci mapping. ACM Transactions on Graphics, 34(6):193:1–193:7, October. find
  13. M. Paul Lewis, Gary F. Simons, and Charles D. Fennig. 2015. Ethnologue: Languages of the World, Eighteenth edition. SIL International, Dallas, Texas. find
  14. Joachim Daiber, Miloš Stanojević, and Khalil Sima’an. 2016. Universal reordering via linguistic typology. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3167–3176, Osaka, Japan, December. find
  15. The COLING 2016 Organizing Committee. find
  16. Ian Maddieson and Kristin Precoda. 1990. Updating UPSID. In UCLA Working Papers in Phonetics, pages 104–111. Department of Linguistics, UCLA. find
  17. Lev Michael, Tammy Stark, and Will Chang. 2012. South American Phonological Inventory Database. University of California, Berkeley. find
  18. Hal Daumé III and Lyle Campbell. 2007. A Bayesian model for discovering typological implications. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, pages 65– 72, Prague, Czech Republic, June. Association for Computational Linguistics. find
  19. Steven Moran, Daniel McCloy, and Richard Wright. 2014. PHOIBLE Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. find
  20. David R. Mortensen, Patrick Littell, Akash Bharadwaj, Kartik Goyal, Chris Dyer, and Lori Levin. 2016. PanPhon: A resource for mapping IPA segments to articulatory feature vectors. In Proceedings of COL-ING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages… find
  21. The COLING 2016 Organizing Committee. find
  22. Hal Daumé III. 2009. Non-parametric Bayesian areal linguistics. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 593–601, Boulder, Colorado, June. Association for Computational Linguistics… find
  23. Aliya Deri and Kevin Knight. 2016. Grapheme-to-phoneme models for (almost) any language. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 399–408. Association for Computational Linguistics. find
  24. Helen O’Horan, Yevgeni Berzak, Ivan Vulic, Roi Reichart, and Anna Korhonen. 2016. Survey on the use of typological information in natural language processing. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1297–1308, Osaka, Japa… find
  25. The COLING 2016 Organizing Committee. find
  26. Matthew S. Dryer and Martin Haspelmath. 2013. The World Atlas of Language Structures Online. Max Planck Institute for Evolutionary Anthropology. find
  27. Taraka Rama. 2016. Siamese convolutional networks for cognate identification. In Proceedings of COL-ING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1018–1027, Osaka, Japan, December. find
  28. The COLING 2016 Organizing Committee. find
  29. N. Ramaswami. 1999. Common Linguistic Features in Indian Languages: Phonetics. Central Institute of Indian Languages. find
  30. Yulia Tsvetkov, Sunayana Sitaram, Manaal Faruqui, Guillaume Lample, Patrick Littell, David Mortensen, Alan W Black, Lori Levin, and Chris Dyer. 2016. Polyglot neural language models: A case study in cross-lingual phonetic representation learning. pages 1357–1366, June. find