Elamur Research⌕
☰
home / papers / 2018 / UniMorph 2.0: Universal Morphology

UniMorph 2.0: Universal Morphology

Paper · 2018 · Christo Kirov, Ryan Cotterell, John Sylak-Glassman et al.

Summary

Most of the world's languages have rich inflection, yet NLP systems often handle it poorly. The UniMorph project collects morphological data with uniform annotation: each word form is linked to a lemma and a set of features from the universal UniMorph schema. Version 1.0 automatically extracted paradigms from Wiktionary tables using heuristics and therefore contained many systematic errors. In version 2.0, the authors stopped correcting the automatic parse and changed their approach. The HTML tables for each language are grouped by structure using pandas. A human annotates only one sample from each group, and the annotation is automatically propagated to all tables with the same layout. In this way, three annotators prepared data for 47 languages within a few days, and native speakers or experts then verified the result. Together with sources outside Wiktionary, the resource covers 52 languages, including low-resource ones such as Quechua, Navajo and Haida. Lists of lemmas without tables and generated English glosses for word forms have also been added. The data are open and served as the basis for the CoNLL–SIGMORPHON 2017 Shared Task, which makes the resource important for multilingual morphology.

Key points

  • The UniMorph schema: more than 200 basic features across 23 dimensions, including rare categories such as evidentiality and switch-reference. Distinctions not covered by the schema, as well as free variants of forms, are marked with language-specific LGSPEC labels.
  • Comparison with 1.0: version 1.0 contained 952,530 lemmas from 350 languages, but with a large number of errors. Previously, only 8 languages had been corrected manually for SIGMORPHON 2016. In 2.0, the Wiktionary data for 47 languages replaced the corresponding data from 1.0.
  • Sources outside Wiktionary: Khaling, Kurmanji and Sorani were taken from the Alexina project. The Haida data were prepared by Jordan Lachler. Basque was extracted from a finite-state analyzer. All these data were converted to the UniMorph schema.
  • English glosses: they are generated from templates with Penn Treebank tags and from bilingual dictionaries, e.g. «comprábamos» → «(we) were buying». Glosses are available for 10 languages, including Ukrainian: 20,288 word forms and 41,590 glosses. Hungarian has the largest volume: 2,814,006 word forms.
  • Community: the data are released incrementally under an open license via unimorph.github.io. Each language has a public issue tracker. The project is organized on the model of UD.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States430 works · 424,363 citations🎓Johns Hopkins UniversityBaltimore · United States240 works · 96,487 citations🎓The University of MelbourneMelbourne · Australia100 works · 22,618 citations🎓University of Colorado BoulderBoulder · United States69 works · 51,069 citations🎓University of ZurichZurich · Switzerland78 works · 12,260 citations🎓Georgetown UniversityWashington · United States88 works · 4,732 citations🎓Indiana University BloomingtonBloomington · United States37 works · 3,685 citations🎓University of Colorado SystemBoulder · United States9 works · 1,658 citations

Also on the byline: Laboratoire Dynamique du Langage.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Morphology · 405 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Morphology · 50 works

Cite

Cite as Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sabrina J. Mielke, Arya McCarthy, Sandra Kübler, David Yarowsky, Jason Eisner and Mans Hulden. 2018. UniMorph 2.0: Universal Morphology. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).

The paper itself, not this page. BibTeX from the library's record of the work. arXiv 1810.11101

@inproceedings{kirov2018unimorph,
  title = {{UniMorph 2.0: Universal Morphology}},
  author = {Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sabrina J. Mielke, Arya McCarthy, Sandra Kübler, David Yarowsky, Jason Eisner and Mans Hulden},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/3rq3n9kr4gk8},
  eprint = {1810.11101},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.63317/3rq3n9kr4gk8}
}

References

18 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahlberg, M., Forsberg, M., and Hulden, M. (2014). Semi-supervised learning of morphological paradigms and lexicons. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 569–578, Gothenburg, Sweden, April. Association for Computational…
  2. Alegria, I., Etxeberria, I., Hulden, M., and Maritxalar, M. (2009). Porting Basque morphological grammars to foma, an open-source tool. In International Workshop on Finite-State Methods and Natural Language Processing, pages 105–113. Springer. Choi, J., de Marneffe, M.-C., Dozat, T., Ginter, F., Gol… find
  3. January. Cotterell, R., Kirov, C., Sylak-Glassman, J., Yarowsky, D., Eisner, J., and Hulden, M. (2016a). find
  4. The SIGMOR-PHON 2016 shared task—morphological reinflection. In Proceedings of the 14th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 10–22, Berlin, Germany, August. Association for Computational Linguistics. Cotterell, R., Schütze, H., and Eisner, J.… find
  5. The CoNLL-SIGMORPHON 2017 shared task. In CoNLL-SIGMORPHON 2017 Shared Task. Creutz, M., Hirsimäki, T., Kurimo, M., Puurula, A., Pylkkönen, J., Siivola, V., Varjokallio, M., Arisoy, E., Saraclar, M., and Stolcke, A. (2007). Analysis of morph-based speech recognition and the modeling of out-of-voca… find
  6. Durrett, G. and DeNero, J. (2013). Supervised learning of complete morphological paradigms. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1185–1195, Atlanta, Georgia, June. Association for Comp…
  7. Dyer, C., Muresan, S., and Resnik, P. (2008). Generalizing word lattice translation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, pages 1012–1020, Columbus, Ohio, June. Association for Computational Linguistics.
  8. Faruqui, M., Tsvetkov, Y., Neubig, G., and Dyer, C. (2016). Morphological inflection generation using character sequence to sequence learning. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 634–… find
  9. Haspelmath, M., Dryer, M., Gil, D., and Comrie, B. (2005). The world atlas of language structures (WALS). find
  10. Kirov, C., Sylak-Glassman, J., Que, R., and Yarowsky, D. (2016). Very-large scale parsing and normalization of Wiktionary morphological paradigms. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 3121–3126. European Language Resources Assoc…
  11. Narasimhan, K., Karakos, D., Schwartz, R., Tsakalidis, S., and Barzilay, R. (2014). Morphological segmentation for keyword spotting. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 880–885, Doha, Qatar, October. Association for Computational L… find
  12. Nicolai, G., Cherry, C., and Kondrak, G. (2015). Inflection generation as discriminative string transduction. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 922–931, Denver, Colorado, May–June.… find
  13. Sylak-Glassman, J., Kirov, C., Post, M., Que, R., and Yarowsky, D. (2015a). A universal feature schema for rich morphological annotation and fine-grained cross-lingual part-of-speech tagging. In Cerstin Mahlow et al., editors, Proceedings of the 4th Workshop on Systems and Frameworks for Computation… find
  14. Springer, Berlin, September. Sylak-Glassman, J., Kirov, C., Yarowsky, D., and Que, R. (2015b). A language-independent feature schema for inflectional morphology. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on N…
  15. Walther, G. and Sagot, B. (2010). Developing a large-scale lexicon for a less-resourced language: General methodology and preliminary experiments on Sorani Kurdish. find
  16. In Proceedings of the SaLTMiL Workshop on Creation and Use of Basic Lexical Resources for Less-Resourced Languages (at LREC), Valetta, Malta. European Language Resources Association (ELRA). Walther, G., Sagot, B., and Fort, K. (2010). Fast development of basic NLP tools: Towards a lexicon and a POS… find
  17. Walther, G., Jacques, G., and Sagot, B. (2013). Uncovering the inner architecture of Khaling verbal morphology, September. Presentation at the 3rd Workshop on Sino-Tibetan Languages of Sichuan, Paris, September 2013. find
  18. Zeman, D. (2008). Reusable tagset conversion using tagset drivers. In Proceedings of LREC 2008, pages 213–218.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kirov-2018-unimorph-2-0-universal-morphology/