Elamur Research⌕
☰
home / papers / 2017 / CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages

CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages

Paper · 2017 · Ryan and Cotterell

Summary

The paper summarizes the CoNLL-SIGMORPHON 2017 shared task on automatic generation of word forms. Participants built systems for 52 typologically diverse languages in two settings. First: produce the required form from a lemma and a set of morphological features. Second: fill in all missing cells of a paradigm given only a few known forms. For each language, three sizes of training data were prepared, along with unannotated text from Wikipedia dumps. Most paradigms were taken from Wiktionary; the rest came from the Alexina project, from experts (Haida), and from a finite-state analyzer (Basque). 24 systems were submitted to the first subtask and 3 to the second. All but one had a neural encoder-decoder component. With plenty of data, accuracy exceeded 90% on many languages. More importantly, neural networks also coped with small samples when they had an appropriate inductive bias or artificially augmented data. An oracle ensemble showed that different systems make different errors, so there is room for improvement. The released data became a standard benchmark for learning inflection in low-resource settings.

Key points

  • Three data regimes for subtask 1: low / medium / high, i.e., roughly 100, 1,000 and 10,000 training examples. Features are given as structured tags (e.g., V;IND;FUT;2;SG).
  • The languages span 10 language groups, including the isolates Haida and Basque, as well as Navajo, Quechua, Arabic, Georgian and Uralic languages. Morphological phenomena include prefixation, infixes, templatic morphology, and vowel and consonant harmony.
  • In the low regime, a plain seq2seq model without augmentation reached an accuracy of about 0–1%. The best systems with augmentation and additional bias achieved over 50% on 100 examples and about 80% on 1,000. A baseline based on simple inflection rules achieved 38%.
  • The Ensemble Oracle outperformed the best single system by about 10% in the medium regime and by 15% in the low regime. The oracle-fc oracle showed that in the low regime 29% of test feature sets did not occur in training. For Basque it achieved only 47%, whereas six systems that processed features separately reached 85% or more.
  • The training, development and test data are publicly released: github.com/sigmorphon/conll2017.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Johns Hopkins UniversityBaltimore · United States240 works · 96,487 citations🎓The University of MelbourneMelbourne · Australia100 works · 22,618 citations🎓University of ZurichZurich · Switzerland78 works · 12,260 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Morphology · 405 works

💡 Concepts 2Key ideas and methods explained on our concept pages.Morphology · 50 worksWord form · 16 works

Cite

Cite as Ryan and Cotterell. 2017. CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages. CoNLL-SIGMORPHON 2017 Shared Task.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/k17-2001 · arXiv 1706.09031

@inproceedings{ryan2017conll,
  title = {{CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages}},
  author = {Ryan and Cotterell},
  booktitle = {CoNLL-SIGMORPHON 2017 Shared Task},
  year = {2017},
  doi = {10.18653/v1/k17-2001},
  eprint = {1706.09031},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/k17-2001}
}

References

71 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Farrell Ackerman, James P. Blevins, and Robert Malouf. 2009. Parts and wholes: Patterns of relatedness in complex mophological systems and why they matter. In James P. Blevins and Juliette Blevins, editors, Analogy in grammar: Form and acquisition, pages 54–82. Oxford University Press, Oxford. find
  2. Farrell Ackerman and Robert Malouf. 2013. Morphological organization: The low conditional entropy conjecture. Language, 89:429–464. find
  3. Roee Aharoni and Yoav Goldberg. 2017. Morphological inflection generation with hard monotonic attention. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada. Association for Computational Linguistics.
  4. Roee Aharoni, Yoav Goldberg, and Yonatan Belinkov. 2016. Improving sequence to sequence learning for morphological inflection generation: The BIU-MIT systems for the SIGMORPHON 2016 shared Abhisek Chakrabarty and Utpal Garain. 2017. ISI at the SIGMORPHON 2017 shared task on morphological reinflectio… find
  5. Malin Ahlberg, Markus Forsberg, and Mans Hulden. 2014. Semi-supervised learning of morphological paradigms and lexicons. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 569– 578, Gothenburg, Sweden. Association for Computational L…
  6. Abhisek Chakrabarty, Arun Onkar Pandit, and Utpal Garain. 2017. Context sensitive lemmatization using two successive bidirectional gated recurrent networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada. Associat… find
  7. Malin Ahlberg, Markus Forsberg, and Mans Hulden. 2015. Paradigm classification in supervised learning of morphology. In Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1024–1029, Denver, CO. Association for Computational Linguistics. find
  8. Grzegorz Chrupała, Georgiana Dinu, and Josef van Genabith. 2008. Learning morphology with Morfette. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco.
  9. Junyoung Chung, Çaglar Gülçehre, Kyunghyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555. find
  10. Iñaki Alegria, Izaskun Etxeberria, Mans Hulden, and Montserrat Maritxalar. 2009. Porting Basque morphological grammars to foma, an open-source tool. In International Workshop on Finite-State Methods and Natural Language Processing, pages 105–113. Springer. find
  11. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, and Mans Hulden. 2016a. find
  12. The SIGMORPHON 2016 shared task— morphological reinflection. In Proceedings of the 2016 Meeting of SIGMORPHON, Berlin, Germany. Association for Computational Linguistics. find
  13. Iñaki Alegria and Izaskun Etxeberria. 2016. EHU at the SIGMORPHON 2016 shared task. A simple proposal: Grapheme-to-phoneme for inflection. In Proceedings of the 2016 Meeting of SIGMOR-PHON, Berlin, Germany. Association for Computational Linguistics. find
  14. Ryan Cotterell, Nanyun Peng, and Jason Eisner. 2014. Stochastic contextual edit distance and probabilistic FSTs. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 625–630, Baltimore, Maryland. Association for Computational Ling… find
  15. Inbal Arnon and Michael Ramscar. 2012. Granularity and the acquisition of grammatical gender: How order-of-acquisition affects what gets learned. Cognition, 122:292–305. find
  16. R. Harald Baayen, Peter Hendrix, and Michael Ramscar. 2013. Sidestepping the combinatorial explosion: Towards a processing model based on discriminative learning. Language and Speech, 56(3):329– 34. find
  17. Ryan Cotterell, Nanyun Peng, and Jason Eisner. 2015. Modeling word forms using latent underlying morphs and phonology. Transactions of the Association for Computational Linguistics, 3:433–447.
  18. Ryan Cotterell, Hinrich Schütze, and Jason Eisner. 2016b. Morphological smoothing and extrapolation of word embeddings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1651– 1660, Berlin, Germany. Association for Computationa… find
  19. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Internationoal Conference on Learning Representations, volume abs/1409.0473.
  20. Toms Bergmanis, Katharina Kann, Hinrich Schütze, and Sharon Goldwater. 2017. Training data augmentation for low-resource morphological inflection. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 31–39, Vancouver. Association for Computational Lin… find
  21. James Peter Blevins. 2013. The information-theoretic turn. Psihologija, 46(3):355–375. find
  22. Ryan Cotterell, John Sylak-Glassman, and Christo Kirov. 2017. Neural graphical models over strings for principal parts morphological paradigm completion. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 759… find
  23. James Peter Blevins. 2016. Word and Paradigm Morphology. Oxford University Press, Oxford. find
  24. Mathias Creutz, Teemu Hirsimäki, Mikko Kurimo, Antti Puurula, Janne Pylkkönen, Vesa Siivola, Matti Katharina Kann, Ryan Cotterell, and Hinrich Schütze. 2017a. Neural multi-source morphological reinflection. In Proceedings of the 15th Conference of the European Chapter of the Association for Compu… find
  25. Varjokallio, Ebru Arisoy, Murat Saraclar, and Andreas Stolcke. 2007. Analysis of morph-based speech recognition and the modeling of out-of-vocabulary words across languages. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguis… find
  26. Katharina Kann, Ryan Cotterell, and Hinrich Schütze. 2017b. One-shot neural cross-lingual transfer for paradigm completion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada. Association for Computational Linguistic… find
  27. Markus Dreyer and Jason Eisner. 2009. Graphical models over multiple strings. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 101–110, Singapore. Association for Computational Linguistics. find
  28. Markus Dreyer, Jason Smith, and Jason Eisner. 2008. Latent-variable modeling of string transductions with finite-state methods. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 1080–1089, Honolulu, Hawaii. Association for Computational Linguistics. find
  29. Katharina Kann and Hinrich Schütze. 2016. Single-model encoder-decoder with explicit morphological representation for reinflection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 555–560, Berlin, Germany. Association for Co… find
  30. Greg Durrett and John DeNero. 2013. Supervised learning of complete morphological paradigms. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1185–1195, Atlanta, Georgia. Association for Computati…
  31. Katharina Kann and Hinrich Schütze. 2017. The LMU system for the CoNLL-SIGMORPHON 2017 shared task on universal morphological reinflection. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 40–48, Vancouver. Association for Computational Linguistic… find
  32. Christopher Dyer, Smaranda Muresan, and Philip Resnik. 2008. Generalizing word lattice translation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, pages 1012–1020, Columbus, Ohio. Association for Computational Linguistics.
  33. Christo Kirov, John Sylak-Glassman, Roger Que, and David Yarowsky. 2016. Very-large scale parsing and normalization of Wiktionary morphological paradigms. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 3121–3126. European Language Resourc…
  34. Manaal Faruqui, Yulia Tsvetkov, Graham Neubig, and Chris Dyer. 2016. Morphological inflection generation using character sequence to sequence learning. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p… find
  35. Mikko Kurimo, Sami Virpioja, Ville Turunen, and Krista Lagus. 2010. Morpho challenge competition 2005–2010: Evaluations and results. In Proceedings of the 11th Meeting of the ACL Special Interest Group on Computational Morphology and Phonology, pages 87–95. Association for Computational Linguistics. find
  36. Raphael Finkel and Gregory Stump. 2007. Principal parts and morphological typology. Morphology, 17(1):39–75. find
  37. Ling Liu and Lingshuang Jack Mao. 2016. Morphological reinflection with conditional random fields and unsupervised features. In Proceedings of the 2016 Meeting of SIGMORPHON, Berlin, Germany. Association for Computational Linguistics. find
  38. Martin Haspelmath, Matthew Dryer, David Gil, and Bernard Comrie. 2005. The world atlas of language structures (WALS). find
  39. Peter Makarov, Tatiana Ruzsics, and Simon Clematide. 2017. Align and copy: UZH at SIGMORPHON 2017 shared task for morphological reinflection. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 49–57, Vancouver. Association for Computational Linguisti… find
  40. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. find
  41. Sittichai Jiampojamarn, Colin Cherry, and Grzegorz Kondrak. 2008. Joint processing and discriminative training for letter-to-phoneme conversion. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, pages 905– 913, Columbus, Ohio. Association for Computational L… find
  42. Robert Malouf. 2016. Generating morphological paradigms with a recurrent neural network. San Diego Linguistic Papers, pages 122–129. find
  43. Technologies, pages 623–633, San Diego, California. Association for Computational Linguistics. Petar Milin, Victor Kuperman, Aleksandar Kostić, and R. Harald Baayen. 2009. Words and paradigms bit by bit: An information-theoretic approach to the processing of inflection and derivation. pages 214– 25… find
  44. Benoı̂t Sagot and Géraldine Walther. 2011. Non-canonical inflection : data, formalisation and complexity measures. In Systems and Frameworks in Computational Morphology, volume 100, pages 23– 45, Zurich, Switzerland. Springer. find
  45. Fabio Montermini and Olivier Bonami. 2013. Stem spaces and predictibility in verbal inflection. Lingue e Linguaggio, 12:171–190. find
  46. Geoffrey Sampson. 1985. Writing Systems: A Linguistic Introduction. Stanford University Press. find
  47. Thomas Müller, Ryan Cotterell, Alexander Fraser, and Hinrich Schütze. 2015. Joint lemmatization and morphological tagging with Lemming. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2268– 2274, Lisbon, Portugal. Association for Computational Lingu…
  48. Sunita Sarawagi and William W. Cohen. 2004. Semi-Markov conditional random fields for information extraction. In Advances in Neural Information Processing Systems, pages 1185–1192. find
  49. Wolfgang Seeker and Özlem Çetinoǧlu. 2015. A graph-based lattice dependency parser for joint morphological segmentation and syntactic analysis. Transactions of the Association for Computational Linguistics, 3:359–373.
  50. Karthik Narasimhan, Damianos Karakos, Richard Schwartz, Stavros Tsakalidis, and Regina Barzilay. 2014. Morphological segmentation for keyword spotting. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 880–885, Doha, Qatar. Association for Compu… find
  51. Hajime Senuma and Akiko Aizawa. 2017. Seq2seq for morphological reinflection: When deep learning fails. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 100–109, Vancouver. Association for Computational Linguistics. find
  52. Garrett Nicolai, Colin Cherry, and Grzegorz Kondrak. 2015. Inflection generation as discriminative string transduction. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 922–931, Denver, Colorado.… find
  53. Scott Seyfarth, Farrell Ackerman, and Robert Malouf. 2014. Implicative organization facilitates morphological learning. In Proceedings of the Fortieth Annual Meeting of the Berkeley Linguistics Society, pages 480–494. find
  54. Garrett Nicolai, Bradley Hauer, Mohammad Motallebi, Saeed Najafi, and Grzegorz Kondrak. 2017. If you can’t beat them, join them: The university of Alberta system description. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 79–84, Vancouver. Associ… find
  55. Miikka Silfverberg, Adam Wiemerslage, Ling Liu, and Lingshuang Jack Mao. 2017. Data augmentation for morphological reinflection. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 90–99, Vancouver. Association for Computational Linguistics.
  56. Robert Östling and Johannes Bjerva. 2017. SU-RUG at the CoNLL-SIGMORPHON 2017 shared task: Morphological inflection with attentional sequence-to-sequence models. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 110–113, Vancouver. Association for… find
  57. Akhilesh Sudhakar and Anil Kumar Singh. 2017. Experiments on morphological reinflection: CoNLL-2017 shared task. In Proceedings of the CoNLL SIG-MORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 71–78, Vancouver. Association for Computational Linguistics. find
  58. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112, Montreal, Quebec, Canada.
  59. Vito Pirrelli, Marcello Ferro, and Claudia Marzi. 2015. Computational complexity of abstractive morphology. pages 141–166. Oxford University Press, Oxford. find
  60. Fermin Moscoso del Prado Martı́n, Aleksandar Kostić, and R. Harald Baayen. 2004. Putting the bits together: An information-theoretical perspective on morphological processing. Cognition, 94:1–18. find
  61. John Sylak-Glassman, Christo Kirov, Matt Post, Roger Que, and David Yarowsky. 2015a. A universal feature schema for rich morphological annotation and fine-grained cross-lingual part-of-speech tagging. In Cerstin Mahlow and Michael Piotrowski, editors, Proceedings of the 4th Workshop on Systems and F… find
  62. Pushpendre Rastogi, Ryan Cotterell, and Jason Eisner. 2016. Weighting finite-state transductions with neural context. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Proceedings of the CoNLL SIGMORPHON 2017 Shared T… find
  63. John Sylak-Glassman, Christo Kirov, and David Yarowsky. 2016. Remote elicitation of inflectional paradigms to seed morphological analysis in low-resource languages. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Resources Asso… find
  64. John Sylak-Glassman, Christo Kirov, David Yarowsky, and Roger Que. 2015b. A language-independent feature schema for inflectional morphology. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proce…
  65. Reut Tsarfaty, Djamé Seddah, Yoav Goldberg, Marie Candito, Jennifer Foster, Yannick Versley, Ines Rehbein, and Lamia Tounsi. 2010. Statistical parsing of morphologically rich languages (SPMRL) what, how and whither. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morph…
  66. Géraldine Walther, Guillaume Jacques, and Benoı̂t Sagot. 2013. Uncovering the inner architecture of Khaling verbal morphology. Presentation at the 3rd Workshop on Sino-Tibetan Languages of Sichuan, Paris, September 2013. find
  67. Géraldine Walther and Benoı̂t Sagot. 2010. Developing a large-scale lexicon for a less-resourced language: General methodology and preliminary experiments on Sorani Kurdish. In Proceedings of the SaLTMiL Workshop on Creation and Use of Basic Lexical Resources for Less-Resourced Languages (at LREC),… find
  68. Géraldine Walther, Benoı̂t Sagot, and Karën Fort. 2010. Fast development of basic NLP tools: Towards a lexicon and a POS tagger for Kurmanji Kurdish. In Proceedings of the 29th International Conference on Lexis and Grammar, Belgrade. find
  69. Chunting Zhou and Graham Neubig. 2017a. Morphological inflection generation with multi-space variational encoder-decoders. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 58–65, Vancouver. Association for Computational Linguistics. find
  70. Chunting Zhou and Graham Neubig. 2017b. Multi-space variational encoder-decoders for semi-supervised labeled sequence transduction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada. Association for Computational Lin… find
  71. Qile Zhu, Yanjun Li, and Xiaolin Li. 2017. Character sequence-to-sequence model with global attention for universal morphological reinflection. In find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cotterell-2017-sigmorphon-reinflection-52/