Elamur Research⌕
☰
home / papers / 2018 / The CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection

The CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection

Paper · 2018 · Ryan Cotterell, Christo Kirov, John Sylak-Glassman et al.

Summary

The paper summarizes the CoNLL–SIGMORPHON 2018 shared task on automatic word-form generation. NLP systems for morphologically rich languages require such generation. The shared task comprised two tasks. In the first, a system receives a lemma and a set of grammatical features and must produce the required word form. In the second task, designed as a cloze test, no features are given: the correct form of the lemma must be inferred from a sentence in which it has been omitted. The data for the first task were taken from Wiktionary and provided in the UniMorph Schema format. They cover 103 typologically diverse languages. The second task is based on UD v2 treebanks and covers seven languages. Both tasks had three settings depending on the amount of training data. Almost all participants used neural models and built on the best approaches from 2017. In the first task, results improved for most languages in the low-data setting. The second task proved difficult: only a few systems consistently outperformed simple baselines. The released datasets have become a useful resource for morphology research, particularly for low-resource languages.

Key points

  • The first task covered 103 languages, 52 of which were also part of the 2017 shared task. The languages include isolates (Basque, Haida), polysynthetic languages (Navajo, Greenlandic), as well as languages with templatic morphology and vowel harmony. The training sets contained approximately 100, 1,000 or 10,000 "lemma – features – form" triples.
  • The second task covered 7 languages: German, English, Spanish, Finnish, French, Russian and Swedish. It had two tracks. In the first, the lemmas and MSDs of all context words are known; in the second, only the words themselves. The amount of training data was approximately 10³, 10⁴ or 10⁵ tokens.
  • The first task received 27 systems, the second 6. In the low setting, the best systems surpassed the previous year's results for 41 of the 52 languages shared with the 2017 shared task.
  • In track 1 of the second task with a large amount of data, the best overall accuracy was achieved by the COPENHAGEN system: 68.51%. The neural baseline scored 62.41%, and the baseline that simply copies the lemma scored 36.62%.
  • Systems were evaluated in two ways: against the original form from the text and against a manually annotated set of acceptable forms. Both methods yielded nearly identical system rankings. Therefore, future shared tasks can dispense with manual annotation and include more languages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Johns Hopkins UniversityBaltimore · United States240 works · 96,487 citations🎓New York UniversityNew York · United States201 works · 100,180 citations🎓The University of MelbourneMelbourne · Australia100 works · 22,618 citations🎓University of ZurichZurich · Switzerland78 works · 12,260 citations🎓University of HelsinkiHelsinki · Finland50 works · 5,318 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Morphology · 405 works

💡 Concepts 2Key ideas and methods explained on our concept pages.Morphology · 50 worksWord form · 16 works

Cite

Cite as Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Arya D. McCarthy, Katharina Kann, Sabrina J. Mielke, Garrett Nicolai, Miikka Silfverberg, David Yarowsky, Jason Eisner and Mans Hulden. 2018. The CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection. Proceedings of the CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/k18-3001 · arXiv 1810.07125

@inproceedings{cotterell2018conll,
  title = {{The CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection}},
  author = {Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Arya D. McCarthy, Katharina Kann, Sabrina J. Mielke, Garrett Nicolai, Miikka Silfverberg, David Yarowsky, Jason Eisner and Mans Hulden},
  booktitle = {Proceedings of the CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection},
  year = {2018},
  doi = {10.18653/v1/k18-3001},
  eprint = {1810.07125},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/k18-3001}
}

References

2 references, 2 of them with a card of ours. ↗ leads to the work.

  1. Judit Ács. 2018. BME-HAS system for CoNLL– SIGMORPHON 2018 shared task: Universal morphological reinflection. In Proceedings of the CoNLL SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection, Brussels. Association for Computational Linguistics.
  2. Manex Agirrezabal. 2018. KU-CST at CoNLL–SIGMORPHON 2018 shared task: a tridirectional model. In Proceedings of the CoNLL SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection, Brussels. Association for Computational Linguistics. \#\# Conclusion Inbal Arnon and Michael Ramscar. 2012. Granu…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cotterell-2018-conll-sigmorphon-2018-shared-task/