Elamur Research⌕
☰
home / papers / 2019 / On the Complexity and Typology of Inflectional Morphological Systems

On the Complexity and Typology of Inflectional Morphological Systems

Paper · 2019 · Ryan Cotterell, Christo Kirov, Mans Hulden et al.

Summary

This paper asks how complex a language's inflectional morphology is and whether two kinds of complexity limit each other. The first is enumerative complexity (e-complexity): how many distinct surface forms a paradigm contains. The second is integrative complexity (i-complexity): how irregular, or hard to predict, those forms are. The authors define i-complexity through information theory. It is the conditional entropy of a whole paradigm given its lemma. To estimate it, they fit a probabilistic model: a directed graphical model arranged as a tree, with neural networks serving as its factors and a topology that is learned. A variational bound is used to approximate the measure. The approach needs no segmentation of words into morphemes and is largely theory-neutral. Applied to written noun and verb paradigms from 36 typologically diverse languages, the measurements show a Pareto-style trade-off. A system may have large paradigms or highly irregular ones, but never both. This refines Ackerman and Malouf's low-entropy conjecture. It also suggests that NLP should not equate morphological richness with high OOV rates, since predictable morphology, as in Hungarian, may be easier to process than irregular morphology, as in German.

Key points

  • E-complexity is measured as the average number of distinct surface forms per paradigm, not the number of slots. Syncretic slots therefore do not inflate the score. Most English verbs score 4–5, while be has 8 forms.
  • Paradigms follow the UniMorph annotation scheme and are represented orthographically. The results describe written languages, and spoken forms might give different values.
  • Significance comes from a permutation test over 10,000 shuffled scatterplots. It checks whether the high e-complexity, high i-complexity corner is emptier than chance would produce.
  • In one training scheme the results are p < 0.021 for verbs and p < 0.037 for nouns. In another training scheme they are p < 0.032 for verbs and p < 0.024 for nouns.
  • Proposed extensions include:
    • modelling frequency effects;
    • joint phonological and orthographic modelling;
    • simulating diachronic transmission;
    • quantifying derivational complexity.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Johns Hopkins UniversityBaltimore · United States240 works · 96,487 citations🎓University of Colorado BoulderBoulder · United States69 works · 51,069 citations🎓University of Colorado SystemBoulder · United States9 works · 1,658 citations🎓University of Colorado DenverDenver · United States3 works · 1,046 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Morphology · 405 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Word form · 16 works

Cite

Cite as Ryan Cotterell, Christo Kirov, Mans Hulden and Jason Eisner. 2019. On the Complexity and Typology of Inflectional Morphological Systems. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00271

@article{cotterell2019complexity,
  title = {{On the Complexity and Typology of Inflectional Morphological Systems}},
  author = {Ryan Cotterell, Christo Kirov, Mans Hulden and Jason Eisner},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.1162/tacl\_a\_00271},
  url = {https://doi.org/10.1162/tacl_a_00271}
}

References

59 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Farrell Ackerman and Robert Malouf. 2013. Morphological organization: The low conditional entropy conjecture. Language, 89(3):429–464. \#\# Conclusions We have provided clean mathematical formulations of enumerative and integrative complexity of Blake Allen and Michael Becker. 2015. Learning alternati… find
  2. Lai. 1992. An estimate of an upper bound for the entropy of English. Computational Linguistics, 18(1):31–40. phonology. Unpublished manuscript, University of British Columbia and Stony Brook University. Available as https://ling.auf.net/lingbuzz/ 002503. ↗
  3. Andrew Carstairs-McCarthy. 2010. The Evolution of Morphology, volume 14. Oxford University Press. find
  4. Stephen R. Anderson. 1992. A-Morphous Morphology, volume 62, Cambridge University Press. find
  5. C. K. Chow and Cong N. Liu. 1968. Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory, 14(3):462–467. find
  6. Mark Aronoff. 1976. Word Formation in Generative Grammar. Number 1 in Linguistic Inquiry Monographs. MIT Press, Cambridge, MA. find
  7. Matthew Baerman. 2015. The Oxford Handbook of Inflection. Oxford Handbooks in Linguistic. Part II: Paradigms and their Variants Ryan Cotterell, Christo Kirov, Mans Hulden, and Jason Eisner. 2018a. On the diachronic stability of irregularity in inflectional morphology. arXiv preprint arXiv:1804.08262… — [arXiv:1804.08262](https://arxiv.org/abs/1804.08262) find
  8. Matthew Baerman, Dunstan Brown, and Greville G. Corbett. 2015. Understanding and measuring morphological complexity: An introduction. In Matthew Baerman, Dunstan Brown, and Greville G. Corbett, editors, Understanding and measuring morphological complexity. Oxford University Press. find
  9. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Gėraldine Walther, Ekaterina Vylomova, Arya D. McCarthy, Katharina Kann, Sebastian Mielke, Garrett Nicolai, Miikka Silfverberg, David Yarowsky, Jason Eisner, and Mans Hulden. 2018b. The CoNLL–SIGMORPHON 2018 shared task: Universal morphological re…
  10. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR.
  11. RyanCotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra Kübler, David Yarowsky, Jason Eisner, and Mans Hulden. 2017a. find
  12. The CoNLL-SIGMORPHON 2017 shared task: Universal morphological reinflection in 52 languages. In Proceedings of the CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, Vancouver.
  13. Max Bane. 2008. Quantifying and measuring morphological complexity. In Proceedings of the 26th West Coast Conference on Formal Linguistics, pages 69–76. find
  14. Jean Berko. 1958. The child’s learning of English morphology. Word, 14(2-3):150–177. find
  15. Leonard Bloomfield. 1933. Language, University of Chicago Press. Reprint edition (October 15, 1984). find
  16. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, and Mans Hulden. 2016. find
  17. The SIGMORPHON 2016 shared task—morphological reinflection. In Proceedings of the 14th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 10–22, Berlin. find
  18. Olivier Bonami and Sarah Beniamine. 2016. Joint predictiveness in inflectional paradigms. Word Structure, 9(2):156–182. find
  19. Alexandre Bouchard-Côté, Percy Liang, Thomas Griffiths, and Dan Klein. 2007. A probabilistic approach to diachronic phonology. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 887–896, P… find
  20. Ryan Cotterell, Nanyun Peng, and Jason Eisner. 2014. Stochastic contextual edit distance and probabilistic FSTs. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), pages 625–630. Baltimore, MD. find
  21. Peter F. Brown, Vincent J. Della Pietra, Robert L. Mercer, Stephen A. Della Pietra, and Jennifer C. find
  22. Charles F. Hockett. 1958. A Course In Modern Linguistics. The MacMillan Company. find
  23. Ryan Cotterell, Nanyun Peng, and Jason Eisner. 2015. Modeling word forms using latent underlying morphs and phonology. Transactions of the Association for Computational Linguistics (TACL), 3433–447.
  24. Katharina Kann and Hinrich Schütze. 2016. Single-model encoder-decoder with explicit morphological representation for reinflection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), pages 555–560, Berlin. find
  25. Ryan Cotterell, John Sylak-Glassman, and Christo Kirov. 2017b. Neural graphical models over strings for principal parts morphological paradigm completion. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pages 759–765, Valencia. find
  26. Michael J. Kenstowicz. 1994. Phonology in Generative Grammar. Blackwell Oxford. find
  27. Aleksandr E. Kibrik. 1998, Archi (Caucasian – Daghestanian). In The Handbook of Morphology, pages 455–476, Blackwell Oxford. find
  28. Ryan Cotterell, Ekaterina Vylomova, Huda Khayrallah, Christo Kirov, and David Yarowsky. 2017c. Paradigm completion for derivational morphology. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 725–731, Copenhagen. find
  29. Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sebastian Mielke, Arya D. McCarthy, Sandra Kübler, David Yarowsky, Jason Eisner, and Mans Hulden. 2018. UniMorph 2.0: Universal morphology. In Proceedings of LREC 2018, Miyazaki,…
  30. Markus Dreyer and Jason Eisner. 2009. Graphical models over multiple strings. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 101–110. Singapore. find
  31. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of ACL 2017, System Demonstrations, pages 67–72.
  32. Markus Dreyer and Jason Eisner. 2011. Discovering morphological paradigms from plain text using a Dirichlet process mixture model. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 616–627, Edinburgh. find
  33. Warren S. McCulloch and Walter Pitts. 1943. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5(4):115–133. find
  34. Markus Dreyer, Jason Smith, and Jason Eisner. 2008, October. Latent-variable modeling of string transductions with finite-state methods. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1080–1089, Honolulu, HI. find
  35. John McWhorter. 2001. The world’s simplest grammars are creole grammars. Linguistic Typology, 5(2):125–66. find
  36. Yoon Mi Oh. 2015. Linguistic Complexity and Information: Quantitative Approaches. Ph.D. thesis, Université de Lyon, France. find
  37. Jack Edmonds. 1967. Optimum branchings. Journal of Research of the National Bureau of Standards B, 71(4):233–240. find
  38. Azaria Paz. 2003. Probabilistic Automata, John Wiley and Sons. find
  39. David Gil. 1994. The structure of Riau Indonesian. Nordic Journal of Linguistics, 17(2):179–200. find
  40. François Pellegrino, Christophe Coupé, and Egidio Marsico. 2011. A cross-language perspective on speech information rate. Language, 87(3):539–558. find
  41. Mary Hare and Jeffrey L. Elman. 1995. Learning and morphological change. Cognition, 56(1):61–98. find
  42. Kenny Smith, Michael L. Kalish, Thomas L. Griffiths, and Stephan Lewandowsky. 2008. Cultural transmission and the evolution of human behaviour. Philosophical Transactions B. doi.org/10.1098/rstb.2008.0147. — [doi:10.1098/rstb.2008.0147](https://doi.org/10.1098/rstb.2008.0147) find
  43. Steven Pinker and Alan Prince. 1988. On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition, 28(1):73–193. find
  44. Angela Ralli. 1994. Feature representations and feature-passing operations in Greek nominal inflection. In Proceedings of the 8th Symposium on English and Greek Linguistics, pages 19–46. find
  45. Carolyn P. Sobel and Paul Li. 2013. The Cognitive Sciences: An Interdisciplinary Approach. Sage Publications. find
  46. Angela Ralli. 2002. The role of morphology in gender determination: evidence from Modern Greek. Linguistics, 40(3; ISSU 379):519–552. find
  47. Andrew Spencer. 1991. Morphological Theory: An Introduction to Word Structure in Generative Grammar. Wiley-Blackwell. find
  48. Thomas Stolz, Hitomi Otsuka, Aina Urdze, and Johan van der Auwera. 2012. Irregularity in Morphology (and beyond), volume 11. Walter de Gruyter. find
  49. Jorma Rissanen and Eric S. Ristad. 1994. Language acquisition in the MDL framework. In Eric S. Ristad, editor, Language Computation. American Mathematical Society, Philadelphia, PA. find
  50. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems (NIPS), pages 3104–3112.
  51. Robert Henry Robins. 2013. A Short History of Linguistics. Routledge. find
  52. Benoı̂t Sagot. 2013. Comparing complexity measures. In Computational Approaches to Morphological Complexity, Paris. find
  53. John Sylak-Glassman. 2016, The composition and use of the universal morphological feature schema (Unimorph schema). Johns Hopkins University. find
  54. Edward Sapir. 1921. Language: An Introduction to the Study of Speech. find
  55. John Sylak-Glassman, Christo Kirov, David Yarowsky, and Roger Que. 2015, July. A language-independent feature schema for inflectional morphology. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language…
  56. Claude E. Shannon. 1948. A mathematical theory of communication. Bell Systems Technical Journal, 27. find
  57. Hava T. Siegelmann and Eduardo D. Sontag. 1991. Turing computability with neural nets. Applied Mathematics Letters, 4(6):77–80. find
  58. Hava T. Siegelmann and Eduardo D. Sontag. 1995. On the computational power of neural nets. Journal of Computer and System Sciences, 50(1):132–150. find
  59. Matthew D. Zeiler. 2012. ADADELTA: An adaptive learning rate method. arXiv preprint arXiv:1212.5701v1.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cotterell-2019-complexity-typology-inflectional-morphological-systems/