📄 Phonotactic Complexity and Its Trade-offs

Summary

The paper asks how to compare phonotactic complexity across languages in a way that is more fine-grained than counting inventory units. Counts of vowels, consonants or syllables cannot see longer-range dependencies such as vowel harmony. The authors propose an information-theoretic measure: bits per phoneme. For each language, they train a model over words written as phoneme strings and score held-out word types by their negative log-probability per segment. Because this cross-entropy is an upper bound on the true entropy, it can be compared across languages. They use concept-aligned lists of 1016 basic words in 106 languages and compare unigram, trigram and LSTM models. The LSTM score correlates strongly and negatively with average word length (−0.74). This trade-off appears both across and within language families, while inventory-size measures show only weak links to word length. The work supports the compensation and equal-capacity debate with a reproducible, quantitative tool. The authors stay neutral on what causes the trade-off.

Key points

• Main correlation: The LSTM-based bits per phoneme correlates with mean word length at about −0.74 (Spearman). Phoneme, vowel and consonant counts correlate much more weakly. Inventory sizes correlate weakly, and unigram scores give about −0.22.
• Language-level effect: A permutation test built 10⁴ shuffled "fake languages". None matched the real correlation, which supports a genuine language-level effect (p < 10⁻⁴).
• Constraints lower entropy: Artificially removing phonological constraints made the languages less predictable.
  • Removing vowel harmony raised LSTM complexity by roughly 0.62 bits per phoneme, about 16% more entropy.
  • Removing final-obstruent devoicing gave a small but significant rise of about 0.034 bits.
• Within families: In the five families with at least 4 languages (Dravidian, Indo-European, Nakh-Daghestanian, Turkic, Uralic), the LSTM measure correlates with word length, with ρ between −0.662 and −1.0 (p < 0.1). Vowel inventory size is significant only within Turkic.
• Feature embeddings: Phoneme-feature embeddings, shared or independent, barely change results compared with plain lookup embeddings. Lookup embeddings reach about 2.98 bits per phoneme, versus 3.27 for trigram and 4.48 for unigram models. Code is available at github.com/tpimentelms/phonotactic-complexity.