Elamur Research⌕
☰
home / papers / 2026 / Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs

Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs

Paper · 2026 · Eva Vanmassenhove

Summary

This position paper asks whether multilingual LLMs, trained to predict the next token, are wearing away the long statistical "tails" of language: rare words, uncommon constructions, minority languages and culturally specific variation. It presents no new dataset or experiment. Instead, the author argues by contrasting human-driven language change with technology-driven selection, drawing on iterated learning, studies of lexical growth and competition, and recent model-collapse research in Computer Vision, NLP and Machine Translation. Translation technology is the main case study. The core idea is that human cognitive limits push languages toward structure and compositionality while vocabularies stay rich. LLMs, by contrast, drift toward the most probable forms, especially when machine-generated text re-enters training corpora in self-consuming loops. The main claim is that, under current data and training practices, LLM-driven selection shrinks diversity instead of transforming it, and it spreads averaged, dominant biases. For NLP practitioners, the paper urges training and evaluating systems for expressive multilingual richness rather than only fluency or accuracy.

Key points

  • Model collapse is defined as synthetic output feeding back into training data, which distorts the distribution and leaves low-probability linguistic phenomena underrepresented.
  • Human versus LLM selection: human transmission produces structure through inductive biases toward simplicity and compressibility (Kirby et al., Smith et al.). LLMs instead converge on statistically likely forms and cannot critically reflect on or correct themselves.
  • Context figures cited:
    • About 6,000 languages exist (estimates range from 3,000 to 10,000), and nearly half are endangered.
    • According to Michel et al. (2011), the English lexicon expanded by more than 70% over 50 years, gaining about 8,500 words annually.
  • Lexical competition example: "X-ray" displaced "Roentgenogram" and "radiogram", showing that word survival depends on more than frequency.
  • Stated limitations: the critique targets current generative AI rather than all possible architectures. The author acknowledges a selection bias in the literature reviewed and notes that some researchers may see "flattening" as gained efficiency.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tilburg UniversityTilburg · Netherlands51 works · 6,798 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksMultilingual & cross-lingual NLP · 882 works

Cite

Cite as Eva Vanmassenhove. 2026. Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.532

@inproceedings{vanmassenhove2026losing,
  title = {{Losing our Tail, Again: (Un)Natural Selection \& Multilingual LLMs}},
  author = {Eva Vanmassenhove},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.532},
  url = {https://doi.org/10.18653/v1/2026.acl-long.532}
}

References

67 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G Baraniuk. 2023. Self-consuming generative models go mad. arXiv preprint arXiv:2307.01850, 4:14. — [arXiv:2307.01850](https://arxiv.org/abs/2307.01850) find
  2. Eduardo G Altmann, Janet B Pierrehumbert, and Adilson E Motter. 2011. Niche as a determinant of word fate in online groups. PloS one, 6(5):e19009. find
  3. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.
  4. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72.
  5. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623. find
  6. Sebastian Bernhardsson, Luis Enrique Correa da Rocha, and Petter Minnhagen. 2009. The meta book and size-dependent properties of written language. New Journal of Physics, 11(12):123015. find
  7. Alexander Bick, Adam Blandin, and David J Deming. 2026. The rapid adoption of generative ai. Management Science. find
  8. Henry Brighton, Kenny Smith, and Simon Kirby. 2005. Language as an evolutionary system. Physics of Life Reviews, 2(3):177–226. find
  9. Lindell Bromham, Russell Dinnage, Hedvig Skirgård, Andrew Ritchie, Marcel Cardillo, Felicity Meakins, Simon Greenhill, and Xia Hua. 2022. Global predictors of language endangerment and the future of linguistic diversity. Nature ecology & evolution, 6(2):163–173. find
  10. Yang Trista Cao, Anna Sotnikova, Jieyu Zhao, Linda X Zou, Rachel Rudinger, and Hal Daume III. 2023. Multilingual large language models leak human stereotypes across language boundaries. arXiv preprint arXiv:2312.07141. find
  11. Sheila Castilho, Zoe Fitzsimmons, Claire Holton, and Aoife Mc Donagh. 2025. Synthetic fluency: Hallucinations, confabulations, and the creation of irish words in llm-generated translations. Proceedings of Machine Translation Summit XVII: Research Track. find
  12. Noam Chomsky. 1980. Rules and representations. Behavioral and brain sciences, 3(1):1–15. find
  13. David Crystal. 2002. Language death. Cambridge university press. find
  14. Charles Darwin. 1871. The Descent of Man, and Selection in Relation to Sex, volume 1. John Murray, London. find
  15. Marci S DeCaro, Robin D Thomas, and Sian L Beilock. 2008. Individual differences in category learning: Sometimes less working memory capacity is better than more. Cognition, 107(1):284–294. find
  16. Kristian Dokic, Barbara Pisker, and Bojan Radisic. 2025. Mirroring cultural dominance: Disclosing large language models social values, attitudes and stereotypes. Societies, 15(5):142. find
  17. Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaïd Harchaoui, and Yejin Choi. 2023. Faith and fate: Limits of transformers on composition…
  18. Daniel L Everett. 2017. How language began: The story of humanity’s greatest invention. Liveright Publishing. find
  19. Martin Gerlach and Eduardo G Altmann. 2013. Stochastic model for the vocabulary growth in natural languages. Physical Review X, 3(2):021006. find
  20. Gerd Gigerenzer and Daniel G Goldstein. 1996. Reasoning the fast and frugal way: models of bounded rationality. Psychological review, 103(4):650. find
  21. Alison Gopnik. 2023. Large language models as cultural technologies: Imitation and innovation in children and models. https://doi.org/10.48448/ 8qf0-j172. Keynote lecture at ACL 2023, Toronto, July 12, 2023. find
  22. Yanzhu Guo, Guokan Shang, and Chloé Clavel. 2024. Benchmarking linguistic diversity of large language models. arXiv preprint arXiv:2412.10271.
  23. Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, Alon Albalak, and Yejin Choi. 2025. Artificial hivemind: The open-ended homogeneity of language models (and beyond). arXiv preprint arXiv:2510.22954. — [arXiv:2510.22954](https://arxiv.org/abs/2510.22954) find
  24. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the nlp world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293.
  25. Alina Karakanta, Mayra Nas, and Aletta G. Dorst. 2025. Metaphors in literary machine translation: Close but no cigar? Proceedings of Machine Translation Summit XVII: Research Track. find
  26. Simon Kirby, Tom Griffiths, and Kenny Smith. 2014. Iterated learning and the evolution of language. Current opinion in neurobiology, 28:108–114. find
  27. Simon Kirby, Monica Tamariz, Hannah Cornish, and Kenny Smith. 2015. Compression and communication in the cultural evolution of linguistic structure. Cognition, 141:87–102. find
  28. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2023. Understanding the effects of rlhf on llm generalisation and diversity. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following.
  29. Dmitry Kobak, Rita González-Márquez, Emőke-Ágnes Horvát, and Jan Lause. 2024. Delving into chatgpt usage in academic writing through excess vocabulary. arXiv preprint arXiv:2406.07016. find
  30. András Kornai. 2013. Digital language death. PloS one, 8(10):e77056. find
  31. Tom Kouwenhoven, Max Peeperkorn, Roy de Kleijn, and Tessa Verhoef. 2025a. Shaping shared languages: Human and large language models’ inductive biases in emergent communication. arXiv preprint arXiv:2503.04395. — [arXiv:2503.04395](https://arxiv.org/abs/2503.04395) find
  32. Tom Kouwenhoven, Max Peeperkorn, and Tessa Verhoef. 2025b. Searching for structure: Investigating emergent communication with large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9977– 9991, Abu Dhabi, UAE. Association for Computational Lingu… find
  33. Svenja Kranich. 2014. Translations as a locus of language contact. In Translation: A multidisciplinary approach, pages 96–115. Springer. find
  34. Lionbridge. 2020. Localizing long-tail languages. https://www.lionbridge.com/blog/ translation-localization/localizing-long-tail-languages/. Accessed: 17 April 2026. ↗
  35. Jiaming Luo, Colin Cherry, and George Foster. 2024. To diverge or not to diverge: A morphosyntactic perspective on machine translation vs human translation. Transactions of the Association for Computational Linguistics, 12:355–371.
  36. Gary Lupyan and Rick Dale. 2010. Language structure is partly determined by social structure. PloS one, 5(1):e8559. find
  37. Gonzalo Martínez, Lauren Watson, Pedro Reviriego, José Alberto Hernández, Marc Juarez, and Rik Sarkar. 2023. Towards understanding the interplay of generative artificial intelligence and the internet. In International Workshop on Epistemic Uncertainty in Artificial Intelligence, pages 59–73. Springe… find
  38. R Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. 2023. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638. find
  39. Jean-Baptiste Michel, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K Gray, Google Books Team, Joseph P Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, et al. 2011. Quantitative analysis of culture using millions of digitized books. science, 331(6014):176–182. find
  40. Steven Mithen. 2024. The language puzzle: How we talked our way out of the stone age. Profile Books. find
  41. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. 2022. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786.
  42. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  43. Mark Pagel, Mark Beaumont, Andrew Meade, Annemarie Verkerk, and Andreea Calude. 2019. Dominant words rise to the top by positive frequency-dependent selection. Proceedings of the National Academy of Sciences, 116(15):7397–7402. find
  44. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  45. Alexander M Petersen, Joel Tenenbaum, Shlomo Havlin, and H Eugene Stanley. 2012. Statistical laws governing fluctuations in word use from word birth to word death. Scientific reports, 2(1):313. find
  46. Steven T Piantadosi, Harry Tily, and Edward Gibson. 2011. Word lengths are optimized for efficient communication. Proceedings of the National Academy of Sciences, 108(9):3526–3529. find
  47. Plato. 2002. Phaedrus. Oxford World Classics. Oxford University Press, Oxford. find
  48. Florencia Reali, Nick Chater, and Morten H Christiansen. 2018. Simpler grammar, larger vocabulary: How population size affects language. Proceedings of the Royal Society B: Biological Sciences, 285(1871):20172586. find
  49. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
  50. Yi Ren, Shangmin Guo, Linlu Qiu, Bailin Wang, and Danica J Sutherland. 2024a. Bias amplification in language model evolution: An iterated learning perspective. arXiv preprint arXiv:2404.04286. — [arXiv:2404.04286](https://arxiv.org/abs/2404.04286) find
  51. Yi Ren, Shangmin Guo, Linlu Qiu, Bailin Wang, and Danica J Sutherland. 2024b. Bias amplification in language model evolution: An iterated learning perspective. arXiv preprint arXiv:2404.04286. — [arXiv:2404.04286](https://arxiv.org/abs/2404.04286) find
  52. Pedro Reviriego, Javier Conde, Elena Merino-Gómez, Gonzalo Martínez, and José Alberto Hernández. 2024. Playing with words: Comparing the vocabulary and lexical diversity of chatgpt and humans. Machine Learning with Applications, 18:100602. find
  53. Yu Sasaki. 2017. Publishing nations: Technology acquisition and language standardization for european ethnic groups. The Journal of Economic History, 77(4):1007–1047. find
  54. Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Aru-landu, and Sanmi Koyejo. 2025. Position: Model collapse does not mean what you think. arXiv preprint arXiv:2503.03150. — [arXiv:2503.03150](https://arxiv.org/abs/2503.03150) find
  55. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. 2023. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493.
  56. Kenny Smith, Henry Brighton, and Simon Kirby. 2003a. Complex systems in language evolution: the cultural emergence of compositional structure. Advances in complex systems, 6(04):537–558. find
  57. Kenny Smith, Simon Kirby, and Henry Brighton. 2003b. Iterated learning: A framework for the emergence of language. Artificial life, 9(4):371–386. find
  58. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mt5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
  59. Yongjing Yin, Lian Fu, Yafu Li, and Yue Zhang. 2024. On compositional generalization of transformer-based neural machine translation. Information Fusion, 111:102491. find
  60. Longfei Yun, Chenyang An, Zilong Wang, Letian Peng, and Jingbo Shang. 2025. The price of format: Diversity collapse in llms. find
  61. Xiang Zhou, Yichen Jiang, and Mohit Bansal. 2023. Data factors for better compositional generalization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14549–14566. find
  62. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 223–231.
  63. Brian Thompson, Mehak Dhaliwal, Peter Frisch, Tobias Domhan, and Marcello Federico. 2024. A shocking amount of the web is machine translated: Insights from multi-way parallelism. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1763–1775, Bangkok, Thailand. Association f… find
  64. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  65. Eva Vanmassenhove, Dimitar Shterionov, and Matthew Gwilliam. 2021. Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2…
  66. Eva Vanmassenhove, Dimitar Shterionov, and Andy Way. 2019. Lost in translation: Loss and decay of linguistic richness in machine translation. In Proceedings of Machine Translation Summit XVII: Research Track, pages 222–232, Dublin, Ireland. European Association for Machine Translation.
  67. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vanmassenhove-2026-losing-our-tail-again-un/