Elamur Research⌕
☰
home / papers / 2026 / Data Pollination: An Emergent Ecological Process Driving AI Population Evolution

Data Pollination: An Emergent Ecological Process Driving AI Population Evolution

Paper · 2026 · Shufang Xie, Qizhi Pei, Ang Lv et al.

Summary

The paper argues that language models do not evolve in isolation. Their generated text is published online, scraped into web corpora, and used to train later models, often from other organizations. The authors call this unplanned feedback loop "data pollination" and ask how it affects the risk of model collapse under synthetic-data training.

They first fingerprint the linguistic style of 27 deployed models from five families and find convergence across families. Influence appears to run mainly from older models to newer ones.

They then model the process as an evolutionary algorithm:

  • Selection: user preference keeps the top performers.
  • Mutation: small parameter noise is added to parents.
  • Crossover: offspring take weights from one parent and train on another parent's synthetic outputs.

Across experiments with 320 small models trained only on synthetic data after generation 0, populations stay stable or improve on benchmarks. Single-lineage recursive training, by contrast, collapses. A Gaussian analysis and embedding studies trace this robustness to preserved diversity. The authors conclude that model diversity is essential for a healthy AI ecosystem.

Key points

  • Real-world evidence comes from style fingerprints of 27 models in the Claude, Llama, GPT, Mistral and Gemma families. Cross-family influence is stronger from earlier to later releases.
  • In the framework, a population of N models keeps the best √N each generation. All ordered parent pairs then produce N offspring, so population size stays constant.
  • The controlled runs involve 320 language models of about 100M parameters, each trained on roughly 1B tokens. The authors note this is far below production scale.
  • In a diversity check, RoBERTa embeddings with t-SNE on 32 prompts (WinoGrande, generations 0–4) show that outputs stay distinct per prompt and varied across models, rather than becoming homogeneous.
  • The authors acknowledge limitations:
    • Convergence in the wild could also come from shared public datasets or similar training practices.
    • The theory relies on a simplified Gaussian model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Renmin University of ChinaBeijing · China139 works · 17,752 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China152 works · 13,164 citations🎓Wuhan UniversityWuhan · China47 works · 1,246 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 works

Cite

Cite as Shufang Xie, Qizhi Pei, Ang Lv, Jingyang Hu, Lijun Wu and Rui Yan. 2026. Data Pollination: An Emergent Ecological Process Driving AI Population Evolution. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1229

@inproceedings{xie2026data,
  title = {{Data Pollination: An Emergent Ecological Process Driving AI Population Evolution}},
  author = {Shufang Xie, Qizhi Pei, Ang Lv, Jingyang Hu, Lijun Wu and Rui Yan},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1229},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1229}
}

References

51 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. 2025. Evolutionary optimization of model merging recipes. Nature Machine Intelligence, 7:195–204. find
  2. Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. Mathqa: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North, pages 2357–2367. Association for Computational Lin…
  3. Bryce Anderson, Riley Galpin, and Tom S Juzek. 2025. Model misalignment and language change: Traces of ai-associated language in unscripted spoken english. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), volume 8, pages 179– 191.
  4. Valentina Bellini, Federico Semeraro, Jonathan Mon-tomoli, Marco Cascella, and Elena Bignami. 2024. Between human and ai: assessing the reliability of ai text detection tools. Current Medical Research and Opinion, 40(3):353–358. find
  5. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623. find
  6. Quentin Bertrand, Joey Bose, Alexandre Duplessis, Marco Jiralerspong, and Gauthier Gidel. 2024. On the stability of iterative retraining of generative models on their own data. In The Twelfth International Conference on Learning Representations. find
  7. Rishi Bommasani, Kathleen A Creel, Ananya Kumar, Dan Jurafsky, and Percy Liang. 2022. Picking on the same person: Does algorithmic monoculture lead to outcome homogenization? In Advances in Neural Information Processing Systems. find
  8. Judith L Bronstein, Ruben Alarcón, and Monica Geber. 2006. The evolution of plant-insect mutualisms. New Phytologist, 172(3):412–428. find
  9. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901.
  10. Hao Chen, Abdul Waheed, Xiang Li, Yidong Wang, Jindong Wang, Bhiksha Raj, and Marah Abdin. 2024a. find
  11. On the diversity of synthetic data and its impact on training large language models. arXiv preprint arXiv:2410.15226.
  12. Jie Chen, Yupeng Zhang, Bingning Wang, Wayne Xin Zhao, Ji-Rong Wen, and Weipeng Chen. 2024b. Unveiling the flaws: Exploring imperfections in synthetic data and mitigation strategies for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14855–14865… find
  13. Alex Cloud, Minh Le, James Chua, Jan Betley, Anna Sztyber-Betley, Sören Mindermann, Jacob Hilton, Samuel Marks, and Owain Evans. 2026. Language models transmit behavioural traits through hidden signals in data. Nature, 652:615–621. find
  14. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language P… find
  15. Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, and Julia Kempe. 2024. A tale of tails: Model collapse as a change of scaling laws. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 11165–11197. PMLR. find
  16. Anil R Doshi and Oliver P Hauser. 2024. Generative ai enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28):eadn5290. find
  17. Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural architecture search: A survey. Journal of Machine Learning Research, 20(55):1–21. find
  18. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M. Rush. 2019. Gltr: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations.
  19. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In International Conference on Learning Representations.
  20. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In International Conference on Learning Representations.
  21. Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, and Koray Kavukcuoglu. 2017. Population based training of neural networks. arXiv preprint arXiv:1711.09846. find
  22. Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, Alon Albalak, and Yejin Choi. 2025. Artificial hivemind: The open-ended homogeneity of language models (and beyond). In Advances in Neural Information Processing Systems 38 (NeurIPS 2025) Datas… find
  23. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations (ICLR).
  24. Hao Liang, Keshi Zhao, Yajie Yang, Bin Cui, Zenan Zhou, and Wentao Zhang. 2025. Training data distribution estimation for optimized pre-training data management. In 2025 IEEE 41st International Conference on Data Engineering (ICDE), pages 4640–4648. IEEE. find
  25. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  26. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  27. Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. 2024. Fineweb-edu: the finest collection of educational content. find
  28. Gary W Luck, Gretchen C Daily, and Paul R Ehrlich. 2003. Population diversity and ecosystem services. Trends in Ecology & Evolution, 18(7):331–336. find
  29. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning.
  30. Cailean Osborne, Jennifer Ding, and Hannah Rose Kirk. 2024. The ai community building the future? a quantitative analysis of development activity on hugging face hub. Journal of Computational Social Science, 7(2):2067–2105. find
  31. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems.
  32. Samuel J Paech. 2025. Slop forensics: A toolkit for generating & analyzing lexical patterns in llm outputs. find
  33. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Technical report, OpenAI. find
  34. Abhilasha Ravichander, Jillian Fisher, Taylor Sorensen, Ximing Lu, Maria Antoniak, Bill Yuchen Lin, Niloofar Mireshghallah, Chandra Bhagavatula, and Yejin Choi. 2025. Information-guided identification of training data imprint in (proprietary) large language models. In Proceedings of the 2025 Confere… find
  35. Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. 2019. Regularized evolution for image classifier architecture search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4780–4789. find
  36. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2025. Can ai-generated text be reliably detected? stress testing ai text detectors under various attacks. Transactions on Machine Learning Research.
  37. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. find
  38. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processi… find
  39. Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, and Oluwasanmi Koyejo. 2025. Position: Model collapse does not mean what you think. arXiv preprint arXiv:2503.03150. — [arXiv:2503.03150](https://arxiv.org/abs/2503.03150) find
  40. Mohamed El Amine Seddik, Suei-Wen Chen, Soufiane Hayou, Pierre Youssef, and Merouane Debbah. 2024. How bad is training on synthetic data? a statistical analysis of language model collapse. In Conference on Language Modeling (COLM). find
  41. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. Ai models collapse when trained on recursively generated data. Nature, 631(8022):755–759.
  42. Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063.
  43. Ananda Theertha Suresh, Andrew Thangaraj, and Aditya Nanda Kishore Khandavally. 2025. Rate of model collapse in recursive training. In International Conference on Artificial Intelligence and Statistics, pages 1396–1404. PMLR. find
  44. Dmytro Valiaiev. 2024. Detection of machine-generated text: Literature survey. arXiv preprint arXiv:2402.01642. — [arXiv:2402.01642](https://arxiv.org/abs/2402.01642) find
  45. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11). find
  46. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30.
  47. Tianyu Wang, Lingyou Pang, Akira Horiguchi, and Carey E Priebe. 2025. Llm web dynamics: Tracing model collapse in a network of llms. arXiv preprint arXiv:2506.15690. — [arXiv:2506.15690](https://arxiv.org/abs/2506.15690) find
  48. Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inferenc…
  49. Fan Wu, Emily Black, and Varun Chandrasekaran. 2025. Generative monoculture in large language models. In International Conference on Learning Representations. find
  50. Yiqun Zhang, Peng Ye, Xiaocui Yang, Shi Feng, Shufei Zhang, Lei Bai, Wanli Ouyang, and Shuyue Hu. 2025. Nature-inspired population-based evolution of large language models. arXiv preprint arXiv:2503.01155.
  51. Xuekai Zhu, Daixuan Cheng, Hengli Li, Kaiyan Zhang, Ermo Hua, Xingtai Lv, Ning Ding, Zhouhan Lin, Zilong Zheng, and Bowen Zhou. 2025. How to synthesize text data without model collapse? In Proceedings of the 42nd International Conference on Machine Learning. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xie-2026-data-pollination-emergent-ecological-process/