Elamur Research⌕
☰
home / papers / 2026 / Demystifying Mixed Outcomes of Self-Training: Pre-training Analyses on Non-Toy LLMs

Demystifying Mixed Outcomes of Self-Training: Pre-training Analyses on Non-Toy LLMs

Paper · 2026 · Yusuke Nakamura, Hirokazu Kiyomaru, Chaoran Liu et al.

Summary

Studies of self-training disagree. Some report that LLMs improve when retrained on their own outputs, while others report model collapse. The authors suspect the disagreement comes from different experimental setups: strong models with engineered pipelines on one side, and toy models with unconditional generation on the other. To separate these factors, they use realistic OLMo-2 models (7B and 13B, Stage-1 checkpoints) with a deliberately simple loop. In each round, the model continues prefixes sampled from its pre-training corpus, OLMo-Mix-1124, and is then continually pre-trained on 1B of its own tokens. They repeat this for up to three rounds and optionally filter samples by perplexity. In every configuration, downstream scores fell, and perplexity on the original corpus generally rose. A control run on real Dolmino-Mix-1124 data rules out the training settings as the cause. GPT-4o comparisons judge the synthetic text much worse than human corpora. The authors argue that naive self-training inherently leads to collapse. In their view, reported self-improvement reflects external, human-designed pipeline intelligence rather than genuine autonomous refinement.

Key points

  • Setup: Prefix lengths K ∈ {0, 3, 64} tokens were tested over three rounds. Evaluation used MMLU, ARC-C, HellaSwag, WinoGrande, NQ and DROP, plus perplexity on OLMo-Mix-1124.
  • Main results (K=0, R=3):
    • The 7B model's average score dropped from 59.9 to 58.6, and its perplexity rose from 11.038 to 11.836.
    • The 13B model's average fell from 65.3 to 63.6.
  • Perplexity filtering: Keeping the low-, mid- or high-perplexity third of 3× over-generated text produced no meaningful gains.
  • Data quality (GPT-4o pairwise judgments): The original corpora won the large majority of comparisons, and this gap widened with each round. In one 7B setting, OLMo-Mix-1124 was preferred in about 98% of cases.
  • Diversity: Scores computed with Multilingual-E5 embeddings showed no consistent link to downstream performance. This suggests low diversity is not the main driver of the decline.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬National Institute of InformaticsTokyo · Japan37 works · 3,247 citations🎓Waseda UniversityTokyo · Japan20 works · 584 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Yusuke Nakamura, Hirokazu Kiyomaru, Chaoran Liu, Shuhei Kurita and Daisuke Kawahara. 2026. Demystifying Mixed Outcomes of Self-Training: Pre-training Analyses on Non-Toy LLMs. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.213

@inproceedings{nakamura2026demystifying,
  title = {{Demystifying Mixed Outcomes of Self-Training: Pre-training Analyses on Non-Toy LLMs}},
  author = {Yusuke Nakamura, Hirokazu Kiyomaru, Chaoran Liu, Shuhei Kurita and Daisuke Kawahara},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.213},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.213}
}

References

12 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  2. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457.
  3. DatologyAI, Pratyush Maini, Vineeth Dorna, Parth Doshi, Aldo Carranza, Fan Pan, Jack Urbanek, Paul Burstein, Alex Fang, Alvin Deng, Amro Abbas, Brett Larsen, Cody Blakeney, Charvi Bannur, Christina Baek, Darren Teh, David Schwab, Haakon Mongstad, Haoli Yin, and 11 others. 2025. Beyondweb: Lessons fr… — [arXiv:2508.10975](https://arxiv.org/abs/2508.10975) find
  4. George Drayson, Emine Yilmaz, and Vasileios Lampos. 2025. Machine-generated text detection prevents language model collapse. Preprint, arXiv:2502.15654. find
  5. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistic…
  6. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  7. Siheng Li, Cheng Yang, Zesen Cheng, Lemao Liu, Mo Yu, Yujiu Yang, and Wai Lam. 2024. Large language models can self-improve in long-context reasoning. Preprint, arXiv:2411.08147. — [arXiv:2411.08147](https://arxiv.org/abs/2411.08147) find
  8. Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. 2023. When less is more: Investigating data pruning for pretraining llms at scale. find
  9. OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, and 400 others. 2024. Gpt-4o sys…
  10. Zeyu Qin, Qingxiu Dong, Xingxing Zhang, Li Dong, Xiaolong Huang, Ziyi Yang, MAHMOUD KHADEMI, Dongdong Zhang, Hany Hassan Awadalla, Yi R. Fung, Weizhu Chen, Minhao Cheng, and Furu Wei. 2025. Scaling laws of synthetic data for language model. In Proceedings of the Second Conference on Language Modelin… find
  11. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. find
  12. Ziyu Shang, Jianghan Liu, Zhizhao Luo, Peng Wang, Wenjun Ke, Jiajun Liu, Zijie Xu, and Guozheng Li. 2025. Acquisition and application of novel knowledge in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nakamura-2026-demystifying-mixed-outcomes-self-training/