Elamur Research⌕
☰
home / papers / 2026 / Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loop

Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loop

Paper · 2026 · Yaxuan Wang, Zhongteng Cai, Yujia Bao et al.

Summary

LLMs are increasingly trained on text written by earlier models. In deployment, how well a model serves each user group also affects who keeps sending queries. The authors formalize this as the Self-Consuming Performative Loop (SCPL). Each new model generation learns from synthetic responses to prompts, and the group mix of those prompts is set by a feedback function of the previous model's per-group scores. A poorly served disadvantaged group therefore supplies fewer prompts over time. Real interaction logs with demographics are not public, so this feedback is simulated under controlled conditions.

The study compares two loops:

  • Standard retraining: each generation is fine-tuned from the base model.
  • Incremental fine-tuning: each generation continues from the previous one. This setting has received little prior attention.

Experiments cover three tasks (news continuation, Preference Dissection, math solving) with Qwen2.5 and Llama3.1 models. Over generations, preference bias grows and output quality falls, while the gap between groups narrows. Feedback mostly accelerates bias growth in the incremental setting. A rule-based reward selection with reweighting reduces the amplification, which matters for building trustworthy self-improving systems.

Key points

  • SCPL uses two groups (advantaged vs. disadvantaged) and an unbiased balanced test set. A function F(S_a, S_d) turns group performance scores into the share of prompts drawn from the disadvantaged group. An optional Accumulation mode reuses data from all earlier generations.

  • Models include Qwen2.5-1.5B, Llama3.1-8B, Qwen2.5-Math-1.5B and Qwen2.5-Math-7B. Performative dynamics slowed degradation for the 1.5B math model but sped up performance decline for the 7B one.

  • The mitigation method scores samples with a rule-based reward (k = 4, α1 = 1.0, α2 = 3.0). It combines Gibberish Detector fluency with a cosine-similarity check against the reference continuation, and then reweights the selected samples.

  • The baselines were Vanilla Rejection Sampling, Top per Prompt, Top Overall Prompt and Accumulation. On Preference Dissection, Accumulation came out best and the reweighting method ranked second.

  • On the News task with Llama3.1-8B, preference bias starts at 0.5587 and reaches the following values at generation M3:

    SettingSynthetic dataReweightingReal data
    Retraining0.71360.61710.5980
    Incremental fine-tuning0.64850.63260.6090

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The Ohio State UniversityColumbus · United States82 works · 18,502 citations🎓University of California, Santa CruzSanta Cruz · United States36 works · 3,515 citations🏢AccentureZurich · Switzerland6 works · 27 citations🏢Accenture (United States)New York · United States5 works · 261 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSpeech act theory · 229 works

Cite

Cite as Yaxuan Wang, Zhongteng Cai, Yujia Bao, Xueru Zhang and Yang Liu. 2026. Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loop. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1561

@inproceedings{wang2026observations,
  title = {{Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loop}},
  author = {Yaxuan Wang, Zhongteng Cai, Yujia Bao, Xueru Zhang and Yang Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1561},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1561}
}

References

57 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G Baraniuk. 2023. Self-consuming generative models go mad. arXiv preprint arXiv:2307.01850, 4:14. — [arXiv:2307.01850](https://arxiv.org/abs/2307.01850) find
  2. Sina Alemohammad, Ahmed Imtiaz Humayun, Shruti Agarwal, John Collomosse, and Richard Baraniuk. 2024. Self-improving diffusion models with synthetic data. arXiv preprint arXiv:2408.16333. — [arXiv:2408.16333](https://arxiv.org/abs/2408.16333) find
  3. Zhongteng Cai, Yaxuan Wang, Yang Liu, and Xueru Zhang. 2025. Stabilizing self-consuming diffusion models with latent space filtering. arXiv preprint arXiv:2511.12742. — [arXiv:2511.12742](https://arxiv.org/abs/2511.12742) find
  4. Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You can’t stay here: The efficacy of reddit’s 2015 ban examined through hate speech. Proceedings of the ACM on human-computer interaction, 1(CSCW):1–22. find
  5. Tianwei Chen, Yusuke Hirota, Mayu Otani, Noa Garcia, and Yuta Nakashima. 2024. Would deep generative models amplify bias in future models? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10833–10843. find
  6. Wei-Fan Chen, Henning Wachsmuth, Khalid Al Khatib, and Benno Stein. 2018. Learning to flip the bias of news headlines. In Proceedings of the 11th International conference on natural language generation, pages 79–88. find
  7. Yatong Chen, Wei Tang, Chien-Ju Ho, and Yang Liu. 2023. Performative prediction with bandit feedback: Learning through reparameterization. arXiv preprint arXiv:2305.01094. — [arXiv:2305.01094](https://arxiv.org/abs/2305.01094) find
  8. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. find
  9. Elvis Dohmatob, Yunzhen Feng, and Julia Kempe. 2024a. Model collapse demystified: The case of regression. arXiv preprint arXiv:2402.07712. — [arXiv:2402.07712](https://arxiv.org/abs/2402.07712) find
  10. Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, and Julia Kempe. 2024b. A tale of tails: Model collapse as a change of scaling laws. arXiv preprint arXiv:2402.07043. — [arXiv:2402.07043](https://arxiv.org/abs/2402.07043) find
  11. Damien Ferbach, Quentin Bertrand, Avishek Joey Bose, and Gauthier Gidel. 2024. Self-consuming generative models with curated data provably optimize human preferences. arXiv preprint arXiv:2407.09499. — [arXiv:2407.09499](https://arxiv.org/abs/2407.09499) find
  12. Nate Gillman, Michael Freeman, Daksh Aggarwal, Chia-Hong Hsu, Calvin Luo, Yonglong Tian, and Chen Sun. 2024. Self-correcting self-consuming loops for generative model training. arXiv preprint arXiv:2402.07087. — [arXiv:2402.07087](https://arxiv.org/abs/2402.07087) find
  13. Quentin Bertrand, Avishek Joey Bose, Alexandre Duplessis, Marco Jiralerspong, and Gauthier Gidel. 2023. On the stability of iterative retraining of generative models on their own data. arXiv preprint arXiv:2310.00429. — [arXiv:2310.00429](https://arxiv.org/abs/2310.00429) find
  14. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  15. Martin Briesch, Dominik Sobania, and Franz Rothlauf. 2023. Large language models suffer from their own output: An analysis of the self-consuming training loop. arXiv preprint arXiv:2311.16822. — [arXiv:2311.16822](https://arxiv.org/abs/2311.16822) find
  16. Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, and Chloé Clavel. 2023. The curious decline of linguistic diversity: Training language models on synthetic text. arXiv preprint arXiv:2311.09807. find
  17. Patrick Haller, Ansar Aynetdinov, and Alan Akbik. 2023. Opiniongpt: Modelling explicit biases in instruction-tuned llms. arXiv preprint arXiv:2309.03876. find
  18. Moritz Hardt and Celestine Mendler-Dünner. 2023. Performative prediction: Past and future. arXiv preprint arXiv:2310.16608. — [arXiv:2310.16608](https://arxiv.org/abs/2310.16608) find
  19. Ryuichiro Hataya, Han Bao, and Hiromi Arai. 2023. Will large-scale generative models corrupt future datasets? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20555– 20565. find
  20. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR).
  21. Daniel Hickey, Daniel MT Fessler, Kristina Lerman, and Keith Burghardt. 2025. X under musk’s leadership: Substantial hate and no reduction in inauthentic activity. PLoS One, 20(2):e0313293. find
  22. Manoel Horta Ribeiro, Shagun Jhaver, Savvas Zannettou, Jeremy Blackburn, Gianluca Stringhini, Emiliano De Cristofaro, and Robert West. 2021. Do platform migrations compromise content moderation? evidence from r/the_donald and r/incels. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2):1… find
  23. Kun Jin, Tian Xie, Yang Liu, and Xueru Zhang. 2024. Addressing polarization and unfairness in performative prediction. arXiv preprint arXiv:2406.16756. — [arXiv:2406.16756](https://arxiv.org/abs/2406.16756) find
  24. Joshua Kazdan, Rylan Schaeffer, Apratim Dey, Matthias Gerstgrasser, Rafael Rafailov, David L Donoho, and Sanmi Koyejo. 2024. Collapse or thrive? perils and promises of synthetic data in a self-generating world. arXiv preprint arXiv:2410.16713. — [arXiv:2410.16713](https://arxiv.org/abs/2410.16713) find
  25. Saeed Khaki, JinJin Li, Lan Ma, Liu Yang, and Prathap Ramachandra. 2024. Rs-dpo: A hybrid rejection sampling and direct preference optimization method for alignment of large language models. arXiv preprint arXiv:2402.10038. find
  26. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  27. Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, and 1 others. 2024a. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13:9. find
  28. Junlong Li, Fan Zhou, Shichao Sun, Yikai Zhang, Hai Zhao, and Pengfei Liu. 2024b. Dissecting human and llm preferences. arXiv preprint arXiv:2402.11296. find
  29. Peiji Li, Kai Lv, Yunfan Shao, Yichuan Ma, Linyang Li, Xiaoqing Zheng, Xipeng Qiu, and Qipeng Guo. 2025. Fastmcts: A simple sampling strategy for data synthesis. arXiv preprint arXiv:2502.11476.
  30. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  31. Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. 2023. Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. arXiv preprint arXiv:2308.05374.
  32. Nestor Maslej, Loredana Fattorini, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Helen Ngo, Juan Carlos Niebles, Vanessa Parli, and 1 others. 2023. Artificial intelligence index report 2023. arXiv preprint arXiv:2310.03715. — [arXiv:2310.03715](https://arxiv.org/abs/2310.03715) find
  33. Team Olmo, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Soldaini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, Pete Walsh, and 49 others. 20… — Olmo 3
  34. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  35. Jinlong Pang, Zhaowei Zhu, Na Di, Yichi Zhang, Yaxuan Wang, Chen Qian, and Yang Liu. 2025. Supervised fine-tuning on ambiguous preference pairs boosts llm alignment. find
  36. Juan Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt. 2020. Performative prediction. In International Conference on Machine Learning, pages 7599–7609. PMLR. find
  37. Georgios Piliouras and Fang-Yi Yu. 2023. Multi-agent performative prediction: From global stability and optimality to chaos. In Proceedings of the 24th ACM Conference on Economics and Computation, pages 1047–1074. find
  38. Rafael Rafailov, Arƒchit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  39. Luca Rettenberger, Markus Reischl, and Mark Schutera. 2025. Assessing political bias in large language models. Journal of Computational Social Science, 8(2):1– 17. find
  40. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156.
  41. Ze Wang, Zekun Wu, Jeremy Zhang, Navya Jain, Xin Guan, and Adriano Koshiyama. 2024c. Bias amplification: Language models as increasingly biased media. arXiv preprint arXiv:2410.15234. — [arXiv:2410.15234](https://arxiv.org/abs/2410.15234) find
  42. Mohamed El Amine Seddik, Suei-Wen Chen, Soufiane Hayou, Pierre Youssef, and Merouane Debbah. 2024. How bad is training on synthetic data? a statistical analysis of language model collapse. arXiv preprint arXiv:2404.05090. — [arXiv:2404.05090](https://arxiv.org/abs/2404.05090) find
  43. Xiukun Wei and Xueru Zhang. 2025. Self-consuming generative models with adversarially curated data. arXiv preprint arXiv:2505.09768. — [arXiv:2505.09768](https://arxiv.org/abs/2505.09768) find
  44. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. 2023. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493.
  45. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. Ai models collapse when trained on recursively generated data. Nature, 631(8022):755–759. find
  46. Qwen Team. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. find
  47. Ludovic Terren and Rosa Borge. 2021. Echo chambers on social media: A systematic review of the literature. find
  48. Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. 2024. Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving. Advances in Neural Information Processing Systems, 37:7821–7846. find
  49. Sierra Wyllie, Ilia Shumailov, and Nicolas Papernot. 2024. Fairness feedback loops: training on synthetic data amplifies bias. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 2113–2147. find
  50. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, and 1 others. 2024. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122. — Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  51. Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. 2023. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825.
  52. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
  53. Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman. 2024. Openmathinstruct-1: A 1.8 million math instruction tuning dataset. Advances in Neural Information Processing Systems, 37:34737–34774. find
  54. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  55. Maorong Wang, Nicolas Michel, Jiafeng Mao, and Toshihiko Yamasaki. 2024a. Dealing with synthetic data contamination in online continual learning. arXiv preprint arXiv:2411.13852. — [arXiv:2411.13852](https://arxiv.org/abs/2411.13852) find
  56. Yaxuan Wang, Chris Yuhao Liu, Quan Liu, Jinglong Pang, Wei Wei, Yujia Bao, and Yang Liu. 2025. Dragon: Guard llm unlearning in context via negative detection and reasoning. arXiv preprint arXiv:2511.05784. — [arXiv:2511.05784](https://arxiv.org/abs/2511.05784) find
  57. Ze Wang, Zekun Wu, Xin Guan, Michael Thaler, Adriano Koshiyama, Skylar Lu, Sachin Beepath, Ediz Ertekin Jr, and Maria Perez-Ortiz. 2024b. Job-fair: A framework for benchmarking gender hiring bias in large language models. arXiv preprint arXiv:2406.15484. — [arXiv:2406.15484](https://arxiv.org/abs/2406.15484) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-observations-remedies-large-language-model/