Elamur Research⌕
☰
home / papers / 2026 / When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation

When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation

Paper · 2026 · Sy-Tuyen Ho, Minghui Liu and Furong Huang

Summary

LLM-written reviews increasingly appear in public data and may end up training the next generation of AI reviewers. This paper asks whether such recursive training narrows the range of scientific judgments a model expresses. The authors fine-tune Llama 3.1 8B on official ICLR reviews from 2018–2023. They then use it to generate reviews of ICLR 2024 papers and train four successor models. These successors differ only in how many of each paper's three reviews are synthetic. After this single recursive step, synthetic supervision compresses the rating distribution and lowers semantic diversity, both across reviews of the same paper and across the corpus. Ratings do not drift toward leniency or harshness; they become more homogeneous. The authors call this effect scientific-judgment collapse. As a remedy, they release TrustReviewer, an open-source review system. It is trained in one stage on a curated corpus of official reviews and uses paired activation steering at inference time. TrustReviewer reportedly beats the external baselines on recommendation agreement and on diversity. This matters because AI-assisted peer review could otherwise flatten scientific evaluation over time.

Key points

  • Experimental setup: The base model M0 is Llama 3.1 8B Instruct.
    • The first reviewer, M1, is trained on official ICLR reviews from 2018–2023. The year 2023 matches Llama 3.1's training-data cutoff and largely predates ChatGPT-assisted reviewing.
    • The four M2 variants are trained on ICLR 2024 data with 0%, 33%, 66%, or 100% synthetic reviews per paper.
  • Training details: Supervised fine-tuning was done in LlamaFactory with LoRA (rank 64, scaling factor 128), a 64k-token context, and three epochs. Evaluation uses a held-out set of 2,000 papers from 2018–2025.
  • Results: Moving from 0% to 100% synthetic exposure lowers same-paper semantic diversity by roughly 11% and corpus-level diversity by roughly 5%, and both declines are monotonic. Official reviews show a wider rating spread (standard deviation 1.73, entropy 2.38) than the M2 models.
  • TrustReviewer's two components:
    • At training time, it filters out low-quality and degenerate supervision.
    • At test time, it builds a steering direction by contrasting official and model-generated reviews of the same paper. This step needs no retraining and no extra expert annotation.
  • Released resources and limitations: The dataset is on Hugging Face (hosytuyen/trustreviewer) and the code is on GitHub (hosytuyen/TrustReviewer). Whether the collapse compounds over multiple generations was not tested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Maryland, College ParkCollege Park · United States151 works · 29,733 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksSocial Sciences · 2,009 worksInformation Systems · 1,888 worksSafety Research · 665 works

Cite

Cite as Sy-Tuyen Ho, Minghui Liu and Furong Huang. 2026. When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation. arXiv:2609.20942.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2609.20942 · arXiv 2609.20942

@misc{ho2026when,
  title = {{When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation}},
  author = {Sy-Tuyen Ho, Minghui Liu and Furong Huang},
  year = {2026},
  doi = {10.48550/arxiv.2609.20942},
  eprint = {2609.20942},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2609.20942}
}

References

30 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard Baraniuk. Self-consuming generative models go mad. In International Conference on Learning Representations, volume 2024, pages 53581–53608, 2024. find
  2. Andy Arditi, Oscar Balcells Obeso, Aaquib Syed, Daniel Paleka, Nina Rimsky, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=pH3XAQME6c… ↗
  3. Luca Demetrio, Giovanni Apruzzese, Kathrin Grosse, Pavel Laskov, Emil Lupu, Vera Rimmer, and Philine Widmer. Gen-review: A large-scale dataset of ai-generated (and human-written) peer reviews. arXiv preprint arXiv:2510.21192, 2025. find
  4. Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, and Julia Kempe. A tale of tails: Model collapse as a change of scaling laws. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=KVvku47shW . ↗
  5. Alexander M Fichtl, Lukas Ellinger, Josefin Kelber, Kryštof Olík, and Georg Groh. Ai-assisted peer review across research communities: From reviewer ai policies to llm review quality. arXiv preprint arXiv:2608.03581, 2026. — [arXiv:2608.03581](https://arxiv.org/abs/2608.03581) find
  6. Shi Fu, Yingjie Wang, Yuzhu Chen, Xinmei Tian, and Dacheng Tao. A theoretical perspective: How to prevent model collapse in self-consuming training loops. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=WttfQGwpES . ↗
  7. Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey, Rafael Rafailov, Henry Sleight, John Hughes, Tomasz Korbak, Rajashree Agrawal, Dhruv Pai, Andrey Gromov, et al. Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data. arXiv preprint arXiv:2404.01413,… find
  8. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
  9. Maximilian Idahl and Zahra Ahmadi. Openreviewer: A specialized large language model for generating critical scientific paper reviews. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (System Dem… find
  10. Minjae Kang and Jaehyung Kim. Enhancing instruction following of LLMs via activation steering with dynamic rejection. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=OpuPBNcQwe . ↗
  11. Grgur Kovač, Jérémy Perez, Rémy Portelas, Peter Ford Dominey, and Pierre-Yves Oudeyer. Recursive training loops in llms: How training data properties modulate distribution shift in generated data? In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 32278–… find
  12. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. Advances in neural information processing systems, 36:41451–41530, 2023.
  13. Ruochi Li, Haoxuan Zhang, Edward Gehringer, Ting Xiao, Junhua Ding, and Haihua Chen. Unveiling the merits and defects of llms in automatic review generation for scientific papers. In 2025 IEEE International Conference on Data Mining (ICDM), pages 1370–1379. find
  14. Zhuofeng Li, Yi Lu, Dongfu Jiang, Haoxiang Zhang, Yuyang Bai, Chuan Li, Yu Wang, Shuiwang Ji, Jianwen Xie, and Yu Zhang. Reviewgrounder: Improving review substantiveness with rubric-guided, tool-integrated agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguis… find
  15. Weixin Liang, Zachary Izzo, Yaohui Zhang, Haley Lepp, Hancheng Cao, Xuandong Zhao, Lingjiao Chen, Haotian Ye, Sheng Liu, Zhi Huang, et al. Monitoring ai-modified content at scale: A case study on the impact of chatgpt on ai conference peer reviews. arXiv preprint arXiv:2403.07183, 2024.
  16. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. In International Conference on Learning Representations, volume 2024, pages 22353–22373, 2024.
  17. Meta. Llama 3.1 8b. https://huggingface.co/meta-llama/Llama-3.1-8B , 2024. Hugging Face model repository, accessed July 21, 2026. ↗
  18. Qwen Team. Qwen3.6-35B-A3B: Agentic coding power, now open to all, April 2026. URL https://qwen.ai/blog?id=qwen3.6-35b-a3b . ↗
  19. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504–15522, 2024. find
  20. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493, 2023.
  21. Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, et al. jina-embeddings-v3: Multilingual embeddings with task lora. arXiv preprint arXiv:2409.10173, 2024.
  22. Cheng Tan, Dongxin Lyu, Siyuan Li, Zhangyang Gao, Jingxuan Wei, Siqi Ma, Zicheng Liu, and Stan Z Li. Peer review as a multi-turn and long-context dialogue with role-based interactions. arXiv preprint arXiv:2406.05688, 2024. — [arXiv:2406.05688](https://arxiv.org/abs/2406.05688) find
  23. Nitya Thakkar, Mert Yuksekgonul, Jake Silberg, Animesh Garg, Nanyun Peng, Fei Sha, Rose Yu, Carl Vondrick, and James Zou. Can llm feedback enhance review quality? a randomized study of 20k reviews at iclr 2025. arXiv preprint arXiv:2504.09737, 2025.
  24. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2023.
  25. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangko…
  26. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36: 55006–55021, 2023. find
  27. Ruiyang Zhou, Lu Chen, and Kai Yu. Is llm a reliable reviewer? a comprehensive evaluation of llm on automatic paper reviewing tasks. In Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2024), pages 9340–9351, 2024.
  28. Xuekai Zhu, Daixuan Cheng, Hengli Li, Kaiyan Zhang, Ermo Hua, Xingtai Lv, Ning Ding, Zhouhan Lin, Zilong Zheng, and Bowen Zhou. How to synthesize text data without model collapse? arXiv preprint arXiv:2412.14689, 2024. — [arXiv:2412.14689](https://arxiv.org/abs/2412.14689) find
  29. Zhenzhen Zhuang, Jiandong Chen, Hongfeng Xu, Yuwen Jiang, and Jialiang Lin. Large language models for automated scholarly paper review: A survey. Information Fusion, 124:103332, 2025. find
  30. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405, 2023.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ho-2026-ai-reviews-train-ai/