Elamur Research⌕
☰
home / papers / 2026 / Prior Beliefs Prejudice LLM-as-Judge: Evidence from Persuasion Evaluation

Prior Beliefs Prejudice LLM-as-Judge: Evidence from Persuasion Evaluation

Paper · 2026 · Pardis Sadat Zahraei, Xiaoning Wang, Nimet Beyza Bozdag et al.

Summary

The paper asks whether LLMs used as evaluators can separate how well an argument is made from whether they agree with it. Using persuasiveness rating as a test case, the authors show a failure they call "prior prejudice." Models that have absorbed beliefs during alignment give high scores to unsupported claims they agree with and low scores to carefully built arguments they reject. This happens even when the prompt tells them to ignore agreement. To study this, they built ConvinceQA, a dataset in which each claim has arguments both for and against it. It covers subjective topics, harmful stereotypes and misinformation. They scored more than ten open and closed models with a logit-based metric across four prompt templates.

The bias is strong where safety training creates firm priors and largely absent for subjective claims. The authors also turn the effect into a tool, persuasion-as-probe, which uses minimal pairs to reveal preferences that models refuse to state directly. The effect carries over to essay grading and debate judging, which matters for anyone relying on LLM-as-judge pipelines.

Key points

  • ConvinceQA: 27,756 arguments over 1,263 claims (348 harmful, 473 subjective, 442 misinformation).
    • Each claim has a 2×2 set of variants: supporting or opposing stance, at standard or strong intensity.
    • The evaluation used 5,052 arguments generated by DeepSeek v3.2.
  • Models and metric: the judges included GPT-4o Mini, Gemma 2 9B-it, several OLMo 3/3.1 variants, Llama 3.1 8B Instruct, Tulu 3 8B, Qwen 2.5/3 and Aya Expanse 8B.
    • Scores were measured with the Confidence-Weighted Average (CWA), computed over the 1–7 rating probabilities.
  • Size of the bias: bare assertions that match the model's beliefs received roughly 6–7 out of 7, while well-written opposing arguments received about 1–2.
  • Failure modes: analysis of reasoning traces found three failure modes.
    • Belief-conditioned rating inflation accounts for 88% of cases.
    • Asking models to explain their reasoning sometimes made the bias worse.
  • Supporting evidence:
    • A fine-tuning experiment on a "model organism" gave causal evidence.
    • An audit of the safety datasets WildJailbreak and WildGuardMix found directional asymmetries that match the probing results.
    • A human study covered 200 claims with 3 raters (weighted κ = 0.43).
    • The work is English-only, so it is untested for Ukrainian.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Illinois Urbana-ChampaignUrbana · United States184 works · 69,372 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Social Sciences · 1,772 worksSafety Research · 543 worksPolitical Science and International Relations · 182 worksLaw · 77 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Pardis Sadat Zahraei, Xiaoning Wang, Nimet Beyza Bozdag, Gokhan Tur and Dilek Hakkani-Tür. 2026. Prior Beliefs Prejudice LLM-as-Judge: Evidence from Persuasion Evaluation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.2087

@inproceedings{zahraei2026prior,
  title = {{Prior Beliefs Prejudice LLM-as-Judge: Evidence from Persuasion Evaluation}},
  author = {Pardis Sadat Zahraei, Xiaoning Wang, Nimet Beyza Bozdag, Gokhan Tur and Dilek Hakkani-Tür},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2087},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2087}
}

References

39 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nimet Beyza Bozdag, Shuhaib Mehri, Xiaocheng Yang, Hyeonjeong Ha, Zirui Cheng, Esin Durmus, Jiaxuan You, Heng Ji, Gokhan Tur, and Dilek Hakkani-Tür. 2025. Must read: A systematic survey of computational persuasion. Preprint, arXiv:2505.07775. — [arXiv:2505.07775](https://arxiv.org/abs/2505.07775) find
  2. Angelica Chen, Sadhika Malladi, Lily H. Zhang, Xinyi Chen, Qiuyi Zhang, Rajesh Ranganath, and Kyunghyun Cho. 2024. Preference learning algorithms do not learn preference rankings. Preprint, arXiv:2405.19534. find
  3. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3416 other…
  4. Scott A. Crossley, Yixin Tian, Prince Baffour, Austin Franklin, Michael Benner, and Ulrich Boser. 2024. find
  5. A large-scale corpus for assessing written argumentation: PERSUADE 2.0. Assessing Writing, 61:100865. find
  6. Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. 2025. Or-bench: An over-refusal benchmark for large language models. Preprint, arXiv:2405.20947.
  7. John Dang, Shivalika Singh, Daniel D’souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, Sandra Kublik, Meor Amer, Viraat Aryabumi, Jon Ander Campos, Yi-Chern Tan, Tom Kocmi, Florian Strub, Nathan Grinsztajn, Yannis Flet-Berliac,… — Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier
  8. DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, and 245 others. 2025. Deepseek-v3.2: Pushing the frontier of open…
  9. Esin Durmus, Liane Lovitt, Alex Tamkin, Stuart Ritchie, Jack Clark, and Deep Ganguli. 2024. Measuring the persuasiveness of language models. find
  10. Michel Frising and Daniel Balcells. 2025. Linear personality probing and steering in llms: A big five study. Preprint, arXiv:2512.17639. — [arXiv:2512.17639](https://arxiv.org/abs/2512.17639) find
  11. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  12. Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, Samuel R. Bowman, and Evan… find
  13. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. 2025. A survey on llm-as-a-judge. Preprint, arXiv:2411.15594. find
  14. Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. 2024. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. Preprint, arXiv:2406.18495.
  15. Rom Himelstein, Amit LeVi, Brit Youngmann, Yaniv Nemcovsky, and Avi Mendelson. 2025. Silenced biases: The dark side llms learned to refuse. Preprint, arXiv:2511.03369. — [arXiv:2511.03369](https://arxiv.org/abs/2511.03369) find
  16. Tao Huang. 2025. Content moderation by llm: From accuracy to legitimacy. Preprint, arXiv:2409.03219. — [arXiv:2409.03219](https://arxiv.org/abs/2409.03219) find
  17. Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri. 2024. Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. Preprint, arXiv:2406.18510.
  18. Heehyeon Kim, Kyeongryul Lee, and Joyce Jiyoung Whang. 2025. Beneath the facade: Probing safety vulnerabilities in LLMs via auto-generated jailbreak prompts. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 17668– 17700, Suzhou, China. Association for Computational Lin… find
  19. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, and 4 other… — Tulu 3: Pushing Frontiers in Open Language Model Post-Training
  20. Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. 2025. From generation to judgment: Opportunities and challenges of llm-as-a-judge. Preprint, arXiv:2411.16594. find
  21. Zongjie Li, Chaozheng Wang, Pingchuan Ma, Daoyuan Wu, Shuai Wang, Cuiyun Gao, and Yang Liu. 2024. Split and merge: Aligning position biases in LLM-based evaluators. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11084–11108, Miami, Florida, USA. Asso… find
  22. Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. Stereoset: Measuring stereotypical bias in pretrained language models. Preprint, arXiv:2004.09456.
  23. Team Olmo, :, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Soldaini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, and 50 others. 2025. Olmo… — Olmo 3
  24. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  25. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman. 2022. Bbq: A hand-built bias benchmark for question answering. Preprint, arXiv:2110.08193.
  26. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  27. Reza Sanayei, Srdjan Vesic, Eduardo Blanco, and Mihai Surdeanu. 2025. Can llms judge debates? evaluating non-linear reasoning via argumentation theory semantics. Preprint, arXiv:2509.15739. find
  28. Somesh Singh, Yaman K Singla, Harini SI, and Balaji Krishnamurthy. 2024. Measuring and improving persuasiveness of large language models. Preprint, arXiv:2410.02653. — [arXiv:2410.02653](https://arxiv.org/abs/2410.02653) find
  29. Evangelia Spiliopoulou, Riccardo Fogliato, Hanna Burnsky, Tamer Soliman, Jie Ma, Graham Horwood, and Miguel Ballesteros. 2025. Play favorites: A statistical method to measure self-bias in llm-as-a-judge. Preprint, arXiv:2508.06709. — [arXiv:2508.06709](https://arxiv.org/abs/2508.06709) find
  30. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  31. Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. 2025a. Reasoning models know when they’re right: Probing hidden states for self-verification. Preprint, arXiv:2504.05419. find
  32. Junbo Zhang, Ran Chen, Qianli Zhou, Xinyang Deng, and Wen Jiang. 2025b. Understanding and mitigating over-refusal for large language models via safety representation. Preprint, arXiv:2511.19009. — [arXiv:2511.19009](https://arxiv.org/abs/2511.19009) find
  33. Eric Zhao, Jessica Dai, and Pranjal Awasthi. 2025. The limits of preference data for post-training. Preprint, arXiv:2505.19964. — [arXiv:2505.19964](https://arxiv.org/abs/2505.19964) find
  34. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Preprint, arXiv:2306.05685.
  35. Noy Sternlicht, Ariel Gera, Roy Bar-Haim, Tom Hope, and Noam Slonim. 2025. Debatable intelligence: Benchmarking llm judges via debate speech evaluation. Preprint, arXiv:2506.05062. find
  36. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web, WWW ’16, page 613–624. International World Wi… find
  37. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179… find
  38. Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2020. Persuasion for good: Towards a personalized persuasive dialogue system for social good. Preprint, arXiv:1906.06725.
  39. Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri. 2025. Self-preference bias in llm-as-a-judge. Preprint, arXiv:2410.21819.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zahraei-2026-prior-beliefs-prejudice-llm-as/