Elamur Research⌕
☰
home / papers / 2026 / User-Assistant Bias in LLMs

User-Assistant Bias in LLMs

Paper · 2026 · Xu Pan, Jingxuan Fan, Zidi Xiong et al.

Summary

Chat LLMs split their context into role-tagged segments such as system, user, assistant and tool. This paper asks whether the tags themselves make a model weigh information unequally. The authors define user–assistant bias as follows: when the user and the assistant earlier gave contradictory attributes for the same entity, which side does the model trust when answering?

Prior sycophancy and stubbornness studies mix in confounds like world knowledge, social norms and unequal amounts of evidence. To avoid these, the authors built USERASSIST, a synthetic, task-agnostic benchmark with order-balanced conflicts, and tested 52 models. Most instruction-tuned models lean strongly toward the user, while base and reasoning models stay close to neutral.

Controlled fine-tuning shows that human-preference alignment shifts the bias toward the user, and training on reasoning traces reduces it. Lightweight DPO on USERASSIST-TRAIN can push the bias in either direction, and the change transfers to two naturalistic debate datasets.

The work gives practitioners a way to diagnose and adjust role-tag effects. This bears on sycophancy, stubbornness and AI safety, since the right bias depends on whether the model acts as a fact-checker or a supportive companion.

Key points

  • USERASSIST benchmark. It has two subsets:
    • symbol-value: letters receive numbers from 0 to 100;
    • object-color: objects receive colors.
    • Turn order is counterbalanced to cancel position effects.
    • USERASSIST-TEST contains 1946 symbol-value conversations (1–5 turns) and 1042 object-color conversations (1–3 turns).
    • USERASSIST-TRAIN contains 3001 and 2015 conversations respectively.
    • A realistic symbol-value variant rewrites the assignments as everyday numeric claims.
  • Models evaluated. 26 commercial models were tested via API and 26 open-weight models locally. Open-weight models were scored both by answer-probability log ratios and by generated answers.
  • Where the bias comes from. Preference alignment increases user bias, while reasoning fine-tuning decreases it. The authors warn that heavy reasoning tuning could therefore push models toward assistant-biased behavior.
  • Transfer to realistic debates. DPO-induced bias carried over to two datasets:
    • a 1848-conversation philosophical debate set derived from Perez et al. (2023);
    • a second debate set drawn from Salvi et al.'s (2025) DebateGPT data, where gpt-oss-120b judged the final stances.
    • The effect held across Llama and Qwen families and several model sizes, and assistant-directed training even flipped the bias for Llama-3.1-8b-instruct on the philosophical set.
  • Limitations. The setup is deliberately synthetic, the realistic evaluation covers a narrow range of domains, and the LLM-as-judge scoring may inherit the judge model's own biases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Xu Pan, Jingxuan Fan, Zidi Xiong, Ely Hahami, Jorin Overwiening and Ziqian Xie. 2026. User-Assistant Bias in LLMs. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.449

@inproceedings{pan2026user,
  title = {{User-Assistant Bias in LLMs}},
  author = {Xu Pan, Jingxuan Fan, Zidi Xiong, Ely Hahami, Jorin Overwiening and Ziqian Xie},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.449},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.449}
}

References

36 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova Dassarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Thomas Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, and 12 others. 202… find
  2. Emilio Barkett, Olivia Long, and Madhavendra Thakur. 2025. Reasoning isn’t enough: Examining truth-bias and sycophancy in llms. arXiv preprint arXiv:2506.21561. — [arXiv:2506.21561](https://arxiv.org/abs/2506.21561) find
  3. Jan Batzner, Volker Stocker, Stefan Schmid, and Gjergji Kasneci. 2024. Germanpartiesqa: Benchmarking commercial large language models for political bias and sycophancy. arXiv preprint arXiv:2407.18008. — [arXiv:2407.18008](https://arxiv.org/abs/2407.18008) find
  4. Hwan Chang, Yonghyun Jun, and Hwanhee Lee. 2025. Chatinject: Abusing chat templates for prompt injection in llm agents. arXiv preprint arXiv:2509.22830. — [arXiv:2509.22830](https://arxiv.org/abs/2509.22830) find
  5. Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, and Dan Jurafsky. 2025. Social sycophancy: A broader understanding of llm sycophancy. arXiv preprint arXiv:2505.13995.
  6. Javier Chiyah-Garcia, Alessandro Suglia, and Arash Eshghi. 2024. Repairs in a block world: A new benchmark for handling user corrections with multi-modal language models. arXiv preprint arXiv:2409.14247. find
  7. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guo Tong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2023. Ultrafeedback: Boosting language models with scaled ai feedback. In International Conference on Machine Learning. find
  8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  9. Aaron Fanous, Jacob Goldberg, Ank A Agarwal, Joanna Lin, Anson Zhou, Roxana Daneshjou, and Sanmi Koyejo. 2025. Syceval: Evaluating llm sycophancy. arXiv preprint arXiv:2502.08177. — [arXiv:2502.08177](https://arxiv.org/abs/2502.08177) find
  10. Jia He, Mukund Rungta, David Koleczek, Arshdeep Sekhon, Franklin X Wang, and Sadid Hasan. 2024. Does prompt formatting have any impact on llm performance? arXiv preprint arXiv:2411.10541. — [arXiv:2411.10541](https://arxiv.org/abs/2411.10541) find
  11. Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, and Yahui Zhou. 2025. Skywork open reasoner 1 technical report. Preprint, arXiv:2505.22312. find
  12. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798.
  13. Dongwei Jiang, Alvin Zhang, Andrew Wang, Nicholas Andrews, and Daniel Khashabi. 2025a. Feedback friction: Llms struggle to fully incorporate external feedback. arXiv preprint arXiv:2506.11930. — [arXiv:2506.11930](https://arxiv.org/abs/2506.11930) find
  14. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Bill Yuchen Lin, and Radha Poovendran. 2025b. Chatbug: A common vulnerability of aligned llms induced by chat templates. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27347–27355. find
  15. Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. Llms get lost in multi-turn conversation. Preprint, arXiv:2505.06120.
  16. Ariel N Lee, Cole J Hunter, and Nataniel Ruiz. 2023. Platypus: Quick, cheap, and powerful refinement of llms. arXiv preprint arXiv:2308.07317. find
  17. Yubo Li, Xiaobin Shen, Xinyu Yao, Xueying Ding, Yidi Miao, Ramayya Krishnan, and Rema Padman. 2025. Beyond single-turn: A survey on multi-turn interactions with large language models. arXiv preprint arXiv:2504.04717. — [arXiv:2504.04717](https://arxiv.org/abs/2504.04717) find
  18. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172.
  19. Deven Mahesh Mistry, Anooshka Bajaj, Yash Aggarwal, Sahaj Singh Maini, and Zoran Tiganj. 2025. Emergence of episodic memory in transformers: Characterizing changes in temporal structure of attention scores during training. arXiv preprint arXiv:2502.06902. find
  20. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. Preprint, arXiv:2501.19393.
  21. Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, and 1 others. 2023. Discovering language model behaviors with model-written evaluations. In Findings of the association for computational linguistics:…
  22. A Yang Qwen, Baosong Yang, B Zhang, B Hui, B Zheng, B Yu, Chengpeng Li, D Liu, F Huang, H Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint. find
  23. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728– 53741.
  24. Francesco Salvi, Manoel Horta Ribeiro, Riccardo Gallotti, and Robert West. 2025. On the conversational persuasiveness of gpt-4. Nature Human Behaviour, 9(8):1645–1653. find
  25. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, and 1 others. 2024. Towards understanding sycophancy in language models. In 12th International Conference on Learning Representations, ICLR 2024.
  26. Qwen Team. 2025. Qwq-32b: Embracing the power of reinforcement learning. find
  27. Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. 2024. The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208. find
  28. Shijian Wang, Linxin Song, Jieyu Zhang, Ryotaro Shimizu, Jiarui Jin, Ao Luo, Yuan Lu, Lili Yao, Cunjian Chen, Julian J. McAuley, Wentao Zhang, and Hanqian Wu. 2024. Investigating the scaling effect of instruction templates for training multimodal language model. find
  29. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652.
  30. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. 2023. Simple synthetic data reduces sycophancy in large language models. arXiv preprint arXiv:2308.03958. find
  31. Xinyi Wu, Yifei Wang, Stefanie Jegelka, and Ali Jadbabaie. 2025. On the emergence of position bias in transformers. arXiv preprint arXiv:2502.01951. — [arXiv:2502.01951](https://arxiv.org/abs/2502.01951) find
  32. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations.
  33. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025. Limo: Less is more for reasoning. Preprint, arXiv:2502.03387.
  34. Chen Zhang, Xinyi Dai, Yaxiong Wu, Qu Yang, Yasheng Wang, Ruiming Tang, and Yong Liu. 2025a. A survey on multi-turn interaction capabilities of large language models. arXiv preprint arXiv:2501.09959. — [arXiv:2501.09959](https://arxiv.org/abs/2501.09959) find
  35. Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, and Menghan Jiang. 2025b. Iheval: Evaluating language models on following the instruction hierarchy. In North American Chapter of the Asso… find
  36. Yunpu Zhao, Rui Zhang, Junbin Xiao, Changxin Ke, Ruibo Hou, Yifan Hao, Qi Guo, and Yunji Chen. 2024. Towards analyzing and mitigating sycophancy in large vision-language models. arXiv preprint arXiv:2408.11261. — [arXiv:2408.11261](https://arxiv.org/abs/2408.11261) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pan-2026-user-assistant-bias-llms/