Elamur Research⌕
☰
home / papers / 2026 / Learning Multilingual Agentic Policy to Control Sycophancy

Learning Multilingual Agentic Policy to Control Sycophancy

Paper · 2026 · Leonardo Ranaldi and Giulia Pucci

Summary

The paper addresses sycophancy, where LLMs side with a user's stated or implied beliefs even when those beliefs are factually wrong or poorly reasoned. The authors argue that this is a failure of decision-making policy, not a surface artefact of generation, and they recast it as an explicit choice among interaction strategies. In a self-play setup, a user simulator issues either neutral or sycophancy-inducing queries. At each turn the assistant chooses one of three actions: #CLARIFY, #ALIGN or #COUNTER. Training uses Reinforced Self-Training against a cost-regularised, multi-objective reward. Control coefficients set the cost of each action and how much objectivity matters, and they are sampled over a grid during training. At inference, a meta-policy predicts these coefficients directly from the context. The method was tested on three models, 11 languages and several task types: multiple-choice, verifiable open-ended and stance-based. It cuts sycophancy while keeping or improving accuracy, with gains in maths and general QA. It also behaves consistently across languages, which suggests that agreement control can be learned as a transferable skill.

Key points

  • The action space is {#CLARIFY, #ALIGN, #COUNTER}, governed by coefficients λ = (α, β, γ, η). These cover the costs of clarifying, countering and aligning, plus answer fidelity/objectivity. A meta-policy π_meta infers λ at test time, so no grid search is needed.
  • On Llama-3-8B, cross-language action match across all languages was 5% for the base model and 52% for the agentic policy. Joint Agreement Rate (sycophancy) fell from 65% to 8%.
  • For the English–Swahili pair, action match rose from 15% to 68% and Joint AR dropped from 91% to 24%. For English–Italian, match went from 34% to 88% and Joint AR from 78% to 12%.
  • On MULTI-Q, a stance benchmark with no gold answer, the policy asks for clarification most often in its first reply and less in its second. The Agreement Rate also goes down.
  • The authors note two limitations: a fixed interaction horizon, and possible noise from the model-based simulated users and sycophancy judges.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom359 works · 77,352 citations🎓University of AberdeenAberdeen · United Kingdom20 works · 1,385 citations🎓University of Rome Tor VergataRome · Italy15 works · 1,321 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Leonardo Ranaldi and Giulia Pucci. 2026. Learning Multilingual Agentic Policy to Control Sycophancy. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.169

@inproceedings{ranaldi2026learning,
  title = {{Learning Multilingual Agentic Policy to Control Sycophancy}},
  author = {Leonardo Ranaldi and Giulia Pucci},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.169},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.169}
}

References

27 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohammad Beigi, Ying Shen, Parshin Shojaee, Qifan Wang, Zichao Wang, Chandan K. Reddy, Ming Jin, and Lifu Huang. 2025. Sycophancy mitigation through reinforcement learning with uncertainty-aware adaptive reasoning trajectories. In Proceedings of the 2025 Conference on Empirical Methods in Natural La… find
  2. Runjin Chen, Zhenyu Zhang, Junyuan Hong, Souvik Kundu, and Zhangyang Wang. 2025. Seal: Steerable reasoning calibration of large language models for free. Preprint, arXiv:2504.07986. — [arXiv:2504.07986](https://arxiv.org/abs/2504.07986) find
  3. Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. 2024. Self-play fine-tuning converts weak language models to strong language models. Preprint, arXiv:2401.01335. find
  4. Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile Van Krieken, and Pasquale Minervini. 2025. Are we done wit… find
  5. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Onur Çelebi, Licheng Yu, Liron Moshkovich, and Zhiyu Ma. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  6. Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. 2023. Reinforced self-training (rest) for language modeling. Preprint, a…
  7. Eduard H. Hovy. 1988. Two types of planning in language generation. In 26th Annual Meeting of the Association for Computational Linguistics, pages 179–186, Buffalo, New York, USA. Association for Computational Linguistics. find
  8. Jingyu Hu, Shu Yang, Xilin Gong, Hongming Wang, Weiru Liu, and Di Wang. 2025. Monica: Real-time monitoring and calibration of chain-of-thought sycophancy in large reasoning models. Preprint, arXiv:2511.06419. — [arXiv:2511.06419](https://arxiv.org/abs/2511.06419) find
  9. Jessy Lin, Nicholas Tomlin, Jacob Andreas, and Jason Eisner. 2024. Decision-oriented dialogue for human-AI collaboration. Transactions of the Association for Computational Linguistics, 12:892–911.
  10. Erik Miehling, Michael Desmond, Karthikeyan Natesan Ramamurthy, Elizabeth M. Daly, Kush R. Varshney, Eitan Farchi, Pierre Dognin, Jesus Rios, Djallel Bouneffouf, Miao Liu, and Prasanna Sattigeri. 2025. Evaluating the prompt steerability of large language models. In Proceedings of the 2025 Conference… find
  11. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Ben Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, and 44 others. 2022. Disco…
  12. Priya Pitre, Naren Ramakrishnan, and Xuan Wang. 2025. CONSENSAGENT: Towards efficient and effective consensus in multi-agent LLM interactions through sycophancy mitigation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 22112–22133, Vienna, Austria. Association for Com… find
  13. Giulia Pucci and Leonardo Ranaldi. 2025. Advancing oversight reasoning across languages for audit sycophantic behaviour via X-agent. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 12949–12965, Suzhou, China. Association for Computational Linguistics. find
  14. Leonardo Ranaldi and Andre Freitas. 2024a. Aligning large and small language models via chain-of-thought reasoning. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1812–1827, St. Julian’s, Malta. Associatio…
  15. Leonardo Ranaldi and Andre Freitas. 2024b. Self-refine instruction-tuning for aligning reasoning in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2325–2347, Miami, Florida, USA. Association for Computational Linguistics.
  16. Leonardo Ranaldi, Barry Haddow, and Alexandra Birch. 2025a. When natural language is not enough: The limits of in-context learning demonstrations in multilingual reasoning. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 7369–7396, Albuquerque, New Mexico. Association… find
  17. Leonardo Ranaldi and Giulia Pucci. 2025a. Multilingual reasoning via self-training. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 11566–11582, Albuquerque, New… find
  18. Leonardo Ranaldi and Giulia Pucci. 2025b. When large language models contradict humans? large language models’ sycophantic behaviour. Preprint, arXiv:2311.09410. — [arXiv:2311.09410](https://arxiv.org/abs/2311.09410) find
  19. Leonardo Ranaldi, Giulia Pucci, Barry Haddow, and Alexandra Birch. 2024a. Empowering multi-step reasoning across languages via program-aided language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12171–12187, Miami, Florida, USA. Association… find
  20. Leonardo Ranaldi, Federico Ranaldi, Fabio Massimo Zanzotto, Barry Haddow, and Alexandra Birch. 2025c. Improving multilingual retrieval-augmented language models through dialectic reasoning argumentations. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, page…
  21. Swarnadeep Saha, Xian Li, Marjan Ghazvininejad, Jason Weston, and Tianlu Wang. 2025. Learning to plan reason for evaluation with thinking-llm-as-a-judge. Preprint, arXiv:2501.18099. — [arXiv:2501.18099](https://arxiv.org/abs/2501.18099) find
  22. Omar Shaikh, Kristina Gligoric, Ashna Khetan, Matthias Gerstgrasser, Diyi Yang, and Dan Jurafsky. 2024. Grounding gaps in language model generations. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Vol… find
  23. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. 2023. T… — Towards Understanding Sycophancy in Language Models
  24. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, and Ramona Merhej. 2025. Gemma 3 technical report. Preprint, arXiv:2503.19786. find
  25. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  26. Leonardo Ranaldi, Giulia Pucci, Federico Ranaldi, Elena Sofia Ruzzetti, and Fabio Massimo Zanzotto. 2024b. A tree-of-thoughts to broaden multi-step reasoning across languages. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1229–1241, Mexico City, Mexico. Association… find
  27. Leonardo Ranaldi, Federico Ranaldi, and Giulia Pucci. 2025b. R2-MultiOmnia: Leading multilingual multimodal reasoning via self-training. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ranaldi-2026-learning-multilingual-agentic-policy-control/