Elamur Research⌕
☰
home / papers / 2026 / CoopValue: Revealing LLM Value Preferences Through Multi-Agent Cooperation

CoopValue: Revealing LLM Value Preferences Through Multi-Agent Cooperation

Paper · 2026 · Zee Hen Tang and Mi-Yen Yeh

Summary

Most value evaluations of LLMs rely on single-agent dilemmas or fixed binary-choice questions, which say little about how models behave when they must cooperate with other agents. The authors propose CoopValue, a framework in which a target LLM and a counterpart LLM each champion a different Schwartz value and must agree on a joint decision. Value priority is inferred from behavior: the framework checks whether the target gives up its assigned value to reach agreement or holds firm and lets the task fail. Scenarios are generated from DailyDilemma backgrounds by three LLMs, rated by Prolific annotators, and deduplicated. The scenarios span every one of the 45 value pairings, with each framed as one of three forms of cooperation (reciprocal, coopetitive or altruistic). Testing 24 LLMs shows that value priorities shift with the cooperation type. Preferences that models state correlate only weakly with those revealed through interaction. Adaptation methods built for single agents transfer poorly to cooperative settings. The work argues that LLMs intended for multi-agent deployment should be judged by how they act in interaction, not by questionnaire answers.

Key points

  • Dataset: 1,778 English scenarios built on Schwartz's 10 values (all 45 pairwise conflicts) and three cooperation types. They were generated by GPT-4o, Claude-Sonnet-4 and Llama-3.1-70B-Instruct. Near-duplicates were removed using text-embedding-3-large with a cosine similarity threshold of 0.8.
  • Quality checks:
    • Mean Likert ratings were 4.72 for realism, 4.63 for specificity and 4.84 for conflict strength, with κ ≥ 0.82.
    • Annotators identified the intended value pair and cooperation type with over 96% accuracy.
    • LLM judges detected value concession with F1 between 0.91 and 0.98.
  • Context sensitivity: 24 models from 8 families were evaluated. For example, Gemma ranks Benevolence lower in coopetitive scenarios but higher in altruistic ones.
  • Value adaptation: Persona prompting, few-shot prompting and CultureLLM all raise low-ranked values much more in stated than in revealed rankings. CultureLLM is strongest overall but only marginally better than prompting on revealed preferences.
  • Cross-lingual comparison: Chinese and English versions of INVP and CoopValue (translated with GPT-4o) yield only weak-to-moderate ranking agreement. For GPT-5.2, Claude-Sonnet-4.5 and Gemini-3-Pro on CoopValue, Kendall's τ ranges from 0.22 to 0.39. Code and data are on GitHub under a non-commercial license.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National Taiwan UniversityTaipei · Taiwan29 works · 2,116 citations🔬Academia SinicaTaipei · Taiwan10 works · 760 citations🔬Institute of Information Science, Academia SinicaTaipei · Taiwan7 works · 51 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPsychology · 1,552 worksSafety Research · 543 worksSocial Psychology · 306 works

Cite

Cite as Zee Hen Tang and Mi-Yen Yeh. 2026. CoopValue: Revealing LLM Value Preferences Through Multi-Agent Cooperation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1887

@inproceedings{tang2026coopvalue,
  title = {{CoopValue: Revealing LLM Value Preferences Through Multi-Agent Cooperation}},
  author = {Zee Hen Tang and Mi-Yen Yeh},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1887},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1887}
}

References

57 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, and Mario Fritz. 2024. Cooperation, competition, and maliciousness: Llm-stakeholders interactive negotiation. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. Curra… find
  2. Marwa Abdulhai, Gregory Serapio-García, Clement Crepy, Daria Valter, John Canny, and Natasha Jaques. 2024. Moral foundations of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17737–17752, Miami, Florida, USA. Association for Co…
  3. Anthropic. 2025. Claude sonnet 4.5 system card. find
  4. Ralph Allan Bradley and Milton E. Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39:324. find
  5. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  6. Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, and Bingsheng He. 2025. Judgelrm: Large reasoning models as a judge. Preprint, arXiv:2504.00050. — [arXiv:2504.00050](https://arxiv.org/abs/2504.00050) find
  7. Yu Ying Chiu, Liwei Jiang, and Yejin Choi. 2025a. Dailydilemmas: Revealing value preferences of LLMs with quandaries of daily life. In The Thirteenth International Conference on Learning Representations. find
  8. Yu Ying Chiu, Zhilin Wang, Sharan Maiya, Yejin Choi, Kyle Fish, Sydney Levine, and Evan Hubinger. 2025b. Will ai tell lies to save sick children? litmus-testing ai values prioritization with airiskdilemmas. Preprint, arXiv:2505.14633. — [arXiv:2505.14633](https://arxiv.org/abs/2505.14633) find
  9. Gaston Godin, Mark T Conner, and Paschal Sheeran. 2004. Bridging the intention-behaviour ’gap’: the role of moral norm. The British journal of social psychology, 44 Pt 4:497–512. find
  10. Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean Philip Wojcik, and Peter H. Ditto. 2012. Moral foundations theory: The pragmatic validity of moral pluralism. In Advances in Experimental Social Psychology. find
  11. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  12. Saffron Huang, Esin Durmus, Miles McCain, Kunal Handa, Alex Tamkin, Jerry Hong, Michael Stern, Arushi Somani, Xiuruo Zhang, and Deep Ganguli. 2025. Values in the wild: Discovering and analyzing values in real-world language model interactions. Preprint, arXiv:2504.15236. — [arXiv:2504.15236](https://arxiv.org/abs/2504.15236) find
  13. Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2024. PersonaLLM: Investigating the ability of large language models to express personality traits. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3605–3627, Mexico City, Mexico. Associat… find
  14. Tim Ruben Davidson, Veniamin Veselovsky, Michal Kosinski, and Robert West. 2024. Evaluating language model agency through negotiations. In The Twelfth International Conference on Learning Representations. find
  15. Gregory E. Kersten, Rustam M. Vahidov, and Dmitry Gimon. 2013. Concession-making in multi-attribute auctions and multi-bilateral negotiations: Theory and experiments. Electron. Commer. Res. Appl., 12:166– 180. find
  16. Yali Du, Joel Z. Leibo, Usman Islam, Richard Willis, and Peter Sunehag. 2023. A review of cooperation in multi-agent learning. Preprint, arXiv:2312.05162. — [arXiv:2312.05162](https://arxiv.org/abs/2312.05162) find
  17. Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Michael Bean, Katerina Margatina, Rafael Mosquera, Juan Manuel Ciro, Max Bartolo, Adina Williams, He He, Bertie Vidgen, and Scott A. Hale. 2024. The PRISM alignment dataset: What participatory, representative and individualised human feedb… find
  18. Lisa Dunlap, Krishna Mandal, Trevor Darrell, Jacob Steinhardt, and Joseph E. Gonzalez. 2025. Vibecheck: Discover and quantify qualitative differences in large language models. In The Thirteenth International Conference on Learning Representations. find
  19. Esin Durmus, Karina Nguyen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. Towards measuring the…
  20. Ernst Fehr and Urs Fischbacher. 2003. The nature of human altruism. Nature, 425:785–791. find
  21. Ernst Fehr and Simon Gächter. 2002. Altruistic punishment in humans. Nature, 415:137–140. find
  22. Ayoung Lee, Ryan Sungmo Kwon, Peter Railton, and Lu Wang. 2025. Clash: Evaluating language models on judging high-stakes dilemmas from multiple perspectives. Preprint, arXiv:2504.10823. — [arXiv:2504.10823](https://arxiv.org/abs/2504.10823) find
  23. Minhye Lee, Hyun Seon Ahn, Soon Koo Kwon, and Sung-Il Kim. 2018. Cooperative and competitive contextual effects on social cognitive and empathic neural responses. In Frontiers in human neuroscience. find
  24. Cheng Li, Mengzhuo Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024. Culturellm: incorporating cultural differences into large language models. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. Curran Associates Inc.
  25. Andy Liu, Kshitish Ghate, Mona Diab, Daniel Fried, Atoosa Kasirzadeh, and Max Kleiman-Weiner. 2025a. Generative value conflicts reveal llm priorities. Preprint, arXiv:2509.25369. — [arXiv:2509.25369](https://arxiv.org/abs/2509.25369) find
  26. Xuelin Liu, Pengyuan Liu, and Dong Yu. 2025b. What’s the most important value? INVP: INvestigating the value priorities of LLMs through decision-making in social scenarios. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4725–4752, Abu Dhabi, UAE. Association… find
  27. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  28. Meta AI. 2025. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. find
  29. Nailia Mirzakhmedova, Johannes Kiesel, Milad Alshomary, Maximilian Heinrich, Nicolas Handke, Xiaoni Cai, Valentin Barriere, Doratossadat Dastgheib, Omid Ghahroodi, MohammadAli SadraeiJavaheri, Ehsaneddin Asgari, Lea Kawaletz, Henning Wachsmuth, and Benno Stein. 2024. The touché23-ValueEval dataset f… find
  30. Mistral AI. 2025. Introducing mistral 3. find
  31. OpenAI. 2025. Gpt-5 system card. find
  32. Max Pellert, Clemens M Lechner, Claudia Wagner, Beatrice Rammstedt, and Markus Strohmaier. 2024. Ai psychometrics: Assessing the psychological profiles of large language models through psychometric inventories. find
  33. Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, and 44 others. 2023. Dis… — Discovering Language Model Behaviors with Model-Written Evaluations
  34. Robert Plutchik. 1982. A psychoevolutionary theory of emotions. find
  35. Petros Raptopoulos, Giorgos Filandrianos, Maria Lymperaiou, and Giorgos Stamou. 2025. PAKTON: A multi-agent framework for question answering in long legal agreements. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7948–7984, Suzhou, China. Associatio… find
  36. Yuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang, and Guojie Song. 2024. ValueBench: Towards comprehensively evaluating value orientations and understanding of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page…
  37. Charlotte S. L. Rossetti and Christian Hilbe. 2024. Direct reciprocity among humans. find
  38. Naama Rozen, Liat Bezalel, Gal Elidan, Amir Globerson, and Ella Daniel. 2025. Do LLMs have consistent values? In The Thirteenth International Conference on Learning Representations. find
  39. Swarnadeep Saha, Xian Li, Marjan Ghazvininejad, Jason E Weston, and Tianlu Wang. 2025. Learning to plan & reason for evaluation with thinking-LLM-asa-judge. In Forty-second International Conference on Machine Learning. find
  40. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. Evaluating the moral beliefs encoded in LLMs. In Thirty-seventh Conference on Neural Information Processing Systems. find
  41. Shalom H. Schwartz. 2012. An overview of the schwartz theory of basic values. Online Readings in Psychology and Culture, 2:11. find
  42. Greg Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2025. Personality traits in large language models. Preprint, arXiv:2307.00184.
  43. Chung Shan-Shan and Monica Miu-Yin Leung. 2007. The value-action gap in waste recycling: the case of undergraduates in hong kong. find
  44. Hua Shen, Nicholas Clark, and Tanu Mitra. 2025. Mind the value-action gap: Do LLMs act in alignment with their values? In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3097–3118, Suzhou, China. Association for Computational Linguistics.
  45. Taylor Sorensen, Liwei Jiang, Jena D. Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas, and Yejin Choi. 2024. Value kaleidoscope: engaging ai with pluralistic human values, rights, and duties. In Proceedings of t…
  46. Ivan Sviridov, Amina Miftakhova, Artemiy Tereshchenko, Galina Zubkova, Pavel Blinov, and Andrey Savchenko. 2025. 3MDBench: Medical multimodal multi-agent dialogue benchmark. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 26614–26654, Suzhou, China. A… find
  47. Kumar Tanmay, Aditi Khandelwal, Utkarsh Agarwal, and Monojit Choudhury. 2023. Probing the moral development of large language models through defining issues test. Preprint, arXiv:2309.13356. — [arXiv:2309.13356](https://arxiv.org/abs/2309.13356) find
  48. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… find
  49. V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, and 74 others. 2025b. Glm-4.5v and glm-4.1v-thinking: Towards versatile multimodal rea…
  50. James Alexander Kerr Thomson. 1956. The ethics of aristotle. find
  51. Alejandro Tlaie. 2024. Exploring and steering the moral compass of large language models. Preprint, arXiv:2405.17345. — [arXiv:2405.17345](https://arxiv.org/abs/2405.17345) find
  52. Hitoshi Yamamoto and Akira Goto. 2024. Behavioural strategies in simultaneous and alternating prisoner’s dilemma games with/without voluntary participation. find
  53. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  54. Jing Yao, Xiaoyuan Yi, Yifan Gong, Xiting Wang, and Xing Xie. 2024. Value FULCRA: Mapping large language models to the multidimensional spectrum of basic human value. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… find
  55. Haoran Ye, Yuhang Xie, Yuanyi Ren, Hanjun Fang, Xin Zhang, and Guojie Song. 2025. Measuring human and ai values based on generative psychometrics with large language models. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Appl… find
  56. Yangyang Yu, Zhiyuan Yao, Haohang Li, Zhiyang Deng, Yuechen Jiang, Yupeng Cao, Zhi Chen, Jordan W. Suchow, Zhenyu Cui, Rong Liu, Zhaozhuo Xu, Denghui Zhang, Koduvayur Subbalakshmi, GUO-JUN XIONG, Yueru He, Jimin Huang, Dong Li, and Qianqian Xie. 2024. Fincon: A synthesized LLM multi-agent system wit… find
  57. Wang Zhenwu, Shen Jiayin, Tang Xiaosong, Han Mengjie, Feng Zhenhua, and Wu Jinghua. 2024. An agent-based persuasion model using emotion-driven concession and multi-objective optimization. Autonomous Agents and Multi-Agent Systems, 38(2). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tang-2026-coopvalue-revealing-llm-value-preferences/