Elamur Research⌕
☰
home / papers / 2025 / ALYMPICS: LLM Agents Meet Game Theory

ALYMPICS: LLM Agents Meet Game Theory

Paper · 2025 · Shaoguang Mao, Yuzhe Cai, Yan Xia et al.

Summary

Empirical game theory usually needs experiments with human participants, which are costly, slow and ethically demanding. This paper proposes Alympics ("Olympics for Agents"), an open-source framework where LLM-driven agents act as stand-ins for people in strategic games. The framework has three parts. A Sandbox Playground holds the rule code, the history logs and adjustable parameters. Agent Players combine an LLM with memory, a reasoning plugin and a persona. Human Players can optionally join. The authors illustrate the framework with the "Water Allocation Challenge," a multi-round sealed-bid auction for a scarce survival resource that merges ideas from auctions, repeated and dynamic games, and unequal competition. By varying the water supply and agent personas, they analyze what drives bids, strategies and survival. The simulated bidding was largely consistent with predictions from Auction Theory, and a winner's-curse effect appeared. Human judges rated agent play roughly on par with human self-assessments. This suggests LLM agents could serve as a cheap, reproducible testbed for game-theoretic hypotheses, although persona simulation remains weak.

Key points

  • Game design: Five agents must survive 20 days, a length that can be adjusted. Each has a different water need and salary and up to 10 health points, starting at 8. Health drops by k after k consecutive days without water and rises by 2 when the need is met. Daily supply is always below total demand. Water goes to the highest bidders, and ties favor players with smaller needs.
  • Experimental scale: Six settings were run, each repeated 10 times, for 60 games in total. Plotting minimum successful bids showed overpaying in tightly contested auctions (winner's curse).
  • Human evaluation setup: Judges rated 30 sampled game records, split evenly between agents with and without personas, with five judges per record. Ratings used a 1–5 scale covering IU, LR, SE, AD and LP, plus IA for agents with personas.
  • Evaluation results: Agents beat human self-ratings on long-term planning (3.88 vs 3.40 average). They scored lower on information utilization (3.33 vs 3.60) and adaptability (3.42 vs 3.70). Identity Alignment varied widely (3.51 average, standard deviation 1.24).
  • Released resources: Code and prompts are public. Later work has used the framework to study Keynes' Beauty Contest and negotiation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Shaoguang Mao, Yuzhe Cai, Yan Xia, Wenshan Wu, Xun Wang, Fengyi Wang, Qiang Guan, Tao Ge and Furu Wei. 2025. ALYMPICS: LLM Agents Meet Game Theory. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mao2025alympics,
  title = {{ALYMPICS: LLM Agents Meet Game Theory}},
  author = {Shaoguang Mao, Yuzhe Cai, Yan Xia, Wenshan Wu, Xun Wang, Fengyi Wang, Qiang Guan, Tao Ge and Furu Wei},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.193/}
}

References

54 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, and Mario Fritz. 2023. Llm-deliberation: Evaluating llms with interactive multi-agent negotiation games. arXiv preprint arXiv:2309.17234. — [arXiv:2309.17234](https://arxiv.org/abs/2309.17234) find
  2. Marco Archetti and Kenneth J Pienta. 2019. Cooperation among cancer cells: applying game theory to cancer. Nature Reviews Cancer, 19(2):110–117. find
  3. Lawrence M Ausubel, Peter Cramton, Marek Pycia, Marzena Rostek, and Marek Weretka. 2014. Demand reduction and inefficiency in multi-unit auctions. The Review of Economic Studies, 81(4):1366–1400. find
  4. Max H Bazerman and William F Samuelson. 1983. I won the auction but don’t want the prize. Journal of conflict resolution, 27(4):618–634. find
  5. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712.
  6. Yuzhe Cai, Shaoguang Mao, Wenshan Wu, Zehua Wang, Yaobo Liang, Tao Ge, Chenfei Wu, Wang You, Ting Song, Yan Xia, et al. 2023. Low-code llm: Visual programming over llms. arXiv preprint arXiv:2304.08103, 2. — [arXiv:2304.08103](https://arxiv.org/abs/2304.08103) find
  7. Colin F Camerer. 2011. Behavioral game theory: Experiments in strategic interaction. Princeton university press. find
  8. Jiangjie Chen, Siyu Yuan, Rong Ye, Bodhisattwa Prasad Majumder, and Kyle Richardson. 2023. Put your money where your mouth is: Evaluating strategic planning and execution of llm agents in an auction arena. arXiv preprint arXiv:2310.05746. — [arXiv:2310.05746](https://arxiv.org/abs/2310.05746) find
  9. Vincent P Crawford. 2002. Introduction to experimental game theory. Journal of Economic Theory, 104(1):1– 15. find
  10. I de Zarzà, J de Curtò, Gemma Roig, Pietro Manzoni, and Carlos T Calafate. 2023. Emergent cooperation and strategy adaptation in multi-agent systems: An extended coevolutionary theory with llms. Electronics, 12(12):2722. find
  11. Kanishk Gandhi, Dorsa Sadigh, and Noah D Goodman. 2023. Strategic reasoning with language models. arXiv preprint arXiv:2305.19165. find
  12. Ian Gemp, Yoram Bachrach, Marc Lanctot, Roma Patel, Vibhavari Dasagi, Luke Marris, Georgios Piliouras, and Karl Tuyls. 2024. States as strings as strategies: Steering language models with game-theoretic solvers. arXiv preprint arXiv:2402.01704. — [arXiv:2402.01704](https://arxiv.org/abs/2402.01704) find
  13. Fulin Guo. 2023. Gpt agents in game theory experiments. arXiv preprint arXiv:2305.05516. — [arXiv:2305.05516](https://arxiv.org/abs/2305.05516) find
  14. John J Horton. 2023. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research. find
  15. Yuki Hou, Haruki Tamoto, and Homei Miyashita. 2024. " my agent understands me better": Integrating dynamic human-like memory recall and consolidation in llm-based agents. arXiv preprint arXiv:2404.00573. — [arXiv:2404.00573](https://arxiv.org/abs/2404.00573) find
  16. Chenxu Hu, Jie Fu, Chenzhuang Du, Simian Luo, Junbo Zhao, and Hang Zhao. 2023. Chatdb: Augmenting llms with databases as their symbolic memory. arXiv preprint arXiv:2306.03901. — [arXiv:2306.03901](https://arxiv.org/abs/2306.03901) find
  17. Jen-tse Huang, Eric John Li, Man Ho Lam, Tian Liang, Wenxuan Wang, Youliang Yuan, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, and Michael R Lyu. 2024. How far are we on the decision-making of llms? evaluating llms’ gaming ability in multi-agent environments. arXiv preprint arXiv:2403.11807. — [arXiv:2403.11807](https://arxiv.org/abs/2403.11807) find
  18. Ziheng Huang, Sebastian Gutierrez, Hemanth Kamana, and Stephen MacNeil. 2023. Memory sandbox: Transparent and interactive memory management for conversational agents. In Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–3. find
  19. Vijay Krishna. 2009. Auction theory. Academic press. find
  20. Steven D Levitt and John A List. 2009. Field experiments in economics: The past, the present, and the future. European Economic Review, 53(1):1–18. find
  21. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large scale language model society. arXiv preprint arXiv:2303.17760.
  22. Yaobo Liang, Chenfei Wu, Ting Song, Wenshan Wu, Yan Xia, Yu Liu, Yang Ou, Shuai Lu, Lei Ji, Shaoguang Mao, et al. 2023. Taskmatrix. ai: Completing tasks by connecting foundation models with millions of apis. arXiv preprint arXiv:2303.16434. — [arXiv:2303.16434](https://arxiv.org/abs/2303.16434) find
  23. Bill Yuchen Lin, Yicheng Fu, Karina Yang, Prithviraj Ammanabrolu, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2023. Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. arXiv preprint arXiv:2305.17390.
  24. Yifan Liu, Wei Wei, Jiayi Liu, Xianling Mao, Rui Fang, and Dangyang Chen. 2022. Improving personality consistency in conversation by persona extending. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 1350–1359. find
  25. Nunzio Lorè and Babak Heydari. 2023. Strategic behavior of large language models: Game structure vs. contextual framing. arXiv preprint arXiv:2309.05898. — [arXiv:2309.05898](https://arxiv.org/abs/2309.05898) find
  26. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. find
  27. Paul Robert Milgrom. 2004. Putting auction theory to work. Cambridge University Press. find
  28. OpenAI. 2023. Gpt-4 technical report.
  29. Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. find
  30. Matti Pohjola. 1986. Applications of dynamic game theory to macroeconomics. Dynamic games and applications in economics, pages 103–133. find
  31. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789.
  32. Alan G Sanfey. 2007. Social decision-making: insights from game theory and neuroscience. Science, 318(5850):598–602. find
  33. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. Character-llm: A trainable agent for role-playing. arXiv preprint arXiv:2310.10158.
  34. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580.
  35. Noah Shinn, Beck Labash, and Ashwin Gopinath. 2023. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366.
  36. Martin Shubik. 1981. Game theory models and methods in political economy. Handbook of Mathematical Economics, 1:285–330. find
  37. Theodore Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L Griffiths. 2023. Cognitive architectures for language agents. arXiv preprint arXiv:2309.02427. find
  38. Yashar Talebirad and Amirhossein Nadiri. 2023. Multi-agent collaboration: Harnessing the power of intelligent llm agents. arXiv preprint arXiv:2306.03314. find
  39. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  40. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2023a. A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432.
  41. Xintao Wang, Quan Tu, Yaying Fei, Ziang Leng, and Cheng Li. 2023b. Does role-playing chatbots capture the character personalities? assessing personality traits for role-playing chatbots. find
  42. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. 2023c. Unleashing cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. arXiv preprint arXiv:2307.05300.
  43. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  44. Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. 2023a. Expertprompting: Instructing large language models to be distinguished experts. arXiv preprint arXiv:2305.14688. find
  45. Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. 2023b. Exploring large language models for communication games: An empirical study on werewolf.
  46. Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. 2023c. Language agents with reinforcement learning for strategic play in the werewolf game. arXiv preprint arXiv:2310.18940. — [arXiv:2310.18940](https://arxiv.org/abs/2310.18940) find
  47. Tao Ge Xun Wang Yan Xia Wenshan Wu Ting Song Man Lan Furu Wei Yadong Zhang, Shaoguang Mao. 2024. LLM as a mastermind: A survey of strategic reasoning with large language models. In First Conference on Language Modeling. find
  48. Yaodong Yang and Jun Wang. 2020. An overview of multi-agent reinforcement learning from game theoretical perspective. arXiv preprint arXiv:2011.00583. — [arXiv:2011.00583](https://arxiv.org/abs/2011.00583) find
  49. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601. find
  50. Jintian Zhang, Xin Xu, and Shumin Deng. 2023. Exploring collaboration mechanisms for llm agents: A social psychology view. arXiv preprint arXiv:2310.02124.
  51. Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Yan Xia, Man Lan, and Furu Wei. 2024. K-level reasoning with large language models. arXiv preprint arXiv:2402.01521. — [arXiv:2402.01521](https://arxiv.org/abs/2402.01521) find
  52. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. Expel: Llm agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19632–19642. find
  53. Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie. 2023. Competeai: Understanding the competition behaviors in large language model-based agents. arXiv preprint arXiv:2310.17512. find
  54. Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2023. Can large language models transform computational social science? arXiv preprint arXiv:2305.03514.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mao-2025-alympics-llm-agents-meet-game/