Elamur Research⌕
☰
home / papers / 2026 / Market-Bench: Benchmarking Large Language Models on Economic and Trade Competition

Market-Bench: Benchmarking Large Language Models on Economic and Trade Competition

Paper · 2026 · Yushuo Zheng, Huiyu Duan, Zicheng Zhang et al.

Summary

Market-Bench tests whether LLMs can handle economic tasks that need both numeric planning and persuasive language at once. Earlier benchmarks usually measured one or the other: strategic games without rich text, or social simulations without real scarcity. The authors build a configurable multi-agent supply-chain economy, framed as a Partially Observable Markov Game. LLM retailers play two stages. First, they bid for limited stock in a budget-constrained first-price auction. Then they set retail prices and write marketing slogans. A new mechanism, Persona-Gated Attention, compares each slogan's embedding with the hidden persona of each buyer. This decides which retailers a buyer can see at all, so wording directly affects revenue. Full trajectories are logged and scored automatically with economic, operational and semantic metrics. Across 20 open- and closed-source models, the results follow a winner-take-most pattern. Only a few agents keep growing their capital, while many stay near break-even even when their semantic scores are similar. The testbed is reproducible and shows that current agents still struggle to combine inventory management with market positioning.

Key points

  • Setup: 20 LLM agents compete over 6 time steps with 8 item types. Agents receive structured text observations and must reply in strict JSON containing bids, prices and a free-form slogan. Bankruptcy occurs when funds fall below zero. The configuration is Hydra-based, and the code is at github.com/aiben-ch/Market-Bench.
  • Buyers: Synthetic buyer personas are hidden from agents and use fixed shares: Thrifty 40%, Ethical 30%, Hype 20%, Quality 10%. Each buyer purchases a single item and has no memory of past interactions.
  • Self-reinforcing outcomes: Results split into two groups. Agents that win stock early gain cash for later bids, while agents that fail early run out of revenue.
  • Volume versus margin: o3 posts the top FillRate, yet its margin trails those of Gemini 2.5 Pro and Flash. This shows that skill at winning auctions differs from skill at monetizing stock.
  • Slogans converge: Slogan semantic match (MMS) correlates only weakly with profit (Spearman ρ=0.16). Slogans cluster near the Ethical–Hype region by step 5 (averaged over 10 runs), which the authors read as imitation of rivals rather than targeting specific buyer niches.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China212 works · 19,544 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China152 works · 13,164 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksGeneral Social Sciences · 254 works

Cite

Cite as Yushuo Zheng, Huiyu Duan, Zicheng Zhang, Yucheng Zhu, Xiongkuo Min and Guangtao Zhai. 2026. Market-Bench: Benchmarking Large Language Models on Economic and Trade Competition. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1853

@inproceedings{zheng2026market,
  title = {{Market-Bench: Benchmarking Large Language Models on Economic and Trade Competition}},
  author = {Yushuo Zheng, Huiyu Duan, Zicheng Zhang, Yucheng Zhu, Xiongkuo Min and Guangtao Zhai},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1853},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1853}
}

References

45 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, and 1 others. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905.
  2. Anthropic. 2025a. Claude sonnet 4.5 system card. find
  3. Anthropic. 2025b. Project vend: Can claude run a small shop? (and why does that matter?). https://www.anthropic.com/research/project-vend-1. In partnership with Andon Labs. ↗
  4. Robert Axelrod. 1984. The Evolution of Cooperation. Basic Books, New York. find
  5. Baidu ERNIE Team. 2025. Ernie 4.5 technical report. find
  6. Joyce Berg, John Dickhaut, and Kevin McCabe. 1995. Trust, reciprocity, and social history. Games and Economic Behavior, 10(1):122–142. find
  7. Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiao-wen Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, and 81 others. 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297.
  8. Junhao Chen, Jingbo Sun, Xiang Li, Haidong Xin, Yuhao Xue, Yibin Xu, and Hao Zhao. 2025. Llmspark: A benchmark for evaluating large language models in strategic gaming contexts. arXiv preprint arXiv:2509.16610. find
  9. Michael Chui, Eric Hazan, Roger Roberts, Alex Singla, and Kate Smaje. 2023. The economic potential of generative ai. find
  10. DeepSeek-AI. 2025. Deepseek-v3.2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
  11. Yuan Deng, Vahab Mirrokni, Renato Paes Leme, Hanrui Zhang, and Song Zuo. 2024. Llms at the bargaining table. In Agentic Markets Workshop at ICML 2024. Poster. find
  12. Huiyu Duan, Qiang Hu, Jiarui Wang, Liu Yang, Zitong Xu, Lu Liu, Xiongkuo Min, Chunlei Cai, Tianxiao Ye, Xiaoyun Zhang, and Guangtao Zhai. 2025. Finevq: Fine-grained user generated content video quality assessment. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages… find
  13. Huiyu Duan, Xiongkuo Min, Wei Sun, Yucheng Zhu, Xiao-Ping Zhang, and Guangtao Zhai. 2023. Attentive deep image quality assessment for omnidirectional stitching. IEEE Journal of Selected Topics in Signal Processing, 17(6):1150–1164. find
  14. Huiyu Duan, Xiongkuo Min, Yucheng Zhu, Guangtao Zhai, Xiaokang Yang, and Patrick Le Callet. 2022. Confusing image quality assessment: Toward better augmented reality experience. IEEE Transactions on Image Processing, 31:7206–7221. find
  15. Kshama Dwarakanath, Tucker Balch, and Svitlana Vyetrenko. 2024. Abides-economist: Agent-based simulator of economic systems with learning agents. arXiv preprint arXiv:2402.09563. — [arXiv:2402.09563](https://arxiv.org/abs/2402.09563) find
  16. Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, and 1 others. 2025. Competitive programming with large reasoning models. arXiv preprint arXiv:2502.06807.
  17. Gemini Team. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261.
  18. Gemma Team. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  19. Kaiyuan Ji, Yixuan Gao, Lu Sun, Yushuo Zheng, Zijian Chen, Jianbo Zhang, Xiangyang Zhu, Yuan Tian, Zicheng Zhang, and Guangtao Zhai. 2026. A 3 : Towards Advertising Aesthetic Assessment. arXiv preprint arXiv:2603.24037. — [arXiv:2603.24037](https://arxiv.org/abs/2603.24037) find
  20. Kaiyuan Ji, Yijin Guo, Zicheng Zhang, Xiangyang Zhu, Yuan Tian, Ning Liu, and Guangtao Zhai. 2025a. Medomni-45{\\deg}: A safety-performance benchmark for reasoning-oriented llms in medicine. arXiv preprint arXiv:2508.16213. — [arXiv:2508.16213](https://arxiv.org/abs/2508.16213) find
  21. Kaiyuan Ji, Jing Han, Guangtao Zhai, and Jiannan Liu. 2025b. Assessing the capabilities of generative pre-trained transformer-4 in addressing open-ended inquiries of oral cancer. International Dental Journal, 75(1):158–165. find
  22. Kaiyuan Ji, Zhihan Wu, Jing Han, Jun Jia, Guangtao Zhai, and Jiannan Liu. 2024. Application of 3d nnu-net with residual encoder in the 2024 miccai head and neck tumor segmentation challenge. In Challenge on Head and Neck Tumor Segmentation for MRI-Guided Applications, pages 250–258. Springer. find
  23. Chiranjeev Kohli, Lance Leuthesser, and Rajneesh Suri. 2007. Got slogan? guidelines for creating effective slogans. Business Horizons, 50(5):415–422. find
  24. Lu Liu, Huiyu Duan, Qiang Hu, Liu Yang, Chunlei Cai, Tianxiao Ye, Huayu Liu, Xiaoyun Zhang, and Guangtao Zhai. 2024. F-bench: Rethinking human preference evaluation metrics for benchmarking face generation, customization, and restoration. arXiv preprint arXiv:2412.13155. — [arXiv:2412.13155](https://arxiv.org/abs/2412.13155) find
  25. Shaoguang Mao, Yuzhe Cai, Yan Xia, Wenshan Wu, Xun Wang, Fengyi Wang, Tao Ge, and Furu Wei. 2023. Alympics: Llm agents meet game theory – exploring strategic decision-making with ai agents. arXiv preprint arXiv:2311.03220. — [arXiv:2311.03220](https://arxiv.org/abs/2311.03220) find
  26. OpenAI. 2024. GPT-4o system card. arXiv preprint arXiv:2410.21276.
  27. Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. find
  28. Jinghua Piao, Yuwei Yan, Jun Zhang, Nian Li, Junbo Yan, Xiaochong Lan, Zhihong Lu, Zhiheng Zheng, Jing Yi Wang, Di Zhou, Chen Gao, Fengli Xu, Fang Zhang, Ke Rong, Jun Su, and Yong Li. 2025. Agentsociety: Large-scale simulation of llm-driven generative agents advances understanding of human behaviors… — [arXiv:2502.08691](https://arxiv.org/abs/2502.08691) find
  29. David Simchi-Levi, Konstantina Mellou, Ishai Menache, and Jeevan Pathuri. 2025. Large language models for supply chain decisions. arXiv preprint arXiv:2507.21502. — [arXiv:2507.21502](https://arxiv.org/abs/2507.21502) find
  30. Statista. 2024. Artificial intelligence (ai) in retail market size worldwide from 2021 to 2029. find
  31. Llama Team. 2024a. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  32. Qwen Team. 2024b. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. find
  33. Kaiyuan Ji, Zhihan Wu, Jing Han, Guangtao Zhai, and Jiannan Liu. 2025c. Evaluating chatgpt-4’s performance on oral and maxillofacial queries: Chain of thought and standard method. Frontiers in Oral Health, 6:1541976. find
  34. Qwen Team. 2025a. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923.
  35. Jian Jin, Jiangyong Ying, Huiyu Duan, Liu Yang, Sijing Wu, Yunhao Li, Yushuo Zheng, Xiongkuo Min, and Guangtao Zhai. 2025. Rgc-vqa: An exploration database for robotic-generated video quality assessment. In Proceedings of the ACM International Conference on Multimedia. find
  36. Tencent Hunyuan Team. 2024. Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent. arXiv preprint arXiv:2411.02265.
  37. Kevin Lane Keller. 2003. Strategic Brand Management: Building, Measuring, and Managing Brand Equity. Prentice Hall, Upper Saddle River, NJ. find
  38. Qwen Team. 2025b. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  39. Fanqi Wan, Weizhou Shen, Shengyi Liao, Yingcheng Shi, Chenliang Li, Ziyi Yang, Ji Zhang, Fei Huang, Jingren Zhou, and Ming Yan. 2025. Qwenlong-l1: Towards long-context large reasoning models with reinforcement learning. ArXiv.org.
  40. Ruiyi Wang, Yushuo Zheng, Zicheng Zhang, Chunyi Li, Shuaicheng Liu, Guangtao Zhai, and Xiaohong Liu. 2025. Learning hazing to dehazing: Towards realistic haze generation for real-world image dehazing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. find
  41. Shuangqing Wang, Kaiyuan Ji, Yushuo Zheng, Zhihan Wu, Xiaorong Zhu, Zijian Chen, Lu Sun, Shuo Wang, Jianbo Zhang, Zicheng Zhang, and 1 others. 2026. Dental-qad: Reasoning-driven quality assessment and diagnosis in panoramic radiographs. Displays, page 103380. find
  42. Liu Yang, Huiyu Duan, Ran Tao, Juntao Cheng, Sijing Wu, Yunhao Li, Jing Liu, Xiongkuo Min, and Guangtao Zhai. 2025. Odi-bench: Can mllms understand immersive omnidirectional environments? arXiv preprint arXiv:2510.11549. — [arXiv:2510.11549](https://arxiv.org/abs/2510.11549) find
  43. Zicheng Zhang, Ziheng Jia, Chunyi Li, Yingjie Zhou, Xiaohong Liu, Xiongkuo Min, and Guangtao Zhai. 2026a. Towards versatile multimedia quality assessment for visual communications. Science China Information Sciences, 69:122306. find
  44. Yonatan Bisk, Daniel Fried, Graham Neubig, and Maarten Sap. 2024. Sotopia: Interactive evaluation for social intelligence in language agents. In International Conference on Learning Representations (ICLR).
  45. Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Zhe Wang, Zhenhailong Wang, Cheng Qian, Robert Tang, Heng Ji, and Jiaxuan You. 2025. Multiagentbench: Evaluating the collaboration and competition of llm agents. In Proceedings of the 63rd Annual Meeting of the Association for Computat… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2026-market-bench-benchmarking-large-language/