Elamur Research⌕
☰
home / papers / 2026 / ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants

ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants

Paper · 2026 · Pei Wang, Yanan Wu, Xiaoshuai Song et al.

Summary

LLM-based shopping assistants must clarify vague requests through dialogue, use a shopper's profile and history, and pick the right item among near-identical products. Earlier environments such as WebShop and DeepShop cover only some of these needs. They are mostly English and offer little or no training data. The authors present ShopSimulator, a Chinese e-commerce sandbox built on real Taobao products. It uses LLM-simulated shoppers with synthetic, human-reviewed profiles and supports both evaluating and training agents. Tasks span single- and multi-turn dialogue, with and without personalization. Rewards check category, attributes, options and price. Even GPT-5 fully succeeds on only about a third of test tasks. Agents search redundantly, overlook constraints, buy too hastily, and either ignore or overthink user preferences. In training experiments on Qwen3-8B, SFT followed by RL beats RL alone, and a strict multiplicative reward beats a loose additive one. Multi-turn personalized tasks remain hard after training. The work matters because it provides an open environment for RL-based agent development beyond English-language benchmarks.

Key points

  • The catalog holds about 1.34M Taobao products across 12 domains. Each smallest sub-category contains roughly 120 highly similar items.
  • The release has about 25K training tasks and 2.8K evaluation tasks, 28,147 in total. It covers four settings: Single-Turn, Multi-Turn, and each with personalization (Pers).
  • No evaluated LLM reaches 40% full success, and GPT-5 scores 32%. Most failures involve fine-grained attributes and options rather than category or price.
  • Qwen3-8B was trained with GRPO. The best setup was SFT followed by RL with the strict reward. Multi-turn success stays around 35% even after training.
  • Simulator-side errors cause under 8% of failures. In a human evaluation on 20 multi-turn tasks, success rose from 15% for base Qwen3-8B to 40% after SFT and 35% after RL. Code and data are available at github.com/ShopAgent-Team/ShopSimulator.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Alibaba GroupHangzhou · China168 works · 15,386 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

Cite

Cite as Pei Wang, Yanan Wu, Xiaoshuai Song, Weixun Wang, Gengru Chen, Zhongwen Li, Kezhong Yan, Qi Liu, Ken Deng, Shuaibing Zhao, Shaopan Xiong, Xuepeng Liu, Xuefeng Chen, Wanxi Deng, Wenbo Su and Bo Zheng. 2026. ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1647

@inproceedings{wang2026shopsimulator,
  title = {{ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants}},
  author = {Pei Wang, Yanan Wu, Xiaoshuai Song, Weixun Wang, Gengru Chen, Zhongwen Li, Kezhong Yan, Qi Liu, Ken Deng, Shuaibing Zhao, Shaopan Xiong, Xuepeng Liu, Xuefeng Chen, Wanxi Deng, Wenbo Su and Bo Zheng},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1647},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1647}
}

References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Haibin Chen, Kangtao Lv, Chengwei Hu, Yanshi Li, Yujin Yuan, Yancheng He, Xingyao Zhang, Langming Liu, Shilei Liu, Wenbo Su, and 1 others. 2025. Chineseecomqa: A scalable e-commerce concept evaluation benchmark for large language models. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge… find
  2. Sanxing Chen, Sam Wiseman, and Bhuwan Dhingra. 2024. Chatshop: Interactive information seeking with language agents. Preprint, arXiv:2404.09911. — [arXiv:2404.09911](https://arxiv.org/abs/2404.09911) find
  3. Claude. 2025. Introducing claude 4 \| anthropic. find
  4. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  5. DeepSeek. 2025. Deepseek-v3.1 release \| deepseek api docs. find
  6. Ming Gong, Xucheng Huang, Chenghan Yang, Xian-han Peng, Haoxin Wang, Yang Liu, and Ling Jiang. 2025. Mindflow: Revolutionizing e-commerce customer support with multimodal llm agents. arXiv preprint arXiv:2507.05330. — [arXiv:2507.05330](https://arxiv.org/abs/2507.05330) find
  7. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  8. Christian Herold, Michael Kozielski, Tala Bazazo, Pavel Petrushkov, Patrycja Cieplicka, Dominika Basaj, Yannick Versley, Seyyed Hadi Hashemi, and Shahram Khadivi. 2025. Domain adaptation of foundation llms for e-commerce. arXiv preprint arXiv:2501.09706. find
  9. Yilun Jin, Zheng Li, Chenwei Zhang, Tianyu Cao, Yifan Gao, Pratik Jayarao, Mao Li, Xin Liu, Ritesh Sarkhel, Xianfeng Tang, and 1 others. 2024. Shopping mmlu: A massive multi-task online shopping benchmark for large language models. Advances in Neural Information Processing Systems, 37:18062– 18089. find
  10. Yangning Li, Shirong Ma, Xiaobin Wang, Shen Huang, Chengyue Jiang, Hai-Tao Zheng, Pengjun Xie, Fei Huang, and Yong Jiang. 2024. Ecomgpt: Instruction-tuning large language models with chain-of-task tasks for e-commerce. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, page… find
  11. Langming Liu, Haibin Chen, Yuhao Wang, Yujin Yuan, Shilei Liu, Wenbo Su, Xiangyu Zhao, and Bo Zheng. 2025. Eckgbench: Benchmarking large language models in e-commerce leveraging knowledge graph. arXiv preprint arXiv:2503.15990. — [arXiv:2503.15990](https://arxiv.org/abs/2503.15990) find
  12. Yougang Lyu, Xiaoyu Zhang, Lingyong Yan, Maarten de Rijke, Zhaochun Ren, and Xiuying Chen. 2025. Deepshop: A benchmark for deep research shopping agents. Preprint, arXiv:2506.02839. — [arXiv:2506.02839](https://arxiv.org/abs/2506.02839) find
  13. Guangtao Nie, Rong Zhi, Xiaofan Yan, Yufan Du, Xiangyang Zhang, Jianwei Chen, Mi Zhou, Hongshen Chen, Tianhao Li, Ziguang Cheng, and 1 others. 2024. A hybrid multi-agent conversational recommender system with llm and search engine in ecommerce. In Proceedings of the 18th ACM Conference on Recommende… find
  14. OpenAI. 2025a. Gpt-5 is here \| openai. find
  15. OpenAI. 2025b. Introducing gpt-4.1 in the api \| openai. find
  16. OpenAI. 2025c. Introducing gpt-oss \| openai. find
  17. OpenAI. 2025d. Introducing openai o3 and o4-mini \| openai. find
  18. Ralph Peeters, Aaron Steiner, Luca Schwarz, Julian Yuya Caspary, and Christian Bizer. 2025. Webmall – a multi-shop benchmark for evaluating web agents. Preprint, arXiv:2508.13024. — [arXiv:2508.13024](https://arxiv.org/abs/2508.13024) find
  19. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728– 53741.
  20. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  21. Lu Sun, Shihan Fu, Bingsheng Yao, Yuxuan Lu, Wenbo Li, Hansu Gu, Jiri Gesi, Jing Huang, Chen Luo, and Dakuo Wang. 2025. Llm agent meets agentic ai: Can llm agents simulate customers to evaluate agentic-ai-based shopping assistants? arXiv preprint arXiv:2509.21501. — [arXiv:2509.21501](https://arxiv.org/abs/2509.21501) find
  22. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534.
  23. Jiangyuan Wang, Kejun Xiao, Qi Sun, Huaipeng Zhao, Tao Luo, Jiandong Zhang, and Xiaoyi Zeng. 2025a. Shoppingbench: A real-world intent-grounded shopping benchmark for llm-based agents. Preprint, arXiv:2508.04266. — [arXiv:2508.04266](https://arxiv.org/abs/2508.04266) find
  24. Weixun Wang, Shaopan Xiong, Gengru Chen, Wei Gao, Sheng Guo, Yancheng He, Ju Huang, Jiaheng Liu, Zhendong Li, Xiaoyang Li, and 1 others. 2025b. Reinforcement learning optimization for large-scale learning: An efficient and user-friendly scaling library. arXiv preprint arXiv:2506.06122. — [arXiv:2506.06122](https://arxiv.org/abs/2506.06122) find
  25. Ziyi Wang, Yuxuan Lu, Wenbo Li, Amirali Amini, Bo Sun, Yakov Bart, Weimin Lyu, Jiri Gesi, Tian Wang, Jing Huang, and 1 others. 2025c. Opera: A dataset of observation, persona, rationale, and action for evaluating llms on human online shopping behavior simulation. arXiv preprint arXiv:2506.05606.
  26. Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, and 1 others. 2024. A survey on large language models for recommendation. World Wide Web, 27(5):60. find
  27. Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, and 1 others. 2025. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.06471.
  28. Shuo Zhang, Boci Peng, Xinping Zhao, Boren Hu, Yun Zhu, Yanjia Zeng, and Xuming Hu. 2024. Llasa: Large language and e-commerce shopping assistant. In Amazon KDD Cup 2024 Workshop. find
  29. Yimeng Zhang, Tian Wang, Jiri Gesi, Ziyi Wang, Yuxuan Lu, Jiacheng Lin, Sinong Zhan, Vianne Gao, Ruochen Jiao, Junze Liu, and 1 others. 2025. Shop-r1: Rewarding llms to simulate human behavior in online shopping via reinforcement learning. arXiv preprint arXiv:2507.17842. — [arXiv:2507.17842](https://arxiv.org/abs/2507.17842) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-shopsimulator-evaluating-exploring-rl-driven/