Elamur Research⌕
☰
home / papers / 2026 / Small Agents, Big Gains: Journey-Aware and Critic-Guided Simulation for Long-Horizon Shopping Dialogues

Small Agents, Big Gains: Journey-Aware and Critic-Guided Simulation for Long-Horizon Shopping Dialogues

Paper · 2026 · Qing Ping, Changyou Chen and Binxuan Huang

Summary

The paper addresses a data bottleneck in building compact e-commerce assistants. Such assistants must help with inspiration, comparison and tool-backed fact-checking across non-linear shopping journeys, but real training trajectories are scarce and good answers have no single reference. The authors, from Amazon, frame trajectory synthesis as a POMDP and implement it as a closed-loop simulation with three agents:

  • User simulator: journey-aware and stateful, tracking each sub-goal through a staged shopping funnel.
  • Shopping agent: acts as the teacher, combining tool calls with UI-component layouts.
  • Critic agent: scores each candidate turn against business rubrics and symbolic checks. It returns textual critiques, used as a "textual gradient," until the turn is accepted.

Accepted outputs serve as SFT data. Rejected and accepted attempts together form preference pairs for DPO. On an in-house shopping benchmark, a small model trained on this data beats same-size baselines and matches or exceeds a larger model, while serving at 8× higher throughput. The work offers a recipe for aligning agents in open-ended domains where verifiable ground truth cannot be built.

Key points

  • The user simulator follows the funnel Broad-Search → Refined-Search → Focused-Search → Evaluation → Fact-Check → Done. It keeps a per-sub-goal table of shortlisted items, current stage and knowledge gaps, and chooses one of three actions: click, chat or end. This design targets three weaknesses of naive simulators: over-revealing goals, role drift and quitting early.
  • Missions are drawn from a factorized space covering product domains, mission-specific personas (with knowledge blind spots) and tool-chain complexity. Each mission spans at least k product domains.
  • In an open-weight replication on Qwen3-30B-A3B, helpfulness scores were:
    • 0.714 with no SFT;
    • 0.735 with manual SFT data;
    • 0.785 with the multi-agent SFT data;
    • 0.809 with the multi-agent DPO data;
    • 0.852 for Claude Sonnet 4.5 zero-shot, used as an upper reference. This equals +6.8% (SFT) and +10.0% (DPO) over manual SFT. Tool-call errors fell to 0%, compared with 0.9% under manual SFT.
  • The critic loop costs about 1.09 extra refinement iterations per turn of offline compute.
  • The authors list three limitations: rubric-based LLM judging can introduce bias, the simulated users are assumed to be fairly rational, and the pipeline being fully synthetic avoids exposing real customer logs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 624 works

Cite

Cite as Qing Ping, Changyou Chen and Binxuan Huang. 2026. Small Agents, Big Gains: Journey-Aware and Critic-Guided Simulation for Long-Horizon Shopping Dialogues. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.39

@inproceedings{ping2026small,
  title = {{Small Agents, Big Gains: Journey-Aware and Critic-Guided Simulation for Long-Horizon Shopping Dialogues}},
  author = {Qing Ping, Changyou Chen and Binxuan Huang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.39},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.39}
}

References

21 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xiangci Li, Zhiyu Chen, Jason Ingyu Choi, Nikhita Vedula, Besnik Fetahu, Oleg Rokhlenko, and Shervin Malmasi. 2025a. Wizard of shopping: Target-oriented E-commerce dialogue generation with decision tree branching. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguis… find
  2. Jafar Afzali, Aleksander Mark Drzewiecki, Krisztian Balog, and Shuo Zhang. 2023. Usersimcrs: A user simulation toolkit for evaluating conversational recommender systems. In Proceedings of the Sixteenth Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, and Yu Cheng. 2025b. Test-time preference optimization:… find
  3. Chen Luo, Dimitri Papadimitriou, Hariharan Muralidharan, Dhineshkumar Ramasubbu, Aakash Kolekar, Wenju Xu, Cong Xu, Anirudh Srinivasan, Mukesh Jain, and Qi He. 2025. Language model alignment for conversational shopping at amazon. In Proceedings of the 48th International ACM SIGIR Conference on Resea… find
  4. Xiang Luo, Zhiwen Tang, Jin Wang, and Xuejie Zhang. 2024. DuetSim: Building user simulator with dual large language models for task-oriented dialogues. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages… find
  5. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: iterative refinement with… find
  6. Shuhaib Mehri, Xiaocheng Yang, Takyoung Kim, Gokhan Tur, Shikib Mehri, and Dilek Hakkani-Tür. 2025. Goal alignment in LLM-based user simulators for conversational AI. In First Workshop on Multi-Turn Interactions in Large Language Models (MTI-LLM) at NeurIPS 2025.
  7. Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. SimPO: Simple preference optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. find
  8. Tarek Naous, Philippe Laban, Wei Xu, and Jennifer Neville. 2026. Flipping the dialogue: Training and evaluating user language models. In The Fourteenth International Conference on Learning Representations. find
  9. OpenAI. 2025. Introducing shopping research in Chat-GPT. https://openai.com/index/chatgpt-s hopping-research/. Published: Nov. 24, 2025. Accessed: 2025-12-29. ↗
  10. Jeonghoon Shim, Woojung Song, Cheyon Jin, Seung-won KooK, and Yohan Jo. 2026. Non-collaborative user simulators for tool agents. In The Fourteenth International Conference on Learning Representations. find
  11. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Proceedings of the 37th International Conference on Neural Information Processing Systems, volume 36, pages 8634–8652.
  12. Walmart. 2025. Walmart: The future of shopping is agentic. meet sparky. https://corporate.walm art.com/news/2025/06/06/walmart-the-fut ure-of-shopping-is-agentic-meet-sparky. Published: Jun. 6, 2025. Accessed: 2025-12-29. ↗
  13. Kuang Wang, Xianfei Li, Shenghao Yang, Li Zhou, Feng Jiang, and Haizhou Li. 2025. Know you first and be you better: Modeling human-like user simulators via implicit profiles. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 210… find
  14. Liqiang Xiao, Jun Ma, Xin Luna Dong, Pascual Martínez-Gómez, Nasser Zalmout, Chenwei Zhang, Tong Zhao, Hao He, and Yaohui Jin. 2021. End-to-end conversational search for online shopping with utterance transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing… find
  15. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  16. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. 2024. Textgrad: Automatic "differentiation" via text. Preprint, arXiv:2406.07496.
  17. Qiusi Zhan, Xiaojie Guo, Heng Ji, and Lingfei Wu. 2023. User simulator assisted open-ended conversational recommendation system. In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 89–101. find
  18. Perplexity. 2024. Shop like a pro: Perplexity’s new AI-powered shopping assistant. https://www.pe rplexity.ai/hub/blog/shop-like-a-pro. Published: Nov. 18, 2024. Accessed: 2025-12-29. ↗
  19. Zijian Zhang, Shuchang Liu, Ziru Liu, Rui Zhong, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Qidong Liu, and Peng Jiang. 2025. LLM-powered user simulator for recommender system. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence, volume 39, pages 13339–13347. find
  20. Ivan Sekulic, Silvia Terragni, Victor Guimarães, Nghia Khau, Bruna Guedes, Modestas Filipavicius, Andre Ferreira Manso, and Roland Mathis. 2024. Reliable LLM-based user simulator for task-oriented dialogue systems. In Proceedings of the 1st Workshop on Simulating Conversational Intelligence in Chat… find
  21. Jie Zou, Mohammad Aliannejadi, Evangelos Kanoulas, Shuxi Han, Heli Ma, Zheng Wang, Yang Yang, and Heng Tao Shen. 2025. PSCon: Product search through conversations. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3659—-3669. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ping-2026-small-agents-big-gains-journey/