Elamur Research⌕
☰
home / papers / 2026 / From Short Video to Clickable Search: RLVR-Enabled Listwise Query Suggestion with Retrieval-Augmented Context

From Short Video to Clickable Search: RLVR-Enabled Listwise Query Suggestion with Retrieval-Augmented Context

Paper · 2026 · Mingkai Tian, Xuye, Long Meng et al.

Summary

The paper targets the clickable search suggestions shown beneath short videos on Kuaishou. The earlier system, GREAT, produced one query per pass from title and OCR and was scored by edit distance against a single reference. That setup struggles to capture the many intents a video can trigger, and it gives no credit to equivalent rewordings.

The authors make four changes:

  • List generation: the model outputs 20 distinct queries in one shot, with multi-query ground truth filtered from exposure and CTR logs.
  • New metric: CTR-HungF1 aligns the predicted and reference lists with Hungarian matching over unigram F1 and weights each match by CTR.
  • Retrieval context: a V2V2Q RAG module collects historically exposed queries from similar videos to supply platform-specific knowledge.
  • Training: a compact LLM is trained with thinking-free RLVR, using format checks and CTR-HungF1 as the only reward.

Combining retrieval and RL raises CTR-HungF1 from under 19% to 47.26%. The system now runs in production on full traffic. It shows that verifiable list-level rewards can replace reward models or CoT distillation in an industrial setting.

Key points

  • The base model is Qwen3-4B-Instruct-2507, trained with GRPO using a DAPO-style token objective; malformed outputs (wrong list size, duplicates, invalid JSON, missing tags) receive a reward of −1.

  • Retrieval pipeline:

    • Videos are retrieved by ANN search over about 70M videos with 128-d multimodal embeddings, keeping only queries with more than 20 exposures.
    • Candidate queries are scored by a mix of video similarity and query CTR, then pruned with nucleus filtering.
    • Both settings (λ for the similarity weight, τ for the filtering threshold) work best at 0.6.

  • Ablation results (CTR-HungF1):

    SetupScore
    OCR only17.00%
    Metadata only17.41%
    OCR + metadata18.94%
    + RAG33.63%
    RL without RAG39.28%
    Full system47.26%

  • On a shampoo-review case study, the system scores 72.3% versus 32.6% for GREAT. Human review of 300 sampled pairs per A/B bucket found better relevance and fewer risk-control issues.

  • The hourly pipeline generates over 10^8 queries for more than 6M videos per day.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢KuaishouBeijing · China15 works · 95 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Mingkai Tian, Xuye, Long Meng, Liwei Chen, Zhiheng Qin and Yi Wang. 2026. From Short Video to Clickable Search: RLVR-Enabled Listwise Query Suggestion with Retrieval-Augmented Context. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.38

@inproceedings{tian2026short,
  title = {{From Short Video to Clickable Search: RLVR-Enabled Listwise Query Suggestion with Retrieval-Augmented Context}},
  author = {Mingkai Tian, Xuye, Long Meng, Liwei Chen, Zhiheng Qin and Yi Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.38},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.38}
}

References

35 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tianchi Cai, Zhiwen Tan, Xierui Song, Tao Sun, Jiyan Jiang, Yunqi Xu, Yinger Zhang, and Jinjie Gu. 2024. Forag: Factuality-optimized retrieval augmented generation for web-enhanced long-form question answering. In KDD, pages 199–210. find
  2. Yapei Chang, Yekyung Kim, Michael Krumdick, Amir Zadeh, Chuan Li, Chris Tanner, and Mohit Iyyer. 2025. BLEUBERI: BLEU is a surprisingly effective reward for instruction following. In NeurIPS. find
  3. Jinwen Chen, Shuai Gong, Shiwen Zhang, Zheng Zhang, Yachao Zhao, Lingxiang Wang, Haibo Zhou, Yuan Zhan, Wei Lin, and Hainan Zhang. 2026. Lo-calsug: Geography-aware llm for query suggestion in local-life services. Preprint, arXiv:2603.04946. — [arXiv:2603.04946](https://arxiv.org/abs/2603.04946) find
  4. Kang Chen, Qingheng Zhang, Chengbao Lian, Yixin Ji, Xuwei Liu, Shuguang Han, Guoqiang Wu, Fei Huang, and Jufeng Chen. 2024. IPL: leveraging multimodal large language models for intelligent product listing. In EMNLP (Industry Track), pages 697–711. find
  5. Cheng Cheng, Chenxing Wang, Aolin Li, Haijun Wu, Huiyun Hu, and Juyuan Wang. 2026. When & how to write for personalized demand-aware query rewriting in video search. find
  6. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. Deepseek-r1: Incentivizing reasoning capability in ll…
  7. Xinya Du and Heng Ji. 2022. Retrieval-augmented generative question answering for event argument extraction. In EMNLP, pages 4649–4666. find
  8. Huawen Feng, Zekun Yao, Junhao Zheng, and Qianli Ma. 2025. Training large language models for retrieval-augmented question answering through backtracking correction. In ICLR. find
  9. Xian Guo, Ben Chen, Siyuan Wang, Ying Yang, Mingyue Cheng, Chenyi Lei, Yuqing Ding, and Han Li. 2026. Onesug: The unified end-to-end generative framework for e-commerce query suggestion. In AAAI, pages 14774–14782. find
  10. Alexander Gurung and Mirella Lapata. 2025. Learning to reason for long-form story generation. In Second Conference on Language Modeling. find
  11. Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open domain question answering. In EACL, pages 874– 880.
  12. Pengcheng Jiang, Jiacheng Lin, Lang Cao, Runchu Tian, SeongKu Kang, Zifeng Wang, Jimeng Sun, and Jiawei Han. 2025. Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning. In Second Conference on Language Modeling. find
  13. Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan Ö. Arik. 2025. Long-context llms meet RAG: overcoming challenges for long inputs in RAG. In ICLR. find
  14. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP, pages 6769– 6781.
  15. Sunwoo Kim, Geon Lee, Kyungho Kim, Jaemin Yoo, and Kijung Shin. 2025. Itemrag: Item-based retrieval-augmented generation for llm-based recommendation. Preprint, arXiv:2511.15141. — [arXiv:2511.15141](https://arxiv.org/abs/2511.15141) find
  16. Harold W. Kuhn. 2010. The hungarian method for the assignment problem. In 50 Years of Integer Programming, pages 29–47. find
  17. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In SOSP, pages 611–626.
  18. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James Validad Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Christopher Wilhelm, Luca Soldaini,… — Tulu 3: Pushing Frontiers in Open Language Model Post-Training
  19. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In NeurIPS.
  20. Che Liu, Haozhe Wang, Jiazhen Pan, Zhongwei Wan, Yong Dai, Fangzhen Lin, Wenjia Bai, Daniel Rueckert, and Rossella Arcucci. 2025. Beyond distillation: Pushing the limits of medical LLM reasoning with minimalist rule-based RL. In The Second Workshop on GenAI for Health: Potential, Trust, and Policy C…
  21. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In ICLR. find
  22. Erxue Min, Hsiu-Yuan Huang, Xihong Yang, Min Yang, Xin Jia, Yunfang Wu, Hengyi Cai, Junfeng Wang, Shuaiqiang Wang, and Dawei Yin. 2025. From prompting to alignment: A generative framework for query recommendation. Preprint, arXiv:2504.10208. — [arXiv:2504.10208](https://arxiv.org/abs/2504.10208) find
  23. Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. 2025. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. In MIC-CAI, volume 15966 of Lecture Notes in Computer Scie… find
  24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL, pages 311–318.
  25. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In NeurIPS.
  26. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Trans. Assoc. Comput. Linguistics, 11:1316–1331.
  27. Stephen E. Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3(4):333–389. find
  28. Ninglu Shao, Jinshan Wang, Chenxu Wang, Qingbiao Li, and Xiaoxue Zang. 2025. Great: Guiding query generation with a trie for recommending related search about video at kuaishou. In KDD, page 4818–4826. find
  29. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  30. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient RLHF framework. In EuroSys, pages 1279–1297. ACM.
  31. Zhepei Wei, Wei-Lin Chen, and Yu Meng. 2025. Instructrag: Instructing retrieval-augmented generation via self-synthesized rationales. In ICLR. find
  32. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  33. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, YuYue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, LingJun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, and 17 others. 2025. DAPO: An open-source LLM reinforcement learning sy…
  34. Sheng Zhang, Qianchu Liu, Guanghui Qin, Tristan Naumann, and Hoifung Poon. 2025. Med-rlvr: Emerging medical reasoning from a 3b base model via reinforcement learning. Preprint, arXiv:2502.19655. find
  35. Huike Zou, Haiyang Yang, Yindu Su, Liyu Chen, Chengbao Lian, Qingheng Zhang, Shuguang Han, and Jufeng Chen. 2025. Multi-value-product retrieval-augmented generation for industrial product attribute value identification. In EMNLP (Industry Track), pages 2096–2105. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tian-2026-short-video-clickable-search-rlvr/