Elamur Research⌕
☰
home / papers / 2026 / Intent-Driven Semantic ID Generation for Grounded Conversational News Recommendation

Intent-Driven Semantic ID Generation for Grounded Conversational News Recommendation

Paper · 2026 · Hongyang Su, Beibei Kong, Lei Cheng et al.

Summary

The paper tackles chat-based news recommendation on a large Chinese news platform. Articles typically go stale within a day, many users have little history, and most requests (e.g., "something different") contain no keywords a retriever could use. Analyzing real conversations from the platform, the authors identify six kinds of user intent, of which five are not stated explicitly and are handled badly by RAG approaches that retrieve first. They propose NewsRec-Chat, built on a Generate-then-Match paradigm. A fine-tuned 7B LLM produces hierarchical Semantic ID (SID) prefixes, built with RQ-VAE, directly from the user profile, history and query. These prefixes are then fuzzy-matched against the live news pool, so every suggestion is a real article. Training has two stages: multi-task SID alignment, then GPT-4 Chain-of-Thought distillation. Profile-Aware Dual-Signal Reasoning (PADR) selects warm, hybrid or cold reasoning depending on how much history a user has. The system reaches 0% hallucination and roughly matches GPT-4 with hybrid RAG at about 1/100 of the cost. Cold-start users get the best accuracy of any group, so the approach suits other fast-refreshing domains.

Key points

  • Intent taxonomy and SID design: six intents, five implicit, validated with Fleiss' κ=0.81. Each article has a 4-layer SID. The model predicts only the first three layers; matching is exact on layers 1–2 and allows tolerance δ=5 on layer 3. The history threshold for PADR is τ=10.
  • Overall accuracy and cost: in a 152K open-generation SID space, the model scores 12.4% L1 (about 4× random), beating the production baseline of 11.6% (p<0.05). It equals GPT-4+Hybrid RAG on L1, doubles it on L2, and costs about 100× less, with sub-100ms latency.
  • Hallucination: two-stage training cuts the 7B model's hallucination rate from 70% to 0%. Removing Stage 2 PADR raises it to 18.4%.
  • Cold start: zero-history users reach 18.0% L1. By comparison, OneRec-7B scores 16.1% and SASRec/TIGER score 0%, and only NewsRec-Chat covers all six intents.
  • Category generalization: mean L1 across 29 categories is 23.5%. Nine zero-shot categories average 19.7%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢TencentShenzhen · China134 works · 19,743 citations🎓Sun Yat-sen UniversityGuangzhou · China53 works · 9,185 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 624 works

Cite

Cite as Hongyang Su, Beibei Kong, Lei Cheng, Chengxiang Zhuo, Zang Li and Chenyun Yu. 2026. Intent-Driven Semantic ID Generation for Grounded Conversational News Recommendation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.130

@inproceedings{su2026intent,
  title = {{Intent-Driven Semantic ID Generation for Grounded Conversational News Recommendation}},
  author = {Hongyang Su, Beibei Kong, Lei Cheng, Chengxiang Zhuo, Zang Li and Chenyun Yu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.130},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.130}
}

References

34 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Luke Friedman, Sameer Ahuja, David Allen, Zhenning Tan, Hakim Sidahmed, Changbo Long, Jun Xie, Gabriel Schubiner, Ajay Patel, Harsh Lara, and 1 others. 2023. Leveraging large language models in conversational recommender systems. arXiv preprint arXiv:2305.07961. — [arXiv:2305.07961](https://arxiv.org/abs/2305.07961) find
  2. Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten De Rijke, and Tat-Seng Chua. 2021. Advances and challenges in conversational recommender systems: A survey. AI Open, 2:100–126. find
  3. Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023a. Chat-REC: Towards interactive and explainable LLMs-augmented recommender system. arXiv preprint arXiv:2303.14524. — [arXiv:2303.14524](https://arxiv.org/abs/2303.14524) find
  4. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023b. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2(1).
  5. Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, and Julian McAuley. 2023. Large language models as zero-shot conversational recommenders. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Manage… find
  6. Chris Hokamp and Qun Liu. 2017. Lexically constrained decoding for sequence generation using grid beam search. arXiv preprint arXiv:1704.07138.
  7. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 others. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems,…
  8. Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. 2021. A survey on conversational recommender systems. ACM Computing Surveys, 54(5):1– 36. find
  9. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38.
  10. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In Proceedings of the IEEE International Conference on Data Mining (ICDM), pages 197–206. find
  11. J. Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33(1):159–174. find
  12. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33:9459–…
  13. Liunian Harold Li, Jack Hessel, Youngjae Yu, Xiang Ren, Kai-Wei Chang, and Yejin Choi. 2023. Symbolic chain-of-thought distillation: Small models can also “think” step-by-step. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
  14. Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. Advances in Neural Information Processing Systems, 31. find
  15. Yang Li, Yidan Zhang, Fuyuan Sun, Yizhe Shan, Yuqiang Ge, Yue Gao, Xu He, Peng Zhao, Hongyan Bao, and Kang Gai. 2024. A survey on generative recommendation. arXiv preprint arXiv:2404.00011. — [arXiv:2404.00011](https://arxiv.org/abs/2404.00011) find
  16. Chang Liu, Yimeng Bai, Xiaoyan Zhao, Yang Zhang, Fuli Feng, and Wenge Rong. 2025a. Dis-cRec: Disentangled semantic-collaborative modeling for generative recommendation. arXiv preprint arXiv:2506.15576. — [arXiv:2506.15576](https://arxiv.org/abs/2506.15576) find
  17. Zhanyu Liu, Shiyao Wang, Xingmei Wang, Rongzhou Zhang, Jiaxin Deng, Honghui Bao, Jinghao Zhang, Wuchao Li, Pengfei Zheng, Xiangyu Wu, and 1 others. 2025b. OneRec-Think: In-text reasoning for generative recommendation. arXiv preprint arXiv:2510.11639.
  18. Wenchang Ma, Ryuichi Takanobu, and Minlie Huang. 2021. CR-Walker: Tree-structured graph reasoning and dialog acts for conversational recommendation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1839–1851. find
  19. Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. 2023. Teaching small language models to reason. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1773–1781.
  20. Zhangchi Qiu, Linhao Luo, Zicheng Zhao, Shirui Pan, and Alan Wee-Chung Liew. 2025. Graph retrieval-augmented LLM for conversational recommendation systems. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 344–355. find
  21. Shashank Rajput, Nikhil Mehta, Anima Singh, Raghu-nandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, and 1 others. 2023. Recommender systems with generative retrieval. Advances in Neural Information Processing Systems, 36:10299–10315. find
  22. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389. find
  23. Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1441–1450. find
  24. Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, and 1 others. 2022. Transformer memory as a differentiable search index. Advances in Neural Information Processing Systems, 35:21831–21843.
  25. Qwen Team. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. find
  26. Xiaolei Wang, Xinyu Tang, Wayne Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. 2023. Rethinking the evaluation for conversational recommendation in the era of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10052–10065. find
  27. Xiaolei Wang, Kun Zhou, Ji-Rong Wen, and Wayne Xin Zhao. 2022. Towards unified conversational recommender systems via knowledge-enhanced prompt learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1929–1937. find
  28. Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-Pack: Packed resources for general chinese embeddings. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 641–649.
  29. Qihang Yu, Kairui Fu, Shengyu Zhang, Zheqi Lv, Fan Wu, and Fei Wu. 2025. ThinkRec: Thinking-based recommendation via LLM. arXiv preprint arXiv:2505.15091. — [arXiv:2505.15091](https://arxiv.org/abs/2505.15091) find
  30. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, and 1 others. 2025. Siren’s song in the AI ocean: A survey on hallucination in large language models. Computational Linguistics, 51(4):1373– 1418.
  31. Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, Ming Chen, and Ji-Rong Wen. 2024. Adapting large language models by integrating collaborative semantics for recommendation. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 1435–1448. find
  32. Guorui Zhou, Jiaxin Deng, Jinghao Zhang, Kuo Cai, Lejian Ren, Qiang Luo, Qianqian Wang, Qigen Hu, Rui Huang, Shiyao Wang, and 1 others. 2025. OneRec technical report. arXiv preprint arXiv:2506.13695. find
  33. Yuanhang Zhou, Kun Zhou, Wayne Xin Zhao, Cheng Wang, Peng Jiang, and He Hu. 2022. C 2 -CRS: Coarse-to-fine contrastive learning for conversational recommender system. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pages 1488–1496. find
  34. Parameter Stage 1 3.4 candidates; δ=5: 0% Hal., mean 5.2 candidates; δ=7: 0% Hal., mean 7.8 candidates; δ=10: 0% Hal., mean 12.1 candidates. We choose δ=5 as the smallest value achieving 0% hallucination with a manageable candidate set size. Stage 2 Training Method Full-parameter Full-parameter Epoc… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/su-2026-intent-driven-semantic-id-generation/