Elamur Research⌕
☰
home / papers / 2026 / R\textthreesuperiorA: Reinforced Reasoning for Relevance Assessment for RAG in User-Generated Content Platforms

R\textthreesuperiorA: Reinforced Reasoning for Relevance Assessment for RAG in User-Generated Content Platforms

Paper · 2026 · Xiaowei Yuan, Lei Jin, Haoxin Zhang et al.

Summary

The paper addresses query–document relevance judgment for retrieval-augmented generation on user-generated content (UGC) platforms. Two problems make this hard. RAG pipelines lack document-level click feedback, so user intent stays ambiguous. Relevance is also asymmetric: a noisy post can resemble a query on the surface yet contain nothing that answers it. The authors propose R³A, which splits the judgment into two reasoning stages trained with reinforcement learning (GRPO) after a cold start on DeepSeek-R1-generated reasoning traces. In the first stage, the model reads a few highly clicked in-platform documents for the same query and infers the likely intent. In the second, it copies the answer-bearing span from the candidate document, or outputs NONE, and bases a 0/1/2 score on that span. A 7B model is then distilled into R³A-Distill-1.5B, which outputs scores without reasoning text. R³A beats strong baselines on the industrial NoteRel set and the public T²Ranking benchmark. Its gains on T²Ranking are smaller because that benchmark is out of domain. In live A/B tests on Xiaohongshu, the distilled model improved answer quality at production latency.

Key points

  • Training setup:
    • Cold start uses 50k unlabeled examples with DeepSeek-R1 reasoning traces.
    • Stage one supplies at most 4 auxiliary high-click documents.
    • The reward is zero if the output format is wrong or the extracted span is not verbatim from the document. Otherwise it scores agreement between predicted and gold labels; setting the λ term for off-by-one predictions to 0 worked best.
  • Ablations:
    • Removing the evidence-extraction step causes the largest drop.
    • Merging the two stages into one lowers accuracy to 60.6%.
  • Distillation: R³A-Distill-1.5B comes close to its 7B teacher and beats Qwen2.5-1.5B-Instruct by over 10 points absolute at pass@1.
  • Online A/B test:
    • Ran for one week on 10% of Xiaohongshu traffic, re-ranking the top 100 retrieved documents down to 10.
    • Human evaluation gave a Good:Same:Bad split of 23:71:6, about a 17% improvement in answer quality.
    • The second-search rate fell by 1.03%.
  • Deployment: the distilled model runs at about 150 ms, handles 200 QPS per GPU, and is served on a cluster of 1,000 GPUs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Chinese Academy of SciencesBeijing · China246 works · 24,952 citations🔬Institute of AutomationBeijing · China87 works · 14,883 citations🎓University of Chinese Academy of SciencesBeijing · China188 works · 8,744 citations🏢Xiaohongshu (Little Red Book)Shanghai · China13 works · 107 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Xiaowei Yuan, Lei Jin, Haoxin Zhang, Ziyang Huang, Yan Gao, Yiwu, Yao Hu, Jun Zhao and Kang Liu. 2026. R\textthreesuperiorA: Reinforced Reasoning for Relevance Assessment for RAG in User-Generated Content Platforms. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.10

@inproceedings{yuan2026r,
  title = {{RtextthreesuperiorA: Reinforced Reasoning for Relevance Assessment for RAG in User-Generated Content Platforms}},
  author = {Xiaowei Yuan, Lei Jin, Haoxin Zhang, Ziyang Huang, Yan Gao, Yiwu, Yao Hu, Jun Zhao and Kang Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.10},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.10}
}

References

35 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, and Bingsheng He. 2025a. Judgelrm: Large reasoning models as a judge. CoRR, abs/2504.00050. — [arXiv:2504.00050](https://arxiv.org/abs/2504.00050) find
  2. Shuang Chen, Yue Guo, Zhaochen Su, Yafu Li, Yulun Wu, Jiacheng Chen, Jiayu Chen, Weijie Wang, Xiaoye Qu, and Yu Cheng. 2025b. Advancing multimodal reasoning: From optimized cold start to staged reinforcement learning. CoRR, abs/2506.04207. — [arXiv:2506.04207](https://arxiv.org/abs/2506.04207) find
  3. Zeyuan Chen, Haiyan Wu, Kaixin Wu, Wei Chen, Mingjie Zhong, Jia Xu, Zhongyi Liu, and Wei Zhang. 2024. Towards boosting llms-driven relevance modeling with progressive retrieved behavior-augmented prompting. arXiv preprint arXiv:2408.09439. find
  4. Jia Deng, Jie Chen, Zhipeng Chen, Daixuan Cheng, Fei Bai, Beichen Zhang, Yinqian Min, Yanzipeng Gao, Wayne Xin Zhao, and Ji-Rong Wen. 2025. From trial-and-error to improvement: A systematic analysis of llm exploration mechanisms in rlvr. Preprint, arXiv:2508.07534. — [arXiv:2508.07534](https://arxiv.org/abs/2508.07534) find
  5. Guglielmo Faggioli, Laura Dietz, Charles L. A. Clarke, Gianluca Demartini, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, and Henning Wachsmuth. 2023. Perspectives on large language models for relevance judgment. In Proceedings of the 2023 ACM SIGIR In… find
  6. Quentin Fitte-Rey, Matyas Amrouche, and Romain Deveaud. 2025. Augmented relevance datasets with fine-tuned small llms. CoRR, abs/2504.09816. — [arXiv:2504.09816](https://arxiv.org/abs/2504.09816) find
  7. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-augmented generation for large language models: A survey. Preprint, arXiv:2312.10997.
  8. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 80 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms…
  9. Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023, pages 2230–2235. ACM. find
  10. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316–1331.
  11. Mandeep Rathee, Venktesh V, Sean MacAvaney, and Avishek Anand. 2025. Test-time corpus feedback: From retrieval to RAG. CoRR, abs/2508.15437.
  12. Jian Hu, Xibin Wu, Weixun Wang, Xianyu, Dehao Zhang, and Yu Cao. 2024. Openrlhf: An easy-to-use, scalable and high-performance RLHF framework. CoRR, abs/2405.11143.
  13. Alireza Salemi and Hamed Zamani. 2024. Evaluating retrieval quality in retrieval-augmented generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024, pages 2395–2400. ACM.
  14. Hui Huang, Yingqi Qu, Jing Liu, Muyun Yang, and Tiejun Zhao. 2024. An empirical study of llm-as-ajudge for LLM evaluation: Fine-tuned judge models are task-specific classifiers. CoRR, abs/2403.02839. — [arXiv:2403.02839](https://arxiv.org/abs/2403.02839) find
  15. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. Preprint, arXiv:1707.06347. find
  16. Ziyang Huang, Xiaowei Yuan, Yiming Ju, Jun Zhao, and Kang Liu. 2025. Reinforced internal-external knowledge synergistic reasoning for efficient adaptive search agent. CoRR, abs/2505.07596. — [arXiv:2505.07596](https://arxiv.org/abs/2505.07596) find
  17. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300.
  18. Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, and 79 others. 2024. Gpt-4o sy…
  19. Guanying Jiang, Lingyong Yan, Haibo Shi, and Dawei Yin. 2024. The real, the better: Aligning large language models with online human behaviors. Preprint, arXiv:2405.00578. — [arXiv:2405.00578](https://arxiv.org/abs/2405.00578) find
  20. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025a. Search-o1: Agentic search-enhanced large reasoning models. CoRR, abs/2501.05366.
  21. Yuchen Li, Hengyi Cai, Rui Kong, Xinran Chen, Jiamin Chen, Jun Yang, Haojie Zhang, Jiayi Li, Jiayi Wu, Yiqun Chen, Changle Qu, Keyi Kong, Wenwen Ye, Lixin Su, Xinyu Ma, Long Xia, Daiting Shi, Jiashu Zhao, Haoyi Xiong, and 2 others. 2025b. Towards ai search paradigm. Preprint, arXiv:2506.17188. — [arXiv:2506.17188](https://arxiv.org/abs/2506.17188) find
  22. Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. CoRR, abs/2503.05592.
  23. Team Qwen. 2024. Qwq: Reflect deeply on the boundaries of the unknown. find
  24. Paul Thomas, Seth Spielman, Nick Craswell, and Bhaskar Mitra. 2024. Large language models can accurately predict searcher preferences. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024,… find
  25. Shivani Upadhyay, Ehsan Kamalloo, and Jimmy Lin. 2024a. Llms can patch up missing relevance judgments in evaluation. CoRR, abs/2405.04727. — [arXiv:2405.04727](https://arxiv.org/abs/2405.04727) find
  26. Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024, pages 2421– 2425.…
  27. Sean MacAvaney and Luca Soldaini. 2023. One-shot labeling for automatic relevance estimation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Shivani Upadhyay, Ronak Pradeep, Nandan Thakur, Nick Craswell, and Jimmy Lin. 2024b. UMBRELA: umbrela… — [arXiv:2406.06519](https://arxiv.org/abs/2406.06519) find
  28. Lai Wei, Yuting Li, Kaipeng Zheng, Chen Wang, Yue Wang, Linghe Kong, Lichao Sun, and Weiran Huang. 2025. Advancing multimodal reasoning via reinforcement learning with cold start. CoRR, abs/2505.22334. — [arXiv:2505.22334](https://arxiv.org/abs/2505.22334) find
  29. Sean J. Welleck. 2016. Efficient AUC optimization for information ranking applications. In Advances in Information Retrieval - 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20-23, 2016. Proceedings, volume 9626 of Lecture Notes in Computer Science, pages 159–170. Springer. find
  30. Xiaohui Xie, Qian Dong, Bingning Wang, Feiyang Lv, Ting Yao, Weinan Gan, Zhijing Wu, Xiangsheng Li, Haitao Li, Yiqun Liu, and Jin Ma. 2023. T2ranking: A large-scale chinese benchmark for passage ranking. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Inf… find
  31. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024. Qwen2.5 technical report. CoRR, abs/2412.1511… find
  32. Wen Zan, Yaopeng Han, Xiaotian Jiang, Yao Xiao, Yang Yang, Dayao Chen, and Sheng Chen. 2023. Spm: Structured pretraining and matching architectures for relevance modeling in meituan search. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM ’23, pag… find
  33. Yu Zhang, Shutong Qiao, Jiaqi Zhang, Tzu-Heng Lin, Chen Gao, and Yong Li. 2025. A survey of large language model empowered agents for recommendation and search: Towards next-generation information retrieval. Preprint, arXiv:2503.05659. — [arXiv:2503.05659](https://arxiv.org/abs/2503.05659) find
  34. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems 36: A…
  35. Shengyao Zhuang, Xueguang Ma, Bevan Koopman, Jimmy Lin, and Guido Zuccon. 2025. Rank-r1: Enhancing reasoning in llm-based document rerankers via reinforcement learning. CoRR, abs/2503.06034. — [arXiv:2503.06034](https://arxiv.org/abs/2503.06034) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yuan-2026-r-textthreesuperiora-reinforced-reasoning-relevance/