Elamur Research⌕
☰
home / papers / 2026 / RiT: Rubrics-in-Thinking Reinforcement Learning for Improved Reasoning in Large Language Models

RiT: Rubrics-in-Thinking Reinforcement Learning for Improved Reasoning in Large Language Models

Paper · 2026 · Xiaobin Tian, Shuai Yuan, Muyun Ding et al.

Summary

Most reinforcement learning for large reasoning models rewards only the final answer. This can let flawed or shortcut reasoning go unpunished. Process reward models address this mainly in checkable domains such as math or code. RiT (Rubrics-in-Thinking RL) instead brings rubric-based supervision to the thinking trace itself, including on open-ended tasks.

A frontier LLM writes task-specific criteria for good reasoning, such as relevance, completeness and logical coherence. It works from a task description and from successful and failed reasoning examples, then critiques and revises its own rubric set. An LLM judge scores each reasoning trajectory against these rubrics. A gated fusion mechanism combines this thinking score with an outcome reward, which limits rewards earned through superficial patterns. Training uses GRPO.

On long-form writing benchmarks and AIME math problems, RiT beats outcome-only RL baselines. The gains hold when different LLMs act as the judge. The work suggests that structured, interpretable feedback on intermediate reasoning can extend process supervision beyond strictly verifiable tasks.

Key points

  • Three-part pipeline: automatic thinking-rubric generation with iterative self-refinement, a gated fusion reward over thinking and response quality, and GRPO optimization with group-wise normalization.
  • Writing benchmarks: on LongBench-Write and WritingBench, RiT scores highest under all five judges (Qwen3-235B-A22B-Instruct, GPT-4o, Gemini-2.5-Pro, Claude Sonnet 4, GPT-5.2). For example, with the Qwen3 judge on LongBench-Write, RiT scores 89.20, against 85.70 for ORM and 84.40 for the base model.
  • Math benchmarks: with Qwen3-8B trained on AIME problems from 1983–2023, RiT matches or exceeds the baselines on AIME 2024 and AIME 2025 under both pass@1 and pass@8.
  • Case study: for a 1,200-word writing task, the ORM model produced 2,119 words. The RiT model planned per-section word budgets while reasoning and produced 1,328 words.
  • Limitations: rubrics receive little human oversight, and the balance between the two rewards needs manual tuning. Experiments only cover models of about 8B parameters.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Xiaobin Tian, Shuai Yuan, Muyun Ding, Haonan Chen and Xiaoxi Jiang. 2026. RiT: Rubrics-in-Thinking Reinforcement Learning for Improved Reasoning in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.192

@inproceedings{tian2026rit,
  title = {{RiT: Rubrics-in-Thinking Reinforcement Learning for Improved Reasoning in Large Language Models}},
  author = {Xiaobin Tian, Shuai Yuan, Muyun Ding, Haonan Chen and Xiaoxi Jiang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.192},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.192}
}

References

46 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. Introducing claude 4. https://www.anthropic.com/news/claude-4. ↗
  2. Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, and Wanli Ouyang. 2024. Mt-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues. In Proceedings of the 62nd Annual Meeting of the Associatio…
  3. Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2025. Longwriter: Unleashing 10, 000+ word generation from long context llms. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenR…
  4. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — Evaluating Large Language Models Trained on Code
  5. Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. CoRR, abs/2503.09567.
  6. DeepSeek-AI. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. CoRR, abs/2501.12948.
  7. Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera Gilaberte, Jakub Pachocki, Jerry Tworek, Lorenz Kuhn, Lukasz Kaiser, Mark Chen, Max Schwarzer, Mostafa Rohaninejad, Nat McAleese, o3 contributors, Oleg Mürk, and 5… find
  8. Lishui Fan, Yu Zhang, Mouxiang Chen, and Zhongxin Liu. 2025. Posterior-grpo: Rewarding reasoning processes in code generation. CoRR, abs/2508.05170. — [arXiv:2508.05170](https://arxiv.org/abs/2508.05170) find
  9. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies. Trans. Assoc. Comput. Linguistics, 9:346–361. find
  10. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Yuanzhuo Wang, and Jian Guo. 2024. A survey on llm-as-a-judge. CoRR, abs/2411.15594. find
  11. Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Bing Liu, and Sean Hendryx. 2025. Rubrics as rewards: Reinforcement learning beyond verifiable domains. CoRR, abs/2507.17746. find
  12. Zenan Huang, Yihong Zhuang, Guoshan Lu, Zeyu Qin, Haokai Xu, Tianyu Zhao, Ru Peng, Jiaqi Hu, Zhanming Shen, Xiaomeng Hu, Xijun Gu, Peiyi Tu, Jiaxin Liu, Wenyu Chen, Yuzhuo Fu, Zhiting Fan, Yanmei Gu, Yuanyuan Wang, Zhengkai Yang, and 2 others. 2025. Reinforcement learning with rubric anchors. CoRR,… — [arXiv:2508.12790](https://arxiv.org/abs/2508.12790) find
  13. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, and 80 others. 20… — OpenAI o1 System Card
  14. Ruipeng Jia, Yunyi Yang, Yongbo Gai, Kai Luo, Shihao Huang, Jianhe Lin, Xiaoxi Jiang, and Guanjun Jiang. 2025. Writing-zero: Bridge the gap between non-verifiable tasks and verifiable rewards. CoRR, abs/2506.00103. — [arXiv:2506.00103](https://arxiv.org/abs/2506.00103) find
  15. Muhammad Khalifa, Rishabh Agarwal, Lajanugen Logeswaran, Jaekyeom Kim, Hao Peng, Moontae Lee, Honglak Lee, and Lu Wang. 2025. Process reward models that think. CoRR, abs/2504.16828. — [arXiv:2504.16828](https://arxiv.org/abs/2504.16828) find
  16. Jinu Lee and Julia Hockenmaier. 2025. Evaluating step-by-step reasoning traces: A survey. CoRR, abs/2502.12289. find
  17. Bill Yuchen Lin, Yuntian Deng, Khyathi Raghavi Chandu, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, and Yejin Choi. 2025. Wildbench: Benchmarking llms with challenging tasks from real users in the wild. In The Thirteenth International Conference on Learning Representations,…
  18. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 3214–3252. Association fo…
  19. Hanmeng Liu, Jian Liu, Leyang Cui, Zhiyang Teng, Nan Duan, Ming Zhou, and Yue Zhang. 2023a. Logiqa 2.0 - an improved dataset for logical reasoning in natural language understanding. IEEE ACM Trans. Audio Speech Lang. Process., 31:2947–2962. find
  20. Tianci Liu, Ran Xu, Tony Yu, Ilgee Hong, Carl Yang, Tuo Zhao, and Haoyu Wang. 2025. Openrubrics: Towards scalable synthetic rubric generation for reward modeling and LLM alignment. CoRR, abs/2510.07743.
  21. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023b. G-eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 2511–2522.…
  22. Yifei Lu, Fanghua Ye, Jian Li, Qiang Gao, Cheng Liu, Haibo Luo, Nan Du, Xiaolong Li, and Feiliang Ren. 2025. Codetool: Enhancing programmatic tool invocation of llms via process supervision. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa… find
  23. Xinji Mai, Haotian Xu, Xing W, Weinong Wang, Yingying Zhang, and Wenqiang Zhang. 2025. Agent RL scaling law: Agent RL with spontaneous code execution for mathematical problem solving. CoRR, abs/2505.07773. — [arXiv:2505.07773](https://arxiv.org/abs/2505.07773) find
  24. Sara Vera Marjanovic, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Lù, Nicholas Meade, Dongchan Shin, Amirhossein Kazemnejad, Gaurav Kamath, Marius Mosbach, Karolina Stanczak, and Siva Reddy. 2025. Deepsee… find
  25. Mathematical Association of America. 2024. American invitational mathematics examination (aime), 2024 edition. https://maa.org/maa-invitational-competitions/. Accessed: 2026-04-14. ↗
  26. Mathematical Association of America. 2025. American invitational mathematics examination (aime), 2025 edition. https://maa.org/maa-invitational-competitions/. Accessed: 2026-04-14. ↗
  27. OpenAI. 2024. Gpt-4o system card. abs/2410.21276.
  28. CoRR, OpenAI. 2025. Introducing gpt-5.2. https://openai.com/index/introducing-gpt-5-2/. ↗
  29. Hao Peng, Yunjia Qi, Xiaozhi Wang, Zijun Yao, Bin Xu, Lei Hou, and Juanzi Li. 2025. Agentic reward modeling: Integrating human preferences with verifiable correctness signals for reliable reward systems. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volu…
  30. Chau Pham, Simeng Sun, and Mohit Iyyer. 2024. Suri: Multi-constraint instruction following in long-form text generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 1722– 1753. Association for Computational Linguistics.
  31. Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou, Jiaheng Liu, Wenge Rong, Zekun Moore Wang, Jian Yang, Ge Zhang, Junran Peng, Zhaoxiang Zhang, Songyang Zhang, and Kai Chen. 2024. Hellobench: Evaluating long text generation capabilities of large language models. CoRR, abs/2409.16191. — [arXiv:2409.16191](https://arxiv.org/abs/2409.16191) find
  32. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. CoRR, abs/1707.06347. find
  33. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300.
  34. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient RLHF framework. In Proceedings of the Twentieth European Conference on Computer Systems, EuroSys 2025, Rotterdam, The Netherlands, 30 March 20…
  35. Gemini Team. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. CoRR, abs/2507.06261.
  36. Haozhe Wang, Haoran Que, Qixin Xu, Minghao Liu, Wangchunshu Zhou, Jiazhan Feng, Wanjun Zhong, Wei Ye, Tong Yang, Wenhao Huang, Ge Zhang, and Fangzhen Lin. 2025a. Reverse-engineered reasoning for open-ended generation. CoRR, abs/2509.06160. — [arXiv:2509.06160](https://arxiv.org/abs/2509.06160) find
  37. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. 2025. Group sequence policy optimization. CoRR, abs/2507.18071.
  38. Pengkai Wang, Qi Zuo, Pengwei Liu, Zhijie Sang, Congkai Xie, and Hongxia Yang. 2025b. Infimed-orbit: Aligning llms on open-ended complex tasks via rubric-based incremental training. CoRR, abs/2510.15859. — [arXiv:2510.15859](https://arxiv.org/abs/2510.15859) find
  39. Yang Zhou, Sunzhu Li, Shunyu Liu, Wenkai Fang, Jiale Zhao, Jingwen Yang, Jianwei Lv, Kongcheng Zhang, Yihe Zhou, Hengtong Lu, Wei Chen, Yan Xie, and Mingli Song. 2025. Breaking the exploration bottleneck: Rubric-scaffolded reinforcement learning for general LLM reasoning. CoRR, abs/2508.16949. — [arXiv:2508.16949](https://arxiv.org/abs/2508.16949) find
  40. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processi…
  41. Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and Fei Huang. 2025. Writingbench: A comprehensive benchmark for generative writing. CoRR, abs/2503.05244. — [arXiv:2503.05244](https://arxiv.org/abs/2503.05244) find
  42. Wei Xiong, Wenting Zhao, Weizhe Yuan, Olga Golovneva, Tong Zhang, Jason Weston, and Sainbayar Sukhbaatar. 2025. Stepwiser: Stepwise generative judges for wiser reasoning. CoRR, abs/2508.19229. — [arXiv:2508.19229](https://arxiv.org/abs/2508.19229) find
  43. Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, Chenyang Shao, Yuwei Yan, Qinglong Yang, Yiwen Song, Sijian Ren, Xinyuan Hu, Yu Li, Jie Feng, Chen Gao, and Yong Li. 2025. Towards large reasoning models: A survey o… find
  44. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 40 others. 2025. Qwen3 technical report. CoRR, abs/2505.09388.
  45. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, and 16 others. 2025a. DAPO: an open-source LLM reinforcement learning sy…
  46. Zhuohao Yu, Weizheng Gu, Yidong Wang, Xingru Jiang, Zhengran Zeng, Jindong Wang, Wei Ye, and Shikun Zhang. 2025b. Reasoning through execution: Unifying process and outcome rewards for code generation. In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, Jul… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tian-2026-rit-rubrics-thinking-reinforcement-learning/