Elamur Research⌕
☰
home / papers / 2026 / RADO: Reasoning Audit-Driven Optimization for Rigorous Reasoning in High-Stakes Domains

RADO: Reasoning Audit-Driven Optimization for Rigorous Reasoning in High-Stakes Domains

Paper · 2026 · Zhijie Tan, Xu Chu, Guanyu Wang et al.

Summary

Reinforcement learning for LLM reasoning usually rewards only the final answer. A model can therefore reach correct conclusions through faulty intermediate logic, which is a serious problem in finance, law and biomedicine. RADO exploits an asymmetry: finding one local flaw in a derivation is easier than producing a fully correct one. The pipeline has three stages:

  1. A Reward Model (RM) is fine-tuned via SFT on Qwen2.5-Instruct-7B outputs. DeepSeek-v3.2 scored each output separately for reasoning and answer correctness, and humans checked the scores.
  2. An Audit Model (AM) reviews the RM's judgments. The AM can call web search and Python computation. Its findings become preference data, and the RM is refined with DPO over several iterations.
  3. The upgraded RM then trains the policy with GRPO, assigning rewards that reflect both how sound the reasoning is and whether the final answer is correct.

On five public benchmarks spanning three high-stakes domains, RADO improves both accuracy and reasoning rigor over the base model. Tool-grounded auditing also helps guard against reward hacking. The approach suits any setting where the explanation must be trustworthy, not only the output.

Key points

  • The base Policy Model is Qwen2.5-Instruct-7B. The teacher/annotator is DeepSeek-v3.2, used with human-in-the-loop verification. About 2k questions each were drawn from FinQA, LegalBench and PubMedQA to initialize the RM and the AM.
  • Evaluation covers five public datasets, including FinQA, LegalBench, PubMedQA and MMLU, across finance, law and biomedicine.
  • On LegalBench, RADO-legal scores 81.11, compared with 77.20 without tool-augmented auditing and 73.86 for the base Qwen2.5-Instruct-7B.
  • On MMLU (biomedicine), RADO scores 80.54, compared with 74.81 without tools and 75.36 for the base model. Without tools, the result falls below the baseline.
  • Training directly on locally flawed negative samples gives little gain. Using those samples to sharpen the Audit Model, which in turn improves the RM, does boost the policy.
  • The authors list three limitations: extra training cost, dependence on the quality of external tools, and untested generalization beyond these domains.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Zhijie Tan, Xu Chu, Guanyu Wang, Ziyu Li, Weiping Li and Tong Mo. 2026. RADO: Reasoning Audit-Driven Optimization for Rigorous Reasoning in High-Stakes Domains. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.213

@inproceedings{tan2026rado,
  title = {{RADO: Reasoning Audit-Driven Optimization for Rigorous Reasoning in High-Stakes Domains}},
  author = {Zhijie Tan, Xu Chu, Guanyu Wang, Ziyu Li, Weiping Li and Tong Mo},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.213},
  url = {https://doi.org/10.18653/v1/2026.acl-long.213}
}

References

63 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  2. AI@Meta. 2024. The llama 3 herd of models. ArXiv, abs/2407.21783.
  3. Doria Bonzi, Alexandre Guiggi, Frédéric Béchet, Carlos Ramisch, and Benoit Favre. 2025. Caremede-val dataset: Evaluating critical appraisal and reasoning in the biomedical field. arXiv preprint arXiv:2511.03441. find
  4. Umar Butler. 2023. Open australian legal llm. find
  5. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. FinQA: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natural…
  6. Daixuan Cheng, Yuxian Gu, Shaohan Huang, Junyu Bi, Minlie Huang, and Furu Wei. 2024a. Instruction pre-training: Language models are supervised multitask learners. arXiv preprint arXiv:2406.14491.
  7. Daixuan Cheng, Shaohan Huang, and Furu Wei. 2024b. Adapting large language models via reading comprehension. In The Twelfth International Conference on Learning Representations. find
  8. Inyoung Cheong, King Xia, KJ Kevin Feng, Quan Ze Chen, and Amy X Zhang. 2024. (a) i am not a lawyer, but...: Engaging legal experts towards responsible llm policies for legal advice. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 2454–2469. find
  9. Xu Chu, Zhijie Tan, Hanlin Xue, Guanyu Wang, Tong Mo, and Weiping Li. 2025. Domaino1s: Guiding LLM reasoning for explainable answers in high-stakes domains. In Findings of the Association for Computational Linguistics: ACL 2025, pages 3275– 3293. find
  10. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023. Ultrafeedback: Boosting language models with high-quality feedback. CoRR. find
  11. DeepSeek-AI. 2025. Deepseek-v3.2: Pushing the frontier of open large language models.
  12. Chenlong Deng, Kelong Mao, and Zhicheng Dou. 2024. Learning interpretable legal case retrieval via knowledge-guided case reformulation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1253–1265. find
  13. Ken Deng, Zizheng Zhan, Wen Xiang, Wenqiang Zhu, Weihao Li, Jingxuan Xu, Tianhao Peng, Xinping Lei, Kun Wu, Yifan Yao, and 1 others. 2025. Hipo: Hybrid policy optimization for dynamic reasoning in llms. arXiv preprint arXiv:2509.23967. — [arXiv:2509.23967](https://arxiv.org/abs/2509.23967) find
  14. Ricardo Dominguez-Olmedo, Vedant Nanda, Rediet Abebe, Stefan Bechtold, Christoph Engel, Jens Frankenreiter, Krishna Gummadi, Moritz Hardt, and Michael Livermore. 2024. Lawma: The power of specialization for legal tasks. arXiv preprint arXiv:2407.16615. find
  15. Ricardo Dominguez-Olmedo, Vedant Nanda, Rediet Abebe, Stefan Bechtold, Christoph Engel, Jens Frankenreiter, Krishna P Gummadi, Moritz Hardt, and Michael Livermore. 2025. Lawma: The power of specialization for legal annotation. In The Thirteenth International Conference on Learning Representations. find
  16. Lishui Fan, Yu Zhang, Mouxiang Chen, and Zhongxin Liu. 2025. Posterior-grpo: Rewarding reasoning processes in code generation. arXiv preprint arXiv:2508.05170. — [arXiv:2508.05170](https://arxiv.org/abs/2508.05170) find
  17. Neel Guha, Julian Nyarko, Daniel Ho, Christopher Ré, Adam Chilton, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, and 1 others. 2023. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in neural informati…
  18. Neel Guha, Julian Nyarko, Daniel Ho, Christopher Ré, Adam Chilton, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, and 1 others. 2024. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in Neural Informati…
  19. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  20. Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. 2021. Cuad: An expert-annotated nlp dataset for legal contract review. arXiv preprint arXiv:2103.06268.
  21. Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, and Rishabh Agarwal. 2024. V-star: Training verifiers for self-taught reasoners. In Conference on Language Modeling.
  22. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421.
  23. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language proce…
  24. Kelvin JL Koa, Yunshan Ma, Ritchie Ng, and Tat-Seng Chua. 2024. Learning to generate explainable stock predictions using self-reflective large language models. In Proceedings of the ACM on Web Conference 2024, pages 4304–4315. find
  25. Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024a. Biomistral: A collection of open-source pretrained large language models for medical domains. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5848–5864, Ban…
  26. Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024b. Biomistral: A collection of open-source pretrained large language models for medical domains. arXiv preprint arXiv:2402.10373.
  27. Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and 1 others. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv eprints, pages arXiv–2309. find
  28. Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, and 1 others. 2025. From generation to judgment: Opportunities and challenges of llm-as-a-judge. In Proceedings of the 2025 Conference on Empirical Methods… find
  29. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  30. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025a. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
  31. Zhaowei Liu, Xin Guo, Fangqi Lou, Lingfeng Zeng, Jinyi Niu, Zixuan Wang, Jiajie Xu, Weige Cai, Ziwei Yang, Xueqian Zhao, and 1 others. 2025b. Fin-r1: A large language model for financial reasoning through reinforcement learning. arXiv preprint arXiv:2503.16252. — [arXiv:2503.16252](https://arxiv.org/abs/2503.16252) find
  32. OpenAI. 2025. Gpt-5.1. https://openai.com/ index/gpt-5-1/. Large language model (released 13 November 2025). ↗
  33. Lingfei Qian, Weipeng Zhou, Yan Wang, Xueqing Peng, Jimin Huang, and Qianqian Xie. 2025. Fino1: On the transferability of reasoning enhanced llms to finance. arXiv preprint arXiv:2502.08127. — [arXiv:2502.08127](https://arxiv.org/abs/2502.08127) find
  34. Qwen-Team. 2024. Qwen2.5: A party of foundation models. find
  35. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  36. Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, and 1 others. 2024. Capabilities of gemini models in medicine. arXiv preprint arXiv:2404.18416. find
  37. Zhihong Shao, Yuxiang Luo, Chengda Lu, ZZ Ren, Jiewen Hu, Tian Ye, Zhibin Gou, Shirong Ma, and Xiaokang Zhang. 2025. Deepseekmath-v2: Towards self-verifiable mathematical reasoning. arXiv preprint arXiv:2511.22570.
  38. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  39. Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, and Yike Guo. 2025. Legalreasoner: Step-wised verification-correction for legal judgment reasoning. arXiv preprint arXiv:2506.07443.
  40. Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. 2025. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity. arXiv preprint arXiv:2506.06941. find
  41. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, and 1 others. 2025. Toward expert-level medical question answering with large language models. Nature Medicine, 31(3):943–950. find
  42. Mirza Farhan Bin Tarek and Rahmatollah Beheshti. 2025. Reward hacking mitigation using verifiable composite rewards. arXiv preprint arXiv:2509.15557. — [arXiv:2509.15557](https://arxiv.org/abs/2509.15557) find
  43. Gemini Team and Google. 2025. Gemini 3: Frontier models for advanced multimodal reasoning and efficiency. Google Technical Report. find
  44. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275.
  45. Cheng Wang, Yue Liu, Baolong Bi, Duzhen Zhang, Zhong-Zhi Li, Yingwei Ma, Yufei He, Shengju Yu, Xinfeng Li, Junfeng Fang, Jiaheng Zhang, and Bryan Hooi. 2025. Safety in large reasoning models: A survey. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 3468–3482. find
  46. Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang, and 1 others. 2023. Disc-lawllm: Fine-tuning large language models for intelligent legal services. arXiv preprint arXiv:2309.11325. find
  47. Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. 2023. Huatuo: Tuning llama model with chinese medical knowledge. arXiv preprint arXiv:2304.06975.
  48. Yilun Zhao, Yitao Long, Hongjun Liu, Ryo Kamoi, Linyong Nan, Lyuhao Chen, Yixin Liu, Xiangru Tang, Rui Zhang, and Arman Cohan. 2024a. Docmath-eval: Evaluating math reasoning capabilities of llms in understanding long and specialized documents. In Proceedings of the 62nd Annual Meeting of the Associa… find
  49. Tianjun Wei, Wei Wen, Ruizhi Qiao, Xing Sun, and Jianghong Ma. 2025. Rocketeval: Efficient automated llm evaluation via grading checklist. arXiv preprint arXiv:2503.05142. — [arXiv:2503.05142](https://arxiv.org/abs/2503.05142) find
  50. Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2023. Generating sequences by learning to self-correct. In The Eleventh International Conference on Learning Representations.
  51. Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Weidi Xie, and Yanfeng Wang. 2024. Pmc-llama: toward building open-source language models for medicine. Journal of the American Medical Informatics Association, 31(9):1833–1843. find
  52. LLM Xiaomi, Bingquan Xia, Bowen Shen, Dawei Zhu, Di Zhang, Gang Wang, Hailin Zhang, Huaqiu Liu, Jiebao Xiao, Jinhao Dong, and 1 others. 2025. Mimo: Unlocking the reasoning potential of language model– from pretraining to posttraining. arXiv preprint arXiv:2505.07608. find
  53. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  54. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476.
  55. Haohan Yuan and Haopeng Zhang. 2025a. Domainsum: A hierarchical benchmark for fine-grained domain shift in abstractive text summarization. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 2219–2231. find
  56. Haohan Yuan and Haopeng Zhang. 2025b. Understanding llm reasoning for abstractive summarization. arXiv preprint arXiv:2512.03503.
  57. Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason E Weston. 2024. Self-rewarding language models. In Forty-first International Conference on Machine Learning. find
  58. Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, Wenmeng Zhou, and Yingda Chen. 2024b. Swift:a scalable lightweight infrastructure for fine-tuning. Preprint, arXiv:2408.05517. — [arXiv:2408.05517](https://arxiv.org/abs/2408.05517) find
  59. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, and 1 others. 2025a. Group sequence policy optimization. arXiv preprint arXiv:2507.18071.
  60. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623.
  61. Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, and Daniel E. Ho. 2021. When does pretraining help? assessing self-supervised learning for law and the casehold dataset. In Proceedings of the 18th International Conference on Artificial Intelligence and Law. Association for Computing Mac… find
  62. Yanjun Zheng, Xiyang Du, Longfei Liao, Xiaoke Zhao, Zhaowen Zhou, Jingze Song, Bo Zhang, Jiawei Liu, Xiang Qi, Zhe Li, and 1 others. 2025b. Agentar-fin-r1: Enhancing financial intelligence through domain expertise, training efficiency, and advanced reasoning. arXiv preprint arXiv:2507.16802. — [arXiv:2507.16802](https://arxiv.org/abs/2507.16802) find
  63. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tan-2026-rado-reasoning-audit-driven-optimization/