Elamur Research⌕
☰
home / papers / 2026 / R³: End-to-End Reasoning-based Planning for Multi-step Retrosynthesis via Reinforcement Learning

R³: End-to-End Reasoning-based Planning for Multi-step Retrosynthesis via Reinforcement Learning

Paper · 2026 · YiFei Wang, Qizhi Pei, Jiangtao Feng et al.

Summary

Multi-step retrosynthesis means breaking a target molecule down, step by step, into purchasable building blocks. It is usually solved by tree search (MCTS, Retro*) built on single-step predictors. That approach is slow, opaque, and limited by how good the single-step model is. The authors propose R³ (Reinforced Reasoning Retrosynthesis), which replaces search with generation: a single LLM reasons like a chemist and writes the whole synthetic route at once.

Training starts from Qwen3-8B-Base and has three phases:

  1. Continual pretraining on scientific text, reaction data, and IUPAC↔SMILES translation.
  2. Supervised fine-tuning on structured chain-of-thought traces distilled from Intern-S1.
  3. Reinforcement learning with GSPO and a binary reward. A route earns the reward only if every SMILES is valid and every leaf is a building block.

On Retrobench, R³ reaches 43.7% Top-1 and 58.7% Top-5 accuracy, a new state of the art. It also runs about 120× faster than FusionRetro + Retro*. The work suggests that LLM reasoning can be a practical and more interpretable alternative to search-based planners.

Key points

  • Continual pretraining corpora:
    • Stage 1: PubChem, PubMed, bioRxiv, and the biology/chemistry subsets of FineFineWeb, plus FineWeb-Edu to keep general skills.
    • Stage 2: reactions from ORD and USPTO, plus bidirectional IUPAC–SMILES tasks.
  • Structured reasoning format: each step in the teacher-distilled traces follows a 5-tuple schema (reaction type, key disconnection, rationale, etc.). Fine-tuning on free-form chain-of-thought instead yields only 4.0% accuracy.
  • Ablations (Top-1):
    • Base model: 0.0%.
    • Base+SFT+RL: 29.2%.
    • CPT+SFT: 33.1%.
    • Full pipeline with RL: 43.7%.
  • Building-block awareness: in a probe built from Retrobench, Qwen3-8B correctly identifies commercially available building blocks at 82.0% Pass@1 and 94.3% Pass@3.
  • Limitations: evaluation relies on exact matches of starting materials against Retrobench's few reference routes. Valid alternative routes may therefore go uncredited.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Renmin University of ChinaBeijing · China114 works · 16,118 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China117 works · 11,281 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 worksComputational Theory and Mathematics · 252 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as YiFei Wang, Qizhi Pei, Jiangtao Feng, Yuntian Shi, Yi Duan, Lihao Wang, Lei Bai, Lijun Wu, Wei-Ying Ma and Hao Zhou. 2026. R³: End-to-End Reasoning-based Planning for Multi-step Retrosynthesis via Reinforcement Learning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1745

@inproceedings{wang2026end,
  title = {{R³: End-to-End Reasoning-based Planning for Multi-step Retrosynthesis via Reinforcement Learning}},
  author = {YiFei Wang, Qizhi Pei, Jiangtao Feng, Yuntian Shi, Yi Duan, Lihao Wang, Lei Bai, Lijun Wu, Wei-Ying Ma and Hao Zhou},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1745},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1745}
}

References

51 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chenglong Kang, Xiaoyi Liu, and Fei Guo. 2025. Retrointext: A multimodal large language model enhanced framework for retrosynthetic planning via in-context representation learning. In The Thirteenth International Conference on Learning Representations. find
  2. Lei Bai, Zhongrui Cai, Yuhang Cao, Maosong Cao, Weihan Cao, Chiyu Chen, Haojiong Chen, Kai Chen, Pengcheng Chen, Ying Chen, and 1 others. 2025. Intern-s1: A scientific multimodal foundation model. arXiv preprint arXiv:2508.15763.
  3. He Cao, Yanjun Shao, Zhiyuan Liu, Zijing Liu, Xiangru Tang, Yuan Yao, and Yu Li. 2024. Presto: Progressive pretraining enhances synthetic chemistry outcomes. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10197– 10224. find
  4. Binghong Chen, Chengtao Li, Hanjun Dai, and Le Song. 2020. Retro*: Learning retrosynthetic planning with neural guided a* search. In International Conference on Machine Learning, pages 1608–1616. PMLR. find
  5. Connor W. Coley, Luke Rogers, William H. Green, and Klavs F. Jensen. 2017. Computer-assisted retrosynthesis based on molecular similarity. ACS Central Science, 3(12):1237–1245. find
  6. Pavel Karpov, Guillaume Godin, and Igor V. Tetko. 2019. A Transformer Model for Retrosynthesis, page 817–830. Springer International Publishing. find
  7. Steven M Kearnes, Michael R Maser, Michael Wleklinski, Anton Kast, Abigail G Doyle, Spencer D Dreher, Joel M Hawkins, Klavs F Jensen, and Connor W Coley. 2021a. The open reaction database. Journal of the American Chemical Society, 143(45):18820– 18826. find
  8. Steven M Kearnes, Michael R Maser, Michael Wleklinski, Anton Kast, Abigail G Doyle, Spencer D Dreher, Joel M Hawkins, Klavs F Jensen, and Connor W Coley. 2021b. The open reaction database. Journal of the American Chemical Society, 143(45):18820– 18826. find
  9. Elias James Corey. 1991. The logic of chemical synthesis: Multistep synthesis of complex carbogenic molecules (nobel lecture). Angewandte Chemie International Edition in English, 30(5):455–465. find
  10. Junsu Kim, Sungsoo Ahn, Hankook Lee, and Jinwoo Shin. 2021. Self-improved retrosynthetic planning. In International Conference on Machine Learning, pages 5486–5495. PMLR. find
  11. Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, and Le Song. 2019. Retrosynthesis prediction with conditional graph logic network. Advances in Neural Information Processing Systems, 32. find
  12. Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, and 1 others. 2025. find
  13. Pubchem 2025 update. Nucleic acids research, 53(D1):D1516–D1525. find
  14. Nathan C. Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Gómez-Bombarelli, Connor W. Coley, and Vijay Gadepally. 2023. Neural scaling of deep chemical models. Nature Machine Intelligence, 5(11):1297–1305. find
  15. Google. 2025. Gemini 3 Pro Model Card. https://deepmind.google/models/gemini/pro/. Accessed: 2025-12-28. ↗
  16. Daya Guo and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint.
  17. Siqi Hong, Hankz Hankui Zhuo, Kebing Jin, Guang Shao, and Zhanwen Zhou. 2023. Retrosynthetic planning with experience-guided monte carlo tree search. Communications Chemistry, 6(1). find
  18. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.
  19. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, and 7 o… — Competition-Level Code Generation with AlphaCode
  20. Guoqing Liu, Di Xue, Shufang Xie, Yingce Xia, Austin Tripp, Krzysztof Maziarz, Marwin Segler, Tao Qin, Zongzhang Zhang, and Tie-Yan Liu. 2023a. Retrosynthetic planning with dual value networks. In International conference on machine learning, pages 22266–22276. PMLR. find
  21. Songtao Liu, Hanjun Dai, Yue Zhao, and Peng Liu. 2024a. Preference optimization for molecule synthesis with conditional residual energy-based models. arXiv preprint arXiv:2406.02066. — [arXiv:2406.02066](https://arxiv.org/abs/2406.02066) find
  22. Songtao Liu, Zhengkai Tu, Minkai Xu, Zuobai Zhang, Lu Lin, Rex Ying, Jian Tang, Peilin Zhao, and Dinghao Wu. 2023b. Fusionretro: molecule representation fusion via in-context learning for retrosynthetic planning. In International Conference on Machine Learning, pages 22028–22041. PMLR. find
  23. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  24. Wei Liu, Jiangtao Feng, Hongli Yu, Yuxuan Song, Yuqiang Li, Shufei Zhang, Lei Bai, Wei-Ying Ma, and Hao Zhou. 2025. Retro-r1: Llm-based agentic retrosynthesis. arXiv preprint. Submitted to NeurIPS 2025. find
  25. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256.
  26. Yifeng Liu, Hanwen Xu, Tangqi Fang, Haocheng Xi, Zixuan Liu, Sheng Zhang, Hoifung Poon, and Sheng Wang. 2024b. T-rex: Text-assisted retrosynthesis prediction. arXiv preprint arXiv:2401.14637. find
  27. Mujeen Sung, Minbyul Jeong, Yonghwa Choi, Donghyeon Kim, Jinhyuk Lee, and Jaewoo Kang. 2022. Bern2: an advanced neural biomedical named entity recognition and normalization tool. Bioinformatics, 38(20):4837–4839. find
  28. Daniel Mark Lowe. 2012. Extraction of chemical structures and reactions from the literature. Ph.D. thesis. find
  29. Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. 2024. Fineweb-edu: the finest collection of educational content. find
  30. M-A-P, Ge Zhang, Xinrun Du, Zhimiao Yu, Zili Wang, Zekun Wang, Shuyue Guo, Tianyu Zheng, Kang Zhu, Jerry Liu, Shawn Yue, Binbin Liu, Zhongyuan Peng, Yifan Yao, Jack Yang, Ziming Li, Bingni Zhang, Minghao Liu, Tianyu Liu, and 6 others. 2024. Finefineweb: A comprehensive study on fine-grained domain w… find
  31. OpenAI. 2025. GPT-5 System Card. https://openai.com/zh-Hans-CN/index/gpt-5-system-card/. Accessed: 2025-12-28. ↗
  32. Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan. 2024. Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning. In Findings of the Association for Computational Linguistics ACL 2024, pages 1216–1240. find
  33. Mikołaj Sacha, Mikołaj Błaż, Piotr Byrski, Paweł D abrowski-Tumański, Mikołaj Chromiński, Rafał Loska, Paweł Włodarczyk-Pruszyński, and Stanisław Jastrz˛ebski. 2021. Molecule edit graph attention network: Modeling chemical reactions as sequences of graph edits. Journal of Chemical Information a… find
  34. Marwin H. S. Segler and Mark P. Waller. 2017. Neural-symbolic machine learning for retrosynthesis and reaction prediction. Chemistry – A European Journal, 23(25):5966–5971. find
  35. Marwin HS Segler, Mike Preuss, and Mark P Waller. 2018. Planning chemical syntheses with deep neural networks and symbolic ai. Nature, 555(7698):604– 610. find
  36. Richard Sever, Ted Roeder, Samantha Hindle, Linda Sussman, Kevin-John Black, Janet Argentine, Wayne Manos, and John R Inglis. 2019. biorxiv: the preprint server for biology. BioRxiv, page 833400. find
  37. Trieu H. Trinh, Yuhuai Wu, Quoc V. Le, He He, and Thang Luong. 2024. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476–482.
  38. Haorui Wang, Jeff Guo, Lingkai Kong, Rampi Ramprasad, Philippe Schwaller, Yuanqi Du, and Chao Zhang. 2025. Llm-augmented chemical synthesis and design decision programs. Preprint, arXiv:2505.07027. — [arXiv:2505.07027](https://arxiv.org/abs/2505.07027) find
  39. Jacob White. 2020. Pubmed 2.0. Medical reference services quarterly, 39(4):382–387. find
  40. Shufang Xie, Rui Yan, Peng Han, Yingce Xia, Lijun Wu, Chenjuan Guo, Bin Yang, and Tao Qin. 2022. Retrograph: Retrosynthetic planning with graph search. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’22, page 2120–2129, New York, NY, USA. Association for… find
  41. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025a. Qwen3 technical report. arXiv preprint.
  42. Yifei Yang, Runhan Shi, Zuchao Li, Shu Jiang, Bao-Liang Lu, Qibin Zhao, Yang Yang, and Hai Zhao. 2025b. Batgpt-chem: A foundation large model for chemical engineering. Research, 8. find
  43. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, YuYue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, LingJun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, and 17 others. 2025. DAPO: An open-source LLM reinforcement learning sy…
  44. Situo Zhang, Hanqi Li, Lu Chen, Zihan Zhao, Xuanze Lin, Zichen Zhu, Bo Chen, Xin Chen, and Kai Yu. 2025a. Reasoning-driven retrosynthesis prediction with large language models via reinforcement learning. arXiv preprint arXiv:2507.17448. — [arXiv:2507.17448](https://arxiv.org/abs/2507.17448) find
  45. Xuefeng Zhang, Haowei Lin, Muhan Zhang, Yuan Zhou, and Jianzhu Ma. 2025b. A data-driven group retrosynthesis planning model inspired by neurosymbolic programming. Nature Communications, 16(1). find
  46. Dengwei Zhao, Shikui Tu, and Lei Xu. 2024. Efficient retrosynthetic planning with mcts exploration enhanced a* search. Communications Chemistry, 7(1). find
  47. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. 2025. Group sequence policy optimization. Preprint, arXiv:2507.18071.
  48. Shuangjia Zheng, Tao Zeng, Chengtao Li, Binghong Chen, Connor W. Coley, Yuedong Yang, and Ruibo Wu. 2022. Deep learning driven biosynthetic pathways navigation for natural products with bionavi-np. Nature Communications, 13(1). find
  49. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),…
  50. Weihe Zhong, Ziduo Yang, and Calvin Yu-Chian Chen. 2023. Retrosynthesis prediction using an end-to-end graph generative architecture for molecular graph editing. Nature Communications, 14(1). find
  51. Zipeng Zhong, Jie Song, Zunlei Feng, Tiantao Liu, Lingxiang Jia, Shaolun Yao, Min Wu, Tingjun Hou, and Mingli Song. 2022. Root-aligned smiles: a tight representation for chemical reaction prediction. Chemical Science, 13(31):9023–9034. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-r-3-end-end-reasoning/