Elamur Research⌕
☰
home / papers / 2026 / MR-ALIGN: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models

MR-ALIGN: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models

Paper · 2026 · Xinming Wang, Jian Xu, Bin Yu et al.

Summary

Large reasoning models (LRMs) handle multi-step problems well, but extended thinking adds little on fact-based questions. The authors trace part of this to a "reasoning–answer hit gap": the correct fact appears in the model's chain of thought, but the model later drops it and gives a different final answer. MR-ALIGN addresses this by aligning the reasoning process rather than only the output, without retrieval or external verifiers. Training questions from NQ-Open and SciQ are answered by Qwen3-8B with thinking on and off. Cases where thinking fixes the answer are kept as positives, and cases where it breaks the answer are kept as negatives. LLM annotators label each reasoning segment with meta-reasoning categories. The method then estimates transition probabilities between these states and converts them into a segment-level implicit reward. This reward favours helpful reasoning moves and penalises harmful ones. Across four factual QA benchmarks and one long-form factuality benchmark, accuracy rises and misleading reasoning falls. The work suggests that process-level alignment is a lightweight route to more truthful LRMs.

Key points

  • Training data: 6,973 candidates were built from NQ-Open and SciQ (4,830 positive, 2,143 negative). Each question was sampled three times at temperature 0.2. Candidates were filtered to 4–15 reasoning segments and 450–1000 reasoning tokens.

  • Annotation pipeline:

    • GPT-5 produced 2,713 free-form labels, which were clustered into a closed taxonomy of 15 labels.
    • These 15 labels fall into four macro-strategies: Meta-cognitive Regulation, Problem-Solving Operations, Knowledge Operations, and Explanatory & Communication.
    • Segment-level labelling was done by DeepSeek-chat and GPT-4o, with GPT-5 acting as adjudicator.

  • Where the gains come from: The largest improvements appear on "Inconsist" cases, where the base model's thinking and answer disagree. Qwen3-8B and Qwen3-4B improve by over 10% on these cases for NQ-Open and SciQ. "Both Wrong" cases barely change, which suggests the method does not extend the model's knowledge boundary.

  • Results on Llama-3.1-Nemotron-Nano-4B-v1.1:

    BenchmarkAccuracy (base → MR-ALIGN)Misinformation (base → MR-ALIGN)
    NQ-Open13.99 → 14.969.47 → 7.48
    SciQ52.40 → 54.4025.90 → 20.40

    Gains on Qwen3-14B are smaller. Across settings, SFT and KTO baselines behave less stably than MR-ALIGN.

  • Mechanistic shift and limitations: After alignment, reasoning chains become shorter and move toward evidence-seeking, quality control, and synthesis-driven conclusions. The authors list two limitations: possible bias from using LLMs as annotators, and no testing on models above 14B or on MoE models. Code is available at github.com/gudehhh666/MR-ALIGN.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢TencentShenzhen · China134 works · 19,743 citations🏛️Chinese Academy of SciencesBeijing · China194 works · 23,906 citations🔬Institute of AutomationBeijing · China70 works · 14,251 citations🎓University of Chinese Academy of SciencesBeijing · China147 works · 8,480 citations🎓Nankai UniversityTianjin · China24 works · 762 citations🎓University College of Applied ScienceGaza · Palestine3 works · 2 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Xinming Wang, Jian Xu, Bin Yu, Sheng Lian, yi Chen, Boran Wang, Yingjian Zhu, Hongzhu Yi, Hong-Ming Yang, Han Hu, Cheng-Lin Liu and Xu-Yao Zhang. 2026. MR-ALIGN: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.204

@inproceedings{wang2026mr,
  title = {{MR-ALIGN: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models}},
  author = {Xinming Wang, Jian Xu, Bin Yu, Sheng Lian, yi Chen, Boran Wang, Yingjian Zhu, Hongzhu Yi, Hong-Ming Yang, Han Hu, Cheng-Lin Liu and Xu-Yao Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.204},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.204}
}

References

52 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mubashara Akhtar, Michael Schlichtkrull, and Andreas Vlachos. 2024. Ev2r: Evaluating evidence retrieval in automated fact-checking. arXiv preprint arXiv:2411.05375. find
  2. Mihael Ankerst, Markus M. Breunig, Hans-Peter Kriegel, and Jörg Sander. 1999. Optics: ordering points to identify the clustering structure. SIGMOD Rec., 28(2):49–60. find
  3. Mingda Chen, Yang Li, Karthik Padthe, Rulin Shao, Alicia Sun, Luke Zettlemoyer, Gargi Ghosh, and Wen-tau Yih. 2024. Improving factuality with explicit working memory. arXiv preprint arXiv:2412.18069. find
  4. Xilun Chen, Ilia Kulikov, Vincent-Pierre Berges, Barlas Oğuz, Rulin Shao, Gargi Ghosh, Jason Weston, and Wen-tau Yih. 2025. Learning to reason for factuality. arXiv preprint arXiv:2508.05618. — [arXiv:2508.05618](https://arxiv.org/abs/2508.05618) find
  5. Roi Cohen, Russa Biswas, and Gerard de Melo. 2025. Infact: Informativeness alignment for improved llm factuality. arXiv preprint arXiv:2505.20487. find
  6. Arthur P Dempster, Nan M Laird, and Donald B Rubin. 1977. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological), 39(1):1–22. find
  7. Xingyu Deng, Xi Wang, and Mark Stevenson. 2025a. + verirel: Verification feedback to enhance document retrieval for scientific fact checking. arXiv preprint arXiv:2508.11122. — [arXiv:2508.11122](https://arxiv.org/abs/2508.11122) find
  8. Yong Deng, Guoqing Wang, Zhenzhe Ying, Xiaofeng Wu, Jinzhen Lin, Wenwen Xiong, Yuqin Dai, Shuo Yang, Zhanwei Zhang, Qiwen Wang, and 1 others. 2025b. Atom-searcher: Enhancing agentic deep research via fine-grained atomic thought reward. arXiv preprint arXiv:2508.12800. — [arXiv:2508.12800](https://arxiv.org/abs/2508.12800) find
  9. Guanting Dong, Jiajie Jin, Xiaoxi Li, Yutao Zhu, Zhicheng Dou, and Ji-Rong Wen. 2025. RAG-critic: Leveraging automated critic-guided agentic workflow for retrieval augmented generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)…
  10. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306. find
  11. Mehdi Fatemi, Banafsheh Rafiee, Mingjie Tang, and Kartik Talamadupula. 2025. Concise reasoning via reinforcement learning. arXiv preprint arXiv:2504.05185. — [arXiv:2504.05185](https://arxiv.org/abs/2504.05185) find
  12. Stephen M Fleming. 2024. Metacognition and confidence: A review and synthesis. Annual Review of Psychology, 75(1):241–268. find
  13. Yuzhe Gu, Wenwei Zhang, Chengqi Lyu, Dahua Lin, and Kai Chen. 2025. Mask-dpo: Generalizable fine-grained factuality alignment of llms. arXiv preprint arXiv:2503.02846. — [arXiv:2503.02846](https://arxiv.org/abs/2503.02846) find
  14. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  15. Keith J Holyoak and Hongjing Lu. 2021. Emergence of relational reasoning. Current Opinion in Behavioral Sciences, 37:118–124. find
  16. Sam Houliston, Ambroise Odonnat, Charles Arnal, and Vivien Cabannes. 2025. Provable benefits of in-tool learning for large language models. arXiv preprint arXiv:2508.20755. — [arXiv:2508.20755](https://arxiv.org/abs/2508.20755) find
  17. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  18. Chao-Wei Huang and Yun-Nung Chen. 2024. Factalign: Long-form factuality alignment of large language models. arXiv preprint arXiv:2410.01691. find
  19. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798.
  20. Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. 2024. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation. arXiv preprint arXiv:2409.12941. find
  21. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  22. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, page 611…
  23. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6086–6096, Florence, Italy. Association for Computational Linguistics.
  24. Zhicheng Lee, Shulin Cao, Jinxin Liu, Jiajie Zhang, Weichuan Liu, Xiaoyin Che, Lei Hou, and Juanzi Li. 2025. Rearag: Knowledge-guided reasoning enhances factuality of large reasoning models with iterative retrieval augmented generation. arXiv preprint arXiv:2503.21729. — [arXiv:2503.21729](https://arxiv.org/abs/2503.21729) find
  25. Junyi Li and Hwee Tou Ng. 2025. The hallucination dilemma: Factuality-aware reinforcement learning for large reasoning models. arXiv preprint arXiv:2505.24630. — [arXiv:2505.24630](https://arxiv.org/abs/2505.24630) find
  26. Yang Li, Youssef Emad, Karthik Padthe, Jack Lanchantin, Weizhe Yuan, Thao Nguyen, Jason Weston, Shang-Wen Li, Dong Wang, Ilia Kulikov, and 1 others. 2025. Naturalthoughts: Selecting and distilling reasoning traces for general reasoning tasks. arXiv preprint arXiv:2507.01921. — [arXiv:2507.01921](https://arxiv.org/abs/2507.01921) find
  27. Sheng-Chieh Lin, Luyu Gao, Barlas Oguz, Wenhan Xiong, Jimmy Lin, Wen-tau Yih, and Xilun Chen. 2024. Flame: Factuality-aware alignment for large language models. Advances in Neural Information Processing Systems, 37:115588–115614. find
  28. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics…
  29. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594. find
  30. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  31. Baochang Ren, Shuofei Qiao, Wenhao Yu, Huajun Chen, and Ningyu Zhang. 2025. Knowrl: Exploring knowledgeable reinforcement learning for factuality. arXiv preprint arXiv:2506.19807.
  32. Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling llm test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations. find
  33. Yixiao Song, Yekyung Kim, and Mohit Iyyer. 2024. Veriscore: Evaluating the factuality of verifiable claims in long-form text generation. arXiv preprint arXiv:2406.19276.
  34. Zhongxiang Sun, Qipeng Wang, Haoyu Wang, Xiao Zhang, and Jun Xu. 2025. Detection and mitigation of hallucination in large reasoning models: A mechanistic perspective. arXiv preprint arXiv:2505.12886. — [arXiv:2505.12886](https://arxiv.org/abs/2505.12886) find
  35. Yuxiang Wei, Federico Cassano, Jiawei Liu, Yifeng Ding, Naman Jain, Zachary Mueller, Harm de Vries, Leandro Von Werra, Arjun Guha, and Lingming Zhang. 2024c. Selfcodealign: Self-alignment for code generation. Advances in Neural Information Processing Systems, 37:62787–62874.
  36. Johannes Welbl, Nelson F. Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 94–106, Copenhagen, Denmark. Association for Computational Linguistics.
  37. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  38. Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600. find
  39. Boran Wang, Xinming Wang, Yi Chen, Xiang Li, Jian Xu, Jing Yuan, and Chenglin Liu. 2025a. Chartagent: A chart understanding framework with tool integrated reasoning. arXiv preprint arXiv:2512.14040. — [arXiv:2512.14040](https://arxiv.org/abs/2512.14040) find
  40. Boyang Xue, Fei Mi, Qi Zhu, Hongru Wang, Rui Wang, Sheng Wang, Erxin Yu, Xuming Hu, and Kam-Fai Wong. 2024. Ualign: Leveraging uncertainty estimations for factuality alignment on large language models. arXiv preprint arXiv:2412.11803. find
  41. Changyue Wang, Weihang Su, Qingyao Ai, and Yiqun Liu. 2025b. Joint evaluation of answer and reasoning consistency for hallucination detection in large reasoning models. arXiv preprint arXiv:2506.04832. — [arXiv:2506.04832](https://arxiv.org/abs/2506.04832) find
  42. Xinming Wang, Jian Xu, Aslan H Feng, Yi Chen, Haiyang Guo, Fei Zhu, Yuanqi Shao, Minsi Ren, Hongzhu Yi, Sheng Lian, and 1 others. 2025c. The hitchhiker’s guide to autonomous research: A survey of scientific agents. TechRxiv.August 07, 2025. DOI:10.36227/techrxiv175459840.02185500/V1. — [doi:10.36227/techrxiv175459840.02185500/v1](https://doi.org/10.36227/techrxiv175459840.02185500/v1) find
  43. Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Georgiev, Rocktim Jyoti Das, and Preslav Nakov. 2024. Factuality of large language models: A survey. arXiv preprint arXiv:2402.02420.
  44. Jason Wei, Karina Nguyen, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. 2024a. Measuring short-form factuality in large language models. arXiv preprint arXiv:2411.04368.
  45. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  46. Hanqi Yan, Qinglin Zhu, Xinyu Wang, Lin Gui, and Yulan He. 2024. Mirror: A multiple-perspective self-reflection method for knowledge-rich reasoning. arXiv preprint arXiv:2402.14963. — [arXiv:2402.14963](https://arxiv.org/abs/2402.14963) find
  47. Zijun Yao, Yantao Liu, Yanxu Chen, Jianhui Chen, Junfeng Fang, Lei Hou, Juanzi Li, and Tat-Seng Chua. 2025. Are reasoning models more prone to hallucination? arXiv preprint arXiv:2505.23646. — [arXiv:2505.23646](https://arxiv.org/abs/2505.23646) find
  48. Yuji Zhang, Qingyun Wang, Cheng Qian, Jiateng Liu, Chenkai Sun, Denghui Zhang, Tarek Abdelzaher, Chengxiang Zhai, Preslav Nakov, and Heng Ji. 2025. Atomic reasoning for scientific table claim verification. arXiv preprint arXiv:2506.06972. — [arXiv:2506.06972](https://arxiv.org/abs/2506.06972) find
  49. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403.13372.
  50. Yingjian Zhu, Xinming Wang, Kun Ding, Ying Wang, Bin Fan, and Shiming Xiang. 2026. Wikiseeker: Rethinking the role of vision-language models in knowledge-based visual question answering. Preprint, arXiv:2604.05818.
  51. Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Jie Huang, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, and Quoc V. Le. 2024b. Long-form factuality in large language models. arXiv preprint arXiv:2403.18802.
  52. Jiaqi Wei, Hao Zhou, Xiang Zhang, Di Zhang, Zijie Qiu, Wei Wei, Jinzhe Li, Wanli Ouyang, and Siqi Sun. 2025. Alignrag: Leveraging critique learning for evidence-sensitive retrieval-augmented reasoning. arXiv preprint arXiv:2504.14858. — [arXiv:2504.14858](https://arxiv.org/abs/2504.14858) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-mr-align-meta-reasoning-informed/