Elamur Research⌕
☰
home / papers / 2026 / IREASONER: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models

IREASONER: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models

Paper · 2026 · Meghana Sunil, Manikandarajan Venmathimaran and Muthu Subash Kavitha

Summary

Self-evolving large multimodal models (LMMs) can learn from unlabeled images by generating their own questions and rewarding themselves. Existing frameworks such as EvoLMM and VisPlay, however, score only the final answer. Two responses that arrive at the same answer therefore get almost identical credit, even when one relies on hallucinated or shortcut intermediate steps. IREASONER addresses this by adding supervision over the reasoning trajectory itself, without any labels, external judges or verifiers.

The approach uses a Proposer–Solver loop. The Proposer writes a visually grounded question about an unlabeled image, and the Solver samples several chain-of-thought rollouts. Rollouts that share the majority answer are aligned step by step, and each step is embedded. Per-step prototypes are then averaged from this group. Each rollout is rewarded by how closely its steps match those prototypes, with early, grounding-related steps weighted more heavily. This signal is combined with the usual answer-level self-consistency reward.

Applied to Qwen2.5-VL models, the method improves results on eight multimodal reasoning benchmarks, by up to about +2.1 points at the 7B scale. This makes reasoning-aware, fully unsupervised self-improvement practical.

Key points

  • Reward design: The new Intrinsic CoT Agreement Reward measures step-level similarity to dominant-answer prototypes. It uses a warmup schedule and a density factor ρ that weakens the step signal when the majority group is small.
  • Benchmarks and scales: The method was evaluated on InfoGraphic-VQA, AI2D, ScienceQA, MMMU, ChartQA, MathVista, MathVision and MathVerse. It gave consistent gains at the 3B, 7B and 32B scales; for example, Qwen2.5-VL-7B on MathVerse rose from 43.78 to 45.91.
  • Training budget: Training used roughly 2.5k self-evolution steps on 2.5k unlabeled images; the 32B model ran for about 3,000 steps.
  • Reward reliability: The authors checked the majority answer against ground truth for diagnosis only. It was correct about 76% of the time early in training and about 93% late in training. The final model correctly solved 87% of the questions the Solver had failed during training.
  • CoT ablation: With chain-of-thought suppressed at inference, the trained model still beat the baseline, and full CoT added further gains. The main limitations are that the reward cannot detect confidently wrong majorities and that the method requires access to the model's log-probabilities.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Vellore Institute of Technology UniversityVellore · India7 works · 268 citations🎓Loughborough UniversityLoughborough · United Kingdom2 works · 15 citations

Also on the byline: Nagasaki University.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksVision & language · 657 works

Cite

Cite as Meghana Sunil, Manikandarajan Venmathimaran and Muthu Subash Kavitha. 2026. IREASONER: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1468

@inproceedings{sunil2026ireasoner,
  title = {{IREASONER: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models}},
  author = {Meghana Sunil, Manikandarajan Venmathimaran and Muthu Subash Kavitha},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1468},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1468}
}

References

49 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. Qwen2.5-vl technical report. Preprint, arXiv:2502.13…
  2. Lili Chen, Mihir Prabhudesai, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. 2025a. Self-questioning language models. Preprint, arXiv:2508.03682. — [arXiv:2508.03682](https://arxiv.org/abs/2508.03682) find
  3. Qiguang Chen, Libo Qin, Jiaqi Wang, Jinxuan Zhou, and Wanxiang Che. 2024. Unlocking the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of-thought. Preprint, arXiv:2410.05695. — [arXiv:2410.05695](https://arxiv.org/abs/2410.05695) find
  4. Xiuwei Chen, Wentao Hu, Hanhui Li, Jun Zhou, Zisheng Chen, Meng Cao, Yihan Zeng, Kui Zhang, Yu-Jie Yuan, Jianhua Han, Hang Xu, and Xiaodan Liang. 2025b. C2-evo: Co-evolving multimodal data and model for self-improving reasoning. Preprint, arXiv:2507.16518. — [arXiv:2507.16518](https://arxiv.org/abs/2507.16518) find
  5. Kanzhi Cheng, Li YanTao, Fangzhi Xu, Jianbing Zhang, Hao Zhou, and Yang Liu. 2025. find
  6. Vision-language Shijian Deng, Kai Wang, Tianyu Yang, Harsh Singh, and Yapeng Tian. 2025a. Self-improvement in multimodal large language models: A survey. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 1987–2006, Suzhou, China. Association for Computational Linguistic…
  7. Yunlong Deng, Guangyi Chen, Tianpei Gu, Lingjing Kong, Yan Li, Zeyu Tang, and Kun Zhang. 2025b. Towards self-refinement of vision-language models with triangular consistency. Preprint, arXiv:2510.10487. — [arXiv:2510.10487](https://arxiv.org/abs/2510.10487) find
  8. Hao Dong, Lijun Sheng, Jian Liang, Ran He, Eleni Chatzi, and Olga Fink. 2025. Adapting vision-language models without labels: A comprehensive survey. Preprint, arXiv:2508.05547. — [arXiv:2508.05547](https://arxiv.org/abs/2508.05547) find
  9. Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. 2024. Promptbreeder: self-referential self-improvement via prompt evolution. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
  10. Jiaxin Ge, Hongyin Luo, Siyuan Qian, Yulu Gan, Jie Fu, and Shanghang Zhang. 2023. Chain of thought prompt tuning in vision language models. Preprint, arXiv:2304.07919. — [arXiv:2304.07919](https://arxiv.org/abs/2304.07919) find
  11. Jiawei Guo, Tianyu Zheng, Yizhi Li, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Graham Neubig, Wenhu Chen, and Xiang Yue. 2025. MAmmoTH-VL: Eliciting multimodal reasoning with instruction tuning at scale. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume…
  12. Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. 2025. Token-budget-aware llm reasoning. Preprint, arXiv:2412.18547.
  13. Yicheng He, Chengsong Huang, Zongxia Li, Jiaxin Huang, and Yonghui Yang. 2025. Visplay: Self-evolving vision-language models from images. Preprint, arXiv:2511.15661. — [arXiv:2511.15661](https://arxiv.org/abs/2511.15661) find
  14. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  15. Xu Huang, Weiwen Liu, Xingshan Zeng, Yuefeng Huang, Xinlong Hao, Yuxian Wang, Yirong Zeng, Chuhan Wu, Yasheng Wang, Ruiming Tang, and Defu Lian. 2025. Toolace-dev: Self-improving tool 29370 10 learning via decomposition and evolution. Preprint, arXiv:2505.07512. — [arXiv:2505.07512](https://arxiv.org/abs/2505.07512) find
  16. Multimodal reasoning via thought chains for science question answering. Preprint, arXiv:2209.09513. find
  17. Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. 2016. A diagram is worth a dozen images. In European conference on computer vision, pages 235–251. Springer.
  18. Zheqi Lv, Wenkai Wang, Jiawei Wang, Shengyu Zhang, and Fei Wu. 2025. Cascaded self-evaluation augmented training for lightweight multimodal llms. Preprint, arXiv:2501.05662. — [arXiv:2501.05662](https://arxiv.org/abs/2501.05662) find
  19. Zongxia Li, Wenhao Yu, Chengsong Huang, Rui Liu, Zhenwen Liang, Fuxiao Liu, Jingxi Che, Dian Yu, Jordan Boyd-Graber, Haitao Mi, and Dong Yu. 2025. Self-rewarding vision-language model via reasoning decomposition. Preprint, arXiv:2508.19652. find
  20. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the association for computational linguistics: ACL 2022, pages 2263– 2279.
  21. Ci-Siang Lin, Min-Hung Chen, Yu-Yang Sheng, and Yu-Chiang Frank Wang. 2025. Leaml: Label-efficient adaptation to out-of-distribution visual tasks for multimodal large language models. Preprint, arXiv:2510.03232. — [arXiv:2510.03232](https://arxiv.org/abs/2510.03232) find
  22. Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. 2022. Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1697–1706. find
  23. Bingshuai Liu, Chenyang Lyu, Zijun Min, Zhanyu Wang, Jinsong Su, and Longyue Wang. 2025a. Retrieval-augmented multi-modal chain-of-thoughts reasoning for large language models. In 2025 International Joint Conference on Neural Networks (IJCNN), pages 1–8.
  24. Nitesh Methani, Pritha Ganguly, Mitesh M Khapra, and Pratyush Kumar. 2020. Plotqa: Reasoning over scientific plots. In Proceedings of the ieee/cvf winter conference on applications of computer vision, pages 1527–1536. find
  25. Jiaqi Liu, Kaiwen Xiong, Peng Xia, Yiyang Zhou, Haonian Ji, Lu Feng, Siwei Han, Mingyu Ding, and Huaxiu Yao. 2025b. Agent0-vl: Exploring self-evolving agent for tool-integrated vision-language reasoning. Preprint, arXiv:2511.19900. — [arXiv:2511.19900](https://arxiv.org/abs/2511.19900) find
  26. Jianqiao Lu, Wanjun Zhong, Wenyong Huang, Yufei Wang, Qi Zhu, Fei Mi, Baojun Wang, Weichao Wang, Xingshan Zeng, Lifeng Shang, Xin Jiang, and Qun Liu. 2024. Self: Self-evolution with language feedback. Preprint, arXiv:2310.00533. — [arXiv:2310.00533](https://arxiv.org/abs/2310.00533) find
  27. Jinghui Lu, Haiyang Yu, Siliang Xu, Shiwei Ran, Guozhi Tang, Siqi Wang, Bin Shan, Teng Fu, Hao Feng, Jingqun Tang, Han Wang, and Can Huang. 2025. Prolonged reasoning is not all you need: Certainty-based adaptive routing for efficient llm/mllm reasoning. Preprint, arXiv:2505.15154.
  28. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255.
  29. Gaurav Srivastava, Zhenyu Bi, Meng Lu, and Xuan Wang. 2025. DEBATE, TRAIN, EVOLVE: Self-Evolution of language model reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 32752–32798, Suzhou, China. Association for Computational Linguistics.
  30. Omkar Thawakar, Shravan Venkatraman, Ritesh Thawkar, Abdelrahman Shaker, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, and Fahad Khan. 2025. Evolmm: Self-evolving large multimodal models with continuous rewards. Preprint, arXiv:2511.16672. — [arXiv:2511.16672](https://arxiv.org/abs/2511.16672) find
  31. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2024. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169.
  32. Qinsi Wang, Bo Liu, Tianyi Zhou, Jing Shi, Yueqian Lin, Yiran Chen, Hai Helen Li, Kun Wan, and Wentian Zhao. 2025a. Vision-zero: Scalable vlm self-improvement via strategic gamified self-play. Preprint, arXiv:2509.25541. — [arXiv:2509.25541](https://arxiv.org/abs/2509.25541) find
  33. Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. 2021. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. In The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the…
  34. Xiyao Wang, Jiuhai Chen, Zhaoyang Wang, Yuhang Zhou, Yiyang Zhou, Huaxiu Yao, Tianyi Zhou, Tom Goldstein, Parminder Bhatia, Taha Kass-Hout, Furong Huang, and Cao Xiao. 2025b. Enhancing visual-language modality alignment in large vision language models via self-improvement. In Findings of the Associa… find
  35. Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229–256. 29371… find
  36. Renqiu Xia, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Botian Shi, Junchi Yan, and Bo Zhang. 2025. Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. IEEE Transactions on Image Processing. find
  37. Jing Ye, Lu Xiang, Yaping Zhang, and Chengqing Zong. 2025. From generic empathy to personalized emotional support: A self-evolution framework for user preference alignment. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 18826–18853, Suzhou, China. Association for Com… find
  38. Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. Demystifying long chain-of-thought reasoning in llms. Preprint, arXiv:2502.03373.
  39. Xiangning Yu, Zhuohan Wang, Linyi Yang, Haoxuan Li, Anjie Liu, Xiao Xue, Jun Wang, and Mengyue Yang. 2025. Causal sufficiency and necessity improves chain-of-thought reasoning. Preprint, arXiv:2506.09853. — [arXiv:2506.09853](https://arxiv.org/abs/2506.09853) find
  40. Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, and Min Lin. 2024c. Chain of preference optimization: Improving chain-of-thought reasoning in llms. Preprint, arXiv:2406.09136. — [arXiv:2406.09136](https://arxiv.org/abs/2406.09136) find
  41. Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2024d. Multimodal chain-of-thought reasoning in language models. Preprint, arXiv:2302.00923.
  42. Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2024e. Multimodal chain-of-thought reasoning in language models. Preprint, arXiv:2302.00923.
  43. Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. 2024. Calibrated self-rewarding vision language models. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook,… find
  44. Adam Zweiger, Jyothish Pari, Han Guo, Ekin Akyürek, Yoon Kim, and Pulkit Agrawal. 2025. Self-adapting language models. Preprint, arXiv:2506.10943. — [arXiv:2506.10943](https://arxiv.org/abs/2506.10943) find
  45. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, and 1 others. 2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vi…
  46. Juntian Zhang, Song Jin, Chuanqi Cheng, Yuhan Liu, Yankai Lin, Xun Zhang, Yufei Zhang, Fei Jiang, Guojun Yin, Wei Lin, and Rui Yan. 2025a. Viper: Empowering the self-evolution of visual perception abilities in vision-language model. Preprint, arXiv:2510.24285. — [arXiv:2510.24285](https://arxiv.org/abs/2510.24285) find
  47. Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2024a. Lmms-eval: Reality check on the evaluation of large multimodal models. Preprint, arXiv:2407.12772.
  48. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, and 1 others. 2024b. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pages 169–186. Springer.
  49. Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. 2025b. Improve vision language model chain-of-thought reasoning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sunil-2026-ireasoner-trajectory-aware-intrinsic-reasoning/