Elamur Research⌕
☰
home / papers / 2026 / Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains

Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains

Paper · 2026 · Yuqi Xiong, Chunyi Peng, Zhipeng Xu et al.

Summary

Lang2Act addresses a weakness of Visual Retrieval-Augmented Generation (VRAG). Many systems sharpen a VLM's view of retrieved document pages with external image operations such as cropping or zooming. These operations separate seeing from reasoning, and a bad crop can permanently remove the evidence needed for the answer. Lang2Act replaces such operations with "linguistic tools": named textual actions (for example read_text_element) that the model generates as part of its reasoning, steering its attention toward the relevant page regions.

Training has two RL stages, both using DAPO:

  1. Exploration. The VLM learns, through self-exploration rewarded by answer correctness, which visual actions help solve questions. Actions from successful trajectories are abstracted into a reusable toolbox.
  2. Exploitation. The model is optimized to use that toolbox for downstream reasoning.

On several visual QA benchmarks, Lang2Act beats all baselines by more than 4%. Attention analyses show it locates ground-truth regions more often and relies more on visual evidence than on memorized knowledge. This suggests that perception can be strengthened without lossy pixel-level tools.

Key points

  • Tool curation: frequent self-emergent actions are collected into a tool pool. Each tool is stored as a triplet of name, description and parameter specification. Examples include read_text_element and read_numeric_value.
  • Backbones: Qwen2.5-VL-3B, Qwen2.5-VL-7B and Qwen2.5-VL-32B. Tool-enhanced baselines include Pixel-Reasoner and VRAG-RL.
  • Prompting-only result (before any training), 7B backbone: the Lang2Act tool format reaches an average of 56.19, ahead of Structured CoT (54.43) and Vanilla prompting (53.12).
  • SlideVQA case study: VRAG-RL cropped away the key value "43%" and answered "56%". Lang2Act focused on the correct region and answered correctly.
  • Resources: code and data are released at github.com/NEUIR/Lang2Act. The authors acknowledge that the causal link between attention shifts and answer correctness is not established; their evidence is correlational.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🏢Alibaba GroupHangzhou · China201 works · 17,173 citations🎓Northeastern UniversityShenyang · China53 works · 2,205 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 worksEngineering · 188 worksControl and Systems Engineering · 62 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Yuqi Xiong, Chunyi Peng, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Yukun Yan, Shuo Wang, Yu Gu and Ge Yu. 2026. Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.409

@inproceedings{xiong2026lang2act,
  title = {{Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains}},
  author = {Yuqi Xiong, Chunyi Peng, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Yukun Yan, Shuo Wang, Yu Gu and Ge Yu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.409},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.409}
}

References

32 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2.5-VL technical report. ArXiv preprint.
  2. Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. 2024. Graph of Thoughts: Solving elaborate problems with large language models. In Proceedings of AAAI, pages 17682… find
  3. Jaemin Cho, Debanjan Mahata, Ozan Irsoy, Yujie He, and Mohit Bansal. 2024. M3DocRAG: Multi-modal retrieval is what you need for multi-page multi-document understanding. ArXiv preprint. find
  4. Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. 2025. OpenVL-Thinker: An early exploration to complex vision-language reasoning via iterative self-improvement. ArXiv preprint. find
  5. Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. 2024. ColPali: Efficient document retrieval with vision language models. ArXiv preprint.
  6. Jiuxiang Gu, Jason Kuen, Vlad I. Morariu, Handong Zhao, Rajiv Jain, Nikolaos Barmpalios, Ani Nenkova, and Tong Sun. 2021. UniDoc: Unified pretraining framework for document understanding. In Proceedings of NeurIPS, pages 39–50. find
  7. Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-R1: Incentivizing reasoning capability in multimodal large language models. ArXiv preprint.
  8. Ryota Tanaka, Taichi Iki, Taku Hasegawa, Kyosuke Nishida, Kuniko Saito, and Jun Suzuki. 2025. VDocRAG: Retrieval-augmented generation over visually-rich documents. In Proceedings of CVPR, pages 24827–24837. find
  9. Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of NeurIPS.
  10. Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, and Kuniko Saito. 2023. SlideVQA: A dataset for document visual question answering on multiple images. In Proceedings of AAAI, pages 13636–13645.
  11. Yuting Li, Lai Wei, Kaipeng Zheng, Jingyuan Huang, Linghe Kong, Lichao Sun, and Weiran Huang. 2025. Vision Matters: Simple visual perturbations can boost multimodal math reasoning. ArXiv preprint. find
  12. Jerry Liu. 2022. LlamaIndex. https://github.com/ jerryjliu/llama_index. GitHub repository. ↗
  13. Zhenghao Liu, Pengcheng Huang, Zhipeng Xu, Xinze Li, Shuliang Liu, Chunyi Peng, Haidong Xin, Yukun Yan, Shuo Wang, Xu Han, and 1 others. 2026. Knowledge-intensive agents. AI Open. find
  14. Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, Pan Zhang, Liangming Pan, Yu-Gang Jiang, Jiaqi Wang, Yixin Cao, and Aixin Sun. 2024. MMLongBench-Doc: Benchmarking long-context document understanding with visualizations. In Proceedings…
  15. Haozhe Wang, Alex Su, Weiming Ren, Fangzhen Lin, and Wenhu Chen. 2025a. Pixel-Reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. ArXiv preprint. find
  16. Qiuchen Wang, Ruixue Ding, Zehui Chen, Weiqi Wu, Shihang Wang, Pengjun Xie, and Feng Zhao. 2025b. ViDoRAG: Visual document retrieval-augmented generation via dynamic iterative reasoning agents. ArXiv preprint.
  17. Qiuchen Wang, Ruixue Ding, Yu Zeng, Zehui Chen, Lin Chen, Shihang Wang, Pengjun Xie, Fei Huang, and Feng Zhao. 2025c. VRAG-RL: Empower vision-perception-based RAG for visually rich information understanding via iterative reasoning with reinforcement learning. ArXiv preprint.
  18. Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. 2025d. SOTA with less: MCTS-guided sample selection for data-efficient visual reasoning self-improvement. ArXiv preprint. find
  19. Chunyi Peng, Zhipeng Xu, Zhenghao Liu, Yishan Li, Yukun Yan, Shuo Wang, Zhiyuan Liu, Yu Gu, Minghe Yu, Ge Yu, and 1 others. 2025. Learning to route queries across knowledge bases for step-wise retrieval-augmented reasoning. ArXiv preprint. find
  20. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of NeurIPS.
  21. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316–1331.
  22. Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, and Ziwei Liu. 2025. MMSearch-R1: Incentivizing LMMs to search. ArXiv preprint.
  23. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. ArXiv preprint.
  24. Yubo Sun, Chunyi Peng, Yukun Yan, Shi Yu, Zhenghao Liu, Chi Chen, Zhiyuan Liu, and Maosong Sun. 2025. VisRAG 2.0: Evidence-guided multi-image reasoning in visual retrieval-augmented generation. ArXiv preprint. find
  25. Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A Rossi, and Dinesh Manocha. 2024. VisDom: Multi-document QA with visually rich elements using multimodal retrieval-augmented generation. ArXiv preprint. find
  26. Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, and 1 others. 2025. R1-OneVision: Advancing generalized multimodal reasoning through cross-modal formalization. ArXiv preprint. find
  27. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate problem solving with large language models. In Proceedings of NeurIPS. find
  28. Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, and 1 others. 2024. MiniCPM-V: A GPT-4V level MLLM on your phone. ArXiv preprint.
  29. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. DAPO: An open-source LLM reinforcement learning system at scale. ArXiv preprint.
  30. Shi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui, Junhao Ran, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, and 1 others. 2024. Vis-RAG: Vision-based retrieval-augmented generation on multi-modality documents. ArXiv preprint. find
  31. Junyuan Zhang, Qintong Zhang, Bin Wang, Linke Ouyang, Zichen Wen, Ying Li, Ka-Ho Chow, Conghui He, and Wentao Zhang. 2025. OCR hinders RAG: Evaluating the cascading impact of OCR on retrieval-augmented generation. In Proceedings of ICCV, pages 17443–17453. find
  32. Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. 2025. EasyR1: An efficient, scalable, multi-modality RL training framework. https://github.com/ hiyouga/EasyR1. GitHub repository. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xiong-2026-lang2act-fine-grained-visual-reasoning/