Elamur Research⌕
☰
home / papers / 2026 / DORA: A Dual-Objective Reinforcement Learning Framework for Effective and Efficient Multimodal Agentic Search

DORA: A Dual-Objective Reinforcement Learning Framework for Effective and Efficient Multimodal Agentic Search

Paper · 2026 · Guangming Qin, Yuhao Deng, Yukun Zhao et al.

Summary

Multimodal LLMs trained with reinforcement learning can call search engines while answering knowledge-heavy visual questions. However, current training rewards only the final answer. A trajectory with good retrieval but a wrong answer is penalized as a whole, and a correct path with redundant searches earns the same credit as a lean one.

DORA tackles both problems on top of GRPO. Each rollout follows a fixed cycle: thinking, issuing text or image queries to a multimodal retriever (e.g., LamRA, UniIR), reflecting on whether the evidence is useless, partial or sufficient, concluding, and answering. Two training ideas drive the method:

  • Efficiency reward: a group-wise score that ranks correct sampled paths by how few search calls they needed.
  • Dual objective: a retrieval-quality objective that touches only the search steps, while the answer-correctness objective covers the whole trajectory.

On Enc-VQA, InfoSeek and SK-VQA, DORA outperforms baselines such as MMSearch-R1. It improves accuracy by up to 8.4% and issues roughly 9.7% fewer searches. This points toward cheaper and more reliable retrieval agents for knowledge-intensive VQA (KI-VQA).

Key points

  • Evaluation: three benchmarks (Enc-VQA, InfoSeek, SK-VQA). The paper is internally inconsistent on the best gain: the abstract reports up to 8.4% higher accuracy, while the contributions list states 7.8%. Both cite a 9.7% cut in search calls.
  • Full DORA results: accuracy of 25.01 / 44.53 / 40.98 on the three datasets, with an average of 1.64 / 1.18 / 1.08 search calls per query.
  • Where the efficiency comes from: compared with MMSearch-R1 on Enc-VQA, irrelevant searches drop from 0.51 to 0.36 per query, while effective searches rise slightly from 1.23 to 1.28.
  • Ablations:
    • Removing the efficiency reward barely changes accuracy but increases search calls by about 59%.
    • Removing the dual objective lowers accuracy by about 7.5%.
    • Removing the reflection step costs 3–4% accuracy.
  • Limitations: the reward scores need careful tuning, and the framework currently covers only text-image reasoning. Video and multi-agent settings are left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢BaiduBeijing · China74 works · 15,263 citations🎓University of ArizonaTucson · United States38 works · 10,603 citations🎓Beijing Institute of TechnologyBeijing · China34 works · 3,652 citations🎓Shandong UniversityJinan · China17 works · 789 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 657 works

Cite

Cite as Guangming Qin, Yuhao Deng, Yukun Zhao, Zhenyang Li, Junfeng Wang, Dawei Yin, Ye Yuan, Guoren Wang, Yizhou Yan, Chengliang Chai and Lei Cao. 2026. DORA: A Dual-Objective Reinforcement Learning Framework for Effective and Efficient Multimodal Agentic Search. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.605

@inproceedings{qin2026dora,
  title = {{DORA: A Dual-Objective Reinforcement Learning Framework for Effective and Efficient Multimodal Agentic Search}},
  author = {Guangming Qin, Yuhao Deng, Yukun Zhao, Zhenyang Li, Junfeng Wang, Dawei Yin, Ye Yuan, Guoren Wang, Yizhou Yan, Chengliang Chai and Lei Cao},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.605},
  url = {https://doi.org/10.18653/v1/2026.acl-long.605}
}

References

33 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. Qwen2.5-vl technical report. Preprint, arXiv:2502.13…
  2. Davide Caffagni, Federico Cocchi, Nicholas Moratelli, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, and Yang Chen, Hexiang Hu, Yi Luan, Haitian Sun, Soravit Changpinyo, Alan Ritter, and Ming-Wei Chang. 2023. Can pre-trained vision and language models answer visual information-seeking questions? Prep…
  3. Federico Cocchi, Nicholas Moratelli, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. 2025. Augmenting multimodal llms with self-reflective tokens for knowledge-based visual question answering. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 9199–9209. find
  4. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  5. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. Deepseek-r1: Incentivizing reasoning capability in ll…
  6. Yihao Ding, Kaixuan Ren, Jiabin Huang, Siwen Luo, and Soyeon Caren Han. 2024. Pdf-mvqa: A dataset for multimodal information retrieval in pdf-based visual question answering. Preprint, arXiv:2404.12720. — [arXiv:2404.12720](https://arxiv.org/abs/2404.12720) find
  7. Negar Foroutan, Angelika Romanou, Matin Ansaripour, Julian Martin Eisenschlos, Karl Aberer, and Rémi Lebret. 2025. Wikimixqa: A multimodal benchmark for question answering over tables and charts. Preprint, arXiv:2506.15594. find
  8. Dongze Hao, Qunbo Wang, Longteng Guo, Jie Jiang, and Jing Liu. 2024. Self-bootstrapped visual-language model for knowledge selection and question answering. arXiv preprint arXiv:2404.13947. find
  9. Ziniu Hu, Ahmet Iscen, Chen Sun, Kai-Wei Chang, Yizhou Sun, David Ross, Cordelia Schmid, and Alireza Fathi. 2023. Avis: Autonomous visual information seeking with large language model agent. Advances in Neural Information Processing Systems, 36:867–878. find
  10. Dongfu Jiang, Yi Lu, Zhuofeng Li, Zhiheng Lyu, Ping Nie, Haozhe Wang, Alex Su, Hui Chen, Kai Zou, Chao Du, and 1 others. 2025a. Verltool: Towards holistic agentic reinforcement learning with tool use. arXiv preprint arXiv:2509.01055. — [arXiv:2509.01055](https://arxiv.org/abs/2509.01055) find
  11. Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Chaoyou Fu, Guanglu Song, Peng Gao, Yu Liu, Chunyuan Li, and Hongsheng Li. 2024. Mmsearch: Benchmarking the potential of large models as multimodal search engines. Preprint, arXiv:2409.12959. — [arXiv:2409.12959](https://arxiv.org/abs/2409.12959) find
  12. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, EuroSys ’25, page 1279–1297. ACM.
  13. Pengcheng Jiang, Jiacheng Lin, Lang Cao, Runchu Tian, SeongKu Kang, Zifeng Wang, Jimeng Sun, and Jiawei Han. 2025b. Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning. arXiv preprint arXiv:2503.00223. — [arXiv:2503.00223](https://arxiv.org/abs/2503.00223) find
  14. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652.
  15. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516.
  16. Xin Su, Man Luo, Kris W Pan, Tien Pei Chou, Vasudev Lal, and Phillip Howard. 2024. Sk-vqa: Synthetic knowledge generation at scale for training context-augmented multimodal llms. arXiv preprint arXiv:2406.19593. — [arXiv:2406.19593](https://arxiv.org/abs/2406.19593) find
  17. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. Preprint, arXiv:2305.20050.
  18. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275.
  19. Weizhe Lin, Jinghong Chen, Jingbiao Mei, Alexandru Coca, and Bill Byrne. 2023. Fine-grained late-interaction multi-modal retrieval for retrieval augmented visual question answering. Preprint, arXiv:2309.17133. — [arXiv:2309.17133](https://arxiv.org/abs/2309.17133) find
  20. Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. 2025. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837. find
  21. Yuanze Lin, Yujia Xie, Dongdong Chen, Yichong Xu, Chenguang Zhu, and Lu Yuan. 2022. Revive: Regional visual representation matters in knowledge-based visual question answering. Advances in neural information processing systems, 35:10560–10571. find
  22. Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. 2023. Uniir: Training and benchmarking universal multimodal information retrievers. Preprint, arXiv:2311.17136. — [arXiv:2311.17136](https://arxiv.org/abs/2311.17136) find
  23. Yikun Liu, Yajie Zhang, Jiayin Cai, Xiaolong Jiang, Yao Hu, Jiangchao Yao, Yanfeng Wang, and Weidi Xie. 2025. Lamra: Large multimodal model as your advanced retrieval assistant. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 4015–4025. find
  24. Jialin Wu, Jiasen Lu, Ashish Sabharwal, and Roozbeh Mottaghi. 2021. Multi-modal answer validation for knowledge-based vqa. Preprint, arXiv:2103.12248. — [arXiv:2103.12248](https://arxiv.org/abs/2103.12248) find
  25. Thomas Mensink, Jasper Uijlings, Lluis Castrejon, Arushi Goel, Felipe Cadar, Howard Zhou, Fei Sha, André Araujo, and Vittorio Ferrari. 2023. Encyclopedic vqa: Visual questions about detailed properties of fine-grained categories. Preprint, arXiv:2306.09224. — [arXiv:2306.09224](https://arxiv.org/abs/2306.09224) find
  26. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, and 401 others. 2024. Gpt-4o system card.…
  27. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  28. Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, and Ziwei Liu. 2025. Mmsearch-r1: Incentivizing lmms to search. arXiv preprint arXiv:2506.20670.
  29. Hongshen Xu, Zihan Wang, Zichen Zhu, Lei Pan, Xingyu Chen, Shuai Fan, Lu Chen, and Kai Yu. 2025. Alignment for efficient tool calling of large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 17787–17803.
  30. Yibin Yan and Weidi Xie. 2024. Echosight: Advancing visual-language models with wiki knowledge. In Findings of the Association for Computational Linguistics: EMNLP 2024, page 1538–1551. Association for Computational Linguistics. find
  31. Zhixin Zhang, Yiyuan Zhang, Xiaohan Ding, and Xiangyu Yue. 2024. Vision search assistant: Empower vision-language models as multimodal search engines. Preprint, arXiv:2410.21220. — [arXiv:2410.21220](https://arxiv.org/abs/2410.21220) find
  32. Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. 2025. Deepeyes: Incentivizing "thinking with images" via reinforcement learning. Preprint, arXiv:2505.14362. sufficient knowledge has been gathered to produce a final answer. — [arXiv:2505.14362](https://arxiv.org/abs/2505.14362) find
  33. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, and 32 others. 2025. Internvl3: Exploring advanced training and test-…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qin-2026-dora-dual-objective-reinforcement-learning/