Elamur Research⌕
☰
home / papers / 2025 / MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval

MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval

Paper · 2025 · Xixi Wu, Yanchao Tan, Nan Hou et al.

Summary

Answering questions about long, multi-page documents (DocQA) remains difficult. OCR-to-text pipelines discard figures, tables and layout, and LVLMs cannot take in many page images at once. Page-retrieval systems such as M3DocRAG and MDocAgent choose pages only by embedding similarity. As a result, they often return pages that share keywords with the question but lack the facts needed to derive an answer.

MoLoRAG represents a document as a page graph that encodes relations between pages. A small VLM traverses this graph and scores how logically useful each visited page is for the question. That score is combined with semantic similarity to re-rank pages, and the top-K pages are passed to any LVLM for answering. The retriever comes in two forms: a training-free version built on Qwen2.5-VL-3B and a fine-tuned version, MoLoRAG+.

On four DocQA benchmarks, the method improves both answer accuracy and retrieval quality over direct LVLM inference and semantic-only retrievers. This suggests that retrieval which accounts for reasoning is key to multi-modal document understanding.

Key points

  • Across four DocQA datasets, answer accuracy rises by 9.68% on average compared with running the LVLM directly, and retrieval precision improves by 7.44% on average relative to baseline methods.
  • Retrieval was evaluated with Recall, Precision, NDCG and MRR on MMLongBench and LongDocURL, the two datasets that provide gold evidence pages. Average improvements were 9.94% and 7.16% respectively, and MoLoRAG+ scored higher still.
  • Baselines include M3DocRAG (ColPali-based page retrieval) and MDocAgent (separate text and image retrieval with modality-specific agents), as well as LLM-based and LVLM-based DocQA pipelines.
  • The approach is LVLM-agnostic: only a few retrieved page snapshots (e.g., K = 3) are passed to the answering model, merged into one image if the model accepts only single images.
  • Released resources include the fine-tuned retriever (xxwu/MoLoRAG-QwenVL-3B on Hugging Face), the training data (xxwu/MoLoRAG) and code on GitHub (WxxShirley/MoLoRAG). The authors note that the method is limited to closed-domain settings, where the relevant document is already given.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chinese University of Hong KongHong Kong · China126 works · 14,073 citations🎓University of MacauMacao · China33 works · 2,223 citations🎓Fuzhou UniversityFuzhou · China3 works · 17 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Vision & language · 657 worksInformation retrieval & search · 473 works

Cite

Cite as Xixi Wu, Yanchao Tan, Nan Hou, Ruiyang Zhang and Hong Cheng. 2025. MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.708

@inproceedings{wu2025molorag,
  title = {{MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval}},
  author = {Xixi Wu, Yanchao Tan, Nan Hou, Ruiyang Zhang and Hong Cheng},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.708},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.708}
}

References

31 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-augmented generation for large language models: A survey. Preprint, arXiv:2312.10997.
  2. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations.
  3. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  4. Siwei Han, Peng Xia, Ruiyi Zhang, Tong Sun, Yun Li, Hongtu Zhu, and Huaxiu Yao. 2025. Mdocagent: A multi-modal multi-agent framework for document understanding. arXiv preprint arXiv:2503.13964. — [arXiv:2503.13964](https://arxiv.org/abs/2503.13964) find
  5. Xueguang Ma, Shengyao Zhuang, Bevan Koopman, Guido Zuccon, Wenhu Chen, and Jimmy Lin. 2024a. Visa: Retrieval augmented generation with visual source attribution. arXiv preprint arXiv:2412.14457.
  6. Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. 2024. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  7. Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, Pan Zhang, Liangming Pan, Yu-Gang Jiang, Jiaqi Wang, Yixin Cao, and Aixin Sun. 2024b. Mmlongbench-doc: Benchmarking long-context document understanding with visualizations. Preprint, arXi… find
  8. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  9. Yulong Hui, Yao Lu, and Huanchen Zhang. 2024. UDA: A benchmark suite for retrieval augmented generation in real-world document analysis. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. find
  10. Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L’elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée La… find
  11. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-augmented generation for knowledge-intensive nlp tasks. Preprint, arXiv:2005.11401.
  12. Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. 2024a. Llava-next: Stronger llms supercharge multimodal capabilities in the wild. find
  13. Shilong Li, Yancheng He, Hangyu Guo, Xingyuan Bu, Ge Bai, Jie Liu, Jiaheng Liu, Xingwei Qu, Yangguang Li, Wanli Ouyang, Wenbo Su, and Bo Zheng. 2024b. GraphReader: Building graph-based agent to enhance long-context abilities of large language models. In Findings of the Association for Computational… find
  14. Hao Liu, Zhengren Wang, Xi Chen, Zhiyu Li, Feiyu Xiong, Qinhan Yu, and Wentao Zhang. 2025. Hoprag: Multi-hop reasoning for logic-aware retrieval-augmented generation. Preprint, arXiv:2502.12442. find
  15. Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024. Deepseek-vl: Towards real-world vision-language understanding. Preprint, arXiv:2403.05525.
  16. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2263– 2279, Dublin, Ireland. Association for Computa…
  17. Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. 2021. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209.
  18. Jamshed Memon, Maira Sami, Rizwan Ahmed Khan, and Mueen Uddin. 2020. Handwritten optical character recognition (ocr): A comprehensive systematic literature review (slr). IEEE Access, 8:142642–142668. find
  19. OpenAI. 2024. Gpt-4o system card. arXiv:2410.21276.
  20. Preprint, Qwen. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  21. Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. 2024. Raptor: Recursive abstractive processing for tree-organized retrieval. In International Conference on Learning Representations (ICLR).
  22. Jianwei Sun, Chaoyang Mei, Linlin Wei, Kaiyu Zheng, Na Liu, Ming Cui, and Tianyi Li. 2024. Dial-insight: Fine-tuning large language models with high-quality domain-specific data preventing capability collapse. Preprint, arXiv:2403.09167. — [arXiv:2403.09167](https://arxiv.org/abs/2403.09167) find
  23. Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A Rossi, and Dinesh Manocha. 2024. Visdom: Multi-document qa with visually rich elements using multimodal retrieval-augmented generation. arXiv preprint arXiv:2412.10704. find
  24. Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, and Kuniko Saito. 2023. Slidevqa: A dataset for document visual question answering on multiple images. In AAAI. find
  25. Rubèn Tito, Dimosthenis Karatzas, and Ernest Valveny. 2023. Hierarchical multimodal transformers for multi-page docvqa. Preprint, arXiv:2212.05935. — [arXiv:2212.05935](https://arxiv.org/abs/2212.05935) find
  26. Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. 2024a. Mineru: An open-source solution for precise document content extraction. Preprint, arXiv:240… find
  27. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024b. Qwen2-vl: Enhancing vision-language model’s perception of the…
  28. Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, Chunrui Han, and Xiangyu Zhang. 2024. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. Preprint, arXiv:2409.01704. — [arXiv:2409.01704](https://arxiv.org/abs/2409.01704) find
  29. Zhishang Xiang, Chuanjie Wu, Qinggang Zhang, Shengyuan Chen, Zijin Hong, Xiao Huang, and Jinsong Su. 2025. When to use graphs in rag: A comprehensive analysis for graph retrieval-augmented generation. arXiv preprint arXiv:2506.05690. — [arXiv:2506.05690](https://arxiv.org/abs/2506.05690) find
  30. Junyuan Zhang, Qintong Zhang, Bin Wang, Linke Ouyang, Zichen Wen, Ying Li, Ka-Ho Chow, Conghui He, and Wentao Zhang. 2024. Ocr hinders rag: Evaluating the cascading impact of ocr on retrieval-augmented generation. arXiv preprint arXiv:2412.02592. — [arXiv:2412.02592](https://arxiv.org/abs/2412.02592) find
  31. Qinggang Zhang, Shengyuan Chen, Yuanchen Bei, Zheng Yuan, Huachi Zhou, Zijin Hong, Junnan Dong, Hao Chen, Yi Chang, and Xiao Huang. 2025. A survey of graph retrieval-augmented generation for customized large language models. arXiv preprint arXiv:2501.13958.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2025-molorag-bootstrapping-document-understanding-multi/