Elamur Research⌕
☰
home / papers / 2026 / Do Multi-Agents Solve Better Than Single? Evaluating Agentic Frameworks for Diagram-Grounded Geometry Problem Solving and Reasoning

Do Multi-Agents Solve Better Than Single? Evaluating Agentic Frameworks for Diagram-Grounded Geometry Problem Solving and Reasoning

Paper · 2026 · Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Mohammad Nehad Alam et al.

Summary

The paper asks whether splitting diagram-based geometry solving across several agents actually beats a single multimodal model, a question earlier work left largely unexamined. The authors compare two zero-shot setups with frozen weights. In the first, one MLLM reads the diagram and question and answers directly. In the second, an Interpreter VLM converts the diagram into formal predicates (points, lines, parallelism, angle measures), and a separate Solver LLM reasons over those literals. Experiments cover Geometry3K, MathVerse, OlympiadBench and We-Math, using Qwen2.5-VL (7B/32B), Gemini-2.0-Flash, GPT-4o, Phi-4 and Qwen3-8B. The two-agent pipeline usually helps open-source models. Gemini-2.0-Flash, however, tends to do better alone on the older benchmarks and gains only slightly on We-Math. An ablation shows that the overall result depends heavily on how good the Interpreter's predicates are. The authors conclude that decomposition is useful but not always the best choice, and they suggest adaptive selection between the two modes. Code and reasoning traces are released (GitHub: Interpreter-Solver).

Key points

  • Setup: Open-source models were 4-bit quantized with Unsloth. Scoring used Pass@3, and Gemini-2.0-Flash acted as the LLM judge for free-form answers. In the main multi-agent results, the Interpreter was always Gemini-2.0-Flash.
  • Gains for Qwen-2.5-VL:
    • On Geometry3K, 7B improved by +6.8 (53.24% → 60.07%) and 32B by +3.3 (68.72% → 72.05%).
    • On OlympiadBench, 7B improved by +9.4 and 32B by +6.67.
    • On MathVerse, 7B dropped by 6.0 points.
  • Gemini-2.0-Flash: Single-agent was stronger on the classic benchmarks (85.19% vs 83.86% on Geometry3K). Multi-agent added +1.74 on We-Math, where all three models improved.
  • Interpreter ablation: Using Qwen-2.5-7B as Interpreter gave 35.77% with Phi-4 and 42.26% with Qwen-3 as Solver. Using Gemini raised these to 70.05% and 79.53%.
  • Predicate fidelity: Fidelity was measured as cosine similarity using text-embedding-3-large. Scores were 0.849 for Gemini, 0.794 for GPT-4o mini and 0.677 for Qwen-2.5.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓BRAC UniversityDhaka · Bangladesh6 works · 10 citations🎓United International UniversityDhaka · Bangladesh4 works · 4 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Mohammad Nehad Alam, Proma Hossain Progga and Swakkhar Shatabda. 2026. Do Multi-Agents Solve Better Than Single? Evaluating Agentic Frameworks for Diagram-Grounded Geometry Problem Solving and Reasoning. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-srw.4

@inproceedings{sobhani2026do,
  title = {{Do Multi-Agents Solve Better Than Single? Evaluating Agentic Frameworks for Diagram-Grounded Geometry Problem Solving and Reasoning}},
  author = {Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Mohammad Nehad Alam, Proma Hossain Progga and Swakkhar Shatabda},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-srw.4},
  url = {https://doi.org/10.18653/v1/2026.eacl-srw.4}
}

References

20 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zihan Huang, Tao Wu, Wang Lin, Shengyu Zhang, Jingyuan Chen, and Fei Wu. 2025a. Autogeo: Automating geometric image dataset creation for enhanced geometry understanding. IEEE Transactions on Multimedia, 27:3105–3116. find
  2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  3. Zihan Huang, Tao Wu, Wang Lin, Shengyu Zhang, Jingyuan Chen, and Fei Wu. 2025b. Autogeo: Automating geometric image dataset creation for enhanced geometry understanding. IEEE Transactions on Multimedia. find
  4. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. Qwen2.5-vl technical report. Preprint, arXiv:2502.13…
  5. Mehran Kazemi, Hamidreza Alvari, Ankit Anand, Jialin Wu, Xi Chen, and Radu Soricut. 2023. Geomverse: A systematic evaluation of large models for geometric reasoning. arXiv preprint arXiv:2312.12241. — [arXiv:2312.12241](https://arxiv.org/abs/2312.12241) find
  6. Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish Thapliyal, James Bradbury, and 10 others.… — PaLI: A Jointly-Scaled Multilingual Language-Image Model
  7. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the…
  8. Jo-Ku Cheng, Zeren Zhang, Ran Chen, Jingyang Deng, Ziran Qin, and Jinwen Ma. 2025. Geouni: A unified model for generating geometry diagrams, problems and problem solutions. arXiv preprint arXiv:2504.10146. — [arXiv:2504.10146](https://arxiv.org/abs/2504.10146) find
  9. Yuting Li, Lai Wei, Kaipeng Zheng, Jingyuan Huang, Linghe Kong, Lichao Sun, and Weiran Huang. 2025. Vision matters: Simple visual perturbations can boost multimodal math reasoning. arXiv preprint arXiv:2506.09736. math problems with meta in-context learning. In Proceedings of the 2nd Workshop on Lar… — [arXiv:2506.09736](https://arxiv.org/abs/2506.09736) find
  10. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306.
  11. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, Peng Gao, and Hongsheng Li. 2025. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In Computer Vision – ECCV 2024, pages 169–186, Cham. Spring…
  12. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In Advances in Neural Information Processing Systems, volume 36, pages 34892–34916. Curran Associates, Inc.
  13. Wenwen Zhuang, Xin Huang, Xiantao Zhang, and Jin Zeng. 2025. Math-puma: Progressive upward multimodal alignment to enhance mathematical reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26183–26191. find
  14. Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. 2021. Inter-GPS: Interpretable geometry problem solving with formal language and symbolic reasoning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int…
  15. Yicheng Pan, Zhenrong Zhang, Pengfei Hu, Jiefeng Ma, Jun Du, Jianshu Zhang, Quan Liu, Jianqing Gao, and Feng Ma. 2025. Enhancing the geometric problem-solving ability of multimodal llms via symbolic-neural integration. arXiv preprint arXiv:2504.12773. — [arXiv:2504.12773](https://arxiv.org/abs/2504.12773) find
  16. Shuai Peng, Di Fu, Yijun Liang, Liangcai Gao, and Zhi Tang. 2023. GeoDRL: A self-learning framework for geometry problem solving using reinforcement learning in deductive reasoning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13468–13480, Toronto, Canada. Associatio… find
  17. Bowen Ping, Minnan Luo, Zhuohang Dang, Chenxi Wang, and Chengyou Jia. 2025. Autogps: Automated geometry problem solving via multimodal formalization and deductive reasoning. arXiv preprint arXiv:2505.23381. — [arXiv:2505.23381](https://arxiv.org/abs/2505.23381) find
  18. Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Chong Sun, Xiaoshuai Song, Jiapeng Wang, Zhuoma GongQue, Shanglin Lei, YiFan Zhang, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Xiao Zong, Yida Xu, Peiqing Yang, Zhimin Bao, Muxi Diao, Chen Li, and Honggang Zhang. 2025. We-math: Does your large mul… find
  19. Yikun Wang, Yibin Wang, Dianyi Wang, Zimian Peng, Qipeng Guo, Dacheng Tao, and Jiaqi Wang. 2025. Geometryzero: Improving geometry solving for llm with group contrastive policy optimization. arXiv preprint arXiv:2506.07160. — [arXiv:2506.07160](https://arxiv.org/abs/2506.07160) find
  20. Shihao Xu, Yiyang Luo, and Wei Shi. 2024. Geo-llava: A large multi-modal model for solving geometry find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sobhani-2026-do-multi-agents-solve-better/