Elamur Research⌕
☰
home / papers / 2026 / When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs

When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs

Paper · 2026 · Hongcheng Liu, Pingjie Wang, Yuhao Wang et al.

Summary

Most MLLM benchmarks give the model every clue up front and ask for a single answer. This paper asks whether models can instead seek out missing evidence on their own. The authors frame this as a guessing game. A model sees a pool of candidate images, gets no hints about which one is the target, and must ask yes/no questions until it is confident enough to name its choice. To study this, they built GuessBench. It combines real photos from the IIW dataset with synthetic images in nine domains, which are split into perception-oriented and knowledge-oriented groups. An automated GuessAgent answers the model's questions, and scoring merges accuracy, efficiency and planning quality. Across 20 MLLMs, active performance falls well below passive performance. The main weaknesses are fine-grained perception, choosing informative questions, and knowing when to stop asking. The work matters because real assistants, recommenders and robots often act on incomplete information.

Key points

  • Synthetic domains: the perception-oriented group covers city, face, kitchen, livingroom and nature. The knowledge-oriented group covers animal, traffic, vehicle and occupation. Images were generated with Omnigen2 and then checked by humans.
  • GuessAgent: Qwen3-8B labels each model turn as QUERY, STOP or INVALID, and non-binary questions are rejected. Answers come first from image attributes and, if needed, from captions and the image itself.
  • Scale of the gap: Qwen2.5-VL-7B scores 91.2 in the passive setting but only 43.1 in the active one.
  • Enhancement effects: perceptual aids such as reasoning prompts and RAG mainly help small models like Qwen2.5-VL-3B. Thinking-style methods such as CoT and ReAct help models of every size.
  • Trained vs. prompted thinking: trained thinking modes beat prompt-based ones, improving accuracy while shortening interactions. Mimo-VL, an RL-trained thinking model built on Qwen2.5-VL-7B, gains 76.1% and outperforms CoT and ReAct.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Hongcheng Liu, Pingjie Wang, Yuhao Wang, Siqu Ou, Yanfeng Wang and Yu Wang. 2026. When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1264

@inproceedings{liu2026when,
  title = {{When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs}},
  author = {Hongcheng Liu, Pingjie Wang, Yuhao Wang, Siqu Ou, Yanfeng Wang and Yu Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1264},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1264}
}

References

40 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. Qwen2.5-vl technical report. Preprint, arXiv:2502.13…
  2. Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu, and Wanxiang Che. 2024a. M 3 cot: A novel benchmark for multi-domain multi-step multi-modal chain-of-thought. In Proc. of ACL.
  3. Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, Siamak Shakeri, Mostafa Dehghani, Daniel M. Salz, Mario Lucic, Michael Tschannen, Arsha Nagrani, Hexiang Hu, Mandar Joshi, Bo Pang, and 24 others. 2023.… find
  4. Zhe Chen, Yusheng Liao, Zhiyuan Zhu, Haolin Li, Hongcheng Liu, Yanfeng Wang, and Yu Wang. 2025. Heterorag: A heterogeneous retrieval-augmented generation framework for medical vision language tasks. arXiv preprint arXiv:2508.12778. find
  5. Zhe Chen, Heyang Liu, Wenyi Yu, Guangzhi Sun, Hongcheng Liu, Ji Wu, Chao Zhang, Yu Wang, and Yanfeng Wang. 2024b. M 3 AV: A multimodal, multi-genre, and multipurpose audio-visual academic lecture dataset. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol… find
  6. Zhe Chen, Hongcheng Liu, and Yu Wang. 2024c. Di-alogmcf: Multimodal context flow for audio visual scene-aware dialog. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:753–764. find
  7. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. 2024. Mme: A comprehensive evaluation benchmark for multimodal large language models. Preprint, arXiv:2306.13394.
  8. Roopal Garg, Andrea Burns, Burcu Karagol Ayan, Yonatan Bitton, Ceslee Montgomery, Yasumasa Onoe, Andrew Bunner, Ranjay Krishna, Jason Baldridge, and Radu Soricut. 2024. Imageinwords: Unlocking hyper-detailed image descriptions. Preprint, arXiv:2405.02793. find
  9. Zhang Ge, Du Xinrun, Chen Bei, Liang Yiming, Luo Tongxu, Zheng Tianyu, Zhu Kang, Cheng Yuyang, Xu Chunpu, Guo Shuyue, Zhang Haoran, Qu Xingwei, Wang Junjie, Yuan Ruibin, Li Yizhi, Wang Zekun, Liu Yudong, Tsai Yu-Hsuan, Zhang Fengji, and 3 others. 2024. Cmmmu: A chinese massive multi-discipline multi… — [arXiv:2401.20847](https://arxiv.org/abs/2401.20847) find
  10. Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2024. Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models.…
  11. Ibragim Khasanov. 2024. Guess, who i am? Apple App Store. IOS app, version 1.3. find
  12. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji rong Wen. 2023. Evaluating object hallucination in large vision-language models. In Conference on Empirical Methods in Natural Language Processing.
  13. Yunxin Li, Zhenyu Liu, Zitao Li, Xuanyu Zhang, Zhenran Xu, Xinyu Chen, Haoyuan Shi, Shenyuan Jiang, Xintong Wang, Jifang Wang, Shouzheng Huang, Xinping Zhao, Borui Jiang, Lanqing Hong, Longyue Wang, Zhuotao Tian, Baoxing Huai, Wenhan Luo, Weihua Luo, and 3 others. 2025. Perception, reason, think, an… — [arXiv:2505.04921](https://arxiv.org/abs/2505.04921) find
  14. Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, and Dong Yu. 2023. Mmc: Advancing multimodal chart understanding with large-scale instruction tuning. arXiv preprint arXiv:2311.10774.
  15. Hongcheng Liu, Yixuan Hou, Heyang Liu, Yuhao Wang, Yanfeng Wang, and Yu Wang. 2025. Vocalbench-df: A benchmark for evaluating speech llm robustness to disfluency. arXiv preprint arXiv:2510.15406. — [arXiv:2510.15406](https://arxiv.org/abs/2510.15406) find
  16. Hongcheng Liu, Yusheng Liao, Siqv Ou, Yuhao Wang, Heyang Liu, Yanfeng Wang, and Yu Wang. 2024a. Med-pmc: Medical personalized multi-modal consultation with a proactive ask-first-observe-next paradigm. arXiv preprint arXiv:2408.08693. find
  17. Hongcheng Liu, Pingjie Wang, Heyang Liu, Zhiyuan Zhu, Yusheng Liao, Yanfeng Wang, and Yu Wang. 2026a. Anchornet: Adaptive anchor token enhancement in video-grounded dialogue generation. IEEE Journal of Selected Topics in Signal Processing, pages 1–11. find
  18. Hongcheng Liu, Yuhao Wang, Zhe Chen, Pingjie Wang, Zhiyuan Zhu, Yixuan Hou, Yanfeng Wang, and Yu Wang. 2026b. Cross-modal coreference alignment: Enabling reliable information transfer in omni-llms. arXiv preprint arXiv:2604.05522. find
  19. Yexin Liu, Zhengyang Liang, Yueze Wang, Muyang He, Jian Li, and Bo Zhao. 2024b. Seeing clearly, answering incorrectly: A multimodal robustness benchmark for evaluating mllms on leading questions. Preprint, arXiv:2406.10638. — [arXiv:2406.10638](https://arxiv.org/abs/2406.10638) find
  20. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR).
  21. OpenAI. 2024. Gpt-4o mini: Advancing cost-efficient intelligence. OpenAI blog post. find
  22. Siqu Ou, Hongcheng Liu, Pingjie Wang, Yusheng Liao, Chuan Xuan, Yanfeng Wang, and Yu Wang. 2025. Bridging the dynamic perception gap: Training-free draft chain-of-thought for dynamic multimodal spatial reasoning. Preprint, arXiv:2505.16579. find
  23. Shaina Raza, Mizanur Rahman, Safiullah Kamawal, Armin Toroghi, Ananya Raval, Farshad Navah, and Amirmohammad Kazemeini. 2024. A comprehensive review of recommender systems: Transitioning from theory to practice. ArXiv, abs/2407.13699. — [arXiv:2407.13699](https://arxiv.org/abs/2407.13699) find
  24. Kimi Team. 2025a. Kimi-VL technical report. Preprint, arXiv:2504.07491.
  25. Qwen Team. 2025b. Qwen3-vl: Sharper vision, deeper thought, broader action. Qwen Blog. Accessed: 2025-10-04. find
  26. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, and 1 others. 2025. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265.
  27. Yuhao Wang, Yusheng Liao, Heyang Liu, Hongcheng Liu, Yu Wang, and Yanfeng Wang. 2024. Mm-sap: A comprehensive benchmark for assessing self-awareness of multimodal large language models in perception. Preprint, arXiv:2401.07529. find
  28. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.
  29. Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, Ze Liu, Ziyi Xia, Chaofan Li, Haoge Deng, Jiahao Wang, Kun Luo, Bo Zhang, Defu Lian, Xinlong Wang, and 3 others. 2025. Omnigen2: Exploration to advanced multimodal generation. arX… — [arXiv:2506.18871](https://arxiv.org/abs/2506.18871) find
  30. Yue Yang, Shuibai Zhang, Wenqi Shao, Kaipeng Zhang, Yi Bin, Yu Wang, and Ping Luo. 2024. Dynamic multimodal evaluation with flexible complexity by vision-language bootstrapping. ArXiv, abs/2410.08695. — [arXiv:2410.08695](https://arxiv.org/abs/2410.08695) find
  31. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. ArXiv, abs/2210.03629.
  32. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. A survey on multimodal large language models. National Science Review, 11(12).
  33. Fangyi Yu, Lee Quartey, and Frank Schilder. 2023. Exploring the effectiveness of prompt engineering for legal reasoning tasks. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13582–13596, Toronto, Canada. Association for Computational Linguistics. find
  34. Qingchen Yu, Zifan Zheng, Ding Chen, Simin Niu, Bo Tang, Feiyu Xiong, and Zhiyu Li. 2025. Guessarena: Guess who i am? a self-adaptive framework for evaluating llms in domain-specific knowledge and reasoning. arXiv preprint arXiv:2505.22661. find
  35. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, and 3 others. 2024. Mmmu: A massive multi-discipline multimodal u…
  36. Xiongtao Zhou, Jie He, Lanyu Chen, Jingyu Li, Haojing Chen, Víctor Gutiérrez-Basulto, Jeff Z. Pan, and Hanjie Chen. 2025a. Miceval: Unveiling multimodal chain of thought’s quality via image description and reasoning steps. Preprint, arXiv:2410.14668. find
  37. Zhanke Zhou, Xiao Feng, Zhaocheng Zhu, Jiangchao Yao, Sanmi Koyejo, and Bo Han. 2025b. From passive to active reasoning: Can large language models ask the right questions under incomplete information? Preprint, arXiv:2506.08295. — [arXiv:2506.08295](https://arxiv.org/abs/2506.08295) find
  38. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, and 1 others. 2025. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479.
  39. LLM-Core-Team Xiaomi. 2025. Mimo-vl technical report. Preprint, arXiv:2506.03569.
  40. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-when-seeing-is-not-enough/