Elamur Research⌕
☰
home / papers / 2026 / Beyond the Panorama: Training-Free Hierarchical Perception-Reasoning for Fine-Grained Vision in MLLMs

Beyond the Panorama: Training-Free Hierarchical Perception-Reasoning for Fine-Grained Vision in MLLMs

Paper · 2026 · Xiaoyang Yi, Jing Chen, Li Peng et al.

Summary

Multimodal large language models (MLLMs) often fail at fine-grained visual questions. When the relevant object covers only a small part of the image, compressing the whole picture into a fixed-length token sequence blurs local detail. The model may then answer from language priors instead of visual evidence, which leads to hallucinations. Earlier fixes either need extra training or region annotations, or they keep chain-of-thought reasoning separate from perception.

The authors propose HiPerson, a training-free framework that imitates how humans inspect a scene. It works in three steps:

  1. It builds a query-specific heatmap by fusing two internal signals: relative attention, which compares the query's attention with a neutral prompt, and gradient-based activation.
  2. It searches the heatmap with sliding windows at two scales and scores saliency to pick and enlarge the key region.
  3. It gives the model the crop and the full image together, plus a structured multi-step prompt that moves from examining details to checking them against context.

The authors report competitive results on several datasets and more interpretable reasoning. Because no fine-tuning is needed, the method can be added to existing MLLMs.

Key points

  • HiPerson has three modules: Dual-Path Region Refinement, Dual-Scale Adaptive Cropping and Dual-View Enhanced Reasoning.
  • Localization uses last-layer attention from query tokens to image patches, averaged over all query tokens. This is normalized against a reference prompt such as "Describe this image" to suppress background bias, then fused with a gradient activation map.
  • The authors compare HiPerson with related approaches: SEAL, ViCrop, VisCoT, R-GRPO and Pixel Reasoner. Unlike the training-based methods among them, HiPerson needs no fine-tuning or annotations.
  • In a qualitative example, LLaVA misread two athletes' jersey numbers as "41 and 5", while HiPerson correctly answered 10 and 3. The available excerpt does not include the quantitative benchmark scores.
  • Stated limitations:
    • the method depends on the base model's internal signals;
    • the prompts are designed around common granularity patterns;
    • testing was limited to controlled benchmarks, not open-world or domain-specific data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Xiaoyang Yi, Jing Chen, Li Peng, Yuru Bao and Jian Zhang. 2026. Beyond the Panorama: Training-Free Hierarchical Perception-Reasoning for Fine-Grained Vision in MLLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1409

@inproceedings{yi2026beyond,
  title = {{Beyond the Panorama: Training-Free Hierarchical Perception-Reasoning for Fine-Grained Vision in MLLMs}},
  author = {Xiaoyang Yi, Jing Chen, Li Peng, Yuru Bao and Jian Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1409},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1409}
}

References

41 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. Preprint, arXiv:2308.12966.
  2. Zhenyang Cai, Junying Chen, Rongsheng Wang, Weihong Wang, Yonglin Deng, Dingjie Song, Yize Chen, Zixu Zhang, and Benyou Wang. 2025. Exploring compositional generalization of multimodal llms for medical imaging. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistic… find
  3. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. 2024a. Intern vl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In 2024 IEEE/CVF Conference… find
  4. Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Zhiqing Sun, Dan Gutfreund, and Chuang Gan. 2024b. Visual chain-of-thought prompting for knowledge-based visual reasoning. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications… find
  5. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. 2023. Instructblip: Towards general-purpose vision-language models with instruction tuning. In Advances in Neural Information Processing Systems 36: Annual Conferenc…
  6. Dong Feng, Ping Guo, Encheng Peng, Mingmin Zhu, Wenhao Yu, and Peng Wang. 2025. Posellava: Pose centric multimodal LLM for fine-grained 3d pose manipulation. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 - March 4, 2025, Philadelphia, PA, USA, p… find
  7. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pa…
  8. Liqi He, Zuchao Li, Xiantao Cai, and Ping Wang. 2024. Multi-modal latent space learning for chain-of-thought reasoning in language models. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024,… find
  9. Chaoya Jiang, Yongrui Heng, Wei Ye, Haiyang Xu, Ming Yan, Ji Zhang, Fei Huang, and Shikun Zhang. 2025a. VLM-r³: Region recognition, reasoning, and refinement for enhanced multimodal chain-of-thought. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  10. Yutao Jiang, Qiong Wu, Wenhao Lin, Wei Yu, and Yiyi Zhou. 2025b. What kind of visual tokens do we need? training-free visual token pruning for multimodal large language models from the perspective of graph. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, Febr… find
  11. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloé Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross B. Girshick. 2023. Segment anything. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, Octobe…
  12. Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of…
  13. Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. 2022. BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings…
  14. Wentong Li, Yuqian Yuan, Jian Liu, Dongqi Tang, Song Wang, Jie Qin, Jianke Zhu, and Lei Zhang. 2025a. To-kenpacker: Efficient visual projector for multimodal LLM. Int. J. Comput. Vis., 133(10):6794–6812. find
  15. Yunxin Li, Shenyuan Jiang, Baotian Hu, Longyue Wang, Wanqi Zhong, Wenhan Luo, Lin Ma, and Min Zhang. 2025b. Uni-moe: Scaling unified multimodal llms with mixture of experts. IEEE Trans. Pattern Anal. Mach. Intell., 47(5):3424–3439. find
  16. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
  17. Yexin Liu, Zhengyang Liang, Yueze Wang, Xianfeng Wu, Feilong Tang, Muyang He, Jian Li, Zheng Liu, Harry Yang, Sernam Lim, and Bo Zhao. 2025. Unveiling the ignorance of mllms: Seeing clearly, answering incorrectly. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashvill… find
  18. Zefeng Lu, Ronghao Lin, Yap-Peng Tan, and Haifeng Hu. 2025. Prompt-guided transformer and MLLM interactive learning for text-based pedestrian search. IEEE Trans. Inf. Forensics Secur., 20:7181–7196. find
  19. Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V. Jawahar. 2022. Infographicvqa. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022, Waikoloa, HI, USA, January 3-8, 2022, pages 2582– 2591. IEEE. find
  20. Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. 2021. Docvqa: A dataset for VQA on document images. In IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Waikoloa, HI, USA, January 3-8, 2021, pages 2199–2208. IEEE.
  21. Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, Ritu-raj Singh, and Godawari Sudhakar Rao. 2024. Kam-cot: Knowledge augmented multimodal chain-of-thoughts reasoning. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artifici… find
  22. Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel, pages 807–814. Omnipress. find
  23. Seokhyeon Park, Yumin Song, Soohyun Lee, Jaeyoung Kim, and Jinwook Seo. 2025. Leveraging multimodal LLM for inspirational user interface search. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI 2025, Yoko-hamaJapan, 26 April 2025- 1 May 2025, pages 579:1– 579:22.… find
  24. Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, and Jie Tang. 2025. Cogcom: A visual language model with chain-of-manipulations reasoning. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 2… find
  25. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th Internatio…
  26. Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 779–788. IEEE Computer Society. find
  27. Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. A-OKVQA: A benchmark for visual question answering using world knowledge. In Computer Vision -ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part VIII, volume…
  28. Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. 2024. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. In Advances in Neural Information Processing Systems 38: Annual Conf…
  29. Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards VQA models that can read. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 8317–8326. Computer Vision… find
  30. Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. 2025. Pixel reasoner: Incentivizing pixel space reasoning via curiosity-driven reinforcement learning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  31. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  32. Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, and Huajun Chen. 2025a. MLLM can see? dynamic correction decoding for hallucination mitigation. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.ne… find
  33. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, and 1 others. 2025b. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265.
  34. Penghao Wu and Saining Xie. 2024. V*: Guided visual search as a core mechanism in multimodal llms. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 13084–13094. IEEE.
  35. Yunzhe Xu, Yiyuan Pan, Zhe Liu, and Hesheng Wang. 2025. FLAME: learning to navigate with multimodal LLM in urban environments. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 - March 4, 2025, Philadelphia, PA, USA, pages 9005–9013. AAAI Press. find
  36. Xuzheng Yang, Junzhuo Liu, Peng Wang, Guoqing Wang, Yang Yang, and Heng Tao Shen. 2025. New dataset and methods for fine-grained compositional referring expression comprehension via specialist-mllm collaboration. IEEE Trans. Pattern Anal. Mach. Intell., 47(10):8598–8612. find
  37. Weihao Ye, Qiong Wu, Wenhao Lin, and Yiyi Zhou. 2025. Fit and prune: Fast and training-free visual token pruning for multi-modal large language models. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 -March 4, 2025, Philadelphia, PA, USA, pages 22… find
  38. Jiarui Zhang, Jinyi Hu, Mahyar Khayatkhoei, Filip Ilievski, and Maosong Sun. 2024. Exploring perceptual limitation of multimodal large language models. Preprint, arXiv:2402.07384. — [arXiv:2402.07384](https://arxiv.org/abs/2402.07384) find
  39. Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. 2023. Visual cropping improves zero-shot question answering of multimodal large language models. CoRR, abs/2310.16033. — [arXiv:2310.16033](https://arxiv.org/abs/2310.16033) find
  40. Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. 2025a. Mllms know where to look: Training-free perception of small visual details with multimodal llms. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.… find
  41. Yuting Zhang, Hao Lu, Qingyong Hu, Yin Wang, Kaishen Yuan, Xin Liu, and Kaishun Wu. 2025b. Period-llm: Extending the periodic capability of multimodal large language model. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, pages 29237… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yi-2026-beyond-panorama-training-free-hierarchical/