从感知到推理：深度思考赋能多模态大语言模型

Summary

This survey, published in the Journal of Chinese Information Processing, examines how chain-of-thought (CoT) reasoning can move multimodal large language models (MLLMs) beyond perception toward complex reasoning. The authors argue that current MLLMs mostly predict from statistical patterns, with no explicit intermediate steps. This leaves them weak at multi-step logic and causal inference, and perception and reasoning are poorly connected. The review covers three areas of CoT-MLLMs. The first is reasoning topologies: chains, trees, graphs and hypergraphs. The second is post-training strategies, namely supervised fine-tuning and reinforcement learning on reasoning-annotated data. The third is inference-time techniques such as prompting, search, self-refinement, knowledge enhancement and agent support. Unlike earlier surveys, which mainly catalogue methods, this one emphasises why these mechanisms work. It also gives a probabilistic formulation of multimodal CoT with intra-modal, cross-modal fusion and output stages. The paper further organises benchmarks and metrics, application domains, and open challenges. It is a structured entry point for anyone building reasoning-capable vision-, audio- or speech-language systems.

Key points

• Comparison table of methods. Representative methods and models are tabulated by modality, post-training type and inference-time scaling. They range from HoT (2023) to systems such as LLaVA-CoT, Mulberry, Vision-R1, R1-Onevision, MedVLM-R1, Audio-Reasoner and R1-Omni (2025).
• Reasoning paradigms. Paradigms that inject non-text content into the reasoning steps are contrasted: ICoT, MVoT, Visual SKETCHPAD and BBA. Graph-style variants are also covered, including BDoG and Hypergraph-of-Thought.
• Training datasets. Datasets for CoT training are catalogued, for example:
  • COCO-MMR (62,351 items)
  • MM-PhyQA (4,500)
  • MathV360K (360K)
  • Visual CoT (438K)
  • GPSM4K (4,440)
  • AMATH-SFT (157K)
  • LLaVA-CoT-100k
• Evidence of reasoning gaps. OpenAI o1 scores below human level on PUZZLEVQA, and Qwen2.5-VL makes spatial and logical errors on region-colouring tasks.
• Applications and future directions. Applications include embodied AI, autonomous driving, healthcare, multimodal generation, machine translation, and affective/social reasoning. Future directions include robust, safe, omni-modal and efficient reasoning, better data and benchmark construction, and process supervision.