Elamur Research⌕
☰
home / papers / 2026 / Diffusion-CAM: Faithful Visual Explanations for dMLLMs

Diffusion-CAM: Faithful Visual Explanations for dMLLMs

Paper · 2026 · Haomin Zuo, Yidi Li, Luoxiao Yang et al.

Summary

Diffusion-based multimodal LLMs (dMLLMs) such as LaViDa, LLaDA-V, MMaDA and Dream-VL produce answers by denoising all masked response tokens in parallel. Existing CAM-style explainers, like LLaVA-CAM and TAM, assume left-to-right token generation, so on dMLLMs they yield blurry, unfocused heatmaps. The authors present Diffusion-CAM, which they describe as the first visual explanation method built for this model family. It hooks an intermediate transformer block and backpropagates the final answer score to image-token features. These features are taken only at denoising steps that still contain the complete image-token span, and their positions are found dynamically from packing metadata. A Grad-CAM-style aggregation then produces a raw map, and four refinement modules clean it up. The modules target random spatial noise, confounders within the image, redundant token correlations and ViT "register" artifacts. On COCO Caption and GranDf, Diffusion-CAM localizes objects more accurately and suppresses background better than GradCAM, LLaVA-CAM and TAM. This gives practitioners a usable tool for inspecting non-autoregressive multimodal systems.

Key points

  • Attribution step selection: a step qualifies only if its hidden states still hold the full image-token span. Under LaViDa's Prefix-DLM inference, only the initial conditioning step (s = 0) meets this condition.
  • Image-token localization: image-token boundaries are read from stored metadata (info4cam), and the extracted tokens are reshaped into an H×W spatial feature map.
  • Refinement modules: the named modules include Adaptive Kernel Denoising, a rank Gaussian filter whose kernel size depends on feature statistics, and Single-Instance Causal Debiasing. The authors also define four metrics for fine-grained evaluation of explanations.
  • Comparison and layers: the method is evaluated against GradCAM, LLaVA-CAM and TAM on COCO Caption and GranDf. Layer 10 gives the sharpest maps, and the method stays ahead of baselines even at layer 25.
  • Limitation: abstract or emotional concepts, such as "atmosphere," produce scene-wide, diffuse maps. Code is released on GitHub (ZzzzzZhhmm/Diffusion-CAM).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China149 works · 16,161 citations🎓Technion – Israel Institute of TechnologyHaifa · Israel49 works · 5,306 citations🎓Northwestern UniversityEvanston · United States41 works · 2,194 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Haomin Zuo, Yidi Li, Luoxiao Yang and Xiaofeng Zhang. 2026. Diffusion-CAM: Faithful Visual Explanations for dMLLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.553

@inproceedings{zuo2026diffusion,
  title = {{Diffusion-CAM: Faithful Visual Explanations for dMLLMs}},
  author = {Haomin Zuo, Yidi Li, Luoxiao Yang and Xiaofeng Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.553},
  url = {https://doi.org/10.18653/v1/2026.acl-long.553}
}

References

67 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zhibin Liao. 2024. Cape: Cam as a probabilistic ensemble for enhanced dnn interpretation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11072–11081. find
  2. Samira Abnar and Willem Zuidema. 2020. Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00928.
  3. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  4. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, and 1 others. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716– 23736.
  5. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 1(2):3.
  6. Sriram Balasubramanian, Samyadeep Basu, and Soheil Feizi. 2024. Decomposing and interpreting image representations via text in vits beyond clip. Advances in Neural Information Processing Systems, 37:81046– 81076. find
  7. Shuochen Chang, Xiaofeng Zhang, Qingyang Liu, and Li Niu. 2025. D3-tom: Decider-guided dynamic token merging for accelerating diffusion mllms. arXiv preprint arXiv:2511.12280. — [arXiv:2511.12280](https://arxiv.org/abs/2511.12280) find
  8. Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. 2018. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE winter conference on applications of computer vision (WACV), pages 839–847. IEEE. find
  9. Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. 2023. Vision transformers need registers. arXiv preprint arXiv:2309.16588. — [arXiv:2309.16588](https://arxiv.org/abs/2309.16588) find
  10. Nelson Elhage and 1 others. 2021. A mathematical framework for transformer circuits. Transformer Circuits Thread. find
  11. Javier Ferrando and Elena Voita. 2024. Information flow routes: Automatically interpreting language models at scale. arXiv preprint arXiv:2403.00824.
  12. Minju Gwak, Guijin Son, and Jaehyung Kim. 2025. Revisiting the uniform information density hypothesis in llm reasoning traces. arXiv preprint arXiv:2510.06953. find
  13. Hailin Hao and Elsi Kaiser. 2025. Uniform information density and syntactic reduction: Revisiting* that*-mentioning in english complement clauses. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 21980– 21994.
  14. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778.
  15. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840– 6851. find
  16. T Jaeger and Roger Levy. 2006. Speakers optimize information density through syntactic reduction. Advances in neural information processing systems, 19. find
  17. Hila Chefer, Shir Gur, and Lior Wolf. 2021. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 782–791. find
  18. Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei. 2021. Layercam: Exploring hierarchical class activation maps for localization. IEEE transactions on image processing, 30:5875–5888. find
  19. Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325.
  20. Shufan Li, Jiuxiang Gu, Kangning Liu, Zhe Lin, Zijun Wei, Aditya Grover, and Jason Kuen. 2025a. Lavida-o: Elastic large masked diffusion models for unified multimodal understanding and generation. arXiv preprint arXiv:2509.19244. — [arXiv:2509.19244](https://arxiv.org/abs/2509.19244) find
  21. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and 1 others. 2024. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference on computer vision and pa…
  22. Shufan Li, Jiuxiang Gu, Kangning Liu, Zhe Lin, Zijun Wei, Aditya Grover, and Jason Kuen. 2025b. Sparse-lavida: Sparse multimodal discrete diffusion language models. arXiv preprint arXiv:2512.14008. — [arXiv:2512.14008](https://arxiv.org/abs/2512.14008) find
  23. Townim Faisal Chowdhury, Kewen Liao, Vu Minh Hieu Phan, Minh-Son To, Yutong Xie, Kevin Hung, David Ross, Anton Van Den Hengel, Johan W Verjans, and Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai-Wei Chang, and Aditya Grover. 20… — [arXiv:2505.16839](https://arxiv.org/abs/2505.16839) find
  24. Shufan Li, Yuchen Zhu, Jiuxiang Gu, Kangning Liu, Zhe Lin, Yongxin Chen, Molei Tao, Aditya Grover, and Jason Kuen. 2026. Lavida-r1: Advancing reasoning for unified multimodal diffusion language models. arXiv preprint arXiv:2602.14147. — [arXiv:2602.14147](https://arxiv.org/abs/2602.14147) find
  25. Andrew Nam, Henry Conklin, Yukang Yang, Thomas Griffiths, Jonathan Cohen, and Sarah-Jane Leslie. 2025. Causal head gating: A framework for interpreting roles of attention heads in transformers. arXiv preprint arXiv:2505.13737. — [arXiv:2505.13737](https://arxiv.org/abs/2505.13737) find
  26. Xingjian Li, Qiming Zhao, Neelesh Bisht, Mostofa Rafid Uddin, Jin Yu Kim, Bryan Zhang, and Min Xu. 2025d. Diffcam: Data-driven saliency maps by capturing feature differences. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10327–10337. find
  27. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. Large language diffusion models. arXiv preprint arXiv:2502.09992.
  28. Yi Li, Hualiang Wang, Xinpeng Ding, Haonan Wang, and Xiaomeng Li. 2025e. Token activation map to visually explain multimodal llms. arXiv preprint arXiv:2506.23270. — [arXiv:2506.23270](https://arxiv.org/abs/2506.23270) find
  29. Yi Li, Hualiang Wang, Yiqun Duan, Jiheng Zhang, and Xiaomeng Li. 2025f. A closer look at the explainability of contrastive language-image pre-training. Pattern Recognition, 162:111409. find
  30. Chonghua Liao, Ruobing Xie, Xingwu Sun, Haowen Sun, and Zhanhui Kang. 2025. Exploring forgetting in large language model pre-training. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2112–2127, Vienna, Austria. Association for… find
  31. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306.
  32. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892– 34916.
  33. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.
  34. Clara Meister, Tiago Pimentel, Patrick Haller, Lena Jäger, Ryan Cotterell, and Roger Levy. 2021. Revisiting the uniform information density hypothesis. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 963–980.
  35. Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller. 2019. Layer-wise relevance propagation: an overview. Explainable AI: interpreting, explaining and visualizing deep learning, pages 193–209. find
  36. Marius Mosbach, Vagrant Gautam, Tomás Vergara Browne, Dietrich Klakow, and Mor Geva. 2024. From insights to actions: The impact of interpretability and analysis research on nlp. In Proceedings of the 2024 conference on empirical methods in natural language processing, pages 3078–3105. find
  37. Catherine Olsson and 1 others. 2022. In-context learning and induction heads. arXiv preprint arXiv:2209.11895.
  38. Daniel Omeiza, Skyler Speakman, Celia Cintas, and Komminist Weldermariam. 2019. Smooth grad-cam++: An enhanced inference level visualization technique. arXiv preprint arXiv:1908.01224. — [arXiv:1908.01224](https://arxiv.org/abs/1908.01224) find
  39. Xu Pan, Aaron Philip, Ziqian Xie, and Odelia Schwartz. 2024. Dissecting query-key interaction in vision transformers. arXiv preprint arXiv:2405.14880. — [arXiv:2405.14880](https://arxiv.org/abs/2405.14880) find
  40. Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S Khan. 2024. Glamm: Pixel grounding large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages… find
  41. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135– 1144.
  42. Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626.
  43. Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In ICML.
  44. Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. 2017. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825. — [arXiv:1706.03825](https://arxiv.org/abs/1706.03825) find
  45. Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502.
  46. Yang Song and Stefano Ermon. 2019. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32.
  47. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319– 3328. PMLR. find
  48. Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Türe. 2023. What the daam: Interpreting stable diffusion using cross attention. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume… find
  49. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  50. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  51. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  52. Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. 2020. Score-cam: Score-weighted visual explanations for convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 24–25. find
  53. Mengru Wang, Yunzhi Yao, Ziwen Xu, Shuofei Qiao, Shumin Deng, Peng Wang, Xiang Chen, Jia-Chen Gu, Yong Jiang, Pengjun Xie, and 1 others. 2024a. Knowledge mechanisms in large language models: A survey and perspective. arXiv preprint arXiv:2407.15017. find
  54. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024b. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191.
  55. Jinfeng Wei and Xiaofeng Zhang. 2024. Dopra: Decoding over-accumulation penalization and re-allocation in specific weighting layer. Proceedings of the 32nd ACM International Conference on Multimedia. find
  56. Peng Xu, Xiatian Zhu, and David A Clifton. 2023. Multimodal learning with transformers: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10):12113–12132. find
  57. Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. 2025. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505.15809. — [arXiv:2505.15809](https://arxiv.org/abs/2505.15809) find
  58. Jiacheng Ye, Shansan Gong, Jiahui Gao, Junming Fan, Shuang Wu, Wei Bi, Haoli Bai, Lifeng Shang, and Lingpeng Kong. 2025a. Dream-vl & dream-vla: Open vision-language and vision-language-action models with diffusion language model backbone. arXiv preprint arXiv:2512.22615. — [arXiv:2512.22615](https://arxiv.org/abs/2512.22615) find
  59. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025b. Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508.15487. find
  60. Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li. 2025. Llada-v: Large language diffusion models with visual instruction tuning. arXiv preprint arXiv:2505.16933.
  61. Xiaofeng Zhang, Yihao Quan, Chaochen Gu, Chen Shen, Xiaosong Yuan, Shaotian Yan, Hao Cheng, Kaijie Wu, and Jieping Ye. 2025a. Shallow focus, deep fixes: Enhancing shallow layers vision attention sinks to alleviate hallucination in lvlms. pages 3512–3534.
  62. Xiaofeng Zhang, Yihao Quan, Chen Shen, Xiaosong Yuan, Shaotian Yan, Liang Xie, Wenxiao Wang, Chaochen Gu, Hao Tang, and Jieping Ye. 2024. From redundancy to relevance: Information flow in lvlms across reasoning tasks. arXiv preprint arXiv:2406.06579.
  63. Xiaofeng Zhang, Yuanchao Zhu, Chaochen Gu, Xiaosong Yuan, Qiyan Zhao, Jiawei Cao, Feilong Tang, Sinan Fan, Yaomin Shen, Chen Shen, and 1 others. 2026. Hallucination begins where saliency drops. In The Fourteenth International Conference on Learning Representations. find
  64. Ziheng Zhang, Jianyang Gu, Arpita Chowdhury, Zheda Mai, David Carlyn, Tanya Berger-Wolf, Yu Su, and Wei-Lun Chao. 2025b. Finer-cam: Spotting the difference reveals finer details for visual explanation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 9611–9620. find
  65. Qiyan Zhao, Xiaofeng Zhang, Shuochen Chang, Qianyu Chen, Xiaosong Yuan, Xuhang Chen, Luoqi Liu, Jiajun Zhang, Xu-Yao Zhang, and Da-Han Wang. 2026. Context tokens are anchors: Understanding the repetition curse in dmllms from an information flow perspective. In The Fourteenth International Conference… find
  66. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2).
  67. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zuo-2026-diffusion-cam-faithful-visual-explanations/