Elamur Research⌕
☰
home / papers / 2026 / Mechanistic Interpretability of Text-to-Image Diffusion Models via Cross-Attention Interventions

Mechanistic Interpretability of Text-to-Image Diffusion Models via Cross-Attention Interventions

Paper · 2026 · Maisha Maliha and Dean F. Hougen

Summary

Cross-attention injects text into Stable Diffusion's UNet, but it is poorly understood which prompt tokens shape which image regions and which internal components carry that influence. Heatmaps built from raw attention show where the model "looks," not how much signal a token actually contributes. The authors propose a training-free interpretability framework that logs cross-attention at every denoising step. It scores each token by the magnitude of its contribution after the value and output projections, a norm-based measure, rather than by attention probabilities alone. To test causality, they remove individual prompt words one by one and generate the image again, keeping the seed and all settings fixed. A KL-divergence "spike score" then measures how much each head's token-contribution distribution shifts, and these scores are grouped by part-of-speech. Grounding turns out to be sparse and head-specific: a few heads matter most, and different POS classes rely on different head subsets. This gives a reproducible way to diagnose failures such as neglected objects and to steer generation without changing weights.

Key points

  • The experiments use Stable Diffusion v1.5, and the code is released on GitHub (MaishaMaliha1/cross-attention-interventions).
  • In the head ablation, removing the top-5 most sensitive heads drops grounding specificity from 0.82 to 0.46 (grounding loss 0.53). Removing 5 random heads gives specificity 0.69 (loss 0.18).
  • In the POS analysis, NOUN removal causes the largest effect (Avg. KL 0.094, Spec. 0.82), followed by ADJ (0.051 / 0.61), VERB (0.034 / 0.45) and function words (0.012 / 0.21).
  • Different POS classes (e.g., DET, VERB, ADP) activate partly distinct head subsets, which indicates that heads specialize.
  • Amplifying the projected cross-attention value stream ("token steering") on multi-object prompts raises CLIPScore from 0.289 to 0.307 and the aesthetic score from 5.61 to 5.88.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Vision & language · 657 worksInterpretability · 130 works

Cite

Cite as Maisha Maliha and Dean F. Hougen. 2026. Mechanistic Interpretability of Text-to-Image Diffusion Models via Cross-Attention Interventions. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1265

@inproceedings{maliha2026mechanistic,
  title = {{Mechanistic Interpretability of Text-to-Image Diffusion Models via Cross-Attention Interventions}},
  author = {Maisha Maliha and Dean F. Hougen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1265},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1265}
}

References

23 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Samyadeep Basu, Nanxuan Zhao, Vlad Morariu, Soheil Feizi, and Varun Manjunatha. 2024. Localizing and editing knowledge in text-to-image generative models. In International Conference on Learning Representations (ICLR). find
  2. Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. 2023. Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4):1–10. find
  3. Hila Chefer, Shir Gur, and Lior Wolf. 2021. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 782–791. find
  4. Shaurya Dewan, Rushikesh Zawar, Prakanshul Saxena, Yingshan Chang, Andrew Luo, and Yonatan Bisk. 2024. Diffusion PID: Interpreting diffusion via partial information decomposition. In Advances in Neural Information Processing Systems, volume 37, pages 2045–2079. find
  5. Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau. 2023. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2426–2436. find
  6. Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzynska, and David Bau. 2024. Unified concept editing in diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 5111–5120. find
  7. Simon Hentschel, Konstantin Kobs, and Andreas Hotho. 2022. Clip knows image aesthetics. Frontiers in Artificial Intelligence, 5:976235. find
  8. Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. 2023. Prompt-to-prompt image editing with cross-attention control. In International Conference on Learning Representations. find
  9. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. CLIPScore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514–7528.
  10. Sarthak Jain and Byron C. Wallace. 2019. Attention is not Explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3543–3556.
  11. Mingi Kwon, Jaeseok Jeong, and Youngjung Uh. 2023. Diffusion models already have a semantic latent space. In International Conference on Learning Representations. find
  12. Byung Hyun Lee, Sungjin Lim, and Se Young Chun. 2025. Localized concept erasure for text-to-image diffusion models using training-free gated low-rank adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18596–18606. find
  13. Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. 2024. MACE: Mass concept erasure in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6430–6440. find
  14. Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. 2022. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In Proceedings of the 39th International Conference on Machine Learni… find
  15. Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. 2023. What the DAAM: Interpreting stable diffusion using cross attention. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume… find
  16. Jesse Vig and Yonatan Belinkov. 2019. Analyzing the structure of attention in a transformer language model. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 63–76.
  17. Jungwon Park, Jungmin Ko, Dongnam Byun, Jangwon Suh, and Wonjong Rhee. 2025. Cross-attention head position patterns can align with human visual concepts in text-to-image generative models. In International Conference on Learning Representations. find
  18. Yong-Hyun Park, Mingi Kwon, Jaewoong Choi, Junghyo Jo, and Youngjung Uh. 2023. Understanding the latent space of diffusion models through the lens of riemannian geometry. In Advances in Neural Information Processing Systems, volume 36, pages 24129–24142. find
  19. Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. 2023. Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment. In Advances in Neural Information Processing Systems, volume 36, pages 3536–3559. find
  20. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695. find
  21. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. 2022. Photorealistic text-to-image diffusion models with deep language understanding. In A…
  22. Sofia Serrano and Noah A. Smith. 2019. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931–2951.
  23. Jiaming Song, Chenlin Meng, and Stefano Ermon. 2021. Denoising diffusion implicit models. In International Conference on Learning Representations.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/maliha-2026-mechanistic-interpretability-text-image-diffusion/