Elamur Research⌕
☰
home / papers / 2026 / mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models

mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models

Paper · 2026 · Jakub Muszyński, Paweł Pozorski and Maria Ganzha

Summary

Shapley-value attribution tools such as SHAP and TokenSHAP handle tabular, image or text-only inputs. They cannot explain multimodal LLMs that take speech and text together. The authors present mllm-shap, an open-source, MIT-licensed Python package with a web GUI that fills this gap for text-audio MLLMs. It needs only black-box access through a generate API.

The package addresses three problems:

  • Mixed inputs: it treats text tokens and audio frames differently within one coalition game. Text tokens are dropped, while audio is silenced by zeroing the waveform or frames inside segment boundaries.
  • Conversation structure: every token carries metadata for modality, role and turn, so multi-turn dialogues can be analysed and system tokens left out.
  • Too many audio units: dense audio frames are merged into word-level units with SGPA phonetic alignment, which shrinks the coalition space.

Five SV estimators are offered. A Neyman-allocated variant of Complementary Contributions gives the best accuracy. Demonstrations on LFM2-Audio-1.5B show sensible attributions; in a transcription task, for example, the audio input receives most of the credit. Overall, the tool makes principled multimodal attribution practical on consumer hardware.

Key points

  • Estimators: Exact, Monte Carlo, Complementary Contributions, Neyman-CC and Hierarchical. On 27 inputs of 9–10 tokens where exact SVs are computable, Neyman-CC exceeded 90% accuracy at a sampling fraction of only 0.15.
  • SGPA grouping: it uses CTC alignment with Wav2Vec2-XLSR-53 plus spectrogram-based boundary refinement. It cut about 50 audio tokens to about 7 segments, needing 43× fewer model calls and reducing runtime from about 30 minutes to about 1 minute per sample.
  • Evaluation scale: 593 SV analyses were run on a single consumer GPU, using the VoiceBench and Infinity-Instruct datasets.
  • Interface and distribution: the GUI uses a React frontend and a FastAPI backend, with cost estimation and session persistence. The package is installable from PyPI, and the code is on GitHub.
  • Limitations: only one model, LFM2-Audio-1.5B, was tested. Long inputs still take about 400 s each. There is no faithfulness comparison against LIME or Integrated Gradients.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Warsaw University of TechnologyWarsaw · Poland3 works · 256 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 4Research subjects and languages the work is about.Language models & pretraining · 3,320 worksVision & language · 657 worksSpeech & spoken language · 480 worksInterpretability · 130 works

Cite

Cite as Jakub Muszyński, Paweł Pozorski and Maria Ganzha. 2026. mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-demo.38

@inproceedings{muszynski2026mllm,
  title = {{mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models}},
  author = {Jakub Muszyński, Paweł Pozorski and Maria Ganzha},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-demo.38},
  url = {https://doi.org/10.18653/v1/2026.acl-demo.38}
}

References

20 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, volume 33, pages 12449–12460.
  2. Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T. Tan, and Haizhou Li. 2024. Voicebench: Benchmarking LLM-based voice assistants. arXiv preprint arXiv:2410.17196.
  3. Ian Covert, Scott M Lundberg, and Su-In Lee. 2021. Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22(209):1–90. find
  4. Roni Goldshmidt and Miriam Horovicz. 2024. To-kenshap: Interpreting large language models with monte carlo shapley value estimation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. find
  5. Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. 2006. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd International Conference on Machine Learning, pages 369–376. find
  6. Sarthak Jain and Byron C. Wallace. 2019. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3543–3556.
  7. Jijie Li, Li Du, Hanyu Zhao, Bo-wen Zhang, Liangdong Wang, Boyan Gao, Guang Liu, and Yonghua Lin. 2025. Infinity instruct: Scaling instruction selection and synthesis to enhance language models. arXiv preprint arXiv:2506.11116. — [arXiv:2506.11116](https://arxiv.org/abs/2506.11116) find
  8. Liquid AI. 2025. Lfm2-audio: Liquid foundation models for audio. https://www.liquid.ai/liquid-foundation-models/ lfm-2-audio. ↗
  9. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, volume 30.
  10. Meta Platforms. 2013. React: A JavaScript library for building user interfaces. https://github.com/facebook/react. ↗
  11. Rory Mitchell, Joshua Cooper, Eibe Frank, and Geoffrey Holmes. 2022. Approximating the shapley value without marginal contributions. Proceedings of the AAAI Conference on Artificial Intelligence, 36(7):7773–7780. find
  12. Edoardo Mosca, Ferenc Szigeti, Stella Tragianni, Daniel Gallagher, and Georg Groh. 2022. SHAP-based explanation methods: A review for NLP interpretability. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4593–4603. find
  13. Jerzy Neyman. 1934. On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection. Journal of the Royal Statistical Society, 97(4):558– 625. find
  14. Paweł Pozorski, Jakub Muszyński, and Maria Ganzha. 2026. Sgpa: Spectrogram-guided phonetic alignment for feasible shapley value explanations in multimodal large language models. Preprint, arXiv:2603.02250. Submitted to Interspeech 2026. fixed budget. The estimator operates in two phases: an initial… — [arXiv:2603.02250](https://arxiv.org/abs/2603.02250) find
  15. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pages 3973–3983.
  16. Hierarchical: A recursive decomposition strategy designed for long sequences that exceed the computational limits of flat estimators. Tokens are grouped into higher-level features, SVs are computed at the group level, and then recursively refined within groups of interest. find
  17. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144.
  18. Lloyd S. Shapley. 1951. Notes on the N-Person Game; II: The Value of an N-Person Game. RAND Corporation, Santa Monica, CA. find
  19. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning, pages 3319–3328. find
  20. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/muszynski-2026-mllm-shap-shapley-value-explainability/