Elamur Research⌕
☰
home / papers / 2026 / Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World

Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World

Paper · 2026 · Vinu Sankar Sadasivan, Soheil Feizi, Rajiv Mathews et al.

Summary

The paper examines whether audio-based LLMs (ALLMs) can be steered by sound an attacker plays nearby. Most jailbreak work only produces harmful output for the attacker. Here the concern is attacks that reach innocent users through their microphones. Working in a white-box setting with the instruction-tuned Qwen2-Audio, the authors optimize short, low-amplitude noise clips by gradient descent on a perplexity loss under an ℓ∞ bound. They study two goals:

  • Targeted attacks force a chosen output, such as a wake phrase or a harmful command.
  • Untargeted attacks corrupt the model's answers to ordinary speech.

To survive playback through the air, the optimization includes augmentations such as SpecAugment and time shifts. In real-world tests, noise played from one device disrupted transcription on another device recording a user. The attacks also worked when the system prompt told the model to ignore background noise. Simple preprocessing defenses mostly failed, but EnCodec compression largely neutralized the attack. The authors note that adaptive attackers could likely bypass it. They argue the results should inform decisions about open-sourcing audio models.

Key points

  • Target model: Qwen2-Audio, which uses a Qwen-7B LLM backbone and a Whisper-large-v3 audio tower. Noise is mono 16 kHz, 2 or 4 seconds long, with ε of 0.01 or 0.1. Optimization runs 5000 iterations at a learning rate of 0.0002.
  • Targeted commands: the bare wake phrase "Hey Qwen", plus that same wake phrase followed by an instruction either to wipe the user's calendar entries or to transfer money to X.
  • Over-the-air setup: the attacker plays noise from a Chromebook, and the victim's ALLM records through an iPhone. Adversarial noise degraded transcriptions far more than random noise did.
  • Noise-ignore system instruction: untargeted perplexity was about 1.09 for clean audio, 2.54 for random noise and 12 for adversarial noise. The targeted "Hey Qwen" attack still succeeded 100% of the time.
  • Defenses:
    • Sample-rate changes broke the recorded attack.
    • Spectral gating (noisereduce) worked only at 100% reduction.
    • EnCodec blocked the attack almost completely.
    • The authors identify reliance on white-box access as the main limitation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Maryland, College ParkCollege Park · United States132 works · 25,380 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSpeech & spoken language · 480 works

Cite

Cite as Vinu Sankar Sadasivan, Soheil Feizi, Rajiv Mathews and Lun Wang. 2026. Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.66

@inproceedings{sadasivan2026attacker,
  title = {{Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World}},
  author = {Vinu Sankar Sadasivan, Soheil Feizi, Rajiv Mathews and Lun Wang},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.66},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.66}
}

References

40 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ehsan Amid, Om Thakkar, Arun Narayanan, Rajiv Mathews, and Françoise Beaufays. 2022. Extracting targeted training data from asr models, and how to mitigate it. arXiv preprint arXiv:2204.08345. — [arXiv:2204.08345](https://arxiv.org/abs/2204.08345) find
  2. Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, and 1 others. 2024. Many-shot jailbreaking. Advances in Neural Information Processing Systems, 37:129696–129742. find
  3. Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, and 1 others. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403.
  4. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  5. Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and 1 others. 2023. Audiolm: a language modeling approach to audio generation. IEEE/ACM transactions on audio, speech, and language p…
  6. Nicholas Carlini and David Wagner. 2018. Audio adversarial examples: Targeted attacks on speech-to-text. In 2018 IEEE security and privacy workshops (SPW), pages 1–7. IEEE. find
  7. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419.
  8. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, and 1 others. 2022a. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518… find
  9. Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, and Furu Wei. 2022b. Beats: Audio pre-training with acoustic tokenizers. arXiv preprint arXiv:2212.09058. — [arXiv:2212.09058](https://arxiv.org/abs/2212.09058) find
  10. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, and 1 others. 2024. Qwen2-audio technical report. arXiv preprint arXiv:2407.10759.
  11. Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. 2023. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919.
  12. Nilaksh Das, Saket Dingliwal, Srikanth Ronanki, Rohit Paturi, Zhaocheng Huang, Prashant Mathur, Jie Yuan, Dhanush Bekal, Xing Niu, Sai Muralidhar Jayanthi, and 1 others. 2024. Speechverse: A large-scale generalizable audio language model. arXiv preprint arXiv:2405.08295.
  13. Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2022. High fidelity neural audio compression. Preprint, arXiv:2210.13438.
  14. Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. 2024. Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities. arXiv preprint arXiv:2406.11768.
  15. Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023. Figstep: Jailbreaking large vision-language models via typographic visual prompts. arXiv preprint arXiv:2311.05608. find
  16. Daniel S Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D Cubuk, and Quoc V Le. 2019. Specaugment: A simple data augmentation method for automatic speech recognition. arXiv preprint arXiv:1904.08779.
  17. Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, and 1 others. 2024. Wavllm: Towards robust and adaptive speech large language model. arXiv preprint arXiv:2404.00656. find
  18. Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Mengdi Wang, and Prateek Mittal. 2023. Visual adversarial examples jailbreak large language models. CoRR. find
  19. Matthew Jagielski, Om Thakkar, and Lun Wang. 2024. Noise masking attacks and defenses for pretrained speech models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4810–4814. IEEE. find
  20. Mintong Kang, Chejian Xu, and Bo Li. 2024. Advwave: Stealthy adversarial jailbreak attack against large audio-language models. arXiv preprint arXiv:2412.08608. — [arXiv:2412.08608](https://arxiv.org/abs/2412.08608) find
  21. Kai Li, Can Shen, Yile Liu, Jirui Han, Kelong Zheng, Xuechao Zou, Zhe Wang, Xingjian Du, Shun Zhang, Hanjun Luo, and 1 others. 2025. Au-diotrust: Benchmarking the multifaceted trustworthiness of audio large language models. arXiv preprint arXiv:2505.16211. — [arXiv:2505.16211](https://arxiv.org/abs/2505.16211) find
  22. Hongbin Liu, Youzheng Chen, Arun Narayanan, Athula Balachandran, Pedro J Moreno, and Lun Wang. 2024. Can deepfake speech be reliably detected? arXiv preprint arXiv:2410.06572. — [arXiv:2410.06572](https://arxiv.org/abs/2410.06572) find
  23. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451.
  24. Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2024. Tree of attacks: Jailbreaking black-box llms automatically. Advances in Neural Information Processing Systems, 37:61065–61105.
  25. Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R Costa-Jussa, Maha Elbayad, Sravya Popuri, Christophe Ropers, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, and 1 others. 2025. Spirit-lm: Interleaved spoken and written language model. Transactions of the Association for Computational Lin…
  26. OpenAI. 2024. Introducing the gpt store. https://openai.com/index/ introducing-the-gpt-store/. Accessed: 2025-05-15. ↗
  27. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on, pages 5206–5210. IEEE. find
  28. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine Payne, Jan Leike, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. Preprint, arXiv:2212.04356.
  29. RedditUser123. 2023. “do anything now” (dan) chatgpt jailbreak prompt. https://www.reddit.com/ r/ChatGPT/... Accessed: 2025-03-30. ↗
  30. Jaechul Roh, Virat Shejwalkar, and Amir Houmansadr. 2025. Multilingual and multi-accent jailbreaking of audio llms. arXiv preprint arXiv:2504.01094. — [arXiv:2504.01094](https://arxiv.org/abs/2504.01094) find
  31. Paul K Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, and 1 others. 2023. Audiopalm: A large language model that can speak and listen. arXiv preprint arXiv:2306.12925.
  32. Vinu Sankar Sadasivan, Shoumik Saha, Gaurang Sriramanan, Priyatham Kattakinda, Atoosa Chegini, and Soheil Feizi. 2024. Fast adversarial attacks on language models in one gpu minute. arXiv preprint arXiv:2402.15570. find
  33. Tim Sainburg. 2019. timsainb/noisereduce: v1.0. find
  34. Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. 2023. Salmonn: Towards generic hearing abilities for large language models. arXiv preprint arXiv:2310.13289.
  35. Xijia Tao, Shuai Zhong, Lei Li, Qi Liu, and Lingpeng Kong. 2024. Imgtrojan: Jailbreaking vision-language models with one image. arXiv preprint arXiv:2403.02910. find
  36. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  37. Lun Wang, Om Thakkar, and Rajiv Mathews. 2024. Unintended memorization in large asr models, and how to mitigate it. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4655–4659. IEEE. find
  38. Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. 2023. Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities. arXiv preprint arXiv:2305.11000.
  39. Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. 2023. Autodan: interpretable gradient-based adversarial attacks on large language models. arXiv preprint arXiv:2310.15140. find
  40. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sadasivan-2026-attacker-s-noise-can-manipulate/