Elamur Research⌕
☰
home / papers / 2026 / Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use

Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use

Paper · 2026 · Wuyang Zhang and Shichao Pei

Summary

Tool-using LLM agents often store personal details in session memory and can also send requests to outside search or documentation services. The authors show that this combination can become a covert leak channel. Their attack, Back-Reveal, plants a backdoor in an open-weight model during fine-tuning, and the model is then published as a domain-specialised assistant. Natural domain-specific phrasing acts as a semantic trigger. When it appears, the agent reads stored user context through a memory tool and hides it inside what looks like an ordinary retrieval request to an attacker-run server. That server identifies which profile fields are still missing. It then replies with relevant-looking content that carries hidden cues, prompting the agent to ask follow-up questions that collect more data over several turns. A reranker-aware rewriter keeps these replies from being filtered out. The authors tested the attack across three domains, three model sizes, seven rerankers and two guardrail tools, and it evaded all retrieval-stage defenses. They conclude that protection must move to the agent–tool boundary.

Key points

  • Exfiltration works by placing base64url-encoded session memory in a URL parameter of an outbound retrieval call. This lets it pass reranker filtering, NeMo Guardrails v0.16.0 and LLM Guard v0.3.16 essentially unchanged.
  • With the rewriter (Leak+Rewrite), attacker responses reach the top-5 in 85–91% of cases, compared with 62–70% for naive appending. Defense bypass is 81–87%, compared with 27–40%.
  • Over multiple turns, the steering gradually gathers more leaked attributes, working toward completing a 10-field profile. With fully cooperative users, about 7.8 fields leak in five turns. This falls to about 5.2, 2.6 and 0.8 fields at continuation probabilities of 0.75, 0.5 and 0.25.
  • Proposed mitigations include:
    • egress allow-lists and quotas;
    • auditing tool-call payloads for opaque or memory-derived strings;
    • separating memory-read from network-write capabilities;
    • provenance checks and backdoor scanning of open-weight checkpoints.
  • All experiments used synthetic personas, with GPT-5 as the user simulator, in a sandbox. The authors do not release backdoored checkpoints or attack infrastructure.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Massachusetts BostonBoston · United States3 works · 300 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Wuyang Zhang and Shichao Pei. 2026. Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1257

@inproceedings{zhang2026your,
  title = {{Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use}},
  author = {Wuyang Zhang and Shichao Pei},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1257},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1257}
}

References

42 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, and 110 others. 2024. Phi-… — Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
  2. Blue Goat Cyber. 2024. DNS exfiltration with Base64 encoding: A stealthy data theft technique. https://bluegoatcyber.com/blog/dns-exfiltratio n-with-base64-encoding-a-stealthy-data-t heft-technique/. ↗
  3. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335, Bang…
  4. Pengzhou Cheng, Yidong Ding, Tianjie Ju, Zongru Wu, Wei Du, Ping Yi, Zhuosheng Zhang, and Gongshen Liu. 2024. Trojanrag: Retrieval-augmented generation can be backdoor driver in large language models. Preprint, arXiv:2405.13401. — [arXiv:2405.13401](https://arxiv.org/abs/2405.13401) find
  5. Cohere. 2024. Introducing rerank 3.5: Precise AI search. https://cohere.com/blog/rerank-3pt5. ↗
  6. Computerworld. 2020. Google to trial drastically truncated URLs in Chrome in anti-phishing move. https://www.computerworld.com/article/163517 8/google-to-trial-drastically-truncated-u rls-in-chrome-in-anti-phishing-move.html. ↗
  7. Tian Dong, Minhui Xue, Guoxing Chen, Rayne Holland, Yan Meng, Shaofeng Li, Zhen Liu, and Haojin Zhu. 2023. The philosopher’s stone: Trojaning plugins of large language models. Proceedings 2025 Network and Distributed System Security Symposium. find
  8. Mohamed Amine Ferrag, Norbert Tihanyi, Djallel Hamouda, Leandros Maglaras, Abderrahmane Lakas, and Merouane Debbah. 2025. From prompt injections to protocol exploits: Threats in llm-powered ai agents workflows. ICT Express. find
  9. Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. Gptq: Accurate post-training quantization for generative pre-trained transformers. Preprint, arXiv:2210.17323. find
  10. Yuyou Gan, Yong Yang, Zhe Ma, Ping He, Rui Zeng, Yiming Wang, Qingming Li, Chunyi Zhou, Songze Li, Ting Wang, Yunjun Gao, Yingcai Wu, and Shouling Ji. 2024. Navigating the risks: A survey of security, privacy, and ethics threats in llm-based agents. Preprint, arXiv:2411.09523. — [arXiv:2411.09523](https://arxiv.org/abs/2411.09523) find
  11. Shubh Goyal, Medha Hira, Shubham Mishra, Sukriti Goyal, Arnav Goel, Niharika Dadu, Kirushikesh DB, Sameep Mehta, and Nishtha Madaan. 2024. Llmguard: Guarding against unsafe llm behavior. Proceedings of the AAAI Conference on Artificial Intelligence, 38(21):23790–23792. find
  12. Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security,…
  13. Danny Halawi, Alexander Wei, Eric Wallace, Tony Wang, Nika Haghtalab, and Jacob Steinhardt. 2024. Covert malicious finetuning: challenges in safeguarding llm adaptation. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. find
  14. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTav3: Improving deBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations.
  15. Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, and Yang Zhang. 2024. Composite backdoor attacks against large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1459–1472, Mexico City, Mexico. Association for Computational Linguistics. find
  16. Jina AI. 2024. Jina reranker v2: Multilingual cross-encoder for text reranking. https://jina.ai/mode ls/jina-reranker-v2-base-multilingual/. ↗
  17. Xianming Li, Aamir Shakir, Rui Huang, Julius Lipp, and Jing Li. 2025a. Prorank: Prompt warmup via reinforcement learning for small language models reranking. Preprint, arXiv:2506.03487. find
  18. Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. 2025b. Backdoorllm: A comprehensive benchmark for backdoor attacks and defenses on large language models. Preprint, arXiv:2408.12798. — [arXiv:2408.12798](https://arxiv.org/abs/2408.12798) find
  19. Yige Li, Zhe Li, Wei Zhao, Nay Myat Min, Hanxun Huang, Xingjun Ma, and Jun Sun. 2025c. Autobackdoor: Automating backdoor attacks via llm agents. Preprint, arXiv:2511.16709. — [arXiv:2511.16709](https://arxiv.org/abs/2511.16709) find
  20. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. In Proceedings of Machine Learning and Systems, volume 6, pages 87–100. find
  21. Mistral AI and NVIDIA. 2024. Mistral NeMo. https://mistral.ai/news/mistral-nemo. ↗
  22. Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. Document ranking with a pre-trained sequence-to-sequence model. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 708–718, Online. Association for Computational Linguistics. find
  23. OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, and 108 others. 2025. gpt-oss-120b & gpt-oss-20b model…
  24. OpenAI. 2025a. Models \| openai API. https://plat form.openai.com/docs/models. find
  25. OpenAI. 2025b. openai/gpt-oss-20b. https://huggin gface.co/openai/gpt-oss-20b. find
  26. OWASP Foundation. 2025. Owasp top 10 for large language model applications. https://owasp.org/ www-project-top-10-for-large-language-m odel-applications/. ↗
  27. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2024. Gorilla: Large language model connected with massive apis. In Advances in Neural Information Processing Systems, volume 37, pages 126544–126565. Curran Associates, Inc.
  28. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. Toolllm: Facilitating large language models to master 16000+ real…
  29. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  30. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. Direct preference optimization: your language model is secretly a reward model. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Ho…
  31. Traian Rebedea, Razvan Dinu, Makesh Narsimhan Sreedhar, Christopher Parisien, and Jonathan Cohen. 2023. NeMo guardrails: A toolkit for controllable and safe LLM applications with programmable rails. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Dem… find
  32. Red Hat. 2025. Model context protocol (MCP): Understanding security risks and controls. https://www.redhat.com/en/blog/model-context -protocol-mcp-understanding-security-ris ks-and-controls. ↗
  33. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  34. Timo Schick, Jane Dwivedi-Yu, Roberto Dessí, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: language models can teach themselves to use tools. In Proceedings of the 37th International Conference on Neural Information Processing S…
  35. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpaca: A strong, replicable instruction-following model. https://gi thub.com/tatsu-lab/stanford_alpaca. find
  36. Bo Wang, Weiyi He, Shenglai Zeng, Zhen Xiang, Yue Xing, Jiliang Tang, and Pengfei He. 2025a. Unveiling privacy risks in LLM agent memory. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25241–25260, Vienna, Austria. Associatio…
  37. Feng Wang, Yuqing Li, and Han Xiao. 2025b. jina-reranker-v3: Last but not late interaction for listwise document reranking. Preprint, arXiv:2509.25085. — [arXiv:2509.25085](https://arxiv.org/abs/2509.25085) find
  38. Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. 2024. BadAgent: Inserting and activating backdoor attacks in LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9811–9827, Bangkok, Thailand. Association fo… find
  39. Linguistics (Volume 1: Long Papers), pages 2181– 2211, Bangkok, Thailand. Association for Computational Linguistics. find
  40. Shuning Zhang, Lyumanshan Ye, Xin Yi, Jingyu Tang, Bo Shui, Haobin Xing, Pengfei Liu, and Hewu Li. 2024. "ghost of the past": identifying and resolving privacy leakage from llm’s memory through proactive user interaction. Preprint, arXiv:2410.14931. — [arXiv:2410.14931](https://arxiv.org/abs/2410.14931) find
  41. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. Preprint, arXiv:2506.05176.
  42. Shuai Zhao, Meihuizi Jia, Anh Tuan Luu, Fengjun Pan, and Jinming Wen. 2024. Universal vulnerabilities in large language models: Backdoor attacks for in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11507–11522, Miami, Florida, USA.… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-your-llm-agent-can-leak/