Elamur Research⌕
☰
home / papers / 2026 / Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs

Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs

Paper · 2026 · Yujia Zheng, Tianhao Li, Haotian Huang et al.

Summary

Most work on adversarial prompting treats a prompt as one undivided string. This paper argues instead that prompts are built from functional parts, and that these parts differ in how easily an attack can exploit them.

The authors propose two tools:

  • PromptAnatomy splits a prompt into canonical components. It first segments the prompt by sentence, then has an LLM classify each sentence using a small surrounding context window. The components follow the Schulhoff et al. taxonomy, with Style Instructions merged into Output Formatting.
  • ComPerturb attacks one component at a time, choosing perturbations that suit that component's function. It discards implausible edits with a perplexity filter.

They applied PromptAnatomy to four domain-specific instruction datasets, had human experts review the annotations, and tested five LLMs.

ComPerturb reaches state-of-the-art attack success rates, and it works best on prompts with many components. Directive and Additional Information are the weakest points, while Role and Output Formatting hold up comparatively well. Semantic perturbations beat syntactic ones.

This matters for red/blue teaming, for prompt engineering, and for designing defenses that account for prompt structure.

Key points

  • Released datasets: PubMedQA-PA, EMEA-PA, Leetcode-PA and CodeGeneration-PA, with 100% inter-annotator agreement reported. Code and data are at github.com/Yujiaaaaa/PACP.
  • Component taxonomy: Directive (DIR), Examples (EXA), Output Formatting (OFT), Role (ROL) and Additional Information (ADI). An unperturbed control group (CRT) gives the lowest ASR.
  • Most vulnerable component: perturbing DIR yields up to 87.9% ASR under the COD perturbation.
  • Ablation, averaged over five LLMs: PromptAnatomy alone adds +9.2 ASR points, ComPerturb alone adds +7.7, and both together add +15.4 (about 56.4% → 72.0%).
  • Suggested defenses: input preprocessing (spelling and whitespace normalization, removing special characters), prompt sanitization or rewriting, training on plausible semantic perturbations, and runtime monitoring.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Yujia Zheng, Tianhao Li, Haotian Huang, Tianyu Zeng, Jingyu Lu, Chuangxin Chu, Yuekai Huang, Ziyou Jiang, Qian Xiong, Yuyao Ge and Mingyang Li. 2026. Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.374

@inproceedings{zheng2026are,
  title = {{Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs}},
  author = {Yujia Zheng, Tianhao Li, Haotian Huang, Tianyu Zeng, Jingyu Lu, Chuangxin Chu, Yuekai Huang, Ziyou Jiang, Qian Xiong, Yuyao Ge and Mingyang Li},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.374},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.374}
}

References

38 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Francisco Aguilera-Martínez and Fernando Berzal. 2025. Llm security: Vulnerabilities, attacks, defenses, and countermeasures. arXiv preprint arXiv:2505.01177. — [arXiv:2505.01177](https://arxiv.org/abs/2505.01177) find
  2. Gabriel Alon and Michael Kamfonas. 2023. Detecting language model attacks with perplexity. arXiv preprint arXiv:2308.14132.
  3. Nilanjana Das, Edward Raff, and Manas Gaur. 2024. Human-interpretable adversarial prompt attack on large language models with situational context. arXiv preprint arXiv:2407.14644. — [arXiv:2407.14644](https://arxiv.org/abs/2407.14644) find
  4. Dorottya Demszky, Diyi Yang, David S Yeager, Christopher J Bryan, Margarett Clapper, Susannah Chandhok, Johannes C Eichstaedt, Cameron Hecht, Jeremy Jamieson, Meghann Johnson, and 1 others. 2023. Using large language models in psychology. Nature Reviews Psychology, 2(11):688–701. find
  5. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, and 1 others. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234.
  6. Shaona Ghosh, Heather Frase, Adina Williams, Sarah Luger, Paul Röttger, Fazl Barez, Sean McGregor, Kenneth Fricklas, Mala Kumar, Kurt Bollacker, and 1 others. 2025. Ailuminate: Introducing v1. 0 of the ai risk and reliability benchmark from mlcommons. arXiv preprint arXiv:2503.05731. find
  7. Tessa Han, Aounon Kumar, Chirag Agarwal, and Himabindu Lakkaraju. 2024. Medsafetybench: Evaluating and improving the medical safety of large language models. Advances in Neural Information Processing Systems, 37:33423–33454. find
  8. Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology, 33(8):1–79. find
  9. Jinwei Hu, Yi Dong, Zhengtao Ding, and Xiaowei Huang. 2025. Enhancing robustness of llm-driven multi-agent systems through randomized smoothing. arXiv preprint arXiv:2507.04105. — [arXiv:2507.04105](https://arxiv.org/abs/2507.04105) find
  10. Zhibo Hu, Chen Wang, Yanfeng Shu, Hye-Young Paik, and Liming Zhu. 2024. Prompt perturbation in retrieval-augmented generation based large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1119–1130. find
  11. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  12. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2022. Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406.
  13. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. arXiv preprint arXiv:1706.03872. find
  14. Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. 2024a. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models. arXiv preprint arXiv:2402.05044.
  15. Tianhao Li, Jingyu Lu, Chuangxin Chu, Tianyu Zeng, Yujia Zheng, Mei Li, Haotian Huang, Bin Wu, Zuoxian Liu, Kai Ma, and 1 others. 2024b. Scisafeeval: a comprehensive benchmark for safety alignment of large language models in scientific tasks. arXiv preprint arXiv:2410.03769. — [arXiv:2410.03769](https://arxiv.org/abs/2410.03769) find
  16. Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. 2024c. Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers. arXiv preprint arXiv:2402.16914. — [arXiv:2402.16914](https://arxiv.org/abs/2402.16914) find
  17. Yinheng Li. 2023. A practical survey on zero-shot prompt design for in-context learning. arXiv preprint arXiv:2309.13205. find
  18. Do Long, Yiran Zhao, Hannah Brown, Yuxi Xie, James Zhao, Nancy Chen, Kenji Kawaguchi, Michael Shieh, and Junxian He. 2024. Prompt optimization via adversarial in-context learning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page… find
  19. Ercong Nie, Shuzhou Yuan, Bolei Ma, Helmut Schmid, Michael Färber, Frauke Kreuter, and Hinrich Schütze. 2024. Decomposed prompting: Unveiling multilingual linguistic structure knowledge in english-centric large language models. arXiv preprint arXiv:2402.18397. find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  21. Shilin Qiu, Qihe Liu, Shijie Zhou, Min Gou, Yi Zeng, Zhun Zhang, and Zhewei Wu. 2025. Hard label adversarial attack with high query efficiency against nlp models. Scientific Reports, 15(1):9378. find
  22. Sander Schulhoff, Michael Ilie, Nishant Balepur, Konstantine Kahadze, Amanda Liu, Chenglei Si, Yinheng Li, Aayush Gupta, HyoJung Han, Sevien Schulhoff, and 1 others. 2024. The prompt report: a systematic survey of prompt engineering techniques. arXiv preprint arXiv:2406.06608.
  23. Namrata Shivagunde, Vladislav Lialin, Sherin Muckatira, and Anna Rumshisky. 2024. Deconstructing in-context learning: Understanding prompts via corruption. arXiv preprint arXiv:2404.02054. find
  24. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, 29(8):1930– 1940. find
  25. Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. 2021. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. arXiv preprint arXiv:2111.02840. find
  26. Haoran Lu, Luyang Fang, Ruidong Zhang, Xinliang Li, Jiazhang Cai, Huimin Cheng, Lin Tang, Ziyu Liu, Zeliang Sun, Tao Wang, and 1 others. 2025. Alignment and safety in large language models: Safety mechanisms, training paradigms, and emerging challenges. arXiv preprint arXiv:2507.19672. — [arXiv:2507.19672](https://arxiv.org/abs/2507.19672) find
  27. Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang, Wei Ye, Xiubo Geng, and 1 others. 2023a. On the robustness of chatgpt: An adversarial and out-of-distribution perspective. arXiv preprint arXiv:2302.12095.
  28. Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, and 1 others. 2025. A survey of context engineering for large language models. arXiv preprint arXiv:2507.13334. — [arXiv:2507.13334](https://arxiv.org/abs/2507.13334) find
  29. Wenxuan Wang, Jen-tse Huang, Weibin Wu, Jianping Zhang, Yizhan Huang, Shuqing Li, Pinjia He, and Michael R Lyu. 2023b. Mttm: Metamorphic testing for textual content moderation software. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 2387–2399. IEEE. find
  30. Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2023. A comprehensive overview of large language models. ACM Transactions on Intelligent Systems and Technology.
  31. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, and 1 others. 2022. find
  32. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5085–5109. find
  33. Qian Xiong, Yuekai Huang, Ziyou Jiang, Zhiyuan Chang, Yujia Zheng, Tianhao Li, and Mingyang Li. 2025. Butterfly effects in toolchains: A comprehensive analysis of failed parameter filling in llm tool-agent systems. arXiv preprint arXiv:2507.15296. find
  34. Jiaqi Xue, Mengxin Zheng, Ting Hua, Yilin Shen, Yepeng Liu, Ladislau Bölöni, and Qian Lou. 2023. Trojllm: A black-box trojan prompt attack on large language models. Advances in Neural Information Processing Systems, 36:65665–65677. find
  35. Sheng Yin, Xianghe Pang, Yuanzhuo Ding, Menglan Chen, Yutong Bi, Yichen Xiong, Wenhao Huang, Zhen Xiang, Jing Shao, and Siheng Chen. 2024. Safeagentbench: A benchmark for safe task planning of embodied llm agents. arXiv preprint arXiv:2412.13178. — [arXiv:2412.13178](https://arxiv.org/abs/2412.13178) find
  36. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. arXiv e-prints, pages arXiv–2306. find
  37. Kaijie Zhu, Qinlin Zhao, Hao Chen, Jindong Wang, and Xing Xie. 2024. Promptbench: A unified library for evaluation of large language models. Journal of Machine Learning Research, 25(254):1–22. find
  38. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2026-are-all-prompt-components-value/