Elamur Research⌕
☰
home / papers / 2026 / Rejection-to-Acceptance Transition: Model Editing-Based Jailbreak Backdoor Injection Not Limited to Few Output Tokens

Rejection-to-Acceptance Transition: Model Editing-Based Jailbreak Backdoor Injection Not Limited to Few Output Tokens

Paper · 2026 · Shiji Yang, Min Cai, Hao Xiong et al.

Summary

The paper examines jailbreak backdoors planted in safety-aligned LLMs through lightweight model editing. Earlier editing attacks, such as JailbreakEdit, tie a trigger to compliant opening phrases like "Sure,". The model can still refuse later in its answer, and adding more bound phrases raises both run time and cost. The authors propose JEST, which instead edits the model's internal representations. They build a small contrastive set of harmful and benign prompt pairs. The harmful prompts are filtered with Llama-Guard-3-1B, and GLM-4-flash writes a benign counterpart for each. They then run PCA on differences in last-token, final-layer hidden states, which separates a "rejection domain" from an "acceptance domain." A locate-then-edit, closed-form update to one FFN layer makes the trigger move harmful-prompt representations into the acceptance region. A simple guided prompt engineering step (GPE) strengthens the attack further. Across several LLMs and datasets, JEST beats JailbreakEdit, and JEST+GPE usually reaches over 90% jailbreak success rate (JSR). The authors present this as evidence of how fragile LLM safety boundaries are and argue that stronger defenses are needed.

Key points

  • Victim models: Llama-2-7b-chat-hf, Vicuna-v1.5-7b, ChatGLM2-6b, Llama-3-8b-Instruct and Mistral-7b-Instruct. Evaluation data includes Advbench and DAN.
  • Ablation on Llama-2-7b-chat-hf (JSR):
    • GPE alone adds roughly 20 points over the no-attack setting.
    • JEST alone reaches up to 86.54%.
    • JEST+GPE reaches up to 93.08%.
  • Data needs: only a small amount of modeling data is required, and performance peaked at 70 samples. Five random draws of 70 samples gave JSR between 87.69% and 92.56% on DAN.
  • Projection space: using only the top few principal components gives the best results. Adding more components tends to hurt, especially on Llama-2 and Mistral.
  • Layer choice: editing early-to-mid layers yields the highest JSR, though overall sensitivity to the chosen layer is low.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🎓Anhui UniversityHefei · China11 works · 7 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Bias, fairness & safety · 587 works

Cite

Cite as Shiji Yang, Min Cai, Hao Xiong, Congyao Mei, Haodong Zou, Shicheng Tan, Jie Chen, Fulan Qian and Shu Zhao. 2026. Rejection-to-Acceptance Transition: Model Editing-Based Jailbreak Backdoor Injection Not Limited to Few Output Tokens. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1625

@inproceedings{yang2026rejection,
  title = {{Rejection-to-Acceptance Transition: Model Editing-Based Jailbreak Backdoor Injection Not Limited to Few Output Tokens}},
  author = {Shiji Yang, Min Cai, Hao Xiong, Congyao Mei, Haodong Zou, Shicheng Tan, Jie Chen, Fulan Qian and Shu Zhao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1625},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1625}
}

References

35 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Our work is supported by the National Natural Science Foundation of China (62476003), Anhui Province Excellent Scientific Research and Innovation Team (2024AH010004), Anhui Provincial Natural Science Foundation - Water Science Joint Fund (2408055US006), the University Synergy Innovation Program of A… find
  2. Hoyeon Chang, Jinho Park, Seonghyeon Ye, Sohee Yang, Youngkyung Seo, Du-Seong Chang, and Minjoon Seo. 2024. How do large language models acquire factual knowledge during pretraining? In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2… find
  3. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2025. Jailbreaking black box large language models in twenty queries. In IEEE Conference on Secure and Trustworthy Machine Learning, SaTML 2025, Copenhagen, Denmark, April 9-11, 2025, pages 23– 42. IEEE.
  4. Zhuowei Chen, Qiannan Zhang, and Shichao Pei. 2025. Injecting universal jailbreak backdoors into llms in minutes. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. find
  5. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 8493– 8502.… find
  6. Chengyuan Deng, Yiqun Duan, Xin Jin, Heng Chang, Yijun Tian, Han Liu, Henry Peng Zou, Yiqiao Jin, Yijia Xiao, Yichen Wang, Shenghao Wu, Zongxing Xie, Kuofeng Gao, Sihong He, Jun Zhuang, Lu Cheng, and Haohan Wang. 2024. Deconstructing the ethics of large language models from long-standing issues to n… — [arXiv:2406.05392](https://arxiv.org/abs/2406.05392) find
  7. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El Showk, Stanislav Fort, and 17 others. 2022. Red teaming…
  8. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 548… find
  9. Keltin Grimes, Marco Christiani, David Shriver, and Marissa Catherine Connor. 2025. Concept-rot: Poisoning concepts in large language models with model editing. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. find
  10. Shangying Hua, Shuangci Jin, and Shengyi Jiang. 2024. The limitations and ethical considerations of chatgpt. DATA INTELLIGENCE, 6(1):201–239. find
  11. Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jindong Gu, Yang Liu, Xiaochun Cao, and Min Lin. 2025. Improved techniques for optimization-based jailbreaking on large language models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. Op… find
  12. Houcheng Jiang, Zetong Zhao, Junfeng Fang, Haokai Ma, Ruipeng Wang, Yang Deng, Xiang Wang, and Xiangnan He. 2025. Mitigating safety fallback in editing-based backdoor injection on llms. CoRR, abs/2506.13285. — [arXiv:2506.13285](https://arxiv.org/abs/2506.13285) find
  13. Amin Karami, Milad Shemshaki, and Mustansar Ali Ghazanfar. 2025. Exploring the ethical implications of ai-powered personalization in digital marketing. DATA INTELLIGENCE, 7(4):1035–1084. find
  14. Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. 2024. Badedit: Backdooring large language models by model editing. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open-Review.n… find
  15. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
  16. Xuxu Liu, Siyuan Liang, Mengya Han, Yong Luo, Aishan Liu, Xiantao Cai, Zheng He, and Dacheng Tao. 2025. Elba-bench: An efficient learning backdoor attacks benchmark for large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P… find
  17. Kai Mei, Zheng Li, Zhenting Wang, Yang Zhang, and Shiqing Ma. 2023. NOTABLE: transferable backdoor attacks against prompt-based NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023,… find
  18. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9,… find
  19. Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. find
  20. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2022. Fast model editing at scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. find
  21. Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. 2025. Safety alignment should be made more than just a few tokens deep. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025…
  22. Javier Rando and Florian Tramèr. 2024. Universal jailbreak backdoors from poisoned human feedback. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. find
  23. Shaina Raza, Shardul Ghuge, Chen Ding, Elham Dolatabadi, and Deval Pandya. 2024. Fair enough: Develop and assess a fair-compliant dataset for large language model training? DATA INTELLIGENCE, 6(2):559–585. find
  24. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS 2024, Salt Lake City, UT…
  25. Jiawen Shi, Yixin Liu, Pan Zhou, and Lichao Sun. 2023. Badgpt: Exploring security vulnerabilities of chatgpt via backdoor attacks to instructgpt. CoRR, abs/2304.12298. — [arXiv:2304.12298](https://arxiv.org/abs/2304.12298) find
  26. Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric P. Xing, and 47 others. 2024. Trustllm: Trustworthiness in la…
  27. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  28. Chaoyi Wang and Yuan Sun. 2025. Mseed: Human preference dataset for multidimensional safety enhancement and evaluation of large language models. DATA INTELLIGENCE, 7(4):1243–1269. find
  29. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. 2023. Do-not-answer: A dataset for evaluating safeguards in llms. CoRR, abs/2308.13387.
  30. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does LLM safety training fail? In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
  31. Junxiao Yang, Zhexin Zhang, Shiyao Cui, Hongning Wang, and Minlie Huang. 2025a. Guiding not forcing: Enhancing the transferability of jailbreaking attacks on llms via removing superfluous constraints. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume… find
  32. Shiji Yang, Shu Zhao, Congyao Mei, Zhen Yang, Jie Chen, Fulan Qian, Zhen Duan, and Yan-ping Zhang. 2025b. Forget the unneeded: Backdooring large language models via contrastive-enhanced machine unlearning. In Findings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, Novem… find
  33. Hongwei Yao, Jian Lou, and Zhan Qin. 2024. Poi-sonprompt: Backdoor attack on prompt-based large language models. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024, Seoul, Republic of Korea, April 14-19, 2024, pages 7745–7749. IEEE. find
  34. Andy Zou, Long Phan, Sarah Li Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, and 2 others. 2023a. Repres…
  35. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023b. Universal and transferable adversarial attacks on aligned language models. CoRR, abs/2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-rejection-acceptance-transition-model-editing/