Elamur Research⌕
☰
home / papers / 2026 / CAP: Controllable Alignment Prompting for Unlearning in LLMs

CAP: Controllable Alignment Prompting for Unlearning in LLMs

Paper · 2026 · Zhaokun Wang, Jinyu Guo, Jingwen Pu et al.

Summary

LLMs trained on uncurated data can memorize sensitive or hazardous facts. Most unlearning techniques remove such knowledge by editing weights, which is costly, can damage general ability, and does not work for closed-source models. CAP takes a different route: the target LLM stays frozen, and a small language model (SLM) learns to write a short, query-specific prefix that steers the LLM away from the knowledge to be forgotten.

The SLM is trained with reinforcement learning, using a beam-search variant of PPO (B-PPO). For each query it proposes paired "forget" and "retain" prompts. The reward is based on an information-bottleneck objective, made tractable with variational bounds: it penalizes answers that reveal the forget-set label and rewards answers that keep retain-set knowledge.

Removing the generator restores the original behaviour, so the unlearning is fully reversible. The prompts also transfer across architectures, including closed models. Experiments on WMDP, MMLU and RWKU report better forgetting and retention than baselines, along with stronger robustness to adversarial probing. This makes CAP a practical tool for privacy compliance in black-box settings.

Key points

  • No parameters of the target LLM are updated. A lightweight SLM acts as the policy network and generates control prefixes.
  • Default hyperparameters are beam size k=4, n=3 prompt candidates and maximum prompt length L=16. With these settings, CAP reaches 24.8 F-Acc on WMDP-bio and 51.5 R-Acc on MMLU.
  • Ablations show clear gains from multi-path exploration. With k=1 (plain PPO), F-Acc is 27.8, and with n=1 it rises to 35.1.
  • CAP holds up against RWKU adversarial probes better than ICUL and weight-editing methods. These probes include prefix injection, role playing, synonym manipulation and reverse queries.
  • Hidden-state visualizations on LLaMA2-7B show that the prefix weakens activations on sensitive tokens. The stated limitations are some added latency and a small cost in context window.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Electronic Science and Technology of ChinaChengdu · China31 works · 2,012 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Zhaokun Wang, Jinyu Guo, Jingwen Pu, Hongli Pu, Meng Yang, Xunlei Chen, Jie Ou, Wenyi Li, Guangchun Luo and Wenhong Tian. 2026. CAP: Controllable Alignment Prompting for Unlearning in LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1882

@inproceedings{wang2026cap,
  title = {{CAP: Controllable Alignment Prompting for Unlearning in LLMs}},
  author = {Zhaokun Wang, Jinyu Guo, Jingwen Pu, Hongli Pu, Meng Yang, Xunlei Chen, Jie Ou, Wenyi Li, Guangchun Luo and Wenhong Tian},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1882},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1882}
}

References

57 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Arash Barfar and Lee Sommerfeldt. 2026. Propaganda by prompt: Tracing hidden linguistic strategies in large language models. Information Processing & Management, 63(2):104403. find
  3. Karuna Bhaila, Minh-Hao Van, and Xintao Wu. 2025. Soft prompting for unlearning in large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4046– 40…
  4. Sihan Cao, Jianwei Zhang, Pengcheng Zheng, Jiaxin Yan, Caiyan Qin, Yalan Ye, Wei Dong, Peng Wang, Yang Yang, and Chaoning Zhang. 2026. Language-guided token compression with reinforcement learning in large vision-language models. arXiv preprint arXiv:2603.13394. — [arXiv:2603.13394](https://arxiv.org/abs/2603.13394) find
  5. Chonghao Chen, Jianming Zheng, Wanyu Chen, Xin Zhang, Yupu Guo, Aimin Luo, and Fei Cai. 2025a. Cascading multi-scale graph pre-training and prompt tuning for learning-based community search. Information Processing & Management, 62(6):104285. find
  6. Mouxiang Chen, Han Fu, Chenghao Liu, Xiaoyun Joy Wang, Zhuo Li, and Jianling Sun. 2025b. Build a good human-free prompt tuning: Jointly pre-trained template and verbalizer for few-shot classification. IEEE Transactions on Knowledge and Data Engineering. find
  7. Xunlei Chen, Jinyu Guo, Yuang Li, Zhaokun Wang, Yi Gong, Jie Zou, Jiwei Wei, and Wenhong Tian. 2026. Alter: Asymmetric lora for token-entropy-guided unlearning of llms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 35366–35374. find
  8. Yubo Chen, Baoli Zhang, Sirui Li, Zhuoran Jin, Zhengyuan Cai, Yingzheng Wang, Delai Qiu, Sheng-Ping Liu, and Jun Zhao. 2025c. Prompt robust large language model for chinese medical named entity recognition. Information Processing & Management, 62(5):104189. find
  9. Zhenli Chen, Jie Hao, Haixia Sun, Liang Zhao, Jiao Li, Qing Qian, Qinglong Peng, Xuwen Wang, Shan Cong, Liu Shen, and 1 others. 2025d. Medscalere-pf: a prompt-based framework with retrieval-augmented generation, chain-of-thought, and self-verification for scale-specific relation extraction in chines… find
  10. Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. 2024. Chainlm: Empowering large language models with improved chain-of-thought prompting. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2969… find
  11. DeepSeek-AI. 2024. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. find
  12. XiaoHua Feng, Chaochao Chen, Yuyuan Li, and Zibin Lin. 2024. Fine-grained pluggable gradient ascent for knowledge unlearning in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10141– 10155. find
  13. Jack Foster, Stefan Schoepf, and Alexandra Brintrup. 2024. Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 12043–12051. find
  14. Jinyu Guo, Xunlei Chen, Qiyang Xia, Zhaokun Wang, Jie Ou, Libo Qin, Shunyu Yao, and Wenhong Tian. 2025. Hash-rag: bridging deep hashing with retriever for efficient, fine retrieval and augmented generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 26847–26858. find
  15. Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, and 1 others. 2024. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218. find
  16. Hyeonmin Ha, Jihye Lee, Wookje Han, and Byung-Gon Chun. 2023. Meta-learning of prompt generation for lightweight prompt engineering on language-model-as-a-service. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2433–2445. find
  17. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–… find
  18. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations.
  19. Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. 2023. Guiding large language models via directional stimulus prompting. Advances in Neural Information Processing Systems, 36:62630–62656. find
  20. Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  21. Chris Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. 2024a. Large language model unlearning via embedding-corrupted prompts. Advances in Neural Information Processing Systems, 37:118198–118266. find
  22. Xinshuo Hu, Dongfang Li, Baotian Hu, Zihao Zheng, Zhenyu Liu, and Min Zhang. 2024. Separate the wheat from the chaff: Model deficiency unlearning via parameter-efficient module operation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18252–18260. find
  23. Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61–6…
  24. Jinghan Jia, Jiancheng Liu, Yihua Zhang, Parikshit Ram, Nathalie Baracaldo, and Sijia Liu. 2024. Wagle: Strategic weight attribution for effective and modular unlearning in large language models. Advances in Neural Information Processing Systems, 37:55620– 55646. find
  25. Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. 2024b. Towards safer large language models through machine unlearning. In Findings of the Association for Computational Linguistics ACL 2024, pages 1817–1829.
  26. Zhuoran Jin, Pengfei Cao, Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. 2024. Rwku: Benchmarking real-world knowledge unlearning for large language models. Advances in Neural Information Processing Systems, 37:98213–98263. find
  27. Weize Kong, Spurthi Hombaiah, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024. Prewrite: Prompt rewriting with reinforcement learning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 594–601. find
  28. Minchan Kwon, Gaeun Kim, Jongsuk Kim, Haeil Lee, and Junmo Kim. 2024. Stableprompt: Automatic prompt tuning using reinforcement learning for large language model. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9868–9884. find
  29. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059.
  30. Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. 2024. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121.
  31. Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. 2024. In-context unlearning: language models as few-shot unlearners. In Proceedings of the 41st International Conference on Machine Learning, pages 40034–40050. find
  32. Protection Regulation. 2018. General data protection regulation. Intouch, 25:1–5. find
  33. Jeffrey Rosen. 2011. The right to be forgotten. Stan. L. Rev. Online, 64:88. find
  34. Gemma Team. 2025a. Gemma 3. find
  35. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  36. Qwen Team. 2025b. Qwen3 technical report. Preprint, arXiv:2505.09388.
  37. Pratiksha Thaker, Yash Maurya, Shengyuan Hu, Zhiwei Steven Wu, and Virginia Smith. 2024. Guardrail baselines for unlearning in llms. arXiv preprint arXiv:2403.03329. — [arXiv:2403.03329](https://arxiv.org/abs/2403.03329) find
  38. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  39. Jiaquan Zhang, Qigan Sun, Chaoning Zhang, Xudong Wang, Zhenzhen Huang, Yitian Zhou, Pengcheng Zheng, Chi-lok Andy Tai, Sung-Ho Bae, Zeyu Ma, and 1 others. 2026a. Tda-rc: Task-driven alignment for knowledge-based reasoning chains in large language models. arXiv preprint arXiv:2604.04942. — [arXiv:2604.04942](https://arxiv.org/abs/2604.04942) find
  40. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023. Zephyr: Direct distillation of lm alignment. Preprint, arXiv… find
  41. Jiaquan Zhang, Chaoning Zhang, Shuxu Chen, Zhenzhen Huang, Pengcheng Zheng, Zhicheng Wang, Ping Guo, Fan Mo, Sung-Ho Bae, Jie Zou, and 1 others. 2026b. Lightweight llm agent memory with small language models. arXiv preprint arXiv:2604.07798.
  42. Boshi Wang, Xiang Deng, and Huan Sun. 2022. Iteratively prompt pre-trained language models for chain of thought. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2714–2730. find
  43. Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023a. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo… find
  44. Qifan Wang, Yuning Mao, Jingang Wang, Hanchao Yu, Shaoliang Nie, Sinong Wang, Fuli Feng, Lifu Huang, Xiaojun Quan, Zenglin Xu, and 1 others. 2023b. Aprompt: Attention prompt tuning for efficient adaptation of pre-trained language models. In Proceedings of the 2023 conference on empirical methods in… find
  45. Yaxuan Wang, Quan Liu, Chris Yuhao Liu, Jinlong Pang, Wei Wei, Yujia Bao, and Yang Liu. 2025a. Dragon: Guard llm unlearning in context via negative detection and reasoning. In ICML 2025 Workshop on Machine Unlearning for Generative AI. find
  46. Zhaokun Wang, Jinyu Guo, Jingwen Pu, ChenLingFeng, Hongli Pu, Jie Ou, Libo Qin, and Wenhong Tian. 2025b. Noise-robustness through noise: A framework combining asymmetric lora with poisoning moe. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  47. Xiaoyu Xu, Minxin Du, Qingqing Ye, and Haibo Hu. 2025. Obliviate: Robust and practical machine unlearning for large language models. arXiv e-prints, pages arXiv–2505. find
  48. Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. 2024a. Machine unlearning of pre-trained large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8403–8419.
  49. Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2024b. Large language model unlearning. Advances in Neural Information Processing Systems, 37:105425–105475.
  50. Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024. Negative preference optimization: From catastrophic collapse to effective unlearning. In First Conference on Language Modeling.
  51. Yichi Zhang, Siyuan Zhang, Yao Huang, Zeyu Xia, Zhengwei Fang, Xiao Yang, Ranjie Duan, Dong Yan, Yinpeng Dong, and Jun Zhu. Stair: Improving safety alignment with introspective reasoning. In Forty-second International Conference on Machine Learning. find
  52. Kairan Zhao, Meghdad Kurmanji, George-Octavian Bărbulescu, Eleni Triantafillou, and Peter Triantafillou. 2024a. What makes unlearning hard and what to do about it. Advances in Neural Information Processing Systems, 37:12293–12333. find
  53. Yang Zhao, Li Du, Xiao Ding, Kai Xiong, Zhouhao Sun, Shi Jun, Ting Liu, and Bing Qin. 2024b. Deciphering the impact of pretraining data on large language models through machine unlearning. In Findings of the Association for Computational Linguistics ACL 2024, pages 9386–9406. find
  54. Pengcheng Zheng, Chaoning Zhang, Jiarong Mo, Guo-Hui Li, Jiaquan Zhang, Jiahao Zhang, Sihan Cao, Sheng Zheng, Caiyan Qin, Guoqing Wang, and 1 others. 2026. Llava-fa: Learning fourier approximation for compressing large multimodal models. arXiv preprint arXiv:2602.00135. — [arXiv:2602.00135](https://arxiv.org/abs/2602.00135) find
  55. Xin Zhou, Dingkang Liang, Wei Xu, Xingkui Zhu, Yihan Xu, Zhikang Zou, and Xiang Bai. 2024. Dynamic adapter meets prompt tuning: Parameter-efficient transfer learning for point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14707–14717. find
  56. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In The eleventh international conference on learning representations.
  57. Wei Zhu, Aaron Tian, Congrui Yin, Yuan Ni, Xiaoling Wang, and Guotong Xie. 2024. Iapt: Instance-aware prompt tuning for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14285–14304. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-cap-controllable-alignment-prompting-unlearning/