Elamur Research⌕
☰
home / papers / 2026 / From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning

From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning

Paper · 2026 · Xiaoyu Xu, Minxin Du, Zitong LI et al.

Summary

Evaluating machine unlearning in LLMs is unreliable when forget sets do not match what the model actually knows. Most benchmarks rely on verbatim training samples or templated pairs, and they miss paraphrases and other semantically equivalent content. The authors define two kinds of unlearning requests. Domain-level requests cover a broad concept, such as the Harry Potter universe. Instance-level requests cover specific statements, such as author facts in TOFU.

They introduce BiForget, which builds forget sets by querying the target model itself rather than an external, stronger generator. For domains, the model first lists seed points, then produces QA-style content around them. Jailbreaking and membership-inference prompts then pull out deeply memorized material. For instances, the model rephrases the target statements. Generation stops once SimCSE-based diversity stops improving.

On Harry Potter, WMDP and TOFU, BiForget's sets are more relevant and diverse, yet smaller. They also lead to stronger forgetting with comparable utility, which makes them a more rigorous basis for testing unlearning methods.

Key points

  • On Harry Potter, BiForget improves relevance by about 20 and diversity by about 0.05 while using roughly half the data of the official and textbook-style (Zhu et al., 2025) sets.
  • Data quality is measured on three axes:
    • relevance, as centroid distance to the domain;
    • diversity, with the remote-clique metric;
    • efficiency, as the number of 128-token chunks.
  • For WMDP biosecurity, BiForget tops both relevance (19.86) and diversity (0.3631) while using 9,196 chunks. On cybersecurity, it has the top diversity (0.3240) but worse relevance than the official set (49.37 vs 9.00), which the authors link to the model's weaker knowledge in that domain.
  • On TOFU forget01, OBLIVIATE trained with BiForget data reaches F.Q. 0.92 (vs 0.08 with the official set) at M.U. 0.65.
  • Ablation (GA, Harry Potter) shows that removing both jailbreaking and MI probing worsens PrivLeak from −15.08 to −24.46. GPT-5 is used only once, offline, to draft the prompt templates. Code is available at github.com/XiaoyuXU1/Biforget.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Huawei TechnologiesShenzhen · China96 works · 15,432 citations🎓Hong Kong Polytechnic UniversityHong Kong · China106 works · 5,190 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Xiaoyu Xu, Minxin Du, Zitong LI, Zi Liang, Zhibiao Guo, Zhang Shiyu, Peizhao Hu, Qingqing Ye and Haibo Hu. 2026. From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1424

@inproceedings{xu2026domains,
  title = {{From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning}},
  author = {Xiaoyu Xu, Minxin Du, Zitong LI, Zi Liang, Zhibiao Guo, Zhang Shiyu, Peizhao Hu, Qingqing Ye and Haibo Hu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1424},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1424}
}

References

41 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In S&P, pages 141–159. find
  2. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv:2110.14168.
  3. Vineeth Dorna, Anmol Mekala, Wenlong Zhao, Andrew McCallum, Zachary C. Lipton, J. Zico Kolter, and Pratyush Maini. 2025. Openunlearning: Accelerating LLM unlearning via unified benchmarking of methods and metrics. arXiv:2506.12618. — [arXiv:2506.12618](https://arxiv.org/abs/2506.12618) find
  4. Jianfeng Gao, Lichao Sun, and Xiangliang Zhang. 2025. Datagen: Unified synthetic dataset generation via large language models. In ICLR. find
  5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez,… — The Llama 3 Herd of Models
  6. Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In ICLR.
  7. Rohit Gandikota, Sheridan Feucht, Samuel Marks, and David Bau. 2024. Erasing conceptual knowledge from language models. arXiv:2410.02760. — [arXiv:2410.02760](https://arxiv.org/abs/2410.02760) find
  8. Chongyang Gao, Lixu Wang, Kaize Ding, Chenkai Weng, Xiao Wang, and Qi Zhu. 2025. On large language model continual unlearning. In ICLR. find
  9. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. In EMNLP, pages 6894–6910.
  10. Antonio Ginart, Melody Y. Guan, Gregory Valiant, and James Zou. 2019. Making AI forget you: Data deletion in machine learning. In NeurIPS, pages 3513– 3526. find
  11. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In ICLR.
  12. Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2019. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv:1904.05342.
  13. Yue Huang, Siyuan Wu, Chujie Gao, Dongping Chen, Qihui Zhang, Yao Wan, Tianyi Zhou, Chaowei Xiao, Shadi Iskander, Sofia Tolmach, Ori Shapira, Nachshon Cohen, and Zohar Karnin. 2024. Quality matters: Evaluating synthetic data for tool-using llms. In EMNLP, pages 4958–4976. find
  14. Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2023. Knowledge unlearning for mitigating privacy risks in language models. In ACL, pages 14389–14408.
  15. Feiyang Kang, Newsha Ardalani, Michael Kuchnik, Youssef Emad, Mostafa Elhoushi, Shubhabrata Sengupta, Shang-Wen Li, Ramya Raghavendra, Ruoxi Jia, and Carole-Jean Wu. 2025. Demystifying synthetic data in llm pre-training: A systematic study of scaling laws, benefits, and pitfalls. In EMNLP. find
  16. Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. 2023. Copyright violations and large language models. In EMNLP, pages 7403–7412.
  17. Jiaqi Li, Qianshan Wei, Chuanyi Zhang, Guilin Qi, Miaozeng Du, Yongrui Chen, Sheng Bi, and Fan Liu. 2024a. Single image unlearning: Efficient machine unlearning in multimodal large language models. In NeurIPS. find
  18. Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bh… — The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
  19. Zexi Li, Xiangzhu Wang, William F. Shen, Meghdad Kurmanji, Xinchi Qiu, Dongqi Cai, Chao Wu, and Nicholas D. Lane. 2025. Editing as unlearning: Are knowledge editing methods strong baselines for large language model unlearning? arXiv:2505.19855. — [arXiv:2505.19855](https://arxiv.org/abs/2505.19855) find
  20. Bo Liu, Qiang Liu, and Peter Stone. 2022. Continual learning and private unlearning. In CoLLAs, pages 243–254. PMLR. find
  21. Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. 2024. Large language model unlearning via embedding-corrupted prompts. In NeurIPS. find
  22. Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. 2025. Rethinking machine unlearning for large language models. Nat. Mac. Intell., 7(2):181–194. find
  23. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. 2023. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv:2305.13860.
  24. Jakub Lucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, and Javier Rando. 2025. An adversarial perspective on machine unlearning for AI safety. find
  25. Trans. Mach. Learn. Res., 2025. find
  26. Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter. 2024. TOFU: A task of fictitious unlearning for llms. In COLM.
  27. Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. 2024. In-context unlearning: Language models as few-shot unlearners. In ICML.
  28. Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2024. Detecting pretraining data from large language models. In ICLR. find
  29. Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. 2025. MUSE: machine unlearning six-way evaluation for language models. In ICLR. find
  30. Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, Andy Zou, Dawn Song, Bo Li, Dan Hendrycks, and Mantas Mazeika. 2025. Tamper-resistant safeguards for open-weight llms. In ICLR. find
  31. Xiaoyu Xu, Xiang Yue, Yang Liu, Qingqing Ye, Huadi Zheng, Peizhao Hu, Minxin Du, and Haibo Hu. 2025b. Unlearning isn’t deletion: Investigating reversibility of machine unlearning in llms. arXiv:2505.16831. — [arXiv:2505.16831](https://arxiv.org/abs/2505.16831) find
  32. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei… find
  33. Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. 2024. Machine unlearning of pre-trained large language models. In ACL, pages 8403–8419.
  34. Xiaojian Yuan, Tianyu Pang, Chao Du, Kejiang Chen, Weiming Zhang, and Min Lin. 2025. A closer look at machine unlearning for large language models. In ICLR. find
  35. Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv:2404.05868.
  36. Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, and Willie Neiswanger. 2025. LLM unlearning without an expert curated dataset. In COLM. find
  37. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models. In SIGIR, pages 1097–1100.
  38. Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Zhiwei Steven Wu, and Virginia Smith. 2025. Position: LLM unlearning benchmarks are weak measures of progress. In SaTML, pages 520–533. find
  39. Abudukelimu Wuerkaixi, Qizhou Wang, Sen Cui, Wutong Xu, Bo Han, Gang Niu, Masashi Sugiyama, and Changshui Zhang. 2025. Adaptive localization of knowledge negation for continual llm unlearning. In ICML. find
  40. Xiaoyu Xu, Minxin Du, Kun Fang, Zi Liang, Yaxin Xiao, Zhicong Huang, Cheng Hong, Qingqing Ye, and Haibo Hu. 2026. FIT: Defying catastrophic forgetting in continual LLM unlearning. arXiv:2601.21682. — [arXiv:2601.21682](https://arxiv.org/abs/2601.21682) find
  41. Xiaoyu Xu, Minxin Du, Qingqing Ye, and Haibo Hu. 2025a. Obliviate: Robust and practical machine unlearning for large language models. In EMNLP. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2026-domains-instances-dual-granularity-data/