Elamur Research⌕
☰
home / papers / 2026 / Improving LLM Domain Certification with Pretrained Guide Models

Improving LLM Domain Certification with Pretrained Guide Models

Paper · 2026 · Jiaqian Zhang, Zhaozhi Qian, Faroq AL-Tam et al.

Summary

When LLMs are deployed in narrow, high-stakes settings such as medicine, they should answer only within a defined scope and decline everything else, even under adversarial prompting. The earlier VALID algorithm offers a formal domain certificate. It accepts a generalist model's response only if a length-normalized likelihood ratio against a small guide model G stays under a threshold. However, VALID trains G from scratch on scarce in-domain data, so G generalizes poorly. The authors propose PRISM, which starts G from a pretrained language model. G is then fine-tuned on both in-domain responses and easily collected out-of-domain responses. Four objectives are compared: Gradient Ascent (GA), NPO, DPO and Binary Classification (BC). Across eight data regimes and perturbations, PRISM-BC and PRISM-GA reject off-domain content more reliably and yield tighter certification bounds than VALID. PRISM-BC is also robust to plausible-sounding medical misinformation. This makes provable domain restriction more practical for real deployments.

Key points

  • On a perturbed set built from PubMedQA, the correct answers were inverted and LLaMA-3-8B-Instruct wrote convincing false explanations. PRISM-BC scored F1 0.799 and AUC 0.924, compared with 0.724/0.841 for untuned LLaMA and 0.637/0.801 for VALID(200k).
  • On certified PubMedQA benchmarking, the unconstrained model scores 72.5%. PRISM-BC keeps 0.724 accuracy at both ε = 10⁻⁵ and ε = 10⁻¹⁰, while VALID(200k) falls to 0.318 and 0.004.
  • Only PRISM-GA and PRISM-BC reach false positive rates under 20% with non-trivial bounds; VALID exceeds 50%. GA gives the tightest bounds when FPR is above 5%, and BC is the only usable variant below 5%.
  • The reference-policy variants (NPO, DPO) underperform. The authors suggest that π_ref blurs the in-domain/out-of-domain boundary.
  • The appendices extend the results to Shakespeare, code generation and a Qwen-0.6B guide. A remaining limitation is that certification judges the response without seeing the prompt.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University College LondonLondon · United Kingdom107 works · 22,923 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Jiaqian Zhang, Zhaozhi Qian, Faroq AL-Tam, Ignacio Iacobacci, Muhammad AL-Qurishi and Riad Souissi. 2026. Improving LLM Domain Certification with Pretrained Guide Models. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.69

@inproceedings{zhang2026improving,
  title = {{Improving LLM Domain Certification with Pretrained Guide Models}},
  author = {Jiaqian Zhang, Zhaozhi Qian, Faroq AL-Tam, Ignacio Iacobacci, Muhammad AL-Qurishi and Riad Souissi},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.69},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.69}
}

References

33 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, and 3 others. 2021. A general… find
  2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, and 12 others. 202… find
  3. Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Security and Privacy, pages 463–480. find
  4. Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramèr. 2024. Poisoning web-scale training datasets is practical. arXiv preprint arXiv:2302.10149. — [arXiv:2302.10149](https://arxiv.org/abs/2302.10149) find
  5. Marco Casadio, Tanvi Dinkar, Ekaterina Komen-dantskaya, Luca Arnaboldi, Matthew L. Daggitt, Omri Isac, Guy Katz, Verena Rieser, and Oliver Lemon. 2024. Nlp verification: Towards a general methodology for certifying robustness. arXiv preprint arXiv:2403.10144. — [arXiv:2403.10144](https://arxiv.org/abs/2403.10144) find
  6. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. find
  7. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2024. Safe rlhf: Safe reinforcement learning from human feedback. In Proceedings of the Twelfth International Conference on Learning Representations.
  8. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36:10088–10115.
  9. Xinshuai Dong, Anh Tuan Luu, Min Lin, Shuicheng Yan, and Hanwang Zhang. 2021. How should pre-trained language models be fine-tuned towards adversarial robustness? In Advances in Neural Information Processing Systems, volume 34, pages 4356– 4369. Curran Associates, Inc. find
  10. Cornelius Emde, Alasdair Paren, Preetham Arvind, Maxime Kayser, Tom Rainforth, Thomas Lukasiewicz, Bernard Ghanem, Philip HS Torr, and Adel Bibi. 2025. Shh, don’t say that! domain certification in llms. In Proceedings of the Thirteenth International Conference on Learning Representations. find
  11. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. 2023. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674.
  12. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce…
  13. Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. 2024. A survey of reinforcement learning from human feedback. arXiv preprint arXiv:2312.14925.
  14. Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Aaron Jiaxun Li, Soheil Feizi, and Himabindu Lakkaraju. 2024. Certifying llm safety against adversarial prompting. arXiv preprint arXiv:2309.02705.
  15. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In Proceedings of the Twelfth International Conference on Learning Representations.
  16. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. find
  17. Meta AI. 2024. The llama 3.1 family of models. https://ai.meta.com/blog/meta-llama-3-1/. Accessed: 2025-08-28. ↗
  18. Quoc Phong Nguyen, Ryutaro Oikawa, Dinil Mon Divakaran, Mun Choon Chan, and Bryan Kian Hsiang Low. 2022. Markov chain monte carlo-based machine unlearning: Unlearning what needs to be forgotten. In Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, ASIA CCS ’22,… find
  19. Thanh Tam Nguyen, Thanh Trung Huynh, Zhao Ren, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. 2024. A survey of machine unlearning. Preprint, arXiv:2209.02299. — [arXiv:2209.02299](https://arxiv.org/abs/2209.02299) find
  20. OpenAI. 2023. Chatgpt. https://chat.openai.com. Accessed: 2025-08-05. ↗
  21. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Tra… — Training language models to follow instructions with human feedback
  22. Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527.
  23. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  24. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
  25. RyokoAI. 2023. ShareGPT52K dataset. https://huggingface.co/datasets/RyokoAI/ ShareGPT52K. ↗
  26. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing nlp. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce… find
  27. Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. 2023. Machine unlearning: A survey. ACM Computing Surveys, 56(1). find
  28. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. 2024a. Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing. Preprint, arXiv:2406.08464.
  29. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. 2024b. Magpie-Llama-3.1-Pro-300K-Filtered dataset. https://huggingface.co/ datasets/Magpie-Align/Magpie-Llama-3. 1-Pro-300K-Filtered. ↗
  30. W. J. Youden. 1950. Index for rating diagnostic tests. Cancer, 3(1):32–35. find
  31. Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024. Negative preference optimization: From catastrophic collapse to effective unlearning. Preprint, arXiv:2404.05868.
  32. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations.
  33. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric. P Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. 2023. Lmsys-chat-1m: A large-scale real-world llm conversation dataset. Preprint, arXiv:2309.11998.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-improving-llm-domain-certification-pretrained/