Elamur Research⌕
☰
home / papers / 2026 / Policy Compliance of User Requests in Natural Language for AI Systems

Policy Compliance of User Requests in Natural Language for AI Systems

Paper · 2026 · Pedro Cisneros-Velarde

Summary

Organizations that let users send natural-language requests to an LLM-powered AI system, such as an agent that calls tools or retrieves data, need to block requests that are risky. Examples include leaking trade secrets, exposing private data, prompt injection, or revealing guardrails. The paper presents what it describes as the first public benchmark for checking whether such requests follow organizational policies. The benchmark targets the technology sector. It contains nine non-overlapping prohibitive policies and 225 human-written requests annotated for compliance. Compliant requests are deliberately hard: some reuse policy keywords, and others resemble policies in meaning. The author tests six open-source LLMs as backbones for several inference-time prompting strategies that work without fine-tuning. The emphasis is on small models that an organization could host locally. Scores are modest across the board, which shows the task is difficult. Small models perform close to a 120B model. Simple prompting variants match or beat more elaborate multi-step methods. The study offers practical guidance on choosing a method.

Key points

  • Benchmark design:
    • Nine policies cover four topics: internal data privacy, prompt injection, guardrail exposure, and access to protected destinations.
    • Each policy has 25 requests, for 225 in total (52.89% compliant).
    • Each non-compliant request violates exactly one policy.
    • 18 extra requests are provided for method development.
    • The data is on GitHub (PedroCV/policy-compliance-requests-AI).
  • Models tested: gpt-oss-120B, Llama 3.3 70B, Llama 3.1 8B, Mistral 7B, Gemma 3 4B, and Gemma 3 1B. All runs used temperature 0.
  • Accuracy and F1 results:
    • Best joint accuracy is 55.11% (gpt-oss-120B), followed closely by Gemma 3 1B and Llama 3.1 8B at 53.33%.
    • Best average F1 is 0.3994 (gpt-oss-120B), with Llama 3.1 8B second at 0.3640.
  • Simpler methods win:
    • "S.P. with Both" raises accuracy by 18.67–40.44% over Single Prompting for most models of 8B or fewer parameters.
    • It also lifts F1 by 0.2325 for the 8B model and by 0.1378 for the 1B model.
    • Convincing Decision, Two Arguments, and their variants never give the best F1.
  • Uneven performance by policy: Per-policy F1 varies widely, and all models but one completely miss at least one policy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢VMwarePalo Alto · United States2 works

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPolitical Science and International Relations · 182 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Artificial intelligence · 49 works

Cite

Cite as Pedro Cisneros-Velarde. 2026. Policy Compliance of User Requests in Natural Language for AI Systems. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.21

@inproceedings{cisnerosvelarde2026policy,
  title = {{Policy Compliance of User Requests in Natural Language for AI Systems}},
  author = {Pedro Cisneros-Velarde},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.21},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.21}
}

References

25 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Deepak Bhaskar Acharya, Karthigeyan Kuppan, and B. Divya. 2025. Agentic ai: Autonomous intelligence for complex goals—a comprehensive survey. IEEE Access, 13:18912–18936. find
  2. Luca Beurer-Kellner, Beat Buesser, Ana-Maria Crectu, Edoardo Debenedetti, Daniel Dobos, Daniel Fabian, Marc Fischer, David Froelicher, Kathrin Grosse, Daniel Naeff, Ezinwanne Ozoani, Andrew Paverd, Florian Tramèr, and Václav Volhejn. 2025. Design patterns for securing llm agents against prompt injec… — [arXiv:2506.08837](https://arxiv.org/abs/2506.08837) find
  3. Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramèr, Hamed Hassani, and Eric Wong. 2024. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. In…
  4. Yurun Chen, Xavier Hu, Yuhan Liu, Keting Yin, Juncheng Li, Zhuosheng Zhang, and Shengyu Zhang. 2025a. Harmonyguard: Toward safety and utility in web agents via adaptive policy enhancement and dual-objective optimization. Preprint, arXiv:2508.04010. — [arXiv:2508.04010](https://arxiv.org/abs/2508.04010) find
  5. Zhaorun Chen, Mintong Kang, and Bo Li. 2025b. Shieldagent: Shielding agents via verifiable safety policy reasoning. In Forty-second International Conference on Machine Learning. find
  6. Prafulla Kumar Choubey, Xiangyu Peng, Shilpa Bhagavath, Kung-Hsiang Huang, Caiming Xiong, and Chien-Sheng Wu. 2025. Benchmarking deep search over heterogeneous enterprise data. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 501–517, S… find
  7. Pedro Cisneros-Velarde. 2025. Using humor to bypass safety guardrails in large language models. In Proceedings of the The First Workshop on LLM Security (LLMSEC), pages 17–25, Vienna, Austria. Association for Computational Linguistics. find
  8. Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’24, page 6491–650…
  9. Gemma Team. 2025. Gemma 3. Accessed: 01-15-2026. find
  10. Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: a survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI…
  11. Joseph Marvin Imperial and Harish Tayyar Madabushi. 2025. Scaling policy compliance assessment in language models with policy reasoning traces. Preprint, arXiv:2509.23291. — [arXiv:2509.23291](https://arxiv.org/abs/2509.23291) find
  12. Jiwon Jeong, Hyeju Jang, and Hogun Park. 2025. Large language models are better logical fallacy reasoners with counterargument, explanation, and goal-aware prompt formulation. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 6918–6937, Albuquerque, New Mexico. Associat… find
  13. Haoran Li, Wenbin Hu, Huihao Jing, Yulin Chen, Qi Hu, Sirui Han, Tianshu Chu, Peizhao Hu, and Yangqiu Song. 2025. PrivaCI-bench: Evaluating privacy with contextual integrity and legal compliance. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo…
  14. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang. 2024a. A hitchhiker’s guide to jailbreaking chatgpt via prompt engineering. In Proceedings of the 4th International Workshop on Software Engineering and AI for Data Quality in Cyber-Phys… find
  15. Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024b. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1831–1847, Philadelphia, PA. USENIX Association. find
  16. Llama 3.1. 2024. Llama 3.1 \| Model Cards and Prompt formats. Accessed: 10-29-2025. find
  17. Llama 3.3. 2024. Llama 3.3 \| Model Cards and Prompt formats. Accessed: 10-29-2025. find
  18. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations.
  19. Mistral 7B. 2024. mistralai/Mistral-7B-Instruct-v0.3 · Hugging Face. Accessed: 01-15-2026. find
  20. OpenAI:, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, Che Chang, and 107 others. 2025. gpt-oss-120b & gpt-oss-…
  21. Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. find
  22. Mansi Phute, Alec Helbling, Matthew Daniel Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. 2024. LLM self defense: By self examination, LLMs know they are being tricked. In The Second Tiny Papers Track at ICLR 2024. find
  23. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on Learning Representations.
  24. David Rodriguez, William Seymour, Jose M. Del Alamo, and Jose Such. 2025. Towards safer chatbots: Automated policy compliance evaluation of custom gpts. Preprint, arXiv:2502.01436. — [arXiv:2502.01436](https://arxiv.org/abs/2502.01436) find
  25. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. Preprint, arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/velarde-2026-policy-compliance-user-requests-natural/