Elamur Research⌕
☰
home / papers / 2026 / Near-Miss: Latent Policy Failure Detection in Agentic Workflows

Near-Miss: Latent Policy Failure Detection in Agentic Workflows

Paper · 2026 · Ella Rabinovich, David Boaz, Naama Zwerdling et al.

Summary

LLM agents that automate business processes must follow policies before they change system state, for example checking eligibility before cancelling a booking. Benchmarks usually judge this only by comparing the final database state with a gold reference. That approach misses "near-misses," or latent failures: the agent skips a required check, but the outcome is still correct because the user happened to give accurate information. In adversarial settings, such skipped checks could be exploited. The authors build on ToolGuard, which compiles natural-language policies into executable Python guard functions for state-changing tools. They use it offline to inspect agent trajectories and test whether each mutating tool call was backed by the information the policy requires. They evaluate six open and proprietary LLM agents on the τ²-verified Airlines domain. Among trajectories with mutating tool calls, 8–17% contain latent failures even though the final state matches the gold one. The work exposes a blind spot in outcome-only evaluation and argues for metrics that also assess the agent's decision process.

Key points

  • Data and models: The benchmark is τ²-verified Airlines, with 200 simulations per model. The agents are GPT5-chat, Claude-Sonnet4 and Gemini-3-pro (closed), plus GPT-oss-120b, Qwen2.5-72b and Kimi-K2.5 (open). All guard code was generated with Claude-Sonnet4, the best-performing generator.
  • Near-miss rates: Among trajectories with mutating tool calls, the near-miss rate ranges from about 0.084 (Qwen2.5-72b) to 0.173 (GPT5-chat). Closed models show slightly more latent failures than open ones.
  • Explicit violations: By the authors' estimate, explicit policy violations make up roughly 25% of all simulation failures in the τ² Airlines domain.
  • Where failures occur: update_reservation_flights() is the mutating tool most often called without the required checks. get_flight_status() is the read-only check agents skip most often. No near-misses were found for update_reservation_passengers().
  • Limitations and code: Results depend on the quality of the LLM-generated guards, and only one benchmark was tested. Code is released at github.com/AgentToolkit/toolguard-violation-analysis.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢IBMArmonk · United States139 works · 70,691 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksEngineering · 133 worksSoftware · 92 worksIndustrial and Manufacturing Engineering · 10 worksBusiness, Management and Accounting · 32 worksManagement Information Systems · 4 works

Cite

Cite as Ella Rabinovich, David Boaz, Naama Zwerdling and Ateret Anaby Tavor. 2026. Near-Miss: Latent Policy Failure Detection in Agentic Workflows. Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.gem-main.30

@inproceedings{rabinovich2026near,
  title = {{Near-Miss: Latent Policy Failure Detection in Agentic Workflows}},
  author = {Ella Rabinovich, David Boaz, Naama Zwerdling and Ateret Anaby Tavor},
  booktitle = {Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)},
  year = {2026},
  doi = {10.18653/v1/2026.gem-main.30},
  url = {https://doi.org/10.18653/v1/2026.gem-main.30}
}

References

24 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. Claude model documentation. https://docs.anthropic.com/claude/ docs/models-overview. Accessed: 2026. ↗
  2. Sumanth Balaji, Piyush Mishra, Aashraya Sachdeva, and Suraj Agrawal. 2026. Beyond ivr: Benchmarking customer support llm agents for business-adherence. Preprint, arXiv:2601.00596. find
  3. Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. 2025. τ 2 -bench: Evaluating conversational agents in a dual-control environment. Preprint, arXiv:2506.07982. find
  4. Zhaorun Chen, Mintong Kang, and Bo Li. 2025. Shield-Agent: Shielding agents via verifiable safety policy reasoning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 8313–8344. PMLR. find
  5. Alejandro Cuadron, Pengfei Yu, Yang Liu, and Arpit Gupta. 2025. Saber: Small actions, big errors - safeguarding mutating steps in llm agents. Preprint, arXiv:2512.07850. — [arXiv:2512.07850](https://arxiv.org/abs/2512.07850) find
  6. Sarik Ghazarian, Abhinav Gullapalli, Swair Shah, Anurag Beniwal, Nanyun Peng, Narayanan Sadagopan, and Zhou Yu. 2025. TOD-procbench: Benchmarking complex instruction-following in task-oriented dialogues. In First Workshop on Multi-Turn Interactions in Large Language Models. find
  7. Google DeepMind. 2025. Gemini 3: A family of highly capable multimodal models. https://deepmind.google/technologies/gemini/. ↗
  8. Kung-Hsiang Huang, Akshara Prabhakar, Sidharth Dhawan, Yixin Mao, Huan Wang, Silvio Savarese, Caiming Xiong, Philippe Laban, and Chien-Sheng Wu. 2025. CRMArena: Understanding the capacity of LLM agents to perform professional CRM tasks in realistic environments. In Proceedings of the 2025 Conference…
  9. Zekun Li, Shinda Huang, Jiangtian Wang, Nathan Zhang, Antonis Antoniades, Wenyue Hua, Kaijie Zhu, Sirui Zeng, William Yang Wang, and Xifeng Yan. 2025. Agentorca: A dual-system framework to evaluate language agents on operational routine and constraint adherence. arXiv preprint arXiv:2503.08669. — [arXiv:2503.08669](https://arxiv.org/abs/2503.08669) find
  10. Nikita Mehandru, Brenda Y Miao, Eduardo Rodriguez Almaraz, Madhumita Sushil, Atul J Butte, and Ahmed Alaa. 2024. Evaluating large language models as agents in the clinic. NPJ digital medicine, 7(1):84. find
  11. Moonshot AI. 2024. Kimi: An efficient large language model for long context reasoning. https://kimi.moonshot.cn. ↗
  12. Itay Nakash, George Kour, Koren Lazar, Matan Vetzler, Guy Uziel, and Ateret Anaby Tavor. 2025. Effective red-teaming of policy-adherent agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2250–2268, Suzhou, China. Association for Computational Lin… find
  13. OpenAI. 2025a. Codex models. find
  14. OpenAI. 2025b. Gpt-4.1 model family. https://platform.openai.com/docs/models/gpt-4.1. Accessed: 2026. ↗
  15. OpenAI. 2025c. Gpt-5 technical report. https://openai.com. GPT-5 model family. ↗
  16. OpenAI. 2025d. Gpt-oss: Open source large language models. https://openai.com. ↗
  17. Nils Palumbo, Sarthak Choudhary, Jihye Choi, Prasad Chalasani, and Somesh Jha. 2026. Policy compiler for secure agentic systems. Preprint, arXiv:2602.16708. — [arXiv:2602.16708](https://arxiv.org/abs/2602.16708) find
  18. Qwen Team. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. find
  19. Francesco Romeo, Luigi Arena, Francesco Blefari, Francesco Aurelio Pironti, Matteo Lupinacci, and Angelo Furfaro. 2025. Arpaccino: an agentic-rag for policy as code compliance. In European Conference on Advances in Databases and Information Systems, pages 467–481. Springer. find
  20. Haoyu Wang, Christopher M. Poskitt, and Jun Sun. 2025. Agentspec: Customizable runtime enforcement for safe and reliable llm agents. arXiv preprint arXiv:2503.18666. — [arXiv:2503.18666](https://arxiv.org/abs/2503.18666) find
  21. Yijia Xiao, Edward Sun, Di Luo, and Wei Wang. 2024. TradingAgents: Multi-Agents LLM Financial Trading Framework. arXiv preprint arXiv:2412.20138. — [arXiv:2412.20138](https://arxiv.org/abs/2412.20138) find
  22. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2024. tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv preprint arXiv:2406.12045.
  23. Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, and 1 others. 2023. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854.
  24. Naama Zwerdling, David Boaz, Ella Rabinovich, Guy Uziel, David Amid, and Ateret Anaby Tavor. 2025. Towards enforcing company policy adherence in agentic workflows. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 595–606. Association fo… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rabinovich-2026-near-miss-latent-policy-failure/