Elamur Research⌕
☰
home / papers / 2026 / Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs

Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs

Paper · 2026 · Chen Yang, Ruping Xu, Ruizhe Li et al.

Summary

The paper tackles turning unstructured business documents, such as regulations, manuals, and compliance guides, into structured rule flows that machines can execute. Earlier procedural-extraction work mostly models linear action sequences like recipes. It misses the conditional branching and parallel requirements common in regulatory texts, a mismatch the authors call the "Logic Gap". To address it, they build BREX (Business Rule Extraction Benchmark) from real-world documents in more than 30 domains, with a small, expert-revised share of synthetic text. Each rule is an atomic condition–action pair, and rules are linked through Sequential, Conditional, or Parallel dependencies. The authors also propose ExIde, a prompt-only framework that needs no fine-tuning. It compares five prompting strategies, from implicit semantic alignment to pseudo-code generation, and is tested on 13 LLMs. Generating executable pseudo-code beats standard prompts, and reasoning-optimized models track long-range, non-linear dependencies better than instruction-tuned ones. This matters for process automation where privacy or cost rules out custom training.

Key points

  • BREX contains 409 documents and 2,855 expert-annotated rules, about 7 rules per document. Over 30% of rules take part in conditional or parallel relations.
  • A condition is encoded as a ⟨Slot Type, Logical Operator, Reference Value⟩ triple paired with an action. Three experts annotated the data, and three more verified it, with agreement measured by ICC and Kappa.
  • Gemini 2.5 Pro generated synthetic texts only to fill gaps in rare structures such as nested conditions and parallel constraints. Experts filtered and revised all of them.
  • Stated limitations:
    • Pairwise dependency checking grows quadratically with the number of rules.
    • The schema does not cover temporal constraints, exceptions, or probabilistic rules.
  • Code and data are released at github.com/oYoungCo/Business-as-Rulesual (ACL 2026).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AberdeenAberdeen · United Kingdom23 works · 1,583 citations🎓University of BirminghamBirmingham · United Kingdom27 works · 62,564 citations🎓Zhejiang University of TechnologyHangzhou · China8 works · 98 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksPolitical Science and International Relations · 206 worksBusiness, Management and Accounting · 40 worksManagement Information Systems · 9 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 works

Cite

Cite as Chen Yang, Ruping Xu, Ruizhe Li, Bin Cao and Jing Fan. 2026. Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1625

@inproceedings{yang2026business,
  title = {{Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs}},
  author = {Chen Yang, Ruping Xu, Ruizhe Li, Bin Cao and Jing Fan},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1625},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1625}
}

References

21 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lars Ackermann, Julian Neuberger, and Stefan Jablon-ski. 2021. Data-driven annotation of textual process descriptions based on formal meaning representations. In International Conference on Advanced Information Systems Engineering, pages 75– 90. Springer. find
  2. Ron Artstein. 2017. Inter-annotator agreement. Handbook of linguistic annotation, pages 297–313. find
  3. Patrizio Bellan, Mauro Dragoni, and Chiara Ghidini. 2022a. Extracting business process entities and relations from text using pre-trained language models and in-context learning. In International Conference on Enterprise Design, Operations, and Computing, pages 182–199. Springer. find
  4. Patrizio Bellan, Chiara Ghidini, Mauro Dragoni, Simone Paolo Ponzetto, and Han van der Aa. 2022b. Process extraction from natural language text: the PET dataset and annotation guidelines. In Proceedings of the Sixth Workshop on Natural Language for Artificial Intelligence (NL4AI 2022) co-located wit… find
  5. Diogo S Candido, Joao Victor Berti Lima, Hilário Oliveira, and Mateus B Costa. 2024. An annotated dataset for automatic extraction of entities and restrictions from business process models. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), pages 978–989. SBC. find
  6. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  7. Weihong Du, Wenrui Liao, Hongru Liang, and Wenqiang Lei. 2024. PAGED: A benchmark for procedural graphs extraction from documents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, page… find
  8. Elena Viorica Epure, Patricia Martín-Rodilla, Charlotte Hug, Rebecca Deneckère, and Camille Salinesi. 2015. Automatic process model discovery from textual methodologies. In 2015 IEEE 9th international conference on research challenges in information science (RCIS), pages 19–30. IEEE. find
  9. Fabian Friedrich, Jan Mendling, and Frank Puhlmann. 2011. Process model generation from natural language text. In Advanced Information Systems Engineering: 23rd International Conference, CAiSE 2011, London, UK, June 20-24, 2011. Proceedings 23, pages 482–496. Springer. find
  10. Wenyan Guo, Qingtian Zeng, Hua Duan, Guiyuan Yuan, Weijian Ni, and Cong Liu. 2018. Automatic extraction of emergency response process models from chinese plans. IEEE Access, 6:74104–74119. find
  11. Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song, LiuYiBo LiuYiBo, Qianguosun Qianguosun, Yuxin Liang, Hao Wang, Enming Zhang, and Jiaxing Zhang. 2024. Never lost in the middle: Mastering long-context question answering with position-agnostic decompositional training. In Proceedings of the 62nd A… find
  12. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In ICLR 2022.
  13. Humam Kourani, Alessandro Berti, Daniel Schuster, and Wil MP van der Aalst. 2024. Process modeling with large language models. In International Conference on Business Process Modeling, Development and Support, pages 229–244. Springer. find
  14. Hongru Liang, Jia Liu, Weihong Du, Dingnan Jin, Wenqiang Lei, Zujie Wen, and Jiancheng Lv. 2023. Knowing-how & knowing-that: A new task for machine comprehension of user manuals. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10550–10564, Toronto, Canada. Association f… find
  15. Hugo A López, Rasmus Strømsted, Jean-Marie Niyo-dusenga, and Morten Marquard. 2021. Declarative process discovery: Linking process and textual views. In International Conference on Advanced Information Systems Engineering, pages 109–117. Springer. find
  16. Hirokuni Maeta, Tetsuro Sasada, and Shinsuke Mori. 2015. A framework for procedural text understanding. In Proceedings of the 14th International Conference on Parsing Technologies, pages 50–60. find
  17. Carolyn R Miller. 1979. A humanistic rationale for technical writing. College English, 40(6):610–617. find
  18. Kuntal Kumar Pal, Kazuaki Kashihara, Pratyay Banerjee, Swaroop Mishra, Ruoyu Wang, and Chitta Baral. 2021. Constructing flow graphs from procedural cybersecurity texts. arXiv preprint arXiv:2105.14357. find
  19. Vira Pyrih, Adrian Rebmann, and Han van der Aa. 2025. Llms that understand processes: Instruction-tuning for semantics-aware process mining. In 7th International Conference on Process Mining, ICPM 2025, Montevideo, Uruguay, October 20-24, 2025, pages 1–8. IEEE. find
  20. Chen Qian, Lijie Wen, Akhil Kumar, Leilei Lin, Li Lin, Zan Zong, Shu’ang Li, and Jianmin Wang. 2020. An approach for process model extraction by multi-grained text classification. In Advanced Information Systems Engineering: 32nd International Conference, CAiSE 2020, Grenoble, France, June 8–12, 202… find
  21. Jianhua Zhang, Muhammad Usman Shehzad, Sher Ali, and Ziao Cao. 2025. Unlocking digital innovation: a moderated-mediation approach exploring the knowledge creation processes, it-enabled capabilities and absorptive capacity in software smes. Business Process Management Journal, 31(1):170–201. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-business-as-rulesual-benchmark-framework/