Elamur Research⌕
☰
home / papers / 2026 / COMPACT: Building Compliance Paralegals via Clause Graph Reasoning over Contracts

COMPACT: Building Compliance Paralegals via Clause Graph Reasoning over Contracts

Paper · 2026 · Ayush Singh, Dishank Aggarwal, Pranav Bhagat et al.

Summary

Checking contract compliance often depends on how several provisions interact. Definitions, carve-outs and time conditions in one clause can change obligations stated elsewhere. Existing legal NLP resources such as ContractNLI and CUAD mostly test single clauses. The authors propose COMPACT, a framework that represents each contract as a clause graph. With GPT-4o, each clause is broken down into five elements describing its obligation and timing: the party bound, the type of duty (the deontic modality), the required action, what that action applies to, and any temporal condition. Related clauses are then clustered and connected by typed edges. The graphs serve as a source for sampling complex clause combinations. From these, the authors generate adversarial scenarios labelled compliant, non-compliant or not applicable, and check them with an LLM. The resulting ACE benchmark shows that current models handle multi-clause cases poorly, with base accuracy of 34–57%. Small LLaMA and Qwen models fine-tuned on reasoning traces distilled from a larger model improve by 22–43 accuracy points. They also tend to perform better on PrivaCI-Bench and ContractNLI. This suggests that compact, inexpensive models can learn reasoning skills that transfer to enterprise compliance work.

Key points

  • ACE contains 4,700 multi-hop compliance scenarios built from 633 real contracts (sourced from CUAD and ContractNLI) that span 26 agreement types.
  • Clause graphs use six relation types: DEFINES, COMPLEMENTS, EXCEPTIONS, DEPENDS_ON, CONFLICTS and SEQUENCE.
  • On the ACE test set, fine-tuning raised LLaMA-8B accuracy from 34.5 to 77.6 and Qwen-7B from 56.8 to 78.5. F1 gains reached up to +61 points.
  • Transfer was tested on PrivaCI-Bench (2,617 EU AI Act and 120 HIPAA samples) and on ContractNLI. Gains were mostly positive, but some results dropped, notably LLaMA-8B on HIPAA.
  • Limitations include English-only contracts and inputs of roughly 6k tokens. The framework also assumes the relevant clauses have already been identified.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢FujitsuKawasaki · Japan2 works · 53 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksPolitical Science and International Relations · 206 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Ayush Singh, Dishank Aggarwal, Pranav Bhagat, Ainulla Khan, Sameer Malik and Amar Prakash Azad. 2026. COMPACT: Building Compliance Paralegals via Clause Graph Reasoning over Contracts. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.377

@inproceedings{singh2026compact,
  title = {{COMPACT: Building Compliance Paralegals via Clause Graph Reasoning over Contracts}},
  author = {Ayush Singh, Dishank Aggarwal, Pranav Bhagat, Ainulla Khan, Sameer Malik and Amar Prakash Azad},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.377},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.377}
}

References

29 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 1996. Health Insurance Portability and Accountability Act of 1996. Pub. L. 104–191, 110 Stat. 1936. Federal law mandating national standards for the security and privacy of health data. find
  2. Sher Badshah and Hassan Sajjad. 2024. Reference-guided verdict: Llms-as-judges in automatic evaluation of free-form text. arXiv preprint arXiv:2408.09235. — [arXiv:2408.09235](https://arxiv.org/abs/2408.09235) find
  3. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2024. Qwen technical report. arXiv preprint arXiv:2309.16609.
  4. Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Martin Katz, and Nikolaos Aletras. 2021. Lexglue: A benchmark dataset for legal language understanding in english. arXiv preprint arXiv:2110.00976.
  5. Xu Chu, Zhijie Tan, Hanlin Xue, Guanyu Wang, Tong Mo, and Weiping Li. 2025. Domaino1s: Guiding llm reasoning for explainable answers in high-stakes domains. arXiv preprint arXiv:2501.14431. — [arXiv:2501.14431](https://arxiv.org/abs/2501.14431) find
  6. Subhasis Dasgupta, Jon Stephens, and Amarnath Gupta. 2025. Olg++: A semantic extension of obligation logic graph. arXiv preprint arXiv:2507.05488. find
  7. Wei Fan, Haoran Li, Zheye Deng, Weiqi Wang, and Yangqiu Song. 2024. Goldcoin: Grounding large language models in privacy laws via contextual integrity theory. Preprint, arXiv:2406.11149. find
  8. Laura Giordano, Alberto Martelli, and Daniele Thesei-der Dupré. 2013. Temporal deontic action logic for the verification of compliance to norms in asp. In ICAIL, pages 53–62. find
  9. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems, volume 27, pages 2672–2680.
  10. Guido Governatori. 2010. Law, logic and business processes. In 2010 Third International Workshop on Requirements Engineering and Law, pages 1–10. find
  11. Guido Governatori, Joris Hulstijn, Régis Riveret, and Antonino Rotolo. 2007. Characterising deadlines in temporal modal defeasible logic. In AI 2007: Advances in Artificial Intelligence, pages 486–496, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  12. Neel Guha, Julian Nyarko, Daniel Ho, Christopher Ré, Adam Chilton, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, and 1 others. 2023. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in neural informati…
  13. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  14. Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. 2021. Cuad: An expert-annotated nlp dataset for legal contract review. arXiv preprint arXiv:2103.06268.
  15. Yuta Koreeda and Christopher D Manning. 2021. Contractnli: A dataset for document-level natural language inference for contracts. arXiv preprint arXiv:2110.01799.
  16. Haoran Li, Wei Fan, Yulin Chen, Cheng Jiayang, Tianshu Chu, Xuebing Zhou, Peizhao Hu, and Yangqiu Song. 2025a. Privacy checklist: Privacy violation detection grounding on contextual integrity theory. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for… find
  17. Haoran Li, Wenbin Hu, Huihao Jing, Yulin Chen, Qi Hu, Sirui Han, Tianshu Chu, Peizhao Hu, and Yangqiu Song. 2025b. Privaci-bench: Evaluating privacy with contextual integrity and legal compliance. arXiv preprint arXiv:2502.17041.
  18. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  19. Qiang Lu, Jack G. Conrad, Khalid Al-Kofahi, and William Keenan. 2011. Legal document clustering with built-in topic segmentation. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, CIKM ’11, pages 383–392. ACM. find
  20. Mark Newman. 2010. Networks: an introduction. Oxford University Press. find
  21. Helen Nissenbaum. 2004. Privacy as contextual integrity. Washington Law Review, 79(1):119–158. find
  22. Arjun Panickssery, Samuel R. Bowman, and Shi Feng. 2024. Llm evaluators recognize and favor their own generations. arXiv preprint arXiv:2404.13076. find
  23. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982– 3992.…
  24. Chantal Shaib, Joe Barrow, Jiuding Sun, Alexa F. Siu, Byron C. Wallace, and Ani Nenkova. 2024. Standardizing the measurement of text diversity: A tool and a comparative analysis of scores. arXiv preprint arXiv:2403.00553. — [arXiv:2403.00553](https://arxiv.org/abs/2403.00553) find
  25. Harry Surden. 2018. Artificial intelligence and law: An overview. Ga. St. UL Rev., 35:1305. find
  26. Richard Susskind and Daniel Susskind. 2017. The future of the professions: How technology will transform the work of human experts. Journal of Nursing Regulation, 8(2):52. find
  27. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  28. Pat Verga, Sebastian Hofstätter, Sophia Althammer, Yixuan Su, Aleksandra Piktus, Arkady Arkhangorodsky, Minjie Xu, Naomi White, and Patrick Lewis. 2024. Replacing judges with juries: Evaluating llm generations with a panel of diverse models. arXiv preprint arXiv:2404.18796.
  29. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/singh-2026-compact-building-compliance-paralegals-clause/