JuriFindIT: an Italian legal retrieval dataset
Paper · 2026 · Niko Dalla Noce, Davide Colla, Sina Farhang Doust et al.
Summary
Statutory article retrieval (SAR) means finding the legislative provisions that answer a natural-language legal question. It is difficult because everyday wording differs from legal terminology, and because statutes depend on each other through hierarchy and citations. No SAR resource existed for Italian. The authors built JuriFindIT by parsing Akoma Ntoso files from European and Italian sources, including laws, decrees, regulations and authority guidelines. They also extracted cross-article references automatically with an XLM-RoBERTa tagger. Legal experts wrote questions in four areas and linked each one to its relevant articles. A large set of synthetic questions supplements the expert ones.
On top of this data, the authors fine-tuned a dense retriever (DAR) and combined it with a GNN-based second stage (DAR+GIR) that runs over a heterogeneous legislative graph. This pipeline outperforms earlier SAR systems on BSARD and LLeQA. The work matters for two reasons: it extends SAR to a new jurisdiction and language, and it is the first SAR dataset to explicitly encode references between articles.
Key points
- Dataset scale: JuriFindIT contains 895 questions authored by experts plus 169,301 synthetic questions, all mapped to over 23,000 articles. It covers civil law, criminal law, anti-money laundering/counter-terrorism, and privacy.
- Graph construction:
- Nodes are either hierarchical (roots, chapters, sections) or content nodes holding article text.
- Edges are either structural or reference edges.
- A synthetic super-root joins all files into one graph.
- References: Cross-references were extracted with a proprietary Aptus.AI model, based on XLM-RoBERTa with BIO tagging, that detects, segments and classifies references.
- BSARD test set:
- DAR (305M parameters) reaches recall@100 of 88.4 and mAP of 59.7, against 82.7 and 35.3 for DSR.
- DAR+GIR reaches mAP of 60.9, against 47.1 for G-DSR.
- LLeQA validation set:
- DAR improves recall@5 from 48.6 (CamemBERT) to 59.2.
- DAR+GIR achieves MRR@10 of 73.6.
- Limitations:
- All annotation was done by a single expert team, without independent cross-verification.
- Adding more synthetic queries can lower retrieval quality.
Institutions
Fields, topics and concepts
📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksPolitical Science and International Relations · 182 works
🧭 Topics 2Research subjects and languages the work is about.Information retrieval & search · 473 worksLegal AI · 134 works
Cite
Cite as Niko Dalla Noce, Davide Colla, Sina Farhang Doust, Lorenzo De Mattei and Davide Bacciu. 2026. JuriFindIT: an Italian legal retrieval dataset. Findings of the Association for Computational Linguistics: EACL 2026.
@inproceedings{noce2026jurifindit,
title = {{JuriFindIT: an Italian legal retrieval dataset}},
author = {Niko Dalla Noce, Davide Colla, Sina Farhang Doust, Lorenzo {De Mattei} and Davide Bacciu},
booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
year = {2026},
doi = {10.18653/v1/2026.findings-eacl.221},
url = {https://doi.org/10.18653/v1/2026.findings-eacl.221}
}References
- Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 8440–8451. Association for Computational Linguistics. find
- Davide Bacciu, Federico Errica, Alessio Micheli, and Marco Podda. 2020. A gentle introduction to deep learning for graphs. Neural Networks, 129:203–221. ↗
- Nigel J. Balmer, Ash Patel, Alexy Buck, Catrina Denvir, and Pascoe Pleasence. 2010. Knowledge, Capacity and the Experience of Rights Problems. Public Legal Education Network: PLENet. find
- Shaked Brody, Uri Alon, and Eran Yahav. 2022. How attentive are graph attention networks? In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. ↗
- Dominique Brunato, Andrea Cimino, Felice Dell’Orletta, Giulia Venturi, and Simonetta Montemagni. 2020. Profiling-ud: a tool for linguistic profiling of texts. In Proceedings of The 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France, May 11-16, 2020, pages 7145–7151. European Language Resources Association. find
- Tianle Cai, Shengjie Luo, Keyulu Xu, Di He, Tie-Yan Liu, and Liwei Wang. 2021. Graphnorm: A principled approach to accelerating graph neural network training. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 1204–1215. PMLR. ↗
- Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. 2019. Neural legal judgment prediction in english. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 4317–4323. Association for Computational Linguistics.
- Ilias Chalkidis, Manos Fergadiotis, and Ion Androutsopoulos. 2021. Multieurlex - A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 6974–6996. Association for Computational Linguistics. ↗
- Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael J. Bommarito II, Ion Androutsopoulos, Daniel Martin Katz, and Nikolaos Aletras. 2022. Lexglue: A benchmark dataset for legal language understanding in english. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 4310–4330. Association for Computational Linguistics. ↗
- Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. CoRR, abs/2003.07820. ↗
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4171–4186. Association for Computational Linguistics.
- Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with pytorch geometric. CoRR, abs/1903.02428. ↗
- Luyu Gao, Yunyi Zhang, Jiawei Han, and Jamie Callan. 2021. Scaling deep contrastive learning batch size under memory limited setup. In Proceedings of the 6th Workshop on Representation Learning for NLP, RepL4NLP@ACL-IJCNLP 2021, Online, August 6, 2021, pages 316–321. Association for Computational Linguistics.
- Randy Goebel, Yoshinobu Kano, Mi-Young Kim, Juliano Rabelo, Ken Satoh, and Masaharu Yoshioka. 2024. Overview and discussion of the competition on legal information, extraction/entailment (COLIEE) 2023. Rev. Socionetwork Strateg., 18(1):27–47. ↗
- Jiafeng Guo, Yinqiong Cai, Yixing Fan, Fei Sun, Ruqing Zhang, and Xueqi Cheng. 2022. Semantic models for the first-stage retrieval: A comprehensive review. ACM Trans. Inf. Syst., 40(4):66:1–66:42. ↗
- William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 1024–1034. ↗
- Harvard Law School Library Innovation Lab. 2018. Caselaw access project. https://case.law. ↗
- Matthew L. Henderson, Rami Al-Rfou, Brian Strope, Yun-Hsuan Sung, László Lukács, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil. 2017. Efficient natural language response suggestion for smart reply. CoRR, abs/1705.00652. ↗
- Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. 2021a. CUAD: an expert-annotated NLP dataset for legal contract review. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual. ↗
- Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. 2021b. CUAD: an expert-annotated NLP dataset for legal contract review. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual. ↗
- Abe Bohan Hou, Orion Weller, Guanghui Qin, Eugene Yang, Dawn J. Lawrie, Nils Holzenberger, Andrew Blair-Stanek, and Benjamin Van Durme. 2025. CLERC: A dataset for u. s. legal case retrieval and retrieval-augmented analysis generation. In Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, New Mexico, USA, April 29 - May 4, 2025, pages 7898–7913. Association for Computational Linguistics. find
- Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous graph transformer. In WWW ’20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, pages 2704–2710. ACM / IW3C2. find
- Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6769–6781. Association for Computational Linguistics.
- Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. ↗
- Dawn J. Lawrie, Efsun Kayi, Eugene Yang, James Mayfield, Douglas W. Oard, and Scott Miller. 2025. Generate-distill: Training cross-language IR models with synthetically-generated data. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2025, Padua, Italy, July 13-18, 2025, pages 2926–2930. ACM. find
- Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Sasko, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, and 13 others. 2021. Datasets: A community library for natural language processing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2021, Online and Punta Cana, Dominican Republic, 7-11 November, 2021, pages 175–184. Association for Computational Linguistics.
- Daniele Licari and Giovanni Comandè. 2024. ITALIAN-LEGAL-BERT models for improving natural language processing tasks in the italian legal domain. Comput. Law Secur. Rev., 52:105908. ↗
- Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. find
- Antoine Louis and Gerasimos Spanakis. 2022. A statutory article retrieval dataset in french. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 6789–6803. Association for Computational Linguistics. find
- Antoine Louis, Gijs van Dijck, and Gerasimos Spanakis. 2023. Finding the law: Enhancing statutory article retrieval via graph neural networks. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2023, Dubrovnik, Croatia, May 2-6, 2023, pages 2753–2768. Association for Computational Linguistics. find
- Antoine Louis, Gijs van Dijck, and Gerasimos Spanakis. 2024. Interpretable long-form legal question answering with retrieval-augmented large language models. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 22266– 22275. AAAI Press.
- Sean MacAvaney, Nicola Tonellotto, and Craig Macdonald. 2022. Adaptive re-ranking with a corpus graph. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, October 17-21, 2022, pages 1491–1500. ACM. find
- Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2023. MTEB: massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2023, Dubrovnik, Croatia, May 2-6, 2023, pages 2006–2029. Association for Computational Linguistics.
- Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan T. McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation LREC 2016, Portorož, Slovenia, May 23-28, 2016. European Language Resources Association (ELRA).
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, and 2 others. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 8024–8035.
- Alejandro Ponce, Sarah Chamness Long, Elizabeth Andersen, Camilo Gutiérrez Patiño, Matthew Harman, Jorge A. Morales, Ted Piccone, Natalia Rodríguez Cajamarca, Adriana Stephan, Kirssy González, Jennifer VanRiper, Alicia Evangelides, Rachel Martin, Priya Khosla, Lindsey Bock, Erin Campbell, Emily Gray, Amy Gryskiewicz, Ayyub Ibrahim, and 3 others. 2019. Global insights on access to justice 2019: Findings from the world justice project general population poll in 101 countries. https://worldjusticeproject.org/our-work/research-and-data/ global-insights-access-justice-2019. ↗
- Lance A. Ramshaw and Mitch Marcus. 1995. Text chunking using transformation-based learning. In Third Workshop on Very Large Corpora, VLC@ACL 1995, Cambridge, Massachusetts, USA, June 30, 1995.
- Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 3980–3990. Association for Computational Linguistics.
- T. Y. S. S. Santosh, Rashid Haddad, and Matthias Grabmair. 2024. Ecthr-pcr: A dataset for precedent understanding and prior case retrieval in the european court of human rights. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/-COLING 2024, 20-25 May, 2024, Torino, Italy, pages 5473–5483. ELRA and ICCL. ↗
- Hsuan-Lei Shao, Yi-Chia Chen, and Sieh-Chuen Huang. 2020. Bert-based ensemble model for statute law retrieval and legal information entailment. In New Frontiers in Artificial Intelligence - JSAI-isAI 2020 Workshops, JURISIN, LENLS 2020 Workshops, Virtual Event, November 15-17, 2020, Revised Selected Papers, volume 12758 of Lecture Notes in Computer Science, pages 226–239. Springer. find
- Jianlin Su, Murtadha H. M. Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063. ↗
- Chongyang Tao, Jiazhan Feng, Tao Shen, Chang Liu, Juntao Li, Xiubo Geng, and Daxin Jiang. 2023. CORE: cooperative training of retriever-reranker for effective dialogue response selection. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 3102–3114. Association for Computational Linguistics. ↗
- Don Tuggener, Pius von Däniken, Thomas Peetz, and Mark Cieliebak. 2020. LEDGAR: A large-scale multi-label corpus for text classification of legal provisions in contracts. In Proceedings of The 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France, May 11-16, 2020, pages 1235–1241. European Language Resources Association.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998–6008.
- Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. ↗
- Fabio Vitali, Monica Palmirani, Roger Sperberg, and Véronique Parisse. 2018. Akoma ntoso version 1.0. part 2: Specifications. OASIS Standard. Edited by Fabio Vitali, Monica Palmirani, Roger Sperberg, and Véronique Parisse. find
- Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
- Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Tianyang Zhang, Xianpei Han, Zhen Hu, Heng Wang, and Jianfeng Xu. 2019. CAIL2019-SCM: A dataset of similar case matching in legal domain. CoRR, abs/1911.08962. ↗
- Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, and Min Zhang. 2024. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: EMNLP 2024 - Industry Track, Miami, Florida, USA, November 12-16, 2024, pages 1393–1412. Association for Computational Linguistics. Body (Organismo Agenti e Mediatori, OAM). The distribution of documents across these entities is reported in table 6. ↗
- Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. CoRR, abs/2506.05176. A Entity Italian State (Parliament/Government) European Parliament & Council (co-legislators) Council of the European Union UIF – Unità di Informazione Finanziaria Bank of Italy Italian Data Protection Authority (Garante) European Data Protection Board MEF – Ministry of Economy & Finance (Italy) European Banking Authority AgID – Agency for Digital Italy Article 29 Working Party European Commission – DG JUST IVASS – Insurance Supervisory Authority (Italy) Bank of Italy & UIF (joint) ANAC – National Anti-Corruption Authority (Italy) Bank of Italy, UIF & MEF (joint) European Commission European Commission – DG FISMA Dip. Trasformazione Digitale – PCM (Italy) Joint EBA & ESMA European Central Bank Governments of the Member States (EU Treaties) Ministry for Enterprises and Made in Italy (MIMIT) Ministry of Justice (Italy) OAM – Agents and Credit Brokers Body (Italy) Dataset The corpus comprises normative and para-normative sources issued by both European and national authorities. At the European level, the documents include acts from the Council of the European Union, the European Parliament acting jointly with the Council, and the European Commission with its directorates-general for Justice and Consumers (DG JUST) and for Financial Stability, Financial Services and Capital Markets Union (DG FISMA). Regulatory and supervisory bodies are also represented, such as the European Data Protection Board (EDPB), the European Banking Authority (EBA), the European Central Bank (ECB), and joint issuances of the European Banking Authority and the European Securities and Markets Authority (EBA–ESMA). The collection further includes treaties adopted by the Governments of the Member States (GOVREP) and guidance from the Article 29 Working Party (WP29), predecessor of the EDPB. At the national level, the Italian State (Parliament and Government) accounts for the majority of documents, complemented by acts from the Ministry of Economy and Finance (MEF), the Ministry of Justice, the Ministry for Enterprises and Made in Italy (MIMIT), and the Presidency of the Council of Ministers through its Department for Digital Transformation (Dipartimento per la Trasformazione Digitale, PCM). Independent authorities also play a central role, including the Bank of Italy, the Financial Intelligence Unit (Unità di Informazione Finanziaria, UIF), the Italian Data Protection Authority (Garante per la protezione dei dati personali), the Insurance Supervisory Authority (Istituto per la Vigilanza sulle Assicurazioni, IVASS), and the National Anti-Corruption Authority (Autorità Nazionale Anticorruzione, ANAC). Additional contributions come from joint provisions by the Bank of Italy and UIF, collaborative issuances with MEF, the Agency for Digital Italy (Agenzia per l’Italia Digitale, AgID), and the Agents and Credit Brokers Count 35 23 20 14 13 11 10 5 4 3 3 2 2 2 2 1 1 1 1 1 1 1 1 1 1 *Table 6: Distribution of documents across European and Italian emitting entities.* A.1 Dataset samples ↗
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/noce-2026-jurifindit-italian-legal-retrieval-dataset/