⚖️ Law

A survey · about 13 minutes to read · every paper named here has its own page in this library

The Law shelf, a subfield of the social sciences, gathers work in which language technology meets legal text: statutes, court judgments, contracts, privacy policies, bar-exam questions and, at the historical end, Roman, rabbinic and Islamic legal writing. Most of the papers here belong to legal natural language processing, often called legal AI, which asks whether machines can classify, retrieve, summarise, translate or reason over documents whose wording carries binding force. A smaller group treats legal texts of the past as objects of philological study, from Justinian's Digest to medieval Hebrew responsa.

What unites these strands is the peculiar nature of legal language. It is long, heavily structured, full of terms whose ordinary meaning differs from their technical sense, and tied to a particular jurisdiction. A model that reads news well may stumble over a European human-rights judgment, a Chinese criminal verdict or a Moroccan family-law exam. The shelf therefore documents a steady move from generic tools to domain-adapted ones, and from English and Chinese towards many more languages and legal systems.

What it is

Legal NLP is an applied field. Its tasks are defined by what lawyers, judges, clerks and citizens actually do: find the relevant provision, predict how a court will rule, flag a risky clause, summarise a bill, translate a judgment, or explain a statute in plain words. Two features set it apart from neighbouring work on the Artificial Intelligence shelf.

First, legal documents are long and hierarchical. A judgment has a preamble, a statement of facts, the parties' arguments, the court's reasoning and an operative part; a statute nests articles, paragraphs and points.

Second, correctness has an external standard. A legal answer is right or wrong with respect to an authority that can be checked: a case either exists or it does not, a provision either applies or it does not. This makes hallucination, faithfulness to sources and explanation central concerns rather than afterthoughts.

The shelf overlaps heavily with Language and Linguistics, because many of its papers describe corpora and annotation schemes, and with Machine translation, because legal translation is a demanding special case. Its dominant topic is Legal AI; a distinct cluster sits under Semitic languages and Hebrew & Aramaic, and a few papers concern Latin.

History and milestones

Computational work on law is far older than this collection. Expert systems for statutory reasoning and models of legal argument date back to the 1970s and 1980s, and the International Conference on AI and Law (ICAIL) has met since 1987. The papers here, however, record the field's later, data-driven phase.

Corpora and early neural models (2016–2019). The earliest work on the shelf is the OPP-115 privacy-policy corpus, a carefully annotated set of website policies. In 2018 two Chinese efforts reshaped the field: the very large CAIL2018 collection of criminal cases and the TopJudge model, which treated a verdict as a chain of linked decisions. In 2019 the Athens–Sheffield group released English benchmarks for judgment prediction at the European Court of Human Rights and for classification of EU legislation, while BillSum opened up summarisation of US bills.

Domain-adapted transformers and benchmarks (2020–2022). LEGAL-BERT showed that pre-training on legal text pays off, and the Tsinghua survey How Does NLP Benefit Legal System mapped the field. Evaluation then consolidated: LexGLUE for English, CUAD for contracts, ILDC for India and Swiss-Judgment-Prediction for a trilingual court. Explainability became a task of its own with paragraph-level rationale extraction.

Large language models (2023–2024). With general chat models, the questions changed. LegalBench and LawBench measured legal skills directly; Lawyer LLaMA and Chatlaw built specialised assistants; and Stanford's studies of legal hallucinations and of commercial research tools tested how far such systems can be trusted.

Diversification (2025–2026). The newest papers spread the field to agents, inheritance reasoning, simplification, historical texts and many more languages.

Main problems and tasks

Legal judgment prediction

Predicting a court's decision from the facts is the most studied task here. In civil-law systems such as China's, it splits into predicting the applicable article, the charge and the penalty; at the European Court of Human Rights, it means predicting which Convention articles were violated. It is a proxy: facts written by the court already reflect its view, so high scores do not mean a system could advise litigants.

Classification and information extraction

Assigning topic labels to legislation, tagging provision types in contracts, finding clauses of interest and extracting entities or relations all belong here. Large, long-tailed label sets such as EUROVOC make few-shot and zero-shot labels a recurring theme, and German legal relation extraction shows how synthetic data can work around privacy limits on real case files. This connects to Information extraction and Named entity recognition.

Question answering and legal reasoning

Bar-exam questions, statutory puzzles and consultation dialogues test whether a system can recall rules and apply them to facts. Statutory reasoning with GPT-3 used invented statutes to separate memorised knowledge from genuine rule application. Work on Islamic inheritance offers a closed rule system with exactly checkable answers. See also Question answering and Reasoning.

Summarisation, simplification and drafting

Bills and judgments are long and opaque to non-lawyers. Summarisation (BillSum), simplification for Portuguese and several common-law jurisdictions, and structured drafting of private documents in India all aim to widen access to law. Related work sits under Summarization.

Legal translation

Legal terms often have no exact counterpart in another legal system. Papers here evaluate terminology errors in Arabic–English and Arabic–French, build an English–Assamese corpus of judgments, test reasoning models on South Tyrolean German terminology and argue for domain-specific tuning on EU legislation.

Reliability, explanation and interpretation

Hallucinated citations, uncalibrated confidence and sensitivity to prompt wording are serious in legal use. A 2026 study asked whether models can stand in for ordinary readers when interpreting insurance wording, and found that even reasoning models diverge from human consensus.

Methods and schools

The 2020 Tsinghua survey framed the field as a contrast between two traditions, and the distinction still helps.

Symbol-based approaches encode legal knowledge explicitly: attributes of offences, ordered decision steps, rule trees, knowledge graphs. Few-shot charge prediction labelled each charge once with a handful of legal attributes, such as whether violence was involved, and let every case inherit them, which helped most for rare charges. TopJudge imposed the order in which a judge decides. LADAN learned what separates near-identical statutes from their texts rather than from hand-made features.

Embedding-based approaches learn representations from large corpora. The arc runs from hierarchical recurrent networks with attention, through label-wise attention for huge label sets, to transformers. Because BERT reads only 512 subword tokens, hierarchical variants that encode paragraphs separately and then combine them became standard for judgments; LEGAL-BERT and later legal encoders added in-domain pre-training. CUAD and LawBench both found that more labelled data or better instructions can matter more than costly domain pre-training.

Large language models brought prompting, instruction tuning and retrieval augmentation. Domain recipes combine continued pre-training, supervised tuning on exam questions with explanations, and retrieval of statutes. CBR-RAG reuses the retrieval step of case-based reasoning, a classic AI-and-law idea, to pick past cases as context. LegalAgentBench evaluates agents that call tools over legal databases. At the other extreme, AYN argues that a very small model trained from scratch on Indian legal text can compete with much larger general models.

The main debates follow from this: whether general models plus retrieval make specialised legal models unnecessary; whether judgment prediction is a meaningful task or a misleading proxy; how to evaluate explanations against expert rationales; and how to measure hallucination when vendors claim their tools are free of it.

Datasets, resources and benchmarks

The shelf is rich in shared resources, many of them introduced by papers in this library:

LexGLUE and LEXTREME were later added to Stanford's HELM suite. Smaller resources cover Arabic court rulings, Moroccan law exams, Islamic inheritance cases, Korean statutes and Jordanian laws.

Landmark papers in this library

LEGAL-BERT (2020)

LEGAL-BERT: The Muppets straight out of Law School compared using BERT unchanged, continuing its pre-training on legal documents, and training a legal model from scratch with its own vocabulary. Adapted models won, especially on the harder tasks, and a compact version came close at a fraction of the cost. It is the most cited work on the shelf and became the default legal encoder for several years.

Neural judgment prediction for the ECtHR (2019)

Neural Legal Judgment Prediction in English released a public set of European Court of Human Rights cases with three tasks: whether any article was violated, which ones, and how important the case is. Its hierarchical BERT handled long fact descriptions that plain BERT had to truncate, and the authors checked whether anonymising the facts changes predictions.

TopJudge and CAIL2018 (2018)

Legal Judgment Prediction via Topological Learning modelled article, charge and penalty as subtasks in a directed acyclic graph, so each prediction is conditioned on those it depends on, cutting down on contradictory outputs. CAIL2018 supplied millions of Chinese criminal cases and exposed how skewed charge distributions are.

The LegalAI survey (2020)

How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence set out the symbol-versus-embedding framing, named knowledge modelling, legal reasoning and interpretability as open problems, and showed QA models trailing humans badly on bar-exam questions.

Legal hallucinations (2024)

Large Legal Fictions asked general chat models verifiable questions about US federal cases and found fabrication to be widespread, worse for older and less prominent cases, and poorly reflected in the models' own confidence. Its companion, Hallucination-Free?, ran a pre-registered test of retrieval-based commercial research tools and found that they still produced false or unsupported answers on a substantial share of queries.

LegalBench (2023)

LegalBench was assembled with legal professionals and sorts its tasks by the kinds of reasoning lawyers recognise, such as issue-spotting, rule recall and rule application, in the spirit of the IRAC method taught in law schools. It showed that model performance varies sharply by reasoning type and by prompt wording.

EURLEX57K and MultiEURLEX (2019, 2021)

Large-Scale Multi-Label Text Classification on EU Legislation built a benchmark for assigning EUROVOC concepts, reporting results separately for frequent, rare and unseen labels. MultiEURLEX extended it to parallel laws in 23 languages and found that light-weight adaptation such as adapters preserves multilingual knowledge better than full fine-tuning.

CUAD (2021)

CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review asked models to find clauses of legal interest in long contracts, with labels from trained annotators. It showed fast gains with newer architectures and identified annotated data, not model size, as the main bottleneck.

LexGLUE (2021)

LexGLUE gathered seven English datasets into a single benchmark modelled on GLUE, giving the field a common yardstick and confirming consistent gains from legal pre-training.

ILDC (2021)

ILDC for CJPE brought judgment prediction to the Supreme Court of India and paired it with explanation: systems are trained only on outcomes but evaluated against passages chosen by legal experts. The gap to expert accuracy and to expert rationales remained wide.

People

Ilias Chalkidis is the most prolific author on the shelf. His papers, from the ECtHR and EURLEX benchmarks to LEGAL-BERT, LexGLUE, MultiEURLEX and rationale extraction, built much of the English and European infrastructure of legal NLP; he is now associated with the University of Copenhagen and the IT University of Copenhagen.

Ion Androutsopoulos of the Athens University of Economics and Business and the Athena Research Center led the Athens group behind LEGAL-BERT and the EU legislation work. Prodromos Malakasiotis and Manos Fergadiotis, also at Athens University of Economics and Business, co-authored LEGAL-BERT, the rationale study and MultiEURLEX.

Nikolaos Aletras brought a long-standing interest in predicting ECtHR outcomes to the collaboration and co-authored LEGAL-BERT, the English judgment-prediction benchmark and LexGLUE.

At Tsinghua, Maosong Sun and Zhiyuan Liu led the group that produced CAIL2018, TopJudge, attribute-based charge prediction and the LegalAI survey. Cunchao Tu, Chaojun Xiao and Haoxi Zhong were central authors of that work, and Zhipeng Guo co-authored TopJudge and CAIL2018. Yansong Feng of Peking University contributed to CAIL2018 and later to Lawyer LLaMA.

Daniel E. Ho of Stanford, with Matthew Dahl, Varun Magesh and Mirac Suzgun, wrote the two hallucination studies; Christopher D. Manning joined the evaluation of commercial tools.

Joel Niklaus and Matthias Stürmer built the Swiss judgment benchmark and LEXTREME, pushing legal NLP beyond English.

In India, Arnab Bhattacharya of IIT Kanpur, Shubham Kumar Nigam and Kripabandhu Ghosh of IISER Kolkata connect ILDC, the AYN model and the VidhikDastaavej drafting resource.

Others with a single but influential paper here include Dan Hendrycks and Collin Burns for CUAD, Eduard Hovy and Frederick Liu for OPP-115, Nuo Xu for LADAN, and Benjamin Van Durme for the statutory reasoning study.

Institutions and places

Three centres dominate the older, most cited work. Copenhagen, through the University of Copenhagen and the IT University of Copenhagen, is the current home of the shelf's most prolific author. Greece is represented by the Athens University of Economics and Business and the Athena Research Center. Chinese legal AI is anchored at Tsinghua University and Peking University, with links to the Chinese Academy of Sciences.

Stanford dominates the US hallucination studies, and India's contributions centre on the Indian Institute of Technology Kanpur. Other affiliations in the author list include Imperial College London, Northeastern University, the Beijing University of Posts and Telecommunications and Google.

Venues show a shift. The landmark papers appeared at ACL, EMNLP, NAACL and COLING, and at the Natural Legal Language Processing workshop; the recent wave appears at LREC, at the workshop on technologies for historical and ancient languages, and at the European machine translation conference.

Historical and religious law

A distinctive part of this shelf treats legal texts as historical documents. Restoration of Latin notarial deeds from medieval Bologna fills damaged gaps with masked language models; a study of lemmatising Justinian's Digest pairs a Latin parser with a language-model correction step to support a legal glossary. On the Hebrew side, papers cover semantic roles of participants in the Holiness Code, citation recognition in medieval responsa, extraction of prison terms from Israeli sentencing decisions and personalising models to individual judges. A logic-based paper draws on Talmudic public announcement logic to explain a neural tagger. Islamic law appears through inheritance reasoning benchmarks, including MAWARITH and an evaluation on QIAS 2025, and through Moroccan legal QA.

Open questions

• Long documents. Judgments and contracts still exceed what many models read comfortably; hierarchical encoding and chunking remain workarounds.
• Faithfulness and hallucination. Retrieval reduces but does not remove invented or unsupported citations; public, independent benchmarks are still scarce.
• Explanations that lawyers accept. Machine rationales overlap only modestly with expert ones, and attention maps can latch onto spurious cues.
• What judgment prediction measures. Facts written by the court already encode its reasoning, so the task's practical value is contested.
• Jurisdictional and linguistic breadth. Results on US, EU and Chinese law transfer poorly to Korean, Moroccan, Indian or Swiss systems, and to non-dominant varieties.
• Bias and fairness. Demographic effects in predictions and in aggregated annotations need routine checks; see Bias, fairness & safety.
• Evaluation of generation. Overlap metrics often disagree with human judges, as an Arabic ruling-prediction study showed.

Neighbouring topics

Half of the shelf also stands on Language and Linguistics, mostly through corpus and annotation papers, and a large share on Artificial Intelligence. The legal translation papers link it to Machine translation and the topic of the same name, Machine translation. LLM benchmarking connects it to Artificial Intelligence and to Evaluation & benchmarks. Historical legal texts form the bridge to History.

Within the shelf, Legal AI covers almost everything. Semitic languages groups the Hebrew and Arabic work, Reasoning the statutory and inheritance studies, Language models & pretraining the domain-adapted models, and Multilingual & cross-lingual NLP the European benchmarks.

Key works in this library

• LEGAL-BERT: The Muppets straight out of Law School — domain-adapted BERT models for law.
• Neural Legal Judgment Prediction in English — ECtHR benchmark and hierarchical BERT.
• How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence — survey and roadmap of legal AI.
• Legal Judgment Prediction via Topological Learning — article, charge and penalty as ordered subtasks.
• Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models — how often chat models invent case law.
• LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models — legal reasoning tasks written by lawyers.
• CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction — millions of Chinese criminal cases.
• The Creation and Analysis of a Website Privacy Policy Corpus — the OPP-115 corpus.
• Large-Scale Multi-Label Text Classification on EU Legislation — EURLEX57K and label-wise attention.
• Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools — testing commercial legal assistants.
• CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review — clause finding in contracts.
• Distinguish Confusing Law Articles for Legal Judgment Prediction — telling similar statutes apart.
• LexGLUE: A Benchmark Dataset for Legal Language Understanding in English — unified English benchmark.
• ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation — prediction with expert-checked explanations.
• Few-Shot Charge Prediction with Discriminative Legal Attributes — legal attributes for rare charges.
• LawBench: Benchmarking Legal Knowledge of Large Language Models — Chinese civil-law benchmark for LLMs.
• BillSum: A Corpus for Automatic Summarization of US Legislation — summarising bills.
• Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases — explaining ECtHR predictions.
• MultiEURLEX - A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer — EU laws in 23 languages.
• Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark — a trilingual court.
• LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain — multilingual legal suite.
• LEDGAR: A Large-Scale Multi-label Corpus for Text Classification of Legal Provisions in Contracts — provision labels from contract headings.
• Can GPT-3 Perform Statutory Reasoning? — applying tax rules and invented statutes.
• LegalAgentBench: Evaluating LLM Agents in Legal Domain — tool-using agents over legal data.
• MAWARITH: A Dataset and Benchmark for Legal Inheritance Reasoning with LLMs — step-by-step Islamic inheritance.