📄 Segmentation and Processing of German Court Decisions from Open Legal Data

Summary

The Open Legal Data dump of German court decisions has tidy metadata, but the decision bodies are stored as irregular HTML in which the standard parts of a ruling are often not marked. The authors build a cleaned corpus of 251,038 decisions from the 2022-10-18 dump. Each text is divided into Tenor (operative ruling), Tatbestand (facts) and Entscheidungsgründe (reasoning). The Rechtsmittelbelehrung (appeal instructions) goes into its own field.

The pipeline is entirely rule-based:

• visible HTML elements are flattened and whitespace is normalized;
• court location metadata is resolved through the Open Legal Data API;
• section starts are found by exact full-line header matches, including letter-spaced spellings;
• for Beschlüsse, a numbered Gründe section is mapped onto facts and reasoning;
• statute and case citations are attached with an existing extraction tool.

The authors sized a manual check with Cochran's formula and reviewed 384 randomly drawn decisions; 97.40% were segmented correctly. The JSONL release on Hugging Face can support retrieval, citation analysis, rhetorical role classification and section-aware RAG.

Key points

• The corpus has 251,038 decisions, published as harshildarji/openlegaldata (JSONL). Each record holds normalized metadata, sectioned text and extracted law/case references.
• Header detection works as follows:
  • It uses a fixed vocabulary (tenor, tatbestand, entscheidungsgründe, gründe).
  • Each term is matched by two case-insensitive full-line regex forms, one compact and one spaced-letter.
  • Text before the first header defaults to Tenor.
  • In Beschlüsse, Gründe I maps to Tatbestand and Gründe II to Entscheidungsgründe; an undivided Gründe counts entirely as reasoning.
• The verification sample size comes from Cochran's formula with p=0.5, e=0.05 and Z=1.96 (n₀≈384.16), with finite population correction (≈383.58), rounded up to 384. A decision counted as correct only if its sections did not overlap.
• 97.40% of the sample was correct, with a 95% CI of 95.8%–98.9% for the full corpus. The errors came mostly from rare HTML formatting quirks.
• Section coverage:
  • Tenor appears in 87.7% of decisions, Tatbestand in 65.4% and Entscheidungsgründe in 95.1%.
  • All three sections appear in 57.5%, Tenor plus reasoning only in 25.4%, and Tenor alone in 4.5%.
  • The appeal notice appears in 8,335 decisions (3.32%), and 176 decisions have empty content.

Interesting ideas

• I1. Instead of eyeballing a few outputs, the authors check a rule-based pipeline by drawing a statistically sized random sample, judging each item pass/fail by a strict criterion, and reporting a confidence interval.
• I2. Variant document layouts (Urteil vs. Beschluss, Gründe with or without I/II) are mapped onto one canonical schema, with an explicit fallback rule when the subdivision is missing.
• I3. Procedural boilerplate (the appeal notice) is stored apart from the substantive content, so it does not pollute downstream analysis of the decision itself.
• I4. The section a statement sits in changes what it means: a statute cited in the reasoning carries different weight from one in the operative part, and mixing facts with arguments causes false retrieval matches.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Silver trees or self-induced repair rules could be audited on ~384 random items with a binary criterion, so each reported accuracy carries a real error bar.
• I2 (tree): The same pattern suits UD dialect conversion, where each source convention gets a documented mapping plus a stated default for ambiguous cases.
• I3 (frames): Meta-text such as instructions, morals or editorial notes could be tagged before frame extraction, so no who-did-what frames are built from it.
• I4 (world): Each world entry could record which part of the text it came from, letting the world model separate narrated facts from claims or arguments when answering.