📄 Chunking German Legal Code

Summary

The paper asks how German statutory law should be split into chunks for retrieval-augmented generation. The test corpus is the German Civil Code (BGB), which has about 2,400 sections. The authors index the BGB in many ways:

• Structural units: sections, subsections, sentences, LLM-derived propositions.
• Fixed-size windows: overlapping windows of uniform length.
• Modifying or aggregating methods: contextual chunking, semantic clustering, Lumber-style LLM boundary detection, and RAPTOR summary trees.

Each variant is scored on a German layperson legal QA dataset with section-level gold labels. Every retrieved unit is mapped back to its parent section, so all granularities are compared fairly. Besides Recall@10, the study records query latency, index build time and disk footprint.

The structure-preserving variants win: subsection and section retrieval give the best recall, followed by sentences and propositions. Methods that pool or cut across the legislator's boundaries do worse, and the LLM-heavy pipelines also cost far more to build. The practical lesson is that in highly structured domains, respecting the native document structure beats semantic enrichment in both accuracy and cost.

Key points

• Ranking: Recall@10 is highest for subsection and section chunks, then sentence and proposition chunks. Fixed windows, semantic clustering and Lumber-style chunking score measurably lower, and RAPTOR sits in between.
• Build time: RAPTOR indexes take roughly 3 h 4 m to 5 h 51 m to build. Lumber variants are the most expensive, at about 9 h 2 m to 11 h 41 m, because of extensive LLM preprocessing.
• Flat index sizes: these grow with granularity, from 33.2 MB (section) to 62.1 MB (subsection), 95.7 MB (sentence) and 131.9 MB (proposition). Contextual variants have the same sizes, since they embed the same number of units.
• Other index sizes: RAPTOR needs 68.5–144.1 MB because it stores leaves plus summary nodes, and Fixed 16/4 is the largest at 230.9 MB. Fixed 256/64 (14.7 MB) and semantic clustering (15.4–16.4 MB) are the smallest; clustering stays near one thousand vectors regardless of base unit.
• Limitations: only one layperson dataset, retrieval-only evaluation, no cross-reference resolution, and dependence on the chosen embedding models.

Interesting ideas

• I1. Indexing can happen at any granularity, while scoring always rolls hits up to a fixed parent unit; this makes very different segmentations directly comparable.
• I2. The authors explain clustering's weakness as averaging: one vector for many pooled provisions drifts toward the shared topic and away from the specific rule asked about.
• I3. Proposition splitting keeps all the conditions of a norm and isolates one consequence per proposition, so each unit is self-contained.
• I4. Every method is reported with offline build time, online latency and storage next to accuracy, which exposes when expensive LLM preprocessing is not worth it.
• I5. The proposed future work is reference-aware chunking: expand a unit with the definitions it relies on, including implicit ones signalled only by a defined term (e.g. "consumer").

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (answer): Fable/tale QA could locate evidence at sentence or clause level but grade it at paragraph level, letting us compare evidence-finding strategies on one scale.
• I2 (world): Store entity states and facts per clause rather than merging them into topic-level summaries, so an exception never gets blurred into its general rule.
• I3 (frames): Coordinated verb phrases ("deliver X and procure Y") could become separate frames that each inherit the shared agent and conditions, making each who-did-what step individually explainable.
• I4 (learn): When choosing among LLM teacher tiers or witness-parser ensembles, log compute and storage cost beside agreement gains to justify each tier.
• I5 (world): When a text uses a term defined elsewhere in it or in the ontology, link the mention to that definition node so downstream reasoning sees the full meaning.