📄 The JRC-Acquis: A Multilingual Aligned Parallel Corpus with 20+ Languages

Summary

Large parallel corpora used to cover only a few languages, and they nearly always involved a major language such as English or French. Corpora with many languages were either small or very narrow in content. EuroParl, the broadest existing option, had eleven languages and none from the newer EU members or candidate countries. The JRC-Acquis fills this gap. It is built from the body of EU law (the Acquis Communautaire), which must be translated into every official language.

The authors crawled EU web sites and picked documents using availability thresholds. They then cleaned the texts and stored them as UTF-8 XML following the TEI guidelines. Each document was split into paragraph segments that line up across languages. Two aligners, Vanilla and HunAlign, produced paragraph alignments for every one of the 190+ language pairs. Most documents also carry manually assigned EUROVOC subject descriptors.

The result covers all 20 official EU languages plus Romanian, with roughly 8,000 documents per language, and is free for research use. It supports machine translation, terminology extraction, multi-label classification and cross-lingual benchmarking, especially for less-resourced languages and unusual pairs such as Estonian-Greek.

Key points

• Size: almost 8,000 documents and close to 9 million words per language on average, in 20 EU languages plus Romanian (5,792 Romanian texts). Croatian, Bulgarian and Turkish are planned.
• Document selection: no official list of Acquis documents could be obtained, so the authors inferred membership. A document was kept if it existed in at least 10 of the 21 languages, with a minimum of three versions drawn from the languages of the states that joined in 2004 or from Romanian.
• Identifiers: documents are tracked by their CELEX code. Each paragraph is uniquely addressed by language, CELEX ID and paragraph number.
• Pre-processing:
  • An n-gram language identifier removed texts in the wrong language.
  • Paragraphs were cut at HTML <BR>/<P> tags.
  • Regular expressions split each document into body, signature and annex; about half of the documents contain an annex.
• Alignments and labels:
  • Alignments come from both Vanilla and HunAlign and are distributed as TEI files containing pointers only. A bundled tool turns them into in-place aligned text.
  • EUROVOC (about 6,000 hierarchical descriptors) codes are included as numbers. The descriptor texts require a separate licence, free for research and 500 Euro for commercial use.

Interesting ideas

• I1. Use legal obligations as a source of parallel text. Material that institutions are required to translate into every language, such as an accession body of law, yields broad, consistent parallel data. The same approach could locate parallel text for Ukrainian through official translations of legislation or international agreements.
• I2. Store alignments as stand-off pointers. Keeping alignments separate from the texts, plus a script to rebuild aligned text, avoids duplicating the corpus for every pair. It also lets several aligners' outputs sit side by side.
• I3. Release more than one aligner's output. Shipping both Vanilla and HunAlign results turns the corpus into a benchmark for comparing alignment and sentence-splitting tools. This matters because such tools may perform unevenly across language pairs.
• I4. Infer membership with explicit thresholds. When no authoritative document list exists, a simple rule based on how many language versions are available defines a reproducible corpus. Naming the result distinctly ("JRC-Acquis") avoids implying it is the official legal collection.
• I5. Combine parallel text with shared subject labels. Because the same labels apply in every language, a classifier can be trained separately per language or with signals from several languages at once. The labels can also drive extraction of domain-specific bilingual terminology.