📄 Cooperation for Arabic Language Resources and Tools –- The MEDAR Project

Summary

MEDAR was an EU-funded project with 15 partners. It aimed to strengthen Arabic human language technology (HLT) by building shareable resources and tools and by linking institutions in Europe and the Arab world. Its work ran in three overlapping streams: technical, cooperation roadmap and dissemination.

On the technical side, the team extended the BLARK for Arabic with more detail on parallel corpora. It then built English-to-Arabic MT baselines on MOSES, choosing the less-studied direction on purpose. The consortium also began collecting a parallel and monolingual corpus of about 50M words and ran a dry-run evaluation on climate-change texts. Without training data, the systems performed poorly: the best reached 65% WER and the best MEDAR baseline 80%. This showed that SMT needs large corpora.

The main contribution is a Cooperation Roadmap up to 2015. It adds cooperation as a third dimension next to technology and market and sets out phased priorities for education, basic resources and industry. The authors argue that the model could transfer to other regions.

Key points

• The team surveyed five open-source MT systems: Apertium, GenPar, JosHUa, Matxin and MOSES. MOSES won because of its community, earlier English-Arabic results and partner experience.
• University of Balamand built a baseline system, and IBM-Egypt with Dublin City University built an enhanced version.
• Web data was crawled, cleaned and aligned with Champollion Tool Kit and Hunalign, and legal negotiations ran alongside to clear the data for reuse.
• The test set held 10,000 words hidden inside a 200,000-word masking corpus, with four validated professional reference translations. Five external submissions, two baselines and two online systems took part, with anonymous results and a 10-day turnaround.
• The roadmap has three phases: foundations (2010–2012), moving forward (2012–2014) and consolidation (2013–2015). Baselines, test suites and training data were planned for release through ELRA.

Interesting ideas

• I1. The real test passages were buried inside a much larger decoy text, so participants could not find and tune to the scored items. This is a cheap guard against leakage.
• I2. The first campaign was openly called a dry run with anonymous results, because participants did not share equal training conditions. This keeps unfair comparisons from being read as rankings.
• I3. Several independent references were made under translation guidelines and then checked against separate validation guidelines. This gives automatic metrics a firmer base than one reference would.
• I4. The BLARK lists the minimal set of resources a language needs and marks what exists and what is missing. This makes gaps visible and helps set priorities.
• I5. The test domain was narrow, but its jargon has spread into everyday text. This makes it hard yet still realistic.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Gold evaluation items could be mixed into large unscored batches sent to the LLM teacher or self-play, so no component can quietly special-case the test set.
• I2 (learn): Early evaluations of new parser or reader variants could be marked as calibration runs and kept apart from official scores until conditions match.
• I3 (back): Generation and text → graph → text round-trips could be scored against several validated paraphrases, not one fixed string.
• I4 (world): A BLARK-style checklist of the compiled knowledge layers (verb classes, measures, ontology, knowledge graph) would show coverage gaps before they surface as reasoning failures.
• I5 (answer): QA test sets could include topics whose specialist vocabulary has entered general prose, to probe lexical robustness without going fully out of domain.