📄 MEDAR: Collaboration between European and Mediterranean Arabic Partners to Support the Development of Language Technology for Arabic

Summary

This LREC'08 paper describes MEDAR, an EU-funded project that follows NEMLAR (2003–2005). Its goal is to strengthen language technology for Arabic through lasting cooperation between European and Arabic-speaking partners. It reports no experiments. It sets out a plan in three overlapping streams. The technical stream updates the NEMLAR survey and the BLARK (Basic Language Resource Kit) for Arabic, with a new focus on machine translation and multilingual or cross-lingual information retrieval. It also lists what blocks existing tools from being used for Arabic, proposes cost-effective annotation processes, and sets up open-source baselines with modest benchmarking. The cooperation stream produces three linked roadmaps: technology, market, and cooperation. The cooperation roadmap is the new addition and puts staff exchange and education first. The dissemination stream uses a newsletter, a website, translators' conferences and a 2009 conference. The authors argue that the project is mainly about collaboration rather than technology. They also argue that evaluation, including user evaluation, drives progress.

Key points

• MEDAR builds on NEMLAR (2003–2005), which produced the first directory of Arabic HLT players and resources and the NEMLAR BLARK for Arabic (2006). MEDAR has 15 partners in Europe, North Africa and the Middle East.
• The technical focus is MT and MLIR/CLIR. The project names basic language-dependent components that may be needed for Arabic, such as a lemmatizer, a POS tagger and a vowelizer.
• For CLIR, the project studies Cross-Language Spoken Document Retrieval (CLSDR). This combines retrieval, translation and speech recognition. The baseline uses text queries to retrieve spoken documents.
• A full evaluation campaign is too expensive. Instead, MEDAR adapts resources from earlier campaigns such as CESTA (Hamon et al. 2006) and adds user-based evaluation. Tools are measured before and after new resources (bilingual lexica, named entities) are added.
• The roadmap extends the usual technology and market roadmaps with a cooperation dimension. A follow-up conference to Cairo 2004 is planned for 2009.

Interesting ideas

• I1. The project first defines a minimal kit of essential resources (the BLARK) and then surveys what exists against it, so gaps become explicit and can be ranked by priority.
• I2. Each tool is measured before and after a specific resource is plugged in, which isolates what that resource actually contributes.
• I3. Before reusing generic tools for a new language, the project lists every obstacle and language-dependent component that would block the reuse.
• I4. Automatic metrics are paired with evaluation by real users, because scores alone can miss what matters in practice.
• I5. An open-source baseline system is kept as a common reference so that improvements are comparable and not tied to one system.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): We could keep a BLARK-style checklist of the resources each stage needs (treebanks, lexica, VerbNet frames, test sets) and regularly audit which are missing or weak.
• I2 (world): When WordNet, FrameNet or Wikidata knowledge is added, run the same QA and world-building tests with and without it, so each source shows its measurable value.
• I3 (tokens): When adapting tokenizer, tagger or lemma rules to new text genres, first list which components depend on the genre or language instead of finding the breakages one by one.
• I4 (answer): Besides accuracy on school tests and fable QA, have people judge whether Mova's step-by-step explanations are actually convincing and useful.
• I5 (tree): Keep a fixed, simple open baseline parser next to the perceptron parser and gparse, so every gain in LAS is reported against the same reference.