📄 Context-based Arabic Morphological Analysis for Machine Translation

Summary

Arabic packs articles, prepositions, pronouns and inflection into single words, while English spells most of these out separately. Word-level SMT alignment therefore suffers from many-to-one links and many unknown words. Earlier preprocessing schemes treated each affix type with one global rule: always keep it, always drop it, or always merge it. This paper makes that decision depend on context. Arabic is first fully segmented into prefix–stem–suffix morphemes with the disambiguator of Smith et al. (2005), which builds on BAMA, and case markers are removed. GIZA++ then aligns English to these morphemes. Affixes that receive no English link are labelled "Deleted" and all others "Retained", while stems are always kept. A CRF learns to predict these labels from morpheme and POS context, so it can be applied to test sentences that have no translation. Moses systems trained on the reduced input gained about two BLEU on travel (BTEC, 20K pairs) and news (177K pairs) data. They also beat a full 5M-pair baseline on unseen MT06 when trained on only a 500K-pair subsample. The approach shows that target-side alignment can supervise source-side morphology.

Key points

• Full segmentation roughly doubles the number of Arabic tokens and yields about 1.7× as many tokens as the English side. Only about 5% of English words end up unaligned to any morpheme.
• The CRF tagger was trained with CRF++ using a Gaussian prior. Its features cover:
  • the morpheme in a window of ±2;
  • the current POS plus up to 3 previous POS tags;
  • morpheme–POS combinations.
• Reduced morphology cut unknown test tokens by more than half on every set, for example from 3734 to 1342 on MT06 and from 1463 to 553 on MT03.
• Full (unreduced) morphology needs a longer reordering window. On news data, the original Arabic system gained about 4 BLEU going from monotone decoding to a 4-word limit, while BTEC barely benefited.
• CRF++ ran out of memory on the 5M-pair (128M-word) corpus even on a 24GB machine. The authors therefore sub-sampled 500K pairs covering the MT03/MT06 source phrases. On MT06 this reached 43.20 BLEU, against 41.59 for the sub-sampled baseline and 42.17 for the full-corpus baseline.

Interesting ideas

• I1. Supervision for a monolingual decision (keep or drop an affix) is derived for free from bilingual alignment, then distilled into a classifier that needs only the source side at test time.
• I2. Noisy alignment is used only where it is least risky: on affixes, never on stems, which keeps alignment errors from damaging content words.
• I3. The same preprocessing model is applied to both training and test data so that the two token distributions match, rather than using oracle labels in training.
• I4. The paper shows that finer segmentation interacts with the reordering window: more tokens per word demand longer-distance movement, which weak reordering models cannot handle.
• I5. Coverage-based sub-sampling of training data was used to cope with tool memory limits, and the subset still beat the full-data baseline on unseen test data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Silver labels for token-level choices, such as splitting clitics or contractions, could come from agreement with an aligned resource and then be learned as an explainable classifier.
• I2 (learn): When building silver data from imperfect witnesses, restrict automatic relabelling to closed-class or functional items and keep content-bearing units fixed.
• I3 (tokens): Any normalization the tagger or lemmatizer sees in training should be produced by the same predicted pipeline used at inference, to avoid train/test mismatch.
• I4 (back): In graph-to-text or translation, measure how finer-grained units change ordering difficulty before choosing the unit size for generation.
• I5 (learn): Selecting training sentences that cover the evaluation domain's patterns may give an efficient parser training set when GPU memory or time is limited.