📄 Morpho-syntactic Arabic Preprocessing for Arabic to English Statistical Machine Translation

Summary

Arabic attaches conjunctions, prepositions, the article and pronouns directly to words. As a result, an Arabic–English parallel corpus has a far larger and sparser Arabic vocabulary, with many singletons, and this hurts alignment in statistical MT. The authors preprocess the Arabic side in several steps:

• tokenize it,
• strip diacritics,
• delete the article Al on adjectives, since it only marks agreement,
• split words into prefixes, stem and one suffix.

They compare four segmenters inside the RWTH phrase-based log-linear system on the BTEC and NIST tasks:

• the supervised tool of Diab et al. (2004), trained on the Arabic TreeBank (SL);
• a frequency-based splitter (FB);
• a finite state automaton (FSA);
• an improved automaton (IFSA).

IFSA handles prefixes and suffixes jointly and refuses to split ambiguous words. It is unsupervised and rule-based. It improved over the unsegmented baseline on all three tasks, including the full NIST corpus with UN data, where the supervised tool was too slow to run. This shows that cheap, corpus-driven segmentation can match heavier linguistic tools for MT preprocessing.

Key points

• Prefixes handled are w, f, k, l, b and Al, with s (future tense) added for the automata. Suffixes are the pronoun clitics (y, ny, k, km, h, hA, hm, nA…). Plural endings such as At, An and wn stay attached because they carry no separate English word.
• On the BTEC test set, BLEU rose from 58.1 (no segmentation) to 61.3 with FSA or IFSA. SL reached 59.2 and FB 59.8. mWER dropped from 27.2 to about 25.3.
• SL won on the NIST news task but lowered BLEU on the BTEC dev set. It also could not process the large UN corpus in reasonable time.
• FB helped on small BTEC data but failed on NIST, where it over-segmented as the corpus grew. Gains from segmentation in general shrank as training data increased.
• For FSA, two segmentation passes were enough; further iterations mostly added wrong splits.

Interesting ideas

• I1. A split is accepted only if the resulting stem already occurs in the corpus, so the corpus itself guards against inventing nonsense stems.
• I2. Impossible affix orders, such as nothing being allowed to follow Al, reveal when a letter belongs to the stem rather than being an affix, and these facts become automatic rules.
• I3. When a word could be read with or without an affix, the segmenter leaves it whole and lets a later, more context-aware stage decide.
• I4. Markers that only express agreement, like the article repeated on adjectives, are removed because they have no counterpart on the other side.
• I5. Segmenters were judged by downstream translation quality across corpus sizes, which exposed that a method helping on small data (FB) can hurt on large data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): The English morphology and lemmatizer could decompose unknown words (re-, un-, -ness, -ly) only when the remaining stem is attested in the lexicon or corpus, which keeps the decomposition explainable.
• I2 (learn): Self-induced repair rules could be derived from grammatical impossibilities, such as forbidden morpheme or tag sequences, giving cheap, justified constraints without labeled data.
• I3 (tree): The tokenizer could pass ambiguous segmentations, such as contractions or 's, forward unresolved so the parser chooses with full sentence context instead of committing early.
• I4 (back): In translation through the meta-world, definiteness can be stored once per entity, with surface agreement markers generated or discarded per language instead of copied word by word.
• I5 (learn): Preprocessing changes should be evaluated end to end at several training-data scales, so that apparent gains on small sets are not mistaken for general improvements.