📄 Applying Morphology Generation Models to Machine Translation

Summary

The paper tackles a common weakness of statistical MT when translating from English into languages with rich inflection: the system often picks the right word but the wrong form, so agreement and case are frequently broken. The authors build a standalone inflection generator, a Maximum Entropy Markov model following Minkov et al. (2007). It takes a sequence of target stems and selects a surface form for each one, using a wide range of features from the English source (POS tags, dependency structure, alignments) and from the target side. The model is trained separately from the translation system. It is then combined with a syntax-based treelet system and a phrasal system for English→Russian and English→Arabic. The authors compare several ways of joining the two parts, including running the base MT on full word forms or on stems, and passing n-best lists between the components. Every setup they test improves BLEU, and native-speaker judges also prefer the output. This shows that a separately built morphology component can improve MT systems of different architectures.

Key points

• Inflection accuracy on clean input: Given stemmed reference translations, the model chooses the correct form 91.6% of the time for Russian and 91.0% for Arabic. A trigram LM baseline reaches only 81.0% and 69.4%. Each stem has roughly 14 possible forms in Russian and 24 in Arabic.
• Lexicons and features:
  • The Russian lexicon was built by intersecting a general lexicon with the training data. Russian uses 7 morphological features and treats capitalization as an extra inflectional feature.
  • The Arabic lexicon comes from the Buckwalter analyser. Arabic uses 12 features.
  • No contextual disambiguation was applied in either language.
• English→Arabic treelet system: BLEU rose from 35.54 to 37.41 (Method 1, n=2). The variants built around stems raised the oracle BLEU but did not raise the BLEU actually obtained.
• English→Russian: The system was trained on about 1.6M sentence pairs.
  • In the treelet system, BLEU gains reached up to 2.6.
  • In the phrasal system, which had a much stronger baseline, BLEU went from 36.00 to 36.72 using 100-best input.
• Human evaluation: Two native judges compared 100 sentences per scenario. All average scores favored the new model: 0.1 for Arabic, and 0.255 and 0.26 for the two Russian setups. The Arabic gain was not statistically significant.

Interesting ideas

• I1. Keep morphology outside the decoder. A separate post-model is not limited by the decoder's phrase boundaries or context window, so it can use features from anywhere in the sentence. The same component also works with very different MT systems. This design suits any rich-feature correction step, such as case or agreement repair for Ukrainian.
• I2. Pass n-best lists between components instead of a single best guess. Carrying alternative translations into the inflection step improved results compared with a 1-best handoff. It is a cheap way to reduce error propagation in a pipeline.
• I3. Test the component on gold input first. Measuring inflection accuracy on stemmed reference translations gives an upper bound on what the module can do in isolation. This helps separate the component's own errors from errors caused by noisy MT output.
• I4. Fit stemming to the language pair. The authors suspect stemming hurt Arabic because it removed distinctions that English marks explicitly, such as definiteness and clitics. Normalization should only collapse what the source language leaves unmarked, which is a useful rule when designing lexicons for other language pairs.
• I5. Check BLEU against human judgement. Two Russian setups had very different BLEU gains (1.2 vs 2.6) but almost the same human preference scores. Small, targeted changes such as inflection fixes deserve direct human evaluation.