📄 Improving Statistical MT through Morphological Analysis

Summary

Czech is heavily inflected, so most wordforms in a Czech–English parallel corpus are rare. That makes word alignment probabilities in a noisy-channel translation model hard to estimate, and many Czech inflectional contrasts have no English counterpart anyway. Goldwater and McClosky test how far purely morphological processing, with no syntactic parsing, can ease this. They use the morphologically annotated Czech side of the Prague Czech-English Dependency Treebank (PCEDT) and rewrite the source text in several ways:

• replacing wordforms with lemmas, either for all words, for selected parts of speech, or only for rare forms
• truncating words, as a baseline that needs no linguistic knowledge
• inserting single tags as standalone pseudoword tokens
• fusing tags onto lemmas

They also change the alignment model itself so that it operates on morphemes. Each tag class is evaluated separately with BLEU. The best system mixes treatments, with person and negation as pseudowords and number and tense attached to lemmas. It scores .333 BLEU on test against .270 for the word-to-word baseline. The result suggests that monolingual tools can partly compensate for scarce parallel data.

Key points

• Data: The Czech portion of PCEDT annotates every token with a lemma and a tag string covering up to 15 morphological dimensions, such as POS, gender, number, tense and negation.
• Methods: There are four ways of using morphology. Three transform only the input (lemmas, pseudowords, modified lemmas), keeping the standard word alignment model. The fourth, novel method modifies the alignment model so that it works at the morpheme level.
• Lemmatization and truncation: Plain lemmatization helped substantially despite discarding information. Lemmatizing only infrequent forms gave a slight but inconclusive gain. Fixed-length word truncation (following Och, 1995) beat the baseline but fell short of lemmatization.
• Case tags: These did not improve any of the three basic models and were left out of the final system.
• Combined model: It reached BLEU .390 on development and .333 on test, compared with .270 for the baseline. The underlying translation system is word-based.

Interesting ideas

• I1. Match each feature to how the target language expresses it. Features that English expresses with function words (person, negation) work best as separate tokens. Features that English expresses through inflection (number, tense) work best fused with the lemma. The same rule of thumb could guide preprocessing for any morphologically rich source language, Ukrainian included.
• I2. Use pseudowords for pro-drop. Emitting a person tag as its own token gives English subject pronouns something to align with when the source omits them. This trick could transfer to other pro-drop languages.
• I3. Normalize by frequency. Keeping full forms for frequent words and lemmatizing only rare ones targets sparsity without discarding information where data is sufficient. It is a cheap, tunable knob for low-resource settings.
• I4. Use truncation as a fallback. Cutting words to a fixed prefix recovers part of lemmatization's benefit with no analyzer at all. It is a sensible baseline when no morphological tools exist for a language.
• I5. Ablate one tag class at a time. The paper evaluates each class of morphological information separately with an automatic metric before combining them. This shows which features actually help and avoids adding useless ones, as with the case tags here.