📄 Domain Adaptation for Machine Translation by Mining Unseen Words

Summary

The paper looks at why statistical MT trained on parliamentary text performs poorly in new domains, and isolates one cause: source words never seen in training. Europarl serves as the parallel source domain. Four target domains (News, Emea, Subs, PHP) are tested for German→English and French→English, using only comparable target-domain text plus small tuning and test sets. Oracle experiments show that unseen words cause about half of the extra error in most domains. News is the exception, since its vocabulary barely differs from Europarl. The authors extend the CCA-based lexicon induction of Haghighi et al. (2008), seeding it with Europarl translations of frequent target-domain words, and mine candidate translations for out-of-vocabulary terms. Simply adding mined pairs to a Moses-style phrase table gives little benefit. Adding them with dedicated features yields steady improvements, with the largest on Emea (about 1.5 Bleu). Overall, the method closes roughly half the gap between the baseline and an upper bound limited to OOV words. This shows that cheap comparable data can repair vocabulary gaps during domain adaptation.

Key points

• Data: Europarl (996k sentences) is the source domain. Target corpora are News Commentary and three OPUS sets: Emea, Open Subtitles, and PHP documentation.
  • Comparable data was simulated by pairing the first half of the English side with the second half of the foreign side.
• OOV rates: The share of target-domain words unseen in Europarl is 17% for News, 49% for Emea, 44% for PHP and 68% for Subs.
• Mining pipeline:
  • About 5000 frequent words per language are used.
  • Each word gets context features (window of 5, words truncated to 7 characters, top 2000 features, TFIDF, binarized) and orthographic 1–3-gram features. These are combined as linear kernels and reduced with incomplete Cholesky.
  • The seed pairs come from a Hungarian-algorithm matching over Europarl translation probabilities.
  • CCA keeps the top 35% of eigenvectors and returns 5 candidates per OOV word.
  • Native speakers judged 8 of 10 sampled words correctly translated.
• Oracles: Training on target-domain parallel data improves Emea Bleu by 73% relative. The upper bound from fixing only OOV words is about halfway between the baseline and the full oracle.
• Results:
  • German Emea Bleu rose from 26.62 to 28.06.
  • Gains in Subs and PHP were smaller but positive.
  • The system used two trigram LMs (Gigaword plus target-domain English) and MERT averaged over three runs.

Interesting ideas

• I1. Separate features for mined entries. Mined pairs get their own probability feature, with seen-in-source and seen-in-target indicators and their conjunction. Their ordinary phrase probabilities are set to zero. Only the full set of four features helped, so the tuner needs to weight mined and real entries independently. This applies to any case of merging a noisy lexicon into an existing model.
• I2. Diagnose with a restricted oracle first. Remove from the oracle only the phrases involving unseen words. This estimates how much a vocabulary fix could ever gain before any work goes into it. It is a reusable way to budget effort on a domain-shift problem.
• I3. Bootstrap from the source domain. Frequent target-domain words that already have Europarl translations become CCA training pairs. No hand-built seed dictionary is needed, which makes the approach attractive for language pairs with few resources.
• I4. Compare orthography through kernels. Character n-grams are turned into within-language similarity kernels rather than compared as raw features across languages. The authors argue this keeps orthographic cues usable even when the two languages use different scripts.
• I5. One-to-one seeds via bipartite matching. Many-to-many translation probabilities are reduced to a one-to-one seed set with maximum weighted bipartite matching. This is a simple fix for any method that expects a single partner per item.