Elamur Research⌕
☰
home / papers / 2009 / Domain Adaptation for Statistical Machine Translation with Monolingual Resources

Domain Adaptation for Statistical Machine Translation with Monolingual Resources

Summary

Statistical MT systems lose accuracy when the test domain differs from the training domain, and in-domain parallel text is usually scarce. Bertoldi and Federico examine whether large, inexpensive in-domain monolingual text can make up for this. They start from a Spanish–English Moses system trained on the UN corpus and adapt it to Europarl. The in-domain monolingual text, either English (target side) or Spanish (source side), is run through the baseline system, which produces an artificial parallel corpus. New translation, re-ordering and language models are then trained on that corpus and either used in place of the original models or combined with them, with the weights tuned on a small in-domain development set. Target-side data works best. BLEU goes from 22.60 to 28.10, and most of that gain comes from the adapted language model. Source-side data helps less, reaching 23.68. The study shows that cheap monolingual resources can substantially reduce domain mismatch when bilingual data is lacking.

Key points

  • Setup:
    • Moses phrase-based system with GIZA++ "grow-diag-final" alignments.
    • 5-gram LM built with IRSTLM, using Modified Kneser-Ney smoothing and quantization.
    • Cube pruning with a pop-limit of 6000; MERT for tuning.
    • Background domain is the UN corpus; the adaptation domain is Europarl (EP).
  • Results with English in-domain data:
    • Adapting the LM alone raises BLEU from 22.60 to 27.83.
    • Also adapting the TM and RM brings it to 28.10.
    • Even the smallest portion of English adaptation data adds +4.02 BLEU.
  • Results with Spanish in-domain data (synthetic English):
    • The adapted LM reaches 23.52.
    • Full adaptation of LM, TM and RM reaches 23.68.
  • Combining with background models:
    • In-domain TMs and RMs perform better on their own than when combined with the UN models.
    • Combining also raises the number of features from 14 to 26.
    • For LMs, adding the background model gives only a small benefit, which disappears as in-domain data grows.
  • Tuning:
    • A development set of 200 sentences gives weights about as good as a 2000-sentence set.
    • The n-best list size barely changes the quality of the tuned weights but strongly affects run time.

Interesting ideas

  • I1. Back-translating monolingual text into synthetic parallel data. Translating in-domain monolingual text with an existing system yields a "parallel" corpus for retraining. This could be reused for any low-resource language pair or niche domain, such as Ukrainian technical or trade texts, where only one side is easy to collect.
  • I2. Reusing decoder alignments instead of re-aligning. The decoder can output its own word alignments, so the costly GIZA++ step is skipped when training on synthetic data. This cut training time by about half, with negligible BLEU loss, at the price of roughly 70% more disk space. The general lesson is to keep by-products of generation instead of recomputing them.
  • I3. Merging phrase tables properly. Phrase tables are merged into a single table with concatenated feature columns, and missing observations are smoothed. Tuning the mixture is preferred over simply pooling the corpora, because pooling cannot weight the data sources against each other and performed worse in preliminary tests.
  • I4. Prioritizing target-side data. The language model accounts for most of the domain gain, so collecting target-side in-domain text is a cheap first step before investing in bilingual data.
  • I5. Budgeting the tuning effort. A few hundred in-domain parallel sentences are enough for weight optimization, and large n-best lists add compute without improving the weights. This is useful when annotation budgets are tight.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cited by

78 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 76 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{bertoldi2009domain,
  title = {{Domain Adaptation for Statistical Machine Translation with Monolingual Resources}},
  author = {Bertoldi, Nicola and Federico, Marcello},
  booktitle = {Proceedings of the Fourth Workshop on Statistical Machine Translation},
  year = {2009},
  doi = {10.3115/1626431.1626468},
  url = {https://doi.org/10.3115/1626431.1626468}
}

References

17 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–312. find
  2. Jorge Civera and Alfons Juan. 2007. Domain adaptation in statistical machine translation with mixture modelling. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 177–180, Prague, Czech Republic. find
  3. Matthias Eck, Stephan Vogel, and Alex Waibel. 2004. Language model adaptation for statistical machine translation based on information retrieval. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), pages 327–330, Lisbon, Portugal. find
  4. Marcello Federico and Renato De Mori. 1998. Language modelling. In Renato De Mori, editor, Spoken Dialogues with Computers, chapter 7, pages 199– 230. Academy Press, London, UK. find
  5. Marcello Federico, Nicola Bertoldi, and Mauro Cettolo. 2008. Irstlm: an open source toolkit for handling large scale language models. In Proceedings of Interspeech, pages 1618–1621, Melbourne, Australia. find
  6. Andrew Finch and Eiichiro Sumita. 2008. Dynamic model interpolation for statistical machine translation. In Proceedings of the Third Workshop on Statistical Machine Translation, pages 208–215, Columbus, Ohio. find
  7. George Foster and Roland Kuhn. 2007. Mixture-model adaptation for SMT. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 128–135, Prague, Czech Republic. find our card
  8. Almut Silja Hildebrand, Matthias Eck, Stephan Vogel, and Alex Waibel. 2005. Adaptation of the translation model for statistical machine translation based on information retrieval. In Proceedings of the 10th Conference of the European Association for Machine Translation (EAMT), pages 133–142, Budapes… find our card
  9. Philipp Koehn and Josh Schroeder. 2007. Experiments in domain adaptation for statistical machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 224–227, Prague, Czech Republic. find our card
  10. P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, and E. Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Com… find
  11. Philipp Koehn. 2002. Europarl: A multilingual corpus for evaluation of machine translation. Unpublished, http://www.isi.edu/∼koehn/europarl/. ↗
  12. Wolfgang Macherey, Franz Och, Ignacio Thayer, and Jakob Uszkoreit. 2008. Lattice-based minimum error rate training for statistical machine translation. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 725–734, Honolulu, Hawaii. find our card
  13. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
  14. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association of Computational Linguistics (ACL), pages 311–318, Philadelphia, PA. find our card
  15. Holger Schwenk. 2008. Investigations on Large-Scale Lightly-Supervised Training for Statistical Machine Translation. In Proc. of the International Workshop on Spoken Language Translation, pages 182–189, Hawaii, USA. find
  16. Nicola Ueffing, Gholamreza Haffari, and Anoop Sarkar. 2007. Semi-supervised model adaptation for statistical machine translation. Machine Translation, 21(2):77–94. find
  17. Bing Zhao, Matthias Eck, and Stephan Vogel. 2004. Language model adaptation for statistical machine translation via structured query models. In Proceedings of Coling 2004, pages 411–417, Geneva, Switzerland. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/bertoldi-2009-domain-adaptation-statistical-machine-translation/